CR backend C++ code generator functionality is extracted. This will reduce the logical dependencies and simplifies some sections of the code. There are also improvements to make Generics to generate templates.
Anyway, this functionality cannot be done in all cases so I will have to find in future a work around. In short the problem is the following. Let's say we want to implement a class like MyData<T>, and T can be either a struct or a value. If T is a class type, it has to have separate logic than struct type. The reason is that in future, reference types have to have extra information to identify these types, a "typeId" integer type. As MyData<int> has no typeID necessary, and MyData<T> may need to keep the typeId of the specific type. Even if we share the typeId and in the case of Struct code will be a no-op, it still doesn't offer a nice solution if the MyData<T> has the new() constraint.
In short:
- there will be an improvement in future that will make generics to work with templates, but the today's code with Generics will do generate incomplete types (so you will get compiler errors). In fact many of these errors are medium complexity, and fixes are always welcome
- code is separated to make easier to understand how the C++ code is generated, this component is a smaller Dll
As a future improvement I would expect to try to (incompletely) implement the TypeId system: all classes defined as references will have an int that uniquely identify their types. This implementation detail is critical to make some instructions to work: is/as, boxing/unboxing and virtual/interface calls.
Update: CodeRefractor works with Linux/Mono (I tested with OpenSuse, but other distro do work nice too). If you use Linux you basically have to do the following:
- make a clone of Git repo
- Enable this repository on Suse, Fedora, or any supported distro: http://software.opensuse.org/download/package?project=home:tpokorra:mono&package=monodevelop-opt (or build a recent Mono/MonoDevelop by yourself, at least 4.0.x)
- open CodeRefractor.sln and run it under "Debug | Mixed platforms"
If you prefer command line after the initial build is done you can try to use it with:
mono cr.exe <any Mono/.Net exe>
I am really impressed of the quality of Mono tools and as for you, if you wanted to work on CR but you never knew that it runs on Linux, right now you know!
Saturday, February 15, 2014
Saturday, February 8, 2014
Target 0.0.3: Status 1
One two tree, done!
The compilation step is much faster. Optimizations can work (by uncommenting some lines of code) even by using multiple cores. As for example, the screenshot from the OpenGL application I had presented it earlier was taking like 600-700 ms on an Intel i5-540M. Right now it takes for CR to generate the code around 170-220 ms (the first time is slower, as spinning disks are slower at access). But by all measures is much faster.
Most optimization steps that CR perform do the following: look for a pattern of code that match a property, after that if it matches, it tries to perform the optimization by impacting some instructions, it notifies the compiler that some changes are done. CR after this will try to perform all optimizations up to the point no optimization can be done. CR in a typical case will apply (using the default codebase optimizations) 35+ optimization steps for every function, every instruction, etc.
In typical case let's say there is just one some optimizations that can be done, for example: a variable is nowhere used, so it can be removed. Before noticing that the variable can be removed, the compiler performs other optimizations, and right after will perform another pattern matching. At last when the optimization of the unused variable is match, for every instruction CR will track all declared variables and CR will compare with all used variables. At the end what remains, they can be removed.
But as someone can notice, the step that makes that one optimization succeed is based on some most common knowledge:
- variable usages per instructions
- the instruction kind: if one optimization will remove a declared but never used variable, it will have to take in account if the instruction is a call to a function or if is a simple math operation
- jumps: some optimizations do work just for a sequence of instructions that have no branches and jumps, so looking for jumps and labels are important delimiters for these instructions
Based on this, every time an optimization is performed, the optimization framework in code will recalculate this information so it can be reused. So if the first optimization step needs to check variable usages, and doesn't perform any optimization, the second optimization can use the same usages data, as correct as no change is done.
Some Q&A:
- even 200 ms (for a small) is fast, most time will be used still inside the C++ compiler, so why bother? Because is not always so clear-cut. Also, using a lower optimization level on GCC, or using a fast(er) C++ compiler, makes to matter
- 200 ms is still much (for a 40K application), cannot be reduced to 50 ms (CR doesn't do too much as for me)? In short CR does more than optimizations: it tries to understand the code, to map it to an intermediate representation, after optimizations are done (which is this post is done), CR writes the code on disk. CR have in its design some inneficiencies: for example most CIL instructions are compared using string operations (for clarity), not using the instruction IDs. Using instructions by Id will speedup some (few) milliseconds.
- compilation speed was never an issue for me, so why bother? The CR is intended to be an optimizing compiler/VM. Excluding the fact that most people have fast computers, some people do not have it. Many people develop using an Atom CPU. Also, having a fast compilation speed translates into using these 500 ms (saved) to add in future more optimizations.
The compilation step is much faster. Optimizations can work (by uncommenting some lines of code) even by using multiple cores. As for example, the screenshot from the OpenGL application I had presented it earlier was taking like 600-700 ms on an Intel i5-540M. Right now it takes for CR to generate the code around 170-220 ms (the first time is slower, as spinning disks are slower at access). But by all measures is much faster.
Most optimization steps that CR perform do the following: look for a pattern of code that match a property, after that if it matches, it tries to perform the optimization by impacting some instructions, it notifies the compiler that some changes are done. CR after this will try to perform all optimizations up to the point no optimization can be done. CR in a typical case will apply (using the default codebase optimizations) 35+ optimization steps for every function, every instruction, etc.
In typical case let's say there is just one some optimizations that can be done, for example: a variable is nowhere used, so it can be removed. Before noticing that the variable can be removed, the compiler performs other optimizations, and right after will perform another pattern matching. At last when the optimization of the unused variable is match, for every instruction CR will track all declared variables and CR will compare with all used variables. At the end what remains, they can be removed.
But as someone can notice, the step that makes that one optimization succeed is based on some most common knowledge:
- variable usages per instructions
- the instruction kind: if one optimization will remove a declared but never used variable, it will have to take in account if the instruction is a call to a function or if is a simple math operation
- jumps: some optimizations do work just for a sequence of instructions that have no branches and jumps, so looking for jumps and labels are important delimiters for these instructions
Based on this, every time an optimization is performed, the optimization framework in code will recalculate this information so it can be reused. So if the first optimization step needs to check variable usages, and doesn't perform any optimization, the second optimization can use the same usages data, as correct as no change is done.
Some Q&A:
- even 200 ms (for a small) is fast, most time will be used still inside the C++ compiler, so why bother? Because is not always so clear-cut. Also, using a lower optimization level on GCC, or using a fast(er) C++ compiler, makes to matter
- 200 ms is still much (for a 40K application), cannot be reduced to 50 ms (CR doesn't do too much as for me)? In short CR does more than optimizations: it tries to understand the code, to map it to an intermediate representation, after optimizations are done (which is this post is done), CR writes the code on disk. CR have in its design some inneficiencies: for example most CIL instructions are compared using string operations (for clarity), not using the instruction IDs. Using instructions by Id will speedup some (few) milliseconds.
- compilation speed was never an issue for me, so why bother? The CR is intended to be an optimizing compiler/VM. Excluding the fact that most people have fast computers, some people do not have it. Many people develop using an Atom CPU. Also, having a fast compilation speed translates into using these 500 ms (saved) to add in future more optimizations.
Tuesday, February 4, 2014
Skipping release 0.0.2 - jumping to 0.0.3
As for Git commits, you will see some activity in the last week. Anyway the single key features if you follow the Git repository are all reached (to some extent) and all were proposed for 0.0.2:
- better field matching
- auto-properties do work
- better delegates and generics (top Git code is broken now, look why in the 0.0.3 feature set)
- fixes in some cases like math with various sizes
- most optimizations (especially escape analysis) work over more code patterns
As many programmers know code fixing takes time and I took this time, but I will not pack an installer for 0.0.2 as I found that some features are important to be implemented in a different way and there is no support yet (and request) to make a release package, so why do it?
The changes of 0.0.3 will be:
- visibly faster compilation: use-def code will be computed and updated on changes, and not computed on every optimization. The current code is already in the Git repository
- more fixes (of course expect a breakage in the first time)
- the Dynamic-Invoke mechanism should be more fleshed out
- generics will fully use templates <T> instead of using specializations. This translates into a single compilation of one function (from CR side). If you will define a class List<T>, the old code will create List_Int32, List_String, etc
- expected: a fully inter-procedural optimization which tracks over whole program the function calls. This will allow some very powerful optimizations that are used today into GCC's LTO. This can be done just if is it guaranteed that the compilation speed is improved.
As for this release I would expect a "release" time around May/June 2014.
If you want to get more features, you are encouraged to add every small fix. They really add up and can make the users to enjoy using CR.
So in short, if you think that CR can in future match your needs, and even if you find it a bit overwhelming, try to add some fixes or add a new feature like:
- improve command line flags: to support more compilers/OSes. Many fixes are really one or two days of work (maybe less)
- try to make an application from simple to complex and see where CR crashes. Try to add support for that specific instruction. Is it much easier to follow one instruction than to look for entire cases. Try disabling (by commenting, before looking how to set a lower level in optimizations) all optimizations
- add support for a class in .Net using the Dynamic-invoke support. For example add File.ReadAllText(...) and to work at least for some basic codes. Add after that File.ReadAllLines(...)
- try to make an OpenGL application and see what is not supported (you have a sample application in the Git repository)
- try to add an optimization step: make your class that checks a condition of the instruction set. For example if you divide by 0.5, to replace it by multiply with 2.0. Try to add more optimizations like this.
- automate buids and make yourself a packaged installer
- make a MonoDevelop/VS plugin to automate launching of CR against a project
- anything else (that is really welcomed)
CR is free for use (free as in price and freedom) and you are free to customize it for your future needs.
- better field matching
- auto-properties do work
- better delegates and generics (top Git code is broken now, look why in the 0.0.3 feature set)
- fixes in some cases like math with various sizes
- most optimizations (especially escape analysis) work over more code patterns
As many programmers know code fixing takes time and I took this time, but I will not pack an installer for 0.0.2 as I found that some features are important to be implemented in a different way and there is no support yet (and request) to make a release package, so why do it?
The changes of 0.0.3 will be:
- visibly faster compilation: use-def code will be computed and updated on changes, and not computed on every optimization. The current code is already in the Git repository
- more fixes (of course expect a breakage in the first time)
- the Dynamic-Invoke mechanism should be more fleshed out
- generics will fully use templates <T> instead of using specializations. This translates into a single compilation of one function (from CR side). If you will define a class List<T>, the old code will create List_Int32, List_String, etc
- expected: a fully inter-procedural optimization which tracks over whole program the function calls. This will allow some very powerful optimizations that are used today into GCC's LTO. This can be done just if is it guaranteed that the compilation speed is improved.
As for this release I would expect a "release" time around May/June 2014.
If you want to get more features, you are encouraged to add every small fix. They really add up and can make the users to enjoy using CR.
So in short, if you think that CR can in future match your needs, and even if you find it a bit overwhelming, try to add some fixes or add a new feature like:
- improve command line flags: to support more compilers/OSes. Many fixes are really one or two days of work (maybe less)
- try to make an application from simple to complex and see where CR crashes. Try to add support for that specific instruction. Is it much easier to follow one instruction than to look for entire cases. Try disabling (by commenting, before looking how to set a lower level in optimizations) all optimizations
- add support for a class in .Net using the Dynamic-invoke support. For example add File.ReadAllText(...) and to work at least for some basic codes. Add after that File.ReadAllLines(...)
- try to make an OpenGL application and see what is not supported (you have a sample application in the Git repository)
- try to add an optimization step: make your class that checks a condition of the instruction set. For example if you divide by 0.5, to replace it by multiply with 2.0. Try to add more optimizations like this.
- automate buids and make yourself a packaged installer
- make a MonoDevelop/VS plugin to automate launching of CR against a project
- anything else (that is really welcomed)
CR is free for use (free as in price and freedom) and you are free to customize it for your future needs.
Friday, January 24, 2014
Tutorial of how to test/build Code Refractor locally
I thought that adding a small tutorial how to compile/build CodeRefractor may be useful to be seen for various readers of this blog.
So the steps are in brief:
Go to CR Git page:
https://github.com/ciplogic/CodeRefractor/
Take a git client (if you don't have one)
- http://code.google.com/p/tortoisegit/- http://sourceforge.net/projects/gitextensions/
Open the root solution (.sln) file in the root folder!
To edit the source you should use one of the following IDEs:
Visual Studio or Visual Studio Express
http://www.visualstudio.com/en-us/products/visual-studio-express-vs.aspx
SharpDevelop (I recommend nightly builds)
http://build.sharpdevelop.net/BuildArtefacts/#SD41
MonoDevelop / Xamarin Studio (especially if you want to compile it on a Mac/Linux).
http://monodevelop.com/Download
If you don't know any step, feel free to comment. If you feel there are issues and you want to sort yourself and you want a step-by-step video, this video is here:
CR is still at an early stage but is starting compiling more and more stuff.
If these videos will be popular (at least by make it worth to do more), I will add some more. At the end, I really hope you will like using CR!
Update: there was a small runtime error, this error was fixed in the last Git snapshot. Anyway, look into your code if it compile and try to figure out why it doesn't if there are any problems.
So the steps are in brief:
Go to CR Git page:
https://github.com/ciplogic/CodeRefractor/
Take a git client (if you don't have one)
- http://code.google.com/p/tortoisegit/- http://sourceforge.net/projects/gitextensions/
Open the root solution (.sln) file in the root folder!
To edit the source you should use one of the following IDEs:
Visual Studio or Visual Studio Express
http://www.visualstudio.com/en-us/products/visual-studio-express-vs.aspx
SharpDevelop (I recommend nightly builds)
http://build.sharpdevelop.net/BuildArtefacts/#SD41
MonoDevelop / Xamarin Studio (especially if you want to compile it on a Mac/Linux).
http://monodevelop.com/Download
If you don't know any step, feel free to comment. If you feel there are issues and you want to sort yourself and you want a step-by-step video, this video is here:
CR is still at an early stage but is starting compiling more and more stuff.
If these videos will be popular (at least by make it worth to do more), I will add some more. At the end, I really hope you will like using CR!
Update: there was a small runtime error, this error was fixed in the last Git snapshot. Anyway, look into your code if it compile and try to figure out why it doesn't if there are any problems.
Tuesday, January 21, 2014
Target 0.0.2: Status 7
A milestone was reached!
With minimal changes of a Tao/Sdl/OpenGL sample, the following program is running (and is running well):
The executable has 21KB (when built with Visual Studio on release).
This achievement may motivate me to release the 0.0.2 release somewhere in March.
With minimal changes of a Tao/Sdl/OpenGL sample, the following program is running (and is running well):
The executable has 21KB (when built with Visual Studio on release).
This achievement may motivate me to release the 0.0.2 release somewhere in March.
Subscribe to:
Posts (Atom)