News
LanguageKit Updates
Posted on 31 January 2009 by
Lots of fun things have happened with LanguageKit recently and since I haven't written anything on this blog for a while I thought I'd write about some of them.
Fixing Bugs
A bit before Christmas, I found a subtle bug in clang that was causing the class names not to be NULL-terminated in the support file. This worked fine in some cases and crashed on others, dependent entirely on the location of the generated code in memory. Eric was very confused by the fact that code was working when he added a constant string and failing when he didn't. When this was fixed, a lot of heisenbugs went away.
Returning Structures
I implemented some very basic understanding of calling conventions. It turns out that LLVM doesn't know anything about the host ABI's structure returning mechanism, which was causing some very strange results. This was especially confusing to me since it worked on FreeBSD, but not Linux and everyone I asked told me that they used the same calling convention on x86. It turns out this isn't quite true. FreeBSD uses an in-register mechanism for returning small structures, while Linux uses the old PCC calling convention, passing them on the stack. Adding basic ABI-awareness meant that you can now call functions that return NSRange and similar structures and pass them as arguments. When you get an NSRange returned from a method, it is boxed in an NSValue. When you pass it to a method expecting a range, -rangeValue is called. This allows you to substitute any other object that responds to -rangeValue.
Stack Unwinding
The biggest missing feature before Christmas was support for non-local returns in Smalltalk. It is quite common to want to write code like this:
(some guard condition) ifTrue: [^nil].
The problem here is that the ^nil is inside a block. In Pragmatic Smalltalk, this will be in a nested stack frame. The return, however, is meant to return from the method, not from the block. This is a really ugly bit of Smalltalk, but one that apparently people use.
This month, I finally got around to implementing it. This is done via the zero-cost exception mechanism. The return statement in the block is transformed in the code generator to a self nonLocalReturn:nil. This method is written in Objective-C and looks up the scope chain until it gets to the method. It then throws a new kind of exception which unwinds the stack until it gets to the method.
This means that it will work with intermediate stack frames written in any language. If you have a @finally block and are compiling with native exceptions in Objective-C then this should be called on the way up the stack. Similarly, any cleanup that intervening Smalltalk stack frames need to do will also be done.
This is a very expensive operation. For each stack frame between the block and the method there is a call to the unwinding library, at least two calls to the LanguageKit personality function, and a call to the frame's cleanup code. Fortunately, non-local returns are mainly used for guard clauses so it's not likely to be invoked in the most common code paths. In future versions there will be more aggressive block inlining and some of these returns will be transformed into local returns.
Loading Bundles
One of the things I'd been planning to do from the start is allow bundles of Smalltalk to be run. I've now committed support for loading bundles which contain a LKInfo.plist resource. This specifies a list of Smalltalk (or other language...) files that will be compiled and a list of frameworks that will be loaded.
When used in conjunction with edlc there is some dynamic patching of the NSBundle class so that the loaded bundle will be returned in response to a +mainBundle message. This allows NSApp to find it and load resources easily. As a demonstration of this, Eric has committed the first pure-Smalltalk application to svn recently. This uses a simple shell script as the application binary:
#!/bin/sh
edlc -b `dirname $0`
You can launch this application with openapp or it will run as any Objective-C application would. The class specified by the PrincipalClass field in the plist is instantiated and sent a -run message. This does the same thing as the NSApplicationMain() function that most Objective-C apps use:
NSObject subclass: PhotoManager
[
run [
| nsapp |
nsapp := ETApplication sharedApplication.
NSBundle loadNibNamed: 'MainMenu' owner: nsapp.
nsapp toggleDevelopmentMenu: nil.
nsapp setDelegate: self.
nsapp run.
]
]
Since LanguageKit supports static compiling as well as JIT, I eventually plan on extending this to emit a shared library for each bundle and load it rather than recompiling if the code hasn't changed.
Application Patching
One of the nice features of GNUstep is the idea of an AppKit user bundle. These are specified in defaults and will be loaded when AppKit initializes. Étoilé provides three of these. One relates to theming, one relates to the menu bar, and the final one is for general behaviour.
Part of the refactoring I did recently to LanguageKit moved the code generation component, which is huge, to a separate framework which is now dynamically loaded on demand. This means you can link LanguageKit against other applications without much overhead, and this is exactly what we're now doing.
If LanguageKit is installed then the EtoileBehavior bundle will load it into any GNUstep application. It will then look in $(GNUSTEP_USER_ROOT)/Library/LKPlugins for a directory matching the name of the application. If one exists, then it will load every bundle with a .lkplugin extension. This allows you to add arbitrary code to any application. Using categories, you can even replace existing methods.
Why is this important? Take a look at this from the Free Software Foundation:
The freedom to improve the program, and release your improvements (and modified versions in general) to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this.
In an Étoilé environment, access to the source code is now not a precondition of this freedom. You can improve a program, in a high-level language, without access to the source code.
AST Transforms
There is now a simple API for adding transforms on the LanguageKit abstract syntax tree. You just need to adopt a simple protocol and you will get the opportunity to replace each AST node in the tree. You can also use this mechanism to collect information about the tree.
Plugins using this mechanism will be collected in Languages/LKPlugins in svn. Currently there is just one simple demo, which replaces every comment with a log message. This allows you to record every time a line containing a comment is passed.
Code Monkeys
The final, but possibly most exciting, development recently is CodeMonkey, a new IDE being written by Nicolas. This uses LanguageKit for compiling and for for syntax highlighting using code based on Günther's pretty-printer.
It's still at quite an early stage of development - not even a 0.1 release yet and probably won't make it into 0.4.1 - but it's very promising. This does a few neat things with AST transforms. If you add a new method to a class then it will emit a new category on the class, rather than a new copy of the class (which would confuse the runtime). More interestingly, if you add an instance variable to a loaded class it will turn every reference to this into KVC calls. When you restart the application, the transform won't run and they will be direct ivar accesses.
The medium term plan for CodeMonkey is to turn it into a bundle. EtoileBehavior will then add a new menu item which will load the bundle and run the UI. This will give you Squeak-like introspection and modification of running applications.
