Smalltalk on Roads Less Taken

Benchmarking Spry vs Squeak

Fri, 26 Aug 2016 00:00:00 +0000

Spry is evolving quite nicely during my night hours. Focusing on performance is however still premature, but I do want to verify that I am not barking up the wrong tree, like … in the wrong forest even. So I make trivial benchmarks from time to time, just to see and learn.

Some background on the Spry implementation may be interesting. Spry is implemented in Nim as a direct AST interpreter, it’s not a JIT, in only about 2000 lines of code. It has a recursive classic “naive” design and uses a spaghetti stack of activation records, all allocated on the heap relying fully on Nim’s GC to do it’s work. It also relies on Nim’s method dynamic dispatch in the interpreter loop for dispatching on the different AST nodes. Blocks are true closures and control structures like timesRepeat: are implemented as primitives, normally not cheating. Suffice to say, there are LOTS of things we can do to make Spry run faster!

The philosophy of implementation is to keep Spry very small and “shallow” which means we rely as much as possible on the shoulders of others. In this case, primarily Nim and it’s superb features, performance and standard library.

Enough jibbering, let’s do some silly damn lies - ehrm, I mean silly tests!

Is Spry a Smalltalk?

Tue, 19 Jul 2016 00:00:00 +0000

I love Smalltalk and I have been in love with it since approximately 1994. I have used VisualWorks, VisualAge (IBM Smalltalk), Dolphin Smalltalk, GemStone, Squeak and Pharo quite a lot, and I was very active in the Squeak community for a long period.

But the last few years, finally, I have started to feel the “burn”… as in “Let’s burn our disk packs!” . And last year I started doing something about it - and the result is Spry . Spry is only at version 0.break-your-hd and several key parts are still missing, but its getting interesting already.

Now… is Spry a Smalltalk? And what would that even mean?

I think the reason I am writing this article is because I am feeling a slight frustration that not more people in the Smalltalk community find Spry interesting. :)

And sure, who am I to think Spry is anything remotely interesting… but I would have loved more interest. It may of course change when Spry starts being useful… or perhaps the lack of interest is because it’s not “a Smalltalk”?

Smalltalk family Link to heading

The Smalltalk family of languages has a fair bit of variation, for example Self is clearly in this family, although it doesn’t even have classes, but it maintains a similar “feel” and shares several Smalltalk “values”. There have been a lot of Smalltalks over the years, even at PARC they made different variants before releasing Smalltalk-80.

So… if we look at Spry, can it be considered a member of the Smalltalk family?

There is an ANSI standard of Smalltalk - but not many people care about it, except for some vendors perhaps. I should note however that Seaside apparently (I think) has brought around a certain focus on the ANSI standard since every Smalltalk implementation on earth wants to be able to run Seaside and Seaside tries to enforce relying on the ANSI standard (correct me if I am wrong).

Most Smalltalk implementations share a range of characteristics, and a lot of them also follow the ANSI standard, but they can still differ on pretty major points.

My personal take on things in Smalltalk that are pretty darn important and/or unique are:

Everything is an object including meta levels
A solid model for object oriented programming
The image model
100% live system
The browser based IDE with advanced cross referencing, workspaces and debuggers
The keyword syntax and message cascades
Message based execution model
Dynamic typing and polymorphism
Closures everywhere with lightweight syntax and non local return
Very capable Collections and a good standard library

Not all Smalltalks cover all 10. For example, there are several Smalltalks without the image model and without a browser based IDE. Self and Slate and other prototypical derivatives don’t have classes. Some Smalltalks have much less evolved class libraries for sure, and some are more shallow in the “turtle department”.

In Spry we are deviating on a range of these points, but we are also definitely matching some of them!

Fowltalk - a new Smalltalk

Sun, 19 Jun 2016 00:00:00 +0000

In my quest making Spry I also learned about other efforts in the Nim community to construct Smalltalk-like dynamic languages. The most ambitious one may be Fowltalk by “fowl” who typically hangs out in the #nim channel on Freenode. Fowltalk started out written in Nim but since fowl wanted to learn more C++ it’s now written in C++17.

At the moment he is rewriting the parser and code generator parts in the language itself , following a similar bootstrapping style as Ian Piumarta’s idst . For example, here is the method parsing keyword messages .

Spry Performance

Tue, 24 May 2016 00:00:00 +0000

When writing Spry I am so far mainly ignoring performance. The general execution of Spry code will be a regular interpreter (although stackless I hope) and not a JIT. But that doesn’t prevent us from playing around and learning something!

In this article I do some silly experiments around interpreter startup time and fooling around with 40 million element arrays. As usual, I am fully aware that the languages (Pharo Smalltalk, NodeJS, Python) I compare with a) have lots of other ways to do things b) may not have been used exactly as someone else would have done it. A truck load of salt required. Now… let’s go!

Spry vs Allen

Sat, 14 May 2016 00:00:00 +0000

Allen Wirfs-Brock wrote down a bullet list of what he thought actually made it possible for the Alan Kay team to create Smalltalk , and many other ground breaking things, at Xerox PARC in the 70s. Let’s take a look at his bullets one by one and see how it applies to Spry and my puny little effort around it :)

Spry Modules

Sat, 16 Apr 2016 00:00:00 +0000

As discussed in the previous article I want Spry to have a trivially accessible persistence mechanism enabling something similar to the Smalltalk image model, but based on a database. The memory organisation in Spry is basically nested Maps. After dwelling a bit on the inevitable hard question about modules and namespaces I have decided on a design that I hope will turn out simple and reasonably powerful!

Smalltalk has a Dictionary holding all the globals forming “the roots” of the object memory. In Smalltalk this Dictionary is also itself a global variable accessible as Smalltalk, in other words Smalltalk == (Smalltalk at: #Smalltalk). The primary use of Smalltalk is to hold all classes by name, so they are all reachable as globals. Obviously Smalltalk can also hold any kind of object (not just classes) as a global.

Spry also has such a top level Dictionary, but in Spry we call a Dictionary a Map to be a little bit more aligned in terminology with other languages (and it’s shorter). This top level Map is the root Map and it is accessible via the word root. In Spry the root word is actually bound to a primitive function returning this Map, so in Spry we also have root == (root at: 'root).

Ok, so Spry has a Map of globals and one way of using Spry is simply by populating root with words bound to functions making these functions globally accessible, it’s how I have done it so far. Yeah, yeah, I know, but for smaller systems it probably works just fine!

But…

Spry image model

Sat, 09 Apr 2016 00:00:00 +0000

In developing Spry - renamed from Ni - I am getting closer to the Really Fun Stuff.

As a Smalltalker I dream “bigger” than just managing source code as text in files…

Smalltalk uses the “image model” in which the system is alive and running all the time, the full development environment is also live together with your application, and we are in fact modifying object structures when we develop Smalltalk programs. We can also snapshot that object memory onto disk and fire it up somewhere else. Several Lisp implementations have used a similar approach I think.

The image model has tons of really cool benefits, I don’t have time repeating all of them, but a modern implementation of the idea should take a few things into account that was not considered in the 1970s:

The “image” in Spry will be buildable from source
You should be able to use Spry without the image mechanism (you can already)
Spry code will have a readable text format and file structure
The image model does not have to be all or nothing, it can be partial
The image mechanism will be a module for the Spry VM, so you can skip it entirely

Some argue that the image model has downsides - like being an “ivory tower” incapable of interacting with the outside world. The Smalltalk environments have indeed historically suffered a bit in varying degree, but we can easily find ways around those issues while still reaping the awesomeness of a fully live programming environment, especially if we give the above items proper thought from the start.

With Spry I think I have a beginning to a novel approach… as well as taking the above into account.

Adding objects to Ni!

Fri, 25 Sep 2015 00:00:00 +0000

So… Ni has almost reached the point where I can see objects appearing. The following describes the design I currently have in mind, read it and tell me what you think so I can scrap it and start over ;)

Ni design decisions!

Wed, 23 Sep 2015 00:00:00 +0000

So… my little Ni language got some attention since it was first on Hackernews , then TheRegister , all over Twitter and also Reddit .

But I think it managed to come relatively unscathed out of it, although it REALLY is pre-alpha-not-even-complete-eats-your-harddrive early and you know, I really have no idea if it ever will go the distance since it takes quite a bit of work to get a language to actually be used. But I am going to stick with it.

Anyway, I have been experimenting with “arg words” and “lookup scoping” while thinking about how to add objects, and a few other things. This article doesn’t introduce how I want to do objects, but the next one does (I split it in two). This article however covers a bunch of loose ends and my ideas on how to tackle them in Ni. And I will try to make this understandable even if you don’t know Ni. ;)

Guts of Ni

Tue, 22 Sep 2015 00:00:00 +0000

This article describes some core parts of the current implementation of the Ni (now known as Spry ) language. It’s not a tutorial, introduction or manual. It’s in fact kinda incoherent - but so is Ni :)

Who says Ni?

Wed, 16 Sep 2015 00:00:00 +0000

Ni is my own little language heavily influenced by Smalltalk but also other sources like Rebol, Forth, Lisp, Self and Nim. Ni is a bit strange, but it’s not academic and really meant to become something useful.

So put on your helmet and let me take you to the shrubbery…

Nim seq

Wed, 03 Dec 2014 00:00:00 +0000

One of the most important aspects in a language is how powerful and easy it is to use its collection/container types. At least that is my feeling coming from Smalltalk where the Collection classes and their rather rich protocols are used extensively and also cover String and Array and much more. If you peek into the current leading open source Smalltalk - Pharo - you can see Collection allSubclasses size evaluate to 78. Of course, lots of those are special subclasses and not for general use, but fact remains that a strong language needs a strong library of good collections.

In Smalltalk I can for example run this:

#(1 2 3 4 5) select: [:x | x isOdd ] thenCollect: [:x | x * 3 ]

This is actually a single method call taking two closures, one to perform a “filtering”, and one to perform a “map”. This is not a standard message in “good old” Smaltalk-80, but it has eventually been added since the usage pattern is so common. To me personally a modern language needs similar power or I would go nuts. :)

Squeak to Nim, come in Nim...

Mon, 03 Nov 2014 00:00:00 +0000

In my exploration of Nim the turn has come to see how we can use Nim together with Squeak .

Squeak (and Pharo) has two basic mechanisms of interfacing with the C world:

Squeak VM plugins . That pdf is old, but still fairly accurate I guess.
Squeak FFI , Foreign Function Interface.

The VM plugins are basically a controlled way to introduce new “named” primitives in Smalltalk that can be invoked from Smalltalk. A plugin can be built either linked into the VM binary (statically) or as a dynamically loaded library (dll, so, dylib etc). When “all else fails” a plugin is the way to go, but they are a bit awkward to work with.

Then we have the FFI which is a generic way to dynamically call dynamically loaded libraries. In other words, no compilation step needed - just type the correct Smalltalk line and the library will load and the calls work. Now… sure, the FFI mechanism is a bit slower, since it needs to look at arguments and make the proper type conversions for the call. But the FFI is heavily used in the commercial Terf system, in fact, all the OpenGL calls are done through it. So its quite proven, and not that slow.

NOTE: There are in fact several FFIs today, the old one, the one called Alien and Pharo is brewing a new one called UFFI.

Let’s see if we can use the good old FFI with Nim.

Cog vs LuaJIT2

Mon, 13 Oct 2014 00:00:00 +0000

In the open source Smalltalk community we have a pretty fast VM these days - its called Cog and is written by the highly gifted and experienced Eliot Miranda who also happens to be a really nice guy! Cog is fast and its also still improving with some more developers joining recently.

Another very fast VM is LuaJIT2 for the Lua language (version 5.1), also written by a single individual with extraordinary programming talent - Mike Pall. LuaJIT2 is often mentioned as the fastest dynamically typed language (or VM) and even though Lua is similar to Smalltalk (well, its actually very similar to Javascript) its also clearly a different beast with other characteristics. If you start looking at the world of game development - then Lua appears everywhere.

Here Comes Nim!

Mon, 13 Oct 2014 00:00:00 +0000

I just posted an article comparing some silly benchmarks between Cog Smalltalk and LuaJIT2. Now… let’s take a look at one of the latest “Cool Kids” on the language front, Nimrod - or as it has been renamed - Nim .

ESUG in Annecy Day 4

Thu, 12 Sep 2013 00:00:00 +0000

It starts off with Cincom, Arden Thomas presenting their roadmap. He had a slide mentioning Jules Verne and I must ask him if he is aware of the influence from the books of Jules Verne - as Dan has explained, the ballon and the island in the classic Smalltalk logo comes from the Mysterious Island book .

ESUG in Annecy Day 3

Wed, 11 Sep 2013 00:00:00 +0000

Day three blurred a bit for me - lots of mingling and hacking intermixed with some nice presentations, but I missed quite a lot, I admit, like the presentation on new native UI stuff in VisualWorks and the tODE stuff from Dale Henrichs.

ESUG in Annecy Day 2

Tue, 10 Sep 2013 00:00:00 +0000

Another day in IAE Universite Savoie Mont-Blanc in Annecy.

Only writing about stuff I attended :)

ESUG in Annecy Day 1

Mon, 09 Sep 2013 00:00:00 +0000

Here we go! Time for a week of Smalltalking fun in IAE Universite Savoie Mont-Blanc in Annecy.

While ESUG this year is not super big, I think around 110 people or so, there were still quite a few arms raised when checking how many were here for the first time. The organisation seems impeccable, as always - thank you Stéphane Ducasse, Herve Verjus, Laurent Laffont and of course all volunteers!

Enough chit-chat, let’s get on with the talks…

Moving to SmalltalkHub

Mon, 07 Jan 2013 00:00:00 +0000

As a long time Squeak/Pharo (Smalltalk) developer I have accumulated a set of packages that I have written or co-written and that have been published open source for others to use. Since quite a few years SqueakSource has been the natural hosting place, but it has reached the end of the road and it’s high time to move on .

Oak

Sat, 29 Dec 2012 00:00:00 +0000

In a customer project right now I need to be able to work and evolve code fast, with a relatively complex model. And by fast I mean that I want to cut away as much as possible of the efforts related to persistence. Generally this is what OODBs excel at.

In the Squeak world we have GemStone (commercial), GOODS and Magma as “full fledged” OODBs. Last century :) I worked with GemStone (both Gemstone/S and /J) and its a great product - but I want something lightweight and open source. And simple. And hackable. And new. :)

I also used Magma in the Gjallar project, and while I respect it highly - this time I want to try something with an “externally supported backend”. I also had a mixed performance experience, but this was “pre Cog” and Magma has also surely evolved lots since then, and I am not sure we did everything the way we should have either.

SandstoneDB could also be interesting to look more closely at, but since I have been working with Nicolas Petton on improving Phriak (Riak interface for Pharo) it was natural to take a look at one of his “under the radar” projects - Oak , an “OODBish” solution on top of Riak. At this point I have been doing much more than looking, in fact I am hacking on it! And oh, yeah, of course there are lots more persistence options available too.

Current Smalltalk obsessions...

Tue, 07 Feb 2012 00:15:10 +0000

These days I am, as usual, torn between several interesting technical projects.

Amber Link to heading

The new Smalltalk called Amber (by Nicolas Petton) that compiles to javascript is pretty awesome and there are tons of interesting things one can do with it. My contributions so far include the beginning of a package model, a faster simpler chunk format exporter/importer, a command line compiler, a Makefile system so that Amber can be built fully from the command line and a bunch of examples running on top of Nodejs and webOS , and a few other odds and ends.

ESUG day 4

Thu, 25 Aug 2011 16:50:52 +0000

This day started with some stress, Nicolas and I whipped up the last details of our co-presentation on Jtalk (Nicolas decided to skip Iliad ) - and my Eris demo suddenly got b0rken. But I managed to fix it and our presentation was very well received - it was great fun!

Nicolas managed to do quite a few “on the fly” demonstrations of various Jtalk snippets etc, and running the slides in Jtalk was of course a killer thing. I explained how jtalkc is being run on top of Node.js and quickly proceeded into showing the TrivialServer demo in Node.is - when Apache benchmark showed 1800 requests/second there was a spontaneous applause. :)

ESUG day 3

Wed, 24 Aug 2011 10:07:30 +0000

Suddenly it is Wednesday and we are already on day three at ESUG - a superb software developer conference focused on Smalltalk . Time flies. Yesterday I mainly hacked together with Nicolas Petton on Jtalk , really fun, unfortunately I missed a few interesting presentations, like Fuel and Bifrost etc.

This day starts with Stéphane presenting “Humane assessment”. Mmm, got distracted by my Touchpad, but Stéphane is showing some cool visualizations right now, clearly useful for large systems and organisations that need understand their own “huge legacy software”. Hehe, the browsers shows visual queues on “bad designs” like marking methods as “BrainMethod” or marking a class as “God Class” - that is indeed very slick!

ESUG 2011 in Edinburgh

Thu, 11 Aug 2011 11:59:57 +0000

Each year I try to attend at least one developer conference. Earlier OOPSLA was a given but it lost its appeal quite a few years back and now it is not even called OOPSLA anymore. As a die hard Smalltalker I instead attended the ESUG conference in Brest 2009 and it was easily the most rewarding conference I ever have attended! Missed last year in Barcelona but this year I am going to Edinburgh for a week of Smalltalking.

Tirade, supporting embedded text

Fri, 15 Apr 2011 01:22:58 +0000

Two years ago I ended up creating Tirade - a new “file format” for Smalltalkers. Or rather, a way to serialize stuff into a sequence of Smalltalk messages with literals as arguments. I have written a few blog articles about Tirade so I will not go into details in this one.

One thing that has been disturbing with Tirade is that I wanted it to be the main format for serializing Deltas, the new implementation of “21st Century ChangeSets”. This means I want Tirade to handle Smalltalk source code in the best possible way. Ideally I would want the Tirade file to be editable in a text editor if I wanted, and not being broken by that.

Preaching Smalltalk inside a nuclear reactor

Fri, 08 Apr 2011 12:07:35 +0000

…is what I did yesterday. It was the Stockholm GTUG group having a loose and laid back meetup in a rather special venue - R1, Sweden’s first nuclear reactor ! 27 meters below ground, kinda… funky.

Anyway, I tried doing an ultra compact version of several of my other presentations around Smalltalk and Seaside - didn’t really go 100% since I both had some technical issues (keyboard problems and projector issues too) and ended up taking more time than was planned. Hopefully noone got upset about that.

Node.js vs Nginx/Squeak, part 1

Mon, 14 Mar 2011 16:55:58 +0000

Hmmm, after seeing the Node.js presentation at Dyncon 2011 I couldn’t help installing Nginx and Blackfoot (SimpleCGI) in a Squeak 4.2 image running on the Cog VM to make some performance tests! In fact I started doing that during the presentation. :)

My first run on Nginx/Squeak looked quite unimpressive. Well, one client doing 1300 req/s to a small helloworld was decent although Node.js handled approximately 2x that. With Nginx we have a two tier solution so a factor of 2 is not really surprising in this trivial case. Top showed similar load, both solutions only seem to consume 8-9% of my CPU power on this box, but the Nginx/Squeak solution of course spreads load between them with approximately 1/3 or 1/4 on nginx.

Dyncon 2011, day 2

Sun, 13 Mar 2011 14:04:11 +0000

Day one must have ended with lots of beer because people were quite late for day two. 15 minutes late Carl Lerche finally started his Ruby presentation. One thing I found interesting was Ruby Modules vs Monticello extension methods (in some ways I presume this is how Modules are often used - to extend other classes with behavior). Evidently “method extensions” to the class side in Ruby doesn’t work like extensions to the instance side, it does in Smalltalk of course :). Then Carl described ways to still do this, but it looked complex, and also explaining there are lots of “hooks” when messing with the MOP. Is that a good thing? If Rubyists use this a lot, then I presume utter hopeless confusion might occur.

Skype interview with Dan Ingalls

Fri, 10 Sep 2010 08:49:54 +0000

Before the summer I had the opportunity to do a Skype interview with Dan Ingalls , the “Guido/Larry/Linus” of Smalltalk.

The interview was part of a guest lecture I did at DSV for the DYPL course (dynamic programming languages) held by Beatrice Akerblom. In the end I only used a smaller quote because most of the topics in the interview ended up being more interesting to “true believers” than to Smalltalk newcomers.

Tirade, first trivial use

Mon, 20 Apr 2009 00:43:55 +0000

Last night I started hooking Tirade into Deltas. Quick background: Deltas is “Changesets for the 21st century”, or in other words an intelligent patch system under development for Squeak. Tirade is a Smalltalk/Squeak centric “JSON”-kinda-thingy. I made Tirade in order to get a nice file format for Deltas. Just wanted to share how the first trivial code looks, and thus illustrate simple use of Tirade.

I have a DSDelta (a Delta being almost like a ChangeSet). It consists of some metadata (a UUID, a Dictionary of properties and a TimeStamp) and a DSChangeSequence (which holds the actual DSChange instances). As a first shot I only implemented the metadata bit. So step by step:

Tirade, part 2

Fri, 20 Mar 2009 00:34:07 +0000

In an article recently I described Tirade - a new generic “file format” for Smalltalk/Squeak, or actually a sub language! Since that article I have refined Tirade a bit. Tirade consists today of 4 classes (parser, reader, writer, recorder) totalling about 500 lines of code, excluding tests. Tests are green in 3.10.2, pharo-10231, 3.9, 3.8 and 3.7. It does turn red in 3.6 due to old initialize behavior, some missing methods etc, probably easily fixed if anyone cares. There are no dependencies on other packages. Compared to using the old Compiler»evaluate: it is about 5-7 times faster.

Tirade, a file format for Smalltalkers

Mon, 16 Mar 2009 00:24:07 +0000

In my revived work on Deltastreams in Squeak I ended up facing the choice of native file format for Deltas. Matthew has made an advanced format called InterleavedChangeset which manages to squeeze a binary representation of a Delta into a Changeset file (which is in Smalltalk chunk format). An impressive feat, and it has the advantage of being backwards compatible in the sense that a Delta in this format can be filed in as a plain old Changeset into an old Squeak image.