Roads Less Taken

Recently the ODB vs RDB subject popped up on squeak-dev and I felt like posting about my experiences. Then Todd Blanchard blogged about his experiences and reading that I really wanted to respond "piece by piece".

Todd mentions having worked with ObjectStore, Versant, Poet and a few more. I have worked a lot with GemStone (both the Java and the Smalltalk version) and used Poet with Java in a project when it was first out of the gate with a ODMG binding for Java. I have also used Magma for most of 2006 (the Gjallar project), which is a Squeak specific "GemStone light clone". I have also used one or two "lesser" ones, like Tensegrity (not around anymore?).

About the key concepts described by Todd I generally agree - even though I would probably have mentioned one or two other concepts too. Todd also writes:

"The object reference can only be accessed within a transaction - trying to read data from it outside of a transaction will fail with an exception. This makes redrawing user interfaces problematic."

I don’t share that experience - some ODBs generally think you are in a Transaction "all the time" - the question is if you end it with a commit (saving any changes made) or abort (updating the objects in your session with all changes made by others since you began). And some also allow you to run "outside" of a Transaction - generally giving you the same effect. Exactly how this affects user interfaces I don’t know - I have seen no issues regarding that.

In fact, in one project we built a DTP component that allowed multi-authoring using a domain model that was fully persisted in GemStone, and whenever someone made a modification it was immediately propagated to all concurrent viewers/editors. In fact, the performance we got would IMHO have been really hard to get using an ORM+RDB solution.

Todd then proceeds describing 9 problems with ODBs. Let me give my view on them all - I am not saying Todd is "wrong" - I just have a different opinion based on my particular experiences (especially GemStone and Magma).

1. Poor concurrency. Todd also mentions that he knows of no ODBs that use page locking. AFAIK most offer object locking, and both GemStone and Magma does that - both pessimistic and optimistically. The latter approach offers better concurrency than regular pessimistic locks. GemStone also has "reduced conflict" classes offering defined concurrency characteristics thus avoiding locks completely (and I hope/bet that Magma will get something similar over time). Using GemStone and Magma transactions should tend to be short, and most people using GemStone uses optimistic concurrency AFAIK. So sorry, this one sounds dubious to me - unless you only consider the ODBs that only offer page locking. :)
2. Lack of caching, or as Todd writes: "There is no user level in-memory caching". This also sounds very odd. My impression of all ODBs I have looked at is that almost all of them has caching and GemStone/Magma has very advanced and good cache behavior that offer lots of control. Mmm, ok, I admit that GemStone/J did not have this for non GemStone Java VMs (but if you used the Gem’s themselves as clients, which we did successfully, then it was all dandy. In other words - the GemBuilder part for Java was quite bad, at least when I used GemStone/J).
3. Hard schema migration. Again, both GemStone and Magma offers very good support. Both have the ability to have concurrent versions of classes so that it can lazily migrate objects over time. I do not recall how it worked in Poet, but GemStone and Magma works great in this respect - definitely not one bit "harder" than it "should" be.
4. Upgrades demanding downtime. Well, I am unsure about this one - do RDBs generally offer upgrades of the software without any downtime? Since I haven’t really tried to tackle such a scenario I don’t really know. I guess that one technical approach would be using some kind of online synchronization so that you can smoothly "switch over" to a newer software - and it would be interesting to hear from anyone using GemStone (or Chris that is the author of Magma) how they would deal with it.
5. Lack of support due to limited resources. We had outstanding support for GemStone, but I don’t know about their policies on older versions. Using Poet I had direct contact with the development team. Magma is open source so it is a different ball game, but Chris has been superb so far :). It is generally true that most ODB vendors are smaller - but some of them like GemStone have very high profile customers running mission critical systems - and they demand good support. Personally I have always gotten better support from a smaller vendor, but hey, mileage vary.
6. Bugs are forever. This one sounds really odd too, I guess the products I have used are different. If I screw up my objects they are… screwed up. But I fail to see how that would be different with an ORM+RDB and I fail to see why that would make "read code fail". I think I need more info because I must be misunderstanding this.
7. No security. GemStone has a very nice permission model built in. Magma does not AFAIK. Personally I have never felt a need.
8. Lack of GC. GemStone has very advanced online GC, no problem. Magma has GC, but offline as of yet (I think). ODBs differ in this respect probably because GC in a distributed transactional world is complex stuff. But I agree that GC is something you wish to check when picking an ODB.
9. No ad hoc query capability. This is not true, some ODBs support a specific query language like OQL, but even if they do not then at least when we look at the Smalltalk ODBs we all know that you "have to write a new program" to even create a class. So in fact - Smalltalk is the "query language" and "a new program" can be as short as "myCollection do: [:each | each blabla]" - much more powerful than SQL IMHO. I rather manipulate my objects using messages honouring encapsulation while offering the full capabilities that my programming language gives me - extensible to boot. All that said I still think it is nice that Magma is getting an SQL extension enabling access using ODBC - simply because that offers lots of possibilities. But I would never use it for writing.

So obviously a great deal of experience with ODBs can leave people with very different opinions. I love ODBs but I have had the opportunity to use some good ones that were made for use with Smalltalk. And there is a great lack of good open source ODBs.

But in the end I don’t think it is strange that a mature product built and designed to handle objects is better at it than a "hack" of an ORM on top of an RDB. Of course, the RDBs are very mature at what they do - and some ORMs are really good - but it is still a complicated solution with lots of drawbacks compared to an ODB.

If you already have data in an RDB that you have to work with - typically in a schema not designed to be used as objects - then I would rather like to explore solutions embracing the relational model - like ROE or SimpleORM (for Java) - making it very easy to manipulate the relational model just as it is.