blog: Drinking the Elixir

So, now that we're into December, I'm quite fatalistic about my classes. (If I'm going to do well, well, good, and if not, it's too late now anyway.) As such, I'm not really taking notes anymore, at least not when I have my laptop. This meant that this morning I had some free time - I admit, there was a class also going on around me, but nevertheless the time was free - and I updated my Elixir copy.

Whoa. Elixir has undergone a lot of changes, and, actually, really impressed me.

SQL is Hard, and SQLAlchemy Models That Accurately

Some months ago, Ben Bangert argued that webapp programmers shouldn't use layers like Elixir, because they were a level of indirection. I was half-inclined to agree, but for different reasons: there wasn't a model definition syntax that was intuitive at the time. That's no longer the case, so let's take on Ben's original argument.

It's easy to argue that asking people to buckle down and learn SQLAlchemy is reasonable. But it's also wrong. While Ben's point that databases don't store objects is true, it's quite unrelated to the issue. The reason behind this brings me to the first sub-point:

I Don't Care About the Database

This is important, so read this three times: People do not use databases because they want to deal with rows. (And, by "people", I mean "me", of course. But that's what blogs are for.) Ben's right: databases don't store objects. But I'm not using a database for its conceptual integrity, or lack thereof.

I'm using a database because it's convenient. And, in the event that I should need to scale, the IT world has a lot more experience scaling databases than scaling pickled Python objects - which, yes, would be my second choice.

So the fact that the database doesn't store objects, but rather just a few selected attributes, doesn't matter to me. I don't want to have to deal with specifying which attributes maps to which column or anything of that sort, particularly when I'm rapidly trying out different object models. SQLAlchemy can work all kinds of magic when it's needed, but for RAD situations, the database is just a data store - an implementation detail that I don't care about - at least, for now.

So let's not forget what SQLAlchemy is trying to do: build a complete representation of SQL in Python. SQLAlchemy does a great job. But - and yes, there's a but - SQL isn't the prettiest of paradigms. It's certainly good at what it does, just like HTML, but the fact is that hand-coding quickly gives way to dynamic generation, and for the same reasons (verbosity and vendor quirks) we see with HTML.

And Elixir is More Natural

If you want explicitness, fine, use SQLAlchemy. (Even then, you're giving up an even higher level of explicitness from just hand-writing the SQL - as in most things, it's a matter of scale.) But if you think SQLAlchemy isn't verbose, sorry, try again:

node_t = sa.Table("node", meta, 
            sa.Column("id", sa.Integer, primary_key=True),
            sa.Column("parent_id", sa.Integer, sa.ForeignKey("node.id"), nullable=True),
            )
class Node(object):pass
Session.mapper(Node, node_t, properties={
    'parent': sa.orm.relation(Node, remote_side=[node_t.c.id], backref="children"),
    })

Versus:

class Node(elixir.Entity):
    id = elixir.Field(sa.Integer, primary_key=True)
    parent = elixir.ManyToOne("Node", backref="children")

You're free to consider the first more intuitive. I don't, however, think that's an opinion many people would agree with. The first may seem like that once you grok SQLAlchemy, but, as I said above, application developers - AKA, framework users - don't care about SQLAlchemy: they want their objects, and let the framework handle persistence.

Does it make debugging harder? Well, yeah. But the solution to that is to make the library better, not to give up and conclude everyone should just learn the low-level method.

It's worth noting that while 10-line examples rarely reflect the real world, the above snippet actually does. I can tell you from experience that SQLAlchemy models really end up being three times as verbose as class-based ones. Sure, sometimes - and I've run into more than a few instances of this myself - you need that level of granular control, and SQLAlchemy is great for that situation. But when you don't, having to write three times the code really cuts into the "rapid" part of RAD.

Bad Analogy Time: Debugging generated assembly code is harder than debugging handwritten ASM. The solution there is not to give up and declare C as having too much indirection: it's to correct the compiler. Likewise, the solution to finding debugging your Elixir model too difficult isn't to revert to SQLAlchemy - or to revert to raw SQL, for that matter - it's to correct the library.

This isn't hard, people. Abstraction is the fundamental principle of computer science, and calling it "indirection" instead of "abstraction" doesn't magically make it bad instead of good.

What SQLAlchemy Provides

Mike Bayer has done an incredible job. That guy amazes me. He's written two libraries - SQLAlchemy and Mako - that outdo anything else in the relevant problem domain by a wide margin.

And his libs are actually documented!

But I think it's incorrect to say that SQLAlchemy is the end-all, say-all of ORM in Python. All of Mike's hard work gives us an incredibly powerful tool: Python, and therefore Python's abstractions. Instead of having to do our database abstractions in procedural, nonportable and database-specific code, we can build up everything - each column selection, each portion of the where clause - using Python, which, being object-oriented, gives us the ability to make cool, Pythonic abstractions.

Like Elixir.

We Shouldn't be Afraid to Use it

To put it another way, the advantage of something like SQLAlchemy is more than just that it models SQL in Python: it allows us to apply Python's abstractions to the end result. So why shouldn't we do just that?

No, there's nothing wrong with abstracting SQLAlchemy using something like Elixir - indeed, that's part of the advantage of SQLAlchemy in the first place! Nor is it a slippery slope of infinite indirection; the desire to abstract SQLAlchemy is based on the desire to get to a Pythonic, class-style model definition syntax, the same way we define other objects. Elixir provides that.

And I, for one, am happy to take advantage of the improvements.