Question: How do you get Objects in and out of a Relational Database?

Answer: You don’t. You get Relations in and out of a Relational Database.

Today, I’m going to explain what I’m talking about. In order to do that, I need to back up just a bit and explore in some detail the problem which the Relational Model addresses so brilliantly. Because, fascinatingly, it’s a problem which is incredibly hard to notice, even though it’s right in front of us all the time.

Say Hello To The Frame Problem
Wikipedia somewhat pedantically tells me that “The Frame Problem” has a mathematically precise meaning in AI and logic. I’m going to ignore that bushwa. What I’m interested in is the Frame Problem as a deep and general problem of intelligence. (what’s that, you say, AI and logic professors are more interested in debating arbitrary formalisms than they are in fully understanding intelligence? I know, I know, it’s completely shocking.)

Another good name for it might be “What Is Common Sense And Why Do Computers Suck At It So Badly?”

A useful way to think of the Frame Problem is as a problem of representation (I’m following in Douglas Hofstadter’s footsteps, here). If you look at a given situation, or problem, you have a simply staggering amount of information which you could bring to bear on it. What’s more, you can organize subsets of that information in a dazzling number of different ways.

However, when faced with a given situation, rather than consciously sorting through everything you know and picking out the ‘relevant’ information, you find that… somehow… a concise representation of it springs to mind. If there are further challenges, you can then use that concise representation for conscious reasoning.

The Frame Problem is: in a given situation, how do you find and represent the relevant information, out of all the uncountably vast number of different ways you could represent what you know?

Daniel Dennett has constructed a marvelous fable about this, which is well worth studying and meditating on: The Frame Problem and the Fable of R2D2. It takes some meditating to see the problem, because it just seems so easy. It is something which, literally, a five year old could do. But which we have no way (not even close) of telling a computer how to do.

In fact, this turns out to be so savagely, incredibly hard to get a computer to do that most researchers would be tempted to throw up their hands and say it’s simply impossible. The main thing that stops researchers from doing so is that every neurologically functional human being does it all the time. And they do it without noticing.

This is key: the whole system seems to have been designed by natural selection to not bother your conscious mind. Almost as if there’s a deliberate two-level split for information processing: a subconscious facility for organizing information into concise, relevant representations, and a conscious mind for evaluating and acting on those representations.

(Digression: which of those two facilities would you guess has been more heavily studied? Right — the conscious mind. Because, well, it’s accessible to conscious reflection. And because we are all, I suspect, following evolution’s instructions to ignore the man behind the curtain. This is no small part of why classic AI is an almost compete failure — in focusing on conscious thought, it assumes that information is already stored in a useful representation. And it’s thus pretty much useless in the real world, where problems don’t come neatly packaged up for us in the proper representations. Back to the main story…)

How does all this relate to writing programs?

When you write a program in say, python, you are working with a specific representation of the information in your problem domain. This makes it possible to model certain behaviors, to solve certain problems. To, in a way, imitate some of what conscious thought does.

However, in so doing, you are committing to one concrete representation of the relevant information. And that, inevitably, limits you. If someone comes along and asks for a subtly different behavior, or to solve certain related, but oddly distinct problems, there’s an excellent chance that your particular representation will be completely useless. Even if, as a human, it seems like the other problem is closely related to what you’ve already solved.

This is where the Relational Model comes in — it allows you to store information in a way that flexibly supports a vast range of different representations. Note: it doesn’t do so with the automatic, so-powerful-it-seems-magic system which our subconscious brain uses (if it did, we’d all know it, because we’d be talking to our computers). What the Relational Model does do is allow you, the programmer, to use the same store of information in more than one situation. You can ask your relational data store to give you What It Knows in a new form, with a different emphasis or structure, with some details ignored and others fully fleshed out. And it can do so. In a way, you supply the frame, and it fills it out for you. Not always, not perfectly, but far, far better than just about any other system out there.

I would say: the Relational Model doesn’t solve the Frame Problem, but it does help you, as a programmer, to not get screwed by it.

It allows you to build something like that two-level information processing system I theorized exists in the brain: one level which stores a vast amount of information in a very flexible manner, and another level which, given a concrete representation, can solve specific problems and model specific behaviors.

This is why I’ve never understood why people seem to get so worked up about the so-called Impedance Mismatch between an RDBMS and an OO language. To my mind, the two work in simply brilliant harmony — information is stored in the RDBMS, and behavior is modeled by the objects.

To get the two to play together nicely, the really key thing is to understand the great value that the Relational Model is giving you. If you do so, you won’t be tempted to seal it off beneath a brittle ORM layer which turns it into an concrete represenation store. Instead, you’ll want to bring Relations, with all their marvelous flexibility, up into your OO language.

There are some interesting challenges to doing that. And I’ll get into those in my next post, when I explore the Relational Model in more detail.

I’ve got a sort of long-winded argument to make, so let me jump ahead to the conclusion right away:

You don’t need an object-relational mapping tool/layer/framework.

You really, really don’t.

But what about the famous impedance mismatch? Or the feeble abstract type support in RDBMS’s? And all the useful ORM tools out there? Or the fact that it’s overwhelmingly tempting to build your own ORM, because it just seems like such a Natural, Right Idea, even though all the ones you already know about seem kind of overly complex?

Well, not only do I think that ORM’s are a deep, fundamental mistake, but I also have a hunch as to why it’s so incredibly tempting for humans to make that mistake.

But, first, a joke.

One of My Two Favorite Computer Science Joke Koans

When I was young, I had a fascination with jokes. In some random book of riddles, I read the following:

Question: “How do you get down from an elephant?”
Answer: “You don’t get down from an elephant, you get down from a duck”.

Now, at a young age, I had no idea what the hell this joke meant. I didn’t realize that duck feathers were called down, so the joke felt oddly like it was cheating. As if, someone standing, stuck, on top of an elephant just announced, “I’m going to get down from a duck”, at which point, they’d be standing on a duck, and could casually step down to the ground. What the hell? (Side note: somewhat to my shame, I have memories of telling this joke to adults, and when they laughed, laughing along with them, as if I understood it. Not sure what this says about me, but there it is. Onward…).

In my confusion, I meditated long and hard on this joke in an effort to understand it. Eventually, it became a sort of zen koan for me. Years later, I discovered that it was actually an excellent zen koan for computer science.

Applying the Joke Koan

One of my favorite examples comes from python. If you’re a computer language geek (like myself), here’s a question you’ll run into over and over again, in a variety of guises:

How do you make code written in a high-level, dynamic language run as fast as code written in C?

This is a very, very interesting question, and there are a dazzling number of very interesting answers. To pick just a few: in Common Lisp, you can add optional type declarations as hints to the compiler; in the ML family, there’s sophisticated type inferencing; Java has JIT’s and adaptive, profile-based live recompilation (HotSpot).

All very interesting and very complex, and all sort-of successful.

So how do you do this in python? Well, let’s frame it up:

Question: How do you make code written in python as fast as code written in C?

Answer: You don’t make code written in python as fast as code written in C. You make code written in C as fast as code written in C.

And this is a brilliant effing answer! When I was first learning python, and realized that this was the python answer, I was blown away. You make it easy to call C code from python, and that’s it! Which is why python, legendarily ‘slow’ though it is, is used in the real world to power websites serving tens of millions of hits a day, and for hard-core scientific calculations, and sophisticated image processing, and for, well, all the demanding, real-world applications you can hear about every year at PyCon.

It’s just a fantastic unasking of the question. There you are, standing on the duck, and you just step off onto the ground. Brilliant, no?

So, this is the heart of my answer:

Question: How do you get Objects in and out of a Relational Database?

Answer: You don’t. You get Relations in and out of a Relational Database.

I’ll explain that answer in more detail in part II of this series, coming soon, titled: Let Us All Get Down On Our Knees and Praise the Relational Model.

(Oh, and if you’re curious, my other favorite computer science joke koan comes from The Hitchhiker’s Guide to the Galaxy, and involves the infinite improbability drive. I’ll get to that in another post at some point soon.)

For those interested, some basic facts about me:

• Dan Milstein, born 1971.
• Currently reside in Boston, Mass.
• In sixth grade, someone put me in front of an Apple IIe running… Logo. Never been the same since. Hence the blog title.
• Work at a startup. Quite happily.
• Love that python programming.
• Have an entire other life which I won’t be talking about in this blog, but which just may be of interest to you, if you like the idea of Theater That Doesn’t Suck.