It turned out that relational databases were in the right place at the right time, and the “good enough” implementations and uses took over the world.
Yeah, that's what I've thought for a long time too. It's nice to see that in writing.
But I guess the rest of the essay leaves me a little cold. If the relational model is fundamentally flawed (rather: fundamentally mismatched to the overwhelmingly popular application design paradigm) then isn't the solution just to chuck the SQL database in the trash and start over?
So what has us tied to SQL and afraid of using the filesystem for storage directly? What are the other things we get from a database that we want? As far as I can see, the real need is for atomic transactions and consistent backups (or replication, which is the flip side of those features). No one cares about the "relational" stuff at all, really, as proven by all the awful ORMs out there.
And yet, if that were true, we'd all be happily using Berkeley DB. So what else am I missing? Some part of me thinks that there's a great startup idea in this space, but I don't quite see it yet. Tools like Google's App Engine seem to be attacking part of the problem by at least tweaking around with the SQL assumptions, but they're still fundamentally relational at their core (and they're still services, and my gut tells me that most of the world will never trust their data to someone else's service, no matter how well-justified it might seem).
It may be gratifying to see your hunch written out but the author of the original article is just wrong about the history of relational databases. They were not just in the right place at the right time, or "good enough," or driven by marketing hype as the article claims. You might want to read the well-documented history of relational theory, early RDBMS implementations and how they fit with what was already in use, and the ongoing evolution of RDBMSs and SQL.
Since the relational model predates OOP, to say nothing of flashes in the pan like Rails, it's backwards to say that the model is "fundamentally flawed" because it is mismatched to OOP. The mismatch isn't even real: OOP is a valid way to describe applications but it is not intended to describe databases. The relational model can enforce consistency and data integrity at the whole database level, which is a lot different than the object or single application level.
Programmers who developed large systems pre-Oracle have a very different view of the benefits of RDBMSs than programmers who first tried to use a database while developing a toy program with Rails. A little experience will teach you that for pretty much all real applications in the world the data is the only thing that has value; application code comes and goes. Any business that puts OOP purity before data integrity and consistency will not last. Would you feel better knowing that your bank uses Oracle, or that they use Rails and a half-baked ORM?
You might think "no one cares about the relational stuff at all" based on ORMS and Rails, but that is a peephole view of the world. The sooner you realize that Rails et al. are the Hollywood celebrities of the IT world the sooner you'll figure out how important those things really are.
I had more to say about this subject a while back in my article "Why Programmers Don’t Like Relational Databases" at http://typicalprogrammer.com/?p=14
I think the biggest advantage is that it makes the data somewhat independent from the app. This sounds bad when you design the app, but in practice it's a godsend. Probably because it allows you to refactor the code as you wish, as opposed to having the data tying you up forever.
I remember reading about the way Microsoft did things in office: saving was just dumping the program memory in a file. It's obvious it was bad, but what's interesting is that the most important reason it was so horrible is that it forced you to carry the old code with you forever, just to be able to load and save. Having another abstraction layer in between would have made it much more manageable, even if it saving would still mean dumping the memory somewhere.
Of course, in their particular case that may have been an advantage.
A big part of the reason for Relational DB's continued supremacy, is that it's become a part of the back-end language of business IT. Even though an OODB might be a better fit for an application, it's a handicap that your app is not on top of a relational database. SQL has become a kind of protocol for data exchange. Unfortunately, it makes for a very loose and sloppy protocol that completely spoils encapsulation.
Sure. I'm not expecting something new to displace SQL at a Fortune 500 IT shop. But the question is why are otherwise-groundbreaking technologies like Rails still wedded to the ORM stuff, when clearly there isn't any conservatism at work there. Anyone willing to bet the company on some weird language from Japan isn't going to blink about changing their datastore.
Actually one of the most interesting things at RailsConf was MagLev, which applies the GemStone object persistence engine to Ruby. Apparently there are some pretty big Smalltalk apps out there using GemStone. http://chadfowler.com/2008/6/5/maglev and http://ruby.gemstone.com/
One of the interesting technologies that Gemstone has had in its portfolio for years is the ability to make its data available in relational/SQL DB form. You can have your app running on top of an OODB, but still interface with your Fortune 500 comrades who want to do SQL queries for Crystal Reports.
I had the opportunity to work with Smalltalk/Gemstone in 1994. At that time we thought OODBMS was the next big thing. Programming was easier. However, soon it was clear to me that OODBMS was not the way to go. For example versioning is quite simple in an RDBMS compared to an OODBMS.
It turned out that relational databases were in the right place at the right time, and the “good enough” implementations and uses took over the world.
Yeah, that's what I've thought for a long time too. It's nice to see that in writing.
But I guess the rest of the essay leaves me a little cold. If the relational model is fundamentally flawed (rather: fundamentally mismatched to the overwhelmingly popular application design paradigm) then isn't the solution just to chuck the SQL database in the trash and start over?
So what has us tied to SQL and afraid of using the filesystem for storage directly? What are the other things we get from a database that we want? As far as I can see, the real need is for atomic transactions and consistent backups (or replication, which is the flip side of those features). No one cares about the "relational" stuff at all, really, as proven by all the awful ORMs out there.
And yet, if that were true, we'd all be happily using Berkeley DB. So what else am I missing? Some part of me thinks that there's a great startup idea in this space, but I don't quite see it yet. Tools like Google's App Engine seem to be attacking part of the problem by at least tweaking around with the SQL assumptions, but they're still fundamentally relational at their core (and they're still services, and my gut tells me that most of the world will never trust their data to someone else's service, no matter how well-justified it might seem).