I’m on a project at work now to replace an old trading system with a new one that’s orders of magnitude faster. One of the decisions we made early on was to get rid of relational databases and instead use a distributed caching technology like JBoss Cache or Tangosol Coherence. But we’re repeatedly running into some issues with that approach. Fundamentally, I think they can all be boiled down to the fact that we’re using a technology (caching) for a purpose (data storage) that’s different than its original intended use. This isn’t the first time I’ve been on a team that’s made this mistake; at ePit we were trying to use MQSeries as basically a data store, and were trying to use LDAP as basically a relational database. Those weren’t good ideas then, either.
One thing we’ve done well in this project, and I have to credit our adoption of Agile for this, is force ourselves to prove out our assumptions around caching by implementing some real-life functionality. We’ve discovered early that it might not work quite the way we want it to and are starting to explore alternatives like distributed object databases. We’re failing early and cheaply.
Anyway, it’s all been a very interesting exercise for me. Every system I’ve ever developed had a relational database under it, so I’ve had to really adjust my thought process and test my every assumption in this brave new world.