Friday, July 31, 2009

Tools: experiment with many, master a few of them

I am admittedly a tinkerer: I really enjoy reading other people's code, experimenting with new languages and technologies. That said, I make the effort to really only master a few technologies (e.g., Ruby, Java, and Lisp for programming languages (my paid for work is split fairly evenly between these languages), and specialize in AI, cloud deployments, Rails and Java-based web apps).

I may be transitioning to adopting two new core technologies. I have been using relational databases since 1986 and have a long term liking for PostgreSQL (less so, MySQL). The "non-SQL" meme has become popular with a lot of justification: for many applications you can get easier scalability and/or better performance on a single server using other types of data stores. Google's AppEngine datastore (built on their big table infrastructure) is clearly less convenient to develop with than a relational database but it may be well worth the extra effort to get scalability and very low hosting fees.

I have been spending a fair amount of time using the AppEngine data store this year, and more recently I am learning to effectively use Tokyo Cabinet with Ruby and Java. Different non-relational data stores have different strengths and weaknesses and in an ideal world I would learn several. In a practical world, this is not possible, so I am looking at Tokyo Cabinet as my "new PostgreSQL" - very much good enough for a wide range of projects where a relational database is not a good fit.

I was talking with a new customer yesterday about Ruby deployments, and efficiency issues in general. Ruby is a "slow" language, but Rails apps do not need to run slowly since so much of the heavy lifting is done with highly efficient external processes like Sphinx, PostgreSQL, Tokyo Cabinet, nginx, etc.