Wednesday, November 25, 2009

Playing with Chrome OS

I have to say that even in its alpha (or beta?) version, Chrome OS looks good. I like the home "apps page" and except for not having an application that is a terminal emulator, it is fairly complete: my calendar, google docs, etc., are all available, and web browsing is fast.

Again, given a good terminal program for use cases where I need to quickly SSH to a server, I can see a light weight and low power netbook nicely augmenting my laptop + external monitor setup.

Thursday, November 19, 2009

I am watching the live Chrome OS Webcast

For end users, Chrome OS is a great idea - I would argue that most of my friends and relatives would be better off not running Windows, OS X, or (full) Linux.

What about software developers: still useful, but not a replacement for a laptop. In a pinch, assuming a terminal window to run remote bash shells, Emacs, etc., I could still get work done while travelling. Still, for me, a MacBook with Ubuntu and OS X with RubyMine, IntelliJ, Eclipse, OmniGraffle, etc. is just about perfect for my workflow.

That said, I will buy a Chrome OS netbook when they are available.

Tuesday, November 17, 2009

Hosted MongoDB and CouchDB

After I finish up some client work this morning, I am planning on finishing a DevX article on using Heroku as a deployment platform. Since deploying to Heroku is so simple and so well documented, you might think that I would have a difficult time writing new material :-)

After a short tutorial on getting started, I am writing mostly about using both CouchDB and MongoDB as data store, either hosted yourself on EC2 (or another server external to Heroku, which is itself hosted on EC2) or commercial managed solutions like Cloudant for CouchDB and MongoHQ for a managed MongoDB service.

I like to manage my own and customer deployments on EC2 - frankly, it is fun :-)

That said, I think that there are sometimes business reasons for using hosted solutions like Heroku, Cloudant, and MongoHQ. It is a balance between development and admin costs and paying for managed platform as a service offerings.

nice: Rubymine 2.0 released

I use Rubymine for most of my Ruby/Rails/Sinatra development on Ubuntu, and use it in conjunction with TextMate on OS X. I find it convenient enough to alternate between TextMate when I don't need IDE features, and Rubymine when I do.

One of the biggest improvements is that indexing now occurs in the background and auto-complete and other features become available that depend on knowledge of an application and the gems that it uses.

This is subjective, but once Rubymine 2.0 loads up and is done with any background indexing then the CPU use is minimal, and I think improved from earlier versions (nice to not have the fan kick in on my laptop when the CPU cores heat up). For the Rails application that I am coding on right now, Rubymine is using about 360MB of resident memory - this is OK with me.

Wednesday, November 11, 2009

MongoDB has good support for indexing and search, including prefix matching for AJAX completion lists

I have been spoiled by great support for indexing and search in relational databases (e.g., Sphinx, native search in PostgreSQL and MySQL, etc.)

I was pleased to discover, after a little bit of hacking this morning, how easy it is to do indexing and search using the MongoDB document-centered database. I have two common use cases for search, and MongoDB seems to handle both of them fairly well:
  • Search for words inside of text fields
  • Efficient word prefix search to support AJAX "suggest" style lists
My approach does require combining search results for multiple search terms in application code, but that is OK. Assuming the use of MongoRecord, here is a code snippet:
class Recipe < MongoRecord::Base
collection_name :recipes
fields :name, :directions, :words
def to_s
"recipe: #{name} directions: #{directions[0..20]}..."
end
def Recipe.make collection, name, directions
collection.insert({:_id => Mongo::ObjectID.new, :name => name,
:directions => directions,
:words => (name + ' ' + directions).split.uniq})
end
end

host = 'localhost'
port = Mongo::Connection::DEFAULT_PORT
MongoRecord::Base.connection = Mongo::Connection.new(host,port).db('mongorecord-test')

db = MongoRecord::Base.connection

coll = db.collection('recipes')
coll.remove({})

coll.create_index(:words, Mongo::ASCENDING)

Recipe.make coll, 'Rice Soup', 'Cook the rice, then add extra water to thin it out.'
Recipe.make coll, 'Cheese and Rice Crackers', 'Slice the cheese and layer on top of crackers.'

puts "\nSimple find"
puts Recipe.find_by_name(:name => 'Rice Soup').to_s

puts "\nFind recipe by regular expression (ignoring case) in array of words /water/i"
Recipe.find(:all, :conditions => {:words => /^water/i}).each { |row| puts row.to_s }
According to the MongoDB documentation, a regular expression match like /^water/i will use an index just as a relational database match in the form like 'water%' does.

I am still in a learning mode with MongoDB, so I would appreciate any comments on improving this aproach.

Tuesday, November 10, 2009

How to install CouchDB + nginx + basic authentication on EC2, including a Ruby client

Please note that if want to more secure installation, SSL should also be installed following these instructions (I used these instructions and another web blog to create the following abbreviated instructions). For my purposes, basic HTTP authentication is good enough. I assume that you are used to using nginx and CouchDB and either installed them from source or using apt-get. I am using Ubuntu, so you might have to modify these instructions slightly. On my laptop, I created a simple crypt program because OS X does not include one:
#!/usr/bin/perl
print crypt($ARGV[0],$ARGV[0])."\n";
After giving this script execute permissions, I created an encrypted password:
crypt my12398pass61
You should save the output because on your EC2 instance you need to, as root or sudo, edit the file /etc/nginx/htpasswd adding a line:
couchclient:myEKNgP2ivVVo
where myEKNgP2ivVVo was the output from crypt for the plain text password my12398pass61. Then edit nginx.conf file adding something like:
    server {
listen 9001;
server_name example123.com; # not a real domain name
location / {
auth_basic "Please login to use CouchDB";
auth_basic_user_file /etc/nginx/htpasswd;
proxy_pass http://localhost:5984;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
If you restart nginx, then you should be able to access
http://example123.com:9001/_utils
You will have to enter couchclient as the user name and my12398pass61 as the password. I allowed my browser to set an authentication cookie so I would not have to keep logging in. (Obviously, you should use a different user name and password with crypt and setting up your /etc/nginx/htpasswd file.)

For a ruby client, do a "gem install couchrest" and try this:
require 'rubygems'
require 'couchrest'

db = CouchRest.database!("http://couchclient:my12398pass61@example123.com:9001/testdb")
response = db.save_doc({:key => 'value', 'another key' => 'another value'})
doc = db.get(response['id'])
puts doc.inspect
You should be good to go writing Ruby applications that use your remote CouchDB service.

This installation is not very secure and should probably not be used on a production server containing sensitive data. I am not a security expert; if you are then I would appreciate your comments on this blog entry.

- - -

PS. an hour after writing this blog, I found a simpler solution of using a SSH tunnel. Check this out on the Disco Blog. You set a tunnel like:
ssh -i ~/.ssh/id_rsa-gsg-keypair  -L 5984:localhost:5984 root@ec2-31-111-149-100.compute-1.amazonaws.com
If you use an Elastic IP address so your server always has the same IP address, then this ssh command can be aliased, for fast temporary connections to CouchDB and other services that are confgured for only localhost client connections.

Sunday, November 08, 2009

"always on" MongoDB installation on my laptop

I spend a lot of time experimenting with infrastructure software, sometimes for customer jobs and sometimes just because it is fun to learn new things. For non-SQL data stores, I have spent a lot of time in the last year experimenting with and using CouchDB, AppEngine datastore, Tokyo Cabinet, MongoDB, Cassandra, and SimpleDB. Tokyo Cabinet and SimpleDB store hash values as strings, and don't have the great client APIs that the others have because limitations in string-only hash values. That said, for an Amazon hosted application SimpleDB can be a good choice and Tokyo Cabinet is light weight and easy to install and use. Casandra looks great, and as I have written about here before, Cassandra is easy to use from ruby and has great features.

MongoDB has great performance and similar capabilities as Casandra. Chris Kampmeier has a great writeup that covers installing MongoDB on OS X, including setting it up as a system service. I followed Chris's directions. A pleasant surprise is that MongoDB has a light footprint, and leaving it running as a service like I do with PostgreSQL and MySQL is reasonable.

Along with the mongo and mongo_record gems, MongoDB is an awesome tool, and always keeping it running makes it easy to experiment with. BTW, I also keep Sesame and CouchDB running on my local network for much the same reasons.