Tuesday, November 30, 2010

Python is not such a bad language

I have used Python off and on for about 10 years and have never particularly liked it. This is funny since Ruby is my favorite programming language and Ruby and Python are very similar both in features and in the types of applications that they are best used for. For me the largest shortcoming of Python is the lack of blocks.

That said, I have two small side projects (consultingwith.me and consultingwith.us) that I wanted to host on AppEngine. I have a fair amount of experience with the Java AppEngine SDK but my gut instinct was to go with the Python SDK using the default webapp library. After spending about 5 hours writing code, I find that Python is fairly comfortable. 7/9/2011 edit: I enjoyed getting a basic system working with Python (a great learning experience) but ended up writing a feature complete version in Rails and using that for several months. In the last week or so I have re-implemented it in Java + GWT for the Appengine.

Sunday, November 28, 2010

Must-have tool for understanding your web site and blog: Google Analytics

I use Google Analytics as a feedback mechanism for what people actually read on my blog and to know which pages on my web site people read (which I equate to which pages people find most useful). As an example, 50% of visitors go directly to my Open Content web page so this gives me a strong incentive to write more free web books and post the PDFs.

As a consultant I interact with many developers and companies and it always surprises me when people don't bother to measure performance, in this case understanding what content people find interesting and/or useful.

Speaking of measuring performance:

Another must have tool if you do deployments: get a free 1-server account at cloudkick.com. (Disclosure: if you end up eventually signing up for a non-free multiple server account then I get a small perk.) If one of your web applications starts to slow down it can save a lot of time being able to tell at a glance what the I/O, CPU, network, etc. activity has been in the last few days.

Tuesday, November 23, 2010

Wonderful book: "Land of Lisp" - Conrad Barski is a great author and communicator

I have been enjoying Conrad Barski's web based tutorials for years and I recently received his new book Land of Lisp: Learn to Program in Lisp, One Game at a Time! in the mail.

I have been writing Lisp code since the late 1970s and in ancient times I wrote two Lisp books for Springer Verlag. To be honest, I just bought the book for enjoyment but I find myself getting a new perspective and learning more about Common Lisp.

Recommended!

Sunday, November 21, 2010

Distributed NoSQL datastores: Cassandra and Cloudant's BigCouch

In my work for customers in recent years almost everything that I do uses PostgreSQL (sometimes PostGIS) and/or MongoDB. (I write a lot about the Semantic Web but so far no one has paid me to work on a project with an RDF data store like Sesame or AllegroGraph.)

While I think PostgreSQL and MongoDB are great, their replication stories have not been great. MongoDB's master/slave and replica pairs work OK, and replica sets (MongoDB 1.6 and above) look to be a big improvement (it only takes a few minutes to try the MongoDB 1.6.x replica set tutorial example; follow the instructions.) I have not tried replica sets yet in a production environment but I am looking forward to it! I find MongoDB to be extremely developer friendly with convenient client libraires in Clojure and Ruby (I don't like dealing with JSON data and hashes in Java).

PostgreSQL 9 replication is easier to set up and administer than Slony but I have not had to use it in production. The replication supports master/slave hot stand-by but it is not a distributed data store with no master process.

Cassandra was designed to be distributed with no specific master server. I am almost done reading "Cassandra, The Definitive Guide." I have been enjoying experimenting with Cassandra a lot recently on both my laptop and transient EC2 instances. One negative about Cassandra for me is that I find the Java and Clojure client libraries to be inconvenient to use compared with their MongoDB counterparts. The Ruby client library is very developer friendly!

CouchDB was the first NoSQL datastore that I used (except for RDF data stores) but I have never found it to be as convenient for my work as MongoDB. That may change thanks to Cloudant's BigCouch open-source version of CouchDB that has built in clustering capability. It is extremely easy to set up a test system following the directions on BigCouch github. On both my MacBook and also using EC2s, it only took about 15 minutes to set up a cluster. If I had to set up a fault tolerant distributed data store on small (or even micro) EC2 instances cluster, BigCouch would be a strong candidate because of the relatively low RSIZE memory footprint compared to Cassandra (for empty systems, about 15MB for CouchDB and 150MB for Cassandra - but expect these memory requirements to increase rapidly with large data stores). BTW, check out the people working at Cloudant: interesting that so many physicists work there. Cloudant bases their CouchDB hosting business on BigCouch.

Good to be a programmer. Or: custom solutions are sometimes better

I signed up for a free Evernote account a year ago last April but never really used it. After reading an article in the New York Times this morning about sharing data across computers and handheld devices I decided to install the Evernote app on my Android cellphone. I also installed the web clipper Evernote Chrome browser extension. After setting up notebooks for major interests (writing tools, writing ideas, different technology areas), I spent some time using Evernote (OS X app, browser interface, and Droid app). Really nice.

I thought of using Evernote for my to-do tasks but my own custom web app my-foc.us works better for me. This reminded me how great it is to be a programmer and make things just the way you want. Now, I must admit that my-foc.us is very simple (it has just the functionality that I want and nothing else). This project is open source and took me perhaps 6 hours to write and deploy on Heroku. It is really a great feeling to make something just for yourself (although other people are welcome to use it).

Some background for another "just for me" web app:

About four years ago I almost died very quickly from two large pulmonary embolisms. Fortunately I am almost fully recovered (e.g., I can do 5 or 6 hour hikes, but not the 8 or 9 hour hikes I used to do) but I need to monitor my intake of vitamin K because it interferes with the small doses of blood thinner I take daily. Now, vitamin K is absolutely vital to have in your diet (one example: keeps calcium deposits out of arteries and heart valves and promotes bone growth and general health). So, four years ago I created a simple web app cookingspace.com that contains many of the recipes that I use and some convenience foods that I like. What is different about cookingspace.com is that it uses the USDA National Nutrient Database for Standard Reference, Release 20 to show the nutrients in food I eat. At first, I referred to the nutrients per serving for individual recipes and for complete meal plans but after a while, I trained myself to remember the approximate amounts of vitamin K and about 20 other nutrients in food. You can see displays for random recipes by clicking this link a few times. This web app turned out to be a large project, taking me about 40 hours to write because the recommended USDA database had a lot of errors so I ended up using their much larger "research" database and then trimming out things I didn't need. Anyway, for me this was time well spent.

Anyway, it is great to be a programmer :-)

Saturday, November 20, 2010

Is it better to spend time learning new programming languages or study languages and tools you already use?

I have been thinking about a discusion on Hacker News yesterday about which new programming language people want (or need) to learn. While I definitely enjoy learning new programming languages by writing small applications, I think that I personally get a better productivity boost by reviewing languages and tools I already use.

In the last year I have mostly used Ruby and Clojure for my work. In the last few months I have read two books on Clojure and one on Ruby. Sort of: the more you know, the more you can learn and more deeply understand something.

Recently I reviewed the commands and read another tutorial for the screen utility that I have been using almost everyday for three years. Well worth the time.

I do a lot of work on remote servers and emacs is not always installed so I have also used vi for years. This morning, I saw a reference to learning vim by using vimtutor and spend 20 minutes working through the complete tutor program (learn by doing: you edit the tutorial as part of the tutorial). Time very well spent.

Friday, November 19, 2010

New AppEngine 1.4.0 features a game changer for JVM languages

As I wrote last April you can use Objectify in Java apps to reduce loading request times (avoiding JDO overhead) but running JRuby and Clojure applications on AppEngine has been hindered by still long loading times. The new feature forthcoming to AppEngine (SDK available, but server side support is not in place yet) of allowing paid for apps to keep three compute instances always running will open up the AppEngine platform for other languages and frameworks. On the development side I have had good JRuby and Sinatra experiences with the AppEngine SDK but have never wanted to deploy anything other than experiments - that will change now.

Thursday, November 18, 2010

Publishing to the Kindle, Android, iPhone, and iPad

I just bought a Kindle and I like it more than I thought that I would. The screen is easy to read and it is much lighter than an iPad.

I have a plan to add support for Kindle, Android, iPhone, and iPad file formats in my writing/publishing pipeline. Currently, my "development system" for writing is Latex with some custom Ruby scripts and a Makefile for each writing project. From Latex source files, I can currently generate:
  • Lulu print books
  • PDFs for laptop viewing
  • HTML pages with automatically inserted Google Adsense ads
  • HTML pages
I have been generating fairly good revenue from publishing my Open Content books and selling print copies (some generous people also pay for the PDF instead of using the free downloads). Because I am earning more money from my Open Content writing, I plan on devoting much more time to writing next year. I plan on always making the large format PDFs free for download, and selling print books and versions for hand-held devices.

Benchmarks: memory use can be as important as runtime for some applications

I often look to the benchmark game results. While I am pleased that Clojure is now included, I find the memory use (at least for these benchmark programs) to be disturbing (e.g., Clojure vs. Ruby 1.9 and Clojure vs. Java). When deploying to small servers, VPSs, small EC2 instances, etc. memory use can be critical.

Saturday, November 13, 2010

Clozure Common Lisp 1.6 has been released

Although I have used SBCL for more consulting work than Clozure CL, I have started using Clozure for more of my own projects. One thing I prefer is that standalone applications built with Clozure are about 10 megabytes smaller than apps built with SBCL. Release notes

Thanks to Tom Munnecke for portrait photographs

My friend Tom Munnecke recently took some casual portrait pictures of me, and I am now using one as my Twitter picture. Sometime I will also replace some of the pictures on main web site.

Friday, November 12, 2010

Contact your Congressional Representatives and Senators and ask them to have their families publicly go through the TSA grope experience

Use this link to contact them.

Also, how about President Obama letting his wife and daughters get groped by TSA? Our Senators and Representatives let their wives and kids get groped?

Hate to break it to you, but the government officials who we elect and who get paid off by corporate lobbyists don't live in the same world we do.

People get the government that they deserve. Are you going to do anything about this? An email is good, but also consider taking the extra time to insist on talking to your elected officials.

BTW, I don't think that TSA has every caught a terrorist, or done any good at all. (Not like the FBI and other intelligence services who do worthwhile work.)

I am softening my position on Oracle's stewardship of Java

I have to admit that I got a little carried away with my criticism, fueled largely by my agreeing in substance with the position of the Apache Foundation. However, Apple's announcement and IBM's earlier announcements about working with Oracle and OpenJDK have caused me to soften my position.

To be fair to Oracle, my complaints about lack of tolerance for Apache Harmony and the FSF Gnu Java implementation apply equally to the now defunct Sun Microsystems.

The GPL license for OpenJDK is not all that comforting given the patents wrapped up in Java implementations: only passing the Java Compatibility Kit tests gets a patent waiver. Still, for Java developers, I think that all is right in the world, on a practical level.

Wednesday, November 10, 2010

My nephew died this morning: rest in peace Anthony

My nephew Anthony was hit by a car last night and died early this morning.
The following picture shows Anthony, my sister in law Anita, and me in front of my house in Sedona. Anthony was about 17 years old in this picture.

Here is another picture of Anthony and my Dad when they were visiting Sedona. Anthony was 20 years old in this picture.

Anthony loved talking about politics, his family, and music.

Tuesday, November 09, 2010

Question: what would be the legal ramifications of forking Java, but not calling it Java?

I understand that Oracle owns the trademark and there are patent issues. Still, an alternative pure open source platform named wombat or whatever might end up being necessary.

I understand that Oracle would like to monetize Java but there is that "killing the goose that laid the golden egg" metaphor becoming real life :-)

Ideas?

Monday, November 08, 2010

Downloading all of your data from Facebook

Nice: I just tried this: go to your Facebook Account menu on the upper right corner of the home web page and select Account Settings. Near the bottom of the page, click the learn more link for Download Your Information. Follow the instructions and you will get a ZIP file with your home page, friends list, all messages, wall, and photographs.

Friday, November 05, 2010

Free PDF for the Common Lisp edition of my "Practical Semantic Web and Linked Data Applications" book

I have been working on this book on and off for two years and finally finished it recently while on a long vacation. You can get a free PDF on my Open Content web page or if you want to encourage me to write more material on niche topics, you can buy a print copy.

Although this book covers a small market, I believe that the combination of Common Lisp and the AllegroGraph RDF data store is a great combination for developing knowledge intensive software.

Monday, November 01, 2010

Wow: I think QuickLisp will change my Common Lisp development setup

I have been experimenting with the beta for QuickLisp.org and it looks very good. I never disliked the ASDF package management system, but QuickLisp looks to be easier to use and with a few hundred common Common Lisp already in the repository, getting most dependencies installed is quick and easy. Good job Zach!

For many years I have been using a brute force approach to package management: in every project I work on, I have a utils directory where I un-tar the source code to all dependencies, and their dependencies, etc. I then locally
(push "utils/PACKAGE_DIR/" asdf:*central-registry*)
for all dependencies. This has always worked really well for me because I can ZIP up a project directory, rsync it to a server, and I am good to go with all dependencies. However, it is a pain to keep multiple copies of libraries in multiple projects. For reasons I don't even remember anymore, I never liked to use a global ASDF cache.

It will be a long weekend day project, but if I don't run into any serious problems with QuickLisp, I am going to clean up all of my Common Lisp projects to not use local copies of dependencies. I'll wait until the Common Lisp edition of my Semantic Web book is done, to avoid delaying that effort.