Saturday, December 31, 2005

A trip down memory lane: I have a Prolog programming job

I just accepted a small consulting job that is to be written in Prolog. Ten or fifteen years ago, I was pretty much a Prolog guru (I once rewrote a planning system that had taken 6 weeks to write in Common Lisp to a Prolog version - took just 4 or 5 days -- and the Prolog version had better graphics than the Common Lisp version).

Fortunately, I worked through the excellent book Art of Prolog a few years ago, so I am not that rusty at Prolog programming. When Prolog is an appropriate for a problem domain, probably no other programming language is as productive. Good applications of Prolog include search, circuit layout, planning, and natural language processing (but not statistical natural language processing).

Friday, December 30, 2005

A novel search engine application

During a couple hour conversation today with my publisher, he posed the problem of determining what word sequences can logically go together in natural language. Examples would be:
  • the cow is brown - OK
  • the cow is green - probably does not make sense
  • the recess was one hour - OK
  • the recess is brown - makes no sense if recess is used as a school break, could make sense if the recess is the (Wordnet #2) sense "an enclosure that is set back or indented"
I would normally think of this as a sort-of hidden Markov model problem: look for frequencies of words appearing together, allowing a few wild card intermediate words.

However, a better solution came to me in about 15 seconds: use a search engine like Nutch to index a reasonably large part of the web. To test word sequences like "the cow is brown" and "the cow is green", we would need to look at the number of times the words [cow, brown] appeared in that sequence close to together; same for [cow, green]. This approach would even give reasonable answers for sequences like [recess, brown] that might make sense for some Wordnet senses of the words, but would likely not occur in actual use.

Thursday, December 29, 2005

The world is getting more peaceful

You would not know it from most shoddy "20 second sound bite" news services but this Washington Post article clearly states otherwise. I quote:
A major study by the Rand Corp. published this year found that U.N. peace-building operations had a two-thirds success rate. They were also surprisingly cost-effective. In fact, the United Nations spends less running 17 peace operations around the world for an entire year than the United States spends in Iraq in a single month.
The reality is that, since the end of the Cold War, armed conflict and nearly all other forms of political violence have decreased. The world is far more peaceful than it was.
It turns out that terrorism kills very few people, even small wars kill many people, and as the world becomes more tightly linked economically, war makes even less sense to sane and rational people.

Wednesday, December 28, 2005

New open source project: Ruby Message System

I have relied on guarenteed delivery asynchronous messaging to build
large scale systems for 20 years. I was surprised when I could not find
something simular to Java's JMS for Ruby so I decided to build my own
and release the server under a GPL license and the client libraries
under a LGPL license. When I have code to release it will be in the
usual place (www.markwatson.com/opensource).

If I am reinventing the wheel, please let me know! I only plan on
implementing what I need, but maybe when there is a public code base
other people might contribute. This project is in the planning stage
right now. Here are some rough notes:

1. unlike Java JMS, there is currently no planned support for publish
and subscribe

2. the primary data structure of RMS is a shared collection of named
message queues

3. there is currently no planned support for security: it is
anticipated that enterprise applications will use RMS behind a
firewall. very limited security will be provided by an optional
configuration file that specifies allowed IPs for clients.

4. operations supported:

create_queue(name)
delete_queue(name)
send_message(queue, message)
register_listener(queue, a_listener)

note: a_listener object must be able to respond to the message:

receive_message(message)

5. sent messages are persisted using a database or a flat file and must
be serializable

6. once a message is delivered to all registered listeners for a queue
the message is deleted from persistent store

7. eventually, I would like to support transparent interoperability
with ActiveMQ via stump so Ruby code could interoperate with systems
written in different languages that use ActiveMQ.

I would appreciate any suggestions, ideas, etc.

PS. I was planning on using the stump system for accessing the Java ActiveMQ JMS server but after receiving a recommendation from someone for the stump ruby bindings and ActiveMQ I think that I will experiment with ActiveMQ - it is not too much trouble to keep ActiveMQ running on one of my private servers.

Tuesday, December 27, 2005

Great commentary by Mark Shields

Mark Shields is more than a bit more liberal ("to the left") than I am, but he gets it absolutely correct in this good opinion piece.

I am a "fiscal conservative": I don't believe in large government expenditures in anything but education, defense (but to not use our armed forces in foolish ways), maintaining roads, etc. I voted for Bush in 2000 but I now feel that he is clearly the worst president that the U.S. has had in my lifetime. No one even comes close to both his failed policies that damage our country everyday and his thouroughness in protecting his own privacy but eagerness to strip away privacy and civil liberties of U.S. citizens. He gets away with it the way that corrupt elite-backed rulers have throughout history: dehumanize desired adversaries and do everything that you can to scare your own population instead of making them feel secure like a good leader tries to do. Think of Winston Churchill: "We have nothing to fear but fear itself." Now think how the radical neoconservatives who seem to have hijacked our government do everything that they can to scare people in order to get away with spoiling what our great country stands for.

Monday, December 26, 2005

Java and Ruby integration

One of the chapters of the new book that I am writing is on Java and Ruby integration, mostly through cooperating web services. The idea for this chapter came from a small customer job that I did 18 months ago: integrate a CRM system written in PHP with a document work flow web portal that I had written for them using Tomcat+JSP+custom tag libraries+Prevayler. Sure, one of the great things about Ruby Rails is that the whole stack fits so well together - you lose some of the advantages of Rails when you start mixing in other technologies. Still, at this time a subset of J2EE and Ruby on Rails look to me like the two best tools for building web applications, so it makes sense to have strategies for using both technologies in a company or organization.

Thursday, December 22, 2005

flickr: coolest web portal?

A good friend of mine has been recommending flickr.com for a while and I set up a site with public pictures of Sedona, my family, etc..

Flickr is close to "desktop application functionality". Since I use so many different computers for my work, I like relying on web apps like flickr, gmail, del.itios.us, etc. And with all of my work and writing materials stored one of my servers with CVS, I have the freedom to work just about anywhere.

For just about anything I do on the internet I have sufficient bandwidth. I do look forward to improvements in reducing network latency that will make using web portals and web applications even more compelling. There are drawbacks: for example, saleforce.com had a recent outage that affected some customers for a while - that is the drawback to outsourcing your business infrastructure. Still, I think that the convenience of well implemented web portals trumps any sporadic access problems.

Wednesday, December 21, 2005

RubyPlanet.net is done (for now)

I ended up spending almost 6 hours writing and deploying the RubyPlanet.NET web portal. I would have preferred to write the portal software in Ruby but for cost reasons I used Java: I have a leased server running a single Tomcat instance. With Tomcat, it is trivial to add new virtual hosts so deploying this new web application only took a few minutes and costs me nothing extra. I currently do not have a leased server running Apache and FastCGI but that situation will change in the next week.

There are other features I could add to this web portal, but I basically did it to "scratch an itch": I wanted a single page summary of what people are writing on their blogs about Ruby - this "blog scraping" is automatic so will take no effort on my part. I did add sections "News" and "Resources" that I have to manually maintain but I have a slick administration web interface that lets me effortlessly modify these sections.

Anyway, I keep my new web portal bookmarked and check it myself a few times a day to get Ruby news. Other people are also welcome to use it :-)

Moving to more collaborative web applications

While I was listening to a 90 minute presentation on the Croquet project, I started thinking of what lessons can web portal developers learn from the croquet developers who have no constraints of supporting legacy technology. Croquet, still in early development, uses peer to peer protocols to allow small numbers of users to share a 3D simulation space - 3D graphics, naive physics, IP telephony, etc. are built in.

Users of a web portal do not know who the other users are and can not interact with them. Now, to a limited degree, we could support some interaction between users who have an active session using AJAX: user modifications to a shared data store could be broadcast back out and a Javascript could listen for broadcast changes. However, this requires a central server - Croquet sends changes to an environment (actually, it only broadcasts user inputs and counts on the environment simulation being deterministic - same precision math used on all connected computers, etc.) using peer to peer protocols.

Tuesday, December 20, 2005

Efficient use of time, getting things done

After coming back from a 20 day vacation in Belize and Guatemala, I have been revisiting making my work flow as efficient as possible.

The first thing that I am doing (a "New Year Resolution" made several months ago) is to concentrate on mastering just two technologies: Java and Ruby server side development. I find it fun to spend about 10 hours a week spread out over research and learning about AI in general, trying many software frameworks written in Lisp, Smalltalk, Python, Ruby, and Java - a New Year Resolution is to limit myself to a few hours only to Ruby and Java, and a few hours on AI research.

In the last few years I have tried to streamline my development work flow to minimize time wasted: use CVS for everything, use standard web server setups and automate deployments, etc.

For Ruby development, I just came up with another developer's trick that is saving me time: I set up a Ruby standard class library project with Eclipse (and RDT and Rad Rails) than has all of the source code in $RUBY/lib/ruby (standard Ruby libraries, gem installed libraries, and my local site installed libraries). Now, in a few seconds I can look at the source code for Ruby class libraries that I am using - very handy!

In streamlining my Java server side development, I have a fairly standard setup with IntelliJ and Tomcat (I have for all intents and purposes given up on EJBs) so the startup time on a new project is about 1 minute.

Monday, December 19, 2005

RubyPlanet.NET - all things Ruby

I was looking for a Ruby oriented blog feed aggregator and did not find one so I just created the domain name RubyPlanet.NET and I am currently implementing the site.

I will automatically filter out non-Ruby blog posts so feel free to submit your blog feed URL to me even if you only occasionally post about Ruby programming.

Saturday, December 17, 2005

World peace: yes it is possible

I just returned from a trip to Belize and Guatemala. Visiting other countries always reminds me of how similar people are. I believe that spiritually we are all bound tightly together and on a practical day to day level we all want the same thing: a better life for our children, a safe environment to live in, and time to spend with people we love.

So why do we have wars? The answer has been the same for all of known history: a small group of elites profits from war, whether they are the people behind George W. Bush, Muslim extremist leaders, the Nazis, wealthy Romans, etc. Normal people do not profit from war but often their leaders and the elites in their countries do. The mechanism of going to war has also always been the same: dehumanize the opponent and spread fear among your own population.

The more I travel the more optimistic I get that with better communication, human consciousness will reach a higher point where self-serving leaders will loose the power to spread fear and dehumanize fellow human beings.

Friday, December 16, 2005

I have a new book contract

I need to check with my publisher before talking too much about my new project, but the book will focus on advanced development techniques using Ruby and Ruby on Rails. I assume that readers already have knowledge of Ruby, Ruby on Rails, and some enterprise development experience. Anyway, I love to write and I am very much looking forward to working on this project.

I find myself using Ruby more than Java now. I still think that Java is great for huge enterprise projects but most of my work is for smaller deployments and mid-size projects where Ruby and Ruby on Rails simply seems like a better fit. Since I have a Lisp (I have written 2 Springer-Verlag Lisp books and many years of Lisp use) and Smalltalk background, Ruby is a very comfortable language for me to use. I find it interesting that Ruby might end up being the vehicle for introducing Lisp-like and Smalltalk type programming paradigms to mainstream programmers. Really, the only problem that I have with Ruby is that it is slow - not a high perfomance compiled language like Common Lisp.

Back from Belize

Carol, our friends Tim, Kathleen, Tom, and Cheryl, and I all had a great time. We did 5 days of beach time, then rented a sailboat for a week to explore the second largest coral reef in the world, then spent 4 days inland to the north.

On the spur of the moment, two days ago we rented a car and driver to go into Guatemala to see the ruins of the Mayan super-city at Tikal - awesome! When I was in high school, I took about a 3000 mile train ride down to Merida to see the Mayan ruins at Chichen Itza. A very long day driving to Tikal was easier than that train ride!