Citrusleaf: an interesting (non open source) NoSQL data store

I have been using Citrusleaf for a customer (SiteScout) task. Interesting technology. Maybe because I am excessively frugal, but I almost always favor open source tools (Ruby, Clojure, Java, PostgreSQL, MongoDB, Emacs, Rails, GWT, etc., etc. that I base my businesses on). That said, I also rely on paid for software and services (IntelliJ, Rubymine, Heroku, AWS services, etc.) and it looks like Citrusleaf is a worthy tool because of its speed and scalability (which it gets from Paxos, using lots of memory, efficient multicast when possible for communication between nodes in a cluster, etc.)

Yes, the DynamoDB managed data service is a very big deal

Just announced today: DynamoDB solves several problems for developers:

• No administration except for creating database tables (including some decisions like using simple lookup keys or keys with range indices and whether reads should be consistent or not)
• Fast and predictable performance at any scale (but see comment below on the requirement for provisioning)
• Fault tolerance
• Efficient atomic counters

The probable hassle for developers that I see is in knowing how to provision tables for reasonable numbers of allowed reads and writes per second. When you create tables one option is to get warning emails when you hit 80% of provisioning capacity; I interpret this to mean that you really had better not go over the capacity that you have provisioned. Amazon needs to know how much capacity you need in order to allocate enough computing nodes for your tables. The capacity that you pay for can be raised and lowered to avoid getting runtime exceptions when you go over your provisioned number of reads and/or write per second.

The lastest AWS Java SDK handles DynamoDB. For Ruby, the latest aws-sdk (gem install aws-sdk) supports DynamoDB. I signed up for DynamoDB, looked at the Java example, and wrote a little bit of working Ruby code using documentation - I had to slightly change the example code to get it to work for me:

require 'aws-sdk'

dynamo_db = AWS::DynamoDB.new(
    :access_key_id => ENV['AMAZON_ACCESS_KEY_ID'],
    :secret_access_key => ENV['AMAZON_SECRET_ACCESS_KEY'])
table = dynamo_db.tables.create('my-table', 10, 5)

begin
  sleep 3
  puts "Waiting on status change #{table.status}"
end while table.status == :creating

# add an item
item = table.items.create('id' => '12345', 'foo' => 'bar')

# add attributes to an item
item.attributes.add 'category' => %w(demo), 'tags' => %w(sample item)
p item

# update an item with mixed add, delete, update
item.attributes.update do |u|
  u.add 'colors' => %w(red)
  u.set 'category' => 'demo-category'
  u.delete 'foo'
end
p item.attributes.to_h

# delete attributes
item.attributes.delete 'colors', 'category'

# get attributes
p item.attributes.to_h

# delete an item and all of its attributes
item.delete

I used the AWS web console to then delete the test table to avoid charges.

DynamoDB is a big deal because while it is easy enough to horizontally scale out web applications and back end business applications, it is a real pain to scale out data storage for session handling and application data. Except for paying for the service, Amazon is trying to remove these hassles for developers.

I think that in addition to deployments to EC2s, DynamoDB will be a very big deal for Heroku users because it gives them another data store option in addition to Heroku's excellent managed PostgreSQL service, MongoHQ, Cloudant, and other 3rd party data service providers.

Web 3.0 and the Semantic Web, a slight return

After talking with a friend and a friend of his about the Semantic Web and healthcare yesterday, I re-watched a great video on Web 3.0 by Kate Ray that I bookmarked and blogged about a couple of years ago. I like this video because it frames the problems that the Semantic Web is trying to solve. My last published book (for APress) had Web 3.0 in the title, a term that did not really catch on :-)

At least a little bit of my enthusiasm for Semantic Web technologies has diminished over the last ten years because of problems that I have had on customer projects trying to collect linked data from disparite sources and merge it into something useful. There are (apparently) no silver bullets and any data collection and exploitation activities involve a lot of difficult work.

I would not be surprised if this problem of merging different data sources is not solved by using Ontologies and webs of linked data sites, but rather, by vendors curating data in narrow domains and selling interfaces to this curated data.

In a world of too much information the activity of curation can have a very high value and this value and the market price for these services will determine the amount of resources invested in combinations of automated and manual curation of information.

sleepybird.us site is online

Yesterday I wrote about two web portals I have been working on in Clojure. One of them is online: our stock photo web site. This is a simple web app written with Clojure and Noir. I use the excellent stripe.com system for accepting orders for JPEGs (and soon hi-def video clips). In my tests it seems easy enough to buy JPEG files: you just check the ones you want, go to the purchase page, and in a few seconds you are downloading a ZIP file with the JPEGs you purchased. A simple little web app but I think that my wife and I will have fun with it: we are avid photographers.

My two new projects: both web portals written in Clojure

I have three web portal projects that I have wanted to develop for quite some time. I am close to releasing two of them (a text analytics web service and a stock photos and video clip store. My wife and I are avid photographers and we have been wanting to travel more and do more photography; I started putting together the photo site yesterday morning and hope to have it fully on line in the next day of two - simple to implement. The text analytics web service will be publicly available within a month or so (right now, just the demo page is active - I short circuited the new account login for now). My third project is a web portal for a single consultant to manage multiple customers. Last year I prototyped this for my own use using Java + GWT + AppEngine and then ported it off of AppEngine, using MongoDB for the data store. I have had such a fun and productive time using Clojure and Noir for my two recent projects that I am considering porting this third project to Clojure. I might leave it as-is except that I have already done most of the design for a more complex web app to manage multiple consultants with multiple customers. I know that the development would go much faster using Lisp.

Using Emacs and org-mode in OS X

I recently ran across David O'Toole's org-mode tutorial and I have been experimenting with using org-mode with Emacs instead of the little utility web app I wrote for my own use a few years ago. I decided that I like org-mode better after learning the basic commands even though I can no longer access my to-do lists from ay web browser. Org-mode is useful for more than simply managing to-do lists and tasks but that is what I am mostly using it for.

To make org-mode always easily available I added this to my ~/.profile:

alias orgmode='echo -e "\033]0;org-mode\007";Emacs -nw ~/Documents/org-mode/.'

This will open org-mode in a the current term window tab and change the tab title to "org-mode." I keep all of my org-files in /Users/markw/Documents/org-mode/ so change that bit of bash script to reflect where you want to keep your org data files. You might also want to substitute emacs for Emacs -nw which is what I use for command line Emacs because I prefer the latest version 2.4.x of Emacs.

(re) learning Clojure

I have been using Clojure for about 25% of my consulting work in the last 2 years, read two books on Clojure, and I had some Clojure examples in a book I wrote last year.

That said, I don't really feel "expert" at the language the way I do with Java, Ruby, and Common Lisp.

I am trying to fill in some gaps by carefully reading through one of my customer's Clojure code, and all Clojure libraries that I use like Noir, Compojure, etc. I am trying to pick up more idioms. I enjoy it when I see a new trick in someone else's code and going back to my code to improve it.

Using the New York Times Semantic Web APIs

I am working on a side project of my own in Clojure using the AllegroGraph 4 and Stardog RDF repositories (thanks to Franz and to Clark & Parsia for licenses to use their products!) and my own NLP code. I am using the excellent NYT data access APIs to get research/test data.

I am going to show you some simple examples in Ruby for accessing the NYT Semantic Web APIs that are free to use up to 5000 API calls a day.

I also use other NYT APIs. Each API has an access key that you need to sign up for. I set my access keys as environment variables that I access in my code; for example in Ruby:

# New York Times API Keys:
NYT_SEMANTIC_WEB = ENV['NYT_SEMANTIC_WEB']
NYT_SEARCH = ENV['NYT_SEARCH']
NYT_NEWSWIRE = ENV['NYT_NEWSWIRE']
NYT_PEOPLE = ENV['NYT_PEOPLE']
NYT_TAGS = ENV['NYT_TAGS']

In the following code snippets, I am only using the Semantic Web APIs. I want to first search for available concept types and concept names, based on keyword search:

require 'simple_http'
require 'json'

def semantic_concept_search query
  uri = "http://api.nytimes.com/svc/semantic/v2/" +
        "concept/search.json?" +
        "query=#{CGI.escape(query)}&api-key=" +
        NYT_SEMANTIC_WEB
  JSON.parse(SimpleHttp.get(uri))
end

def  pp_semantic_concept_search query
  json = semantic_concept_search(query)
  puts "Results:\n"
  json["results"].each do |result|
    puts "\n\tconcept_name:\t#{result['concept_name']}"
    puts "\tconcept_type:\t#{result['concept_type']}"
    puts "\tconcept_uri:\t#{result['concept_uri']}" if result['concept_uri']
  end
end

pp_semantic_concept_search("Obama")

The second method "pretty prints" the JSON data that I am interested in. Some of the sample output looks like:

concept_name: Obama, Barack
concept_type: nytd_per
concept_uri: http://data.nytimes.com/47452218948077706853

concept_name: Obama, Malia
concept_type: nytd_per

concept_name: Obama, Michelle
concept_type: nytd_per
concept_uri: http://data.nytimes.com/N13941567618952269073

Once I have a concept type and concept name I can then look up articles:

def lookup_concept_data concept_type, concept_name
  uri = "http://api.nytimes.com/svc/semantic/v2/" +
        "concept/name/#{concept_type}/" +
        "#{CGI.escape(concept_name)}.json?&" +
        "fields=all&api-key=" + NYT_SEMANTIC_WEB
  JSON.parse(SimpleHttp.get(uri))
end
 
def pp_lookup_concept_data concept_type, concept_name
  puts "** type: #{concept_type} name: #{concept_name}"
  json = lookup_concept_data(concept_type, concept_name)
  puts "Results:\n"
  json["results"].each do |result|    puts "\n\tLinks:"
    result["links"].each do |link|
      puts "\t\trelation: #{link['relation']}"
      puts "\t\tlink: #{link['link']}"
      puts "\t\tlink_type: #{link['link_type']}"
    end
    result["article_list"]["results"].each do |article|
      puts "\tTitle: #{article['title']}"
      puts "\tDate: #{article['date']}"
      puts "\tBody: #{article['body']}\n\n"
    end
  end
end

pp_lookup_concept_data('nytd_per', 'Obama, Barack')

Some sample output looks like:

Links:
 relation: sameAs
 link: http://rdf.freebase.com/ns/en.barack_obama
 link_type: freebase_uri
 relation: sameAs
 link: http://dbpedia.org/resource/Barack_Obama
 link_type: dbpedia_uri
 relation: sameAs
 link: http://en.wikipedia.org/wiki/Barack_Obama
 link_type: wikipedia_uri

Title: U.S. Urges Egypt To Let Civilians Govern Quickly
Date: 20111126
Body: WASHINGTON -- Ever since tens of thousands of protesters converged on Tahrir Square in Cairo for the first Day of Revolution exactly 10 months ago, the Obama administration has struggled to strike the right balance between democracy and stability. In the early morning hours on Friday, President Obama came out on the side of the Arab street, issuing

Title: EDITORIAL; The Solyndra Mess
Date: 20111125
Body: The Republicans on the House Energy and Commerce Committee appear to have hit the pause button on their investigation into the failure of Solyndra, a solar panel maker that entered bankruptcy proceedings in September, defaulting on a $528 million federal loan. What have we learned? Nobody comes out of this looking good. Not the Obama

Great to see useful linked data/Semantic Web data sources being made available! Hopefully these little code snippets will save you some time in getting started using the NYT APIs.

Closer to the metal: Clojure, Noir, and plain old Javascript

I am wrapping up a long term engagement over the next five to six weeks that uses Java EE 6 on the backend, and SmartGWT (like GWT, but with very nice commercially supported components) clients. As I have time, I am starting up some new work that uses Clojure and Noir, and it is like a breath of fresh air:

I keep a repl open on the lein project and also separately run the web app so any file changes (including the Javascript in the project) are immediately reflected in the app. Such a nice development environment that I don't even think about it while I am working, and maybe that is the point!

As I have mentioned in previous blog posts, I really like the Clojure Noir web framework that builds on several other excellent projects. Developing in Noir is a lot like using the Ruby Sinatra framework: handles routes, template support options, but it is largely roll your own environment.

Ruby Sinatra web apps with background work threads

In Java-land, I have often used the pattern of writing a servlet with an init() method that starts up one or more background work threads. Then while my web application is handling HTTP requests the background threads can be doing work like fetching RSS feeds for display in the web app, perform periodic maintenance like flushing old data from a database, etc. This is a simple pattern that is robust and easy to implement with a few extra lines of Java code and an extra servlet definition in a web.xml file.

In Ruby-land this pattern is even simpler to implement:

require 'rubygems'
require 'sinatra'

$sum = 0

Thread.new do # trivial example work thread
  while true do
     sleep 0.12
     $sum += 1
  end
end

get '/' do
  "Testing background work thread: sum is #{$sum}"
end

While the main thread is waiting for HTTP requests the background thread can do any other work. This works fine with Ruby 1.8.7 or any 1.9.*, but I would run this in JRuby for a long-running production app since JRuby uses the Java Thread class.

Using the Stardog RDF datastore from JRuby

I was playing with the latest Stardog release during lunch - the way to quickly get going with the included Java examples is to create a project (I use IntelliJ, but use your favorite Java IDE) and include all JAR files in lib/ (included all nested directories) and the source under examples/src.

I took the first Java example class ConnectionAPIExample and converted the RDF loading and query part to JRuby (strange formatting to get it to fit the page width):

require 'java'
Dir.glob("lib/**.jar").each do |fname|
  require fname
end

com.clarkparsia.stardog.security.SecurityUtil.
      setupSingletonSecurityManager()
com.clarkparsia.stardog.StardogDBMS.get().
      createMemory("test")

CONN = com.clarkparsia.stardog.api.
        ConnectionConfiguration.to("test").connect()
CONN.begin()
CONN.add().io().format(org.openrdf.rio.RDFFormat::N3).
  stream(java.io.FileInputStream.new(
            "examples/data/sp2b_10k.n3"))

QUERY = CONN.query("select * where {?s ?p ?o}")
QUERY.limit(10)
RESULTS = QUERY.executeSelect()

while RESULTS.hasNext() do
  result = RESULTS.next()
  result.getBindingNames().toArray().each do |obj|
    puts "#{obj}: #{result.getBinding(obj).getValue().stringValue()}"
  end
  puts
end

This is mostly just a straight conversion from Java to Ruby. The first few lines enumerate all JAR files and require them. The last part, of interpreting the results, took a few minutes to figure out. I used IntelliJ to explore the result values of class MapBindingSet, looking at available methods to call to get the binding names of the variables in my SPARQL query and the values (as strings) for these three variables for each returned result.

Output will look like:

s: http://localhost/vocabulary/bench/Journal
p: http://www.w3.org/2000/01/rdf-schema#subClassOf
o: http://xmlns.com/foaf/0.1/Document

s: http://localhost/vocabulary/bench/Proceedings
p: http://www.w3.org/2000/01/rdf-schema#subClassOf
o: http://xmlns.com/foaf/0.1/Document
...

If you want to run this bit of code, put it in a file test.rb in the top level Stardog distribution directroy and just run

jruby test.rb

I wanted to be able to use Stardog from both JRuby and Clojure. My lunch time hacking today is just a first step.

Experimenting with Google Cloud SQL

I received a beta invite today and had some time to read the documentation and start experimenting with it tonight.

First, the best thing about Google Cloud SQL: when you create an instance you can specify more than one AppEngine application instances that can use it. This should give developers a lot of flexibility for coordinating multiple deployed applications that are in an application family. I think that this is a big deal!

Another interesting thing is that you are allowed some access to the database from outside the AppEngine infrastructure. You are limited to 5 external queries per second but that does offer some coordination with other applications hosted on other platforms or host providers.

Their cloud SQL service is free during beta. It will be interesting to see what the cost will be for different SQL instance types.

It was very simple getting the example Java app built and deployed. I created a separate SQL instance (these are separate from other deployed AppEngine application instances), made a new IntelliJ AppEngine project, pasted in the example code, and it all worked.

Perception of quality is often influenced by price. Since developers now have to pay more for using AppEngine, I find myself looking more at AppEngine as a premium service, which it is. Despite my dislike for MySQL (I use PostgreSQL when given a choice), Google's hosted and managed MySQL cloud data service looks good and provides developers with more options. Their SQL service is synchronously replicated between data centers automatically for you.

It has been a few years now since I had to either set up a physical server or a leased raw server for any deployments. I like that! Thank you Platform as a Service (PaaS) providers like Heroku (built on AWS) and AppEngine - they are the future. I still do a lot of work on "plain AWS" but that is still much more agile than provisioning my own servers.

The quality of new programming languages is apparent by looking at projects using the language

The community growing around the Clojure language is great. While the Clojure platform is still evolving (quickly!) browsing through available libraries, frameworks, and complete projects is amazing.

My "latest" favorite Clojure project is Noir that simply provides a composable mechanism for building web applications (using defpartial). I get to use Noir on two customer web app projects (and some work with HBase + Clojure) over the next month or two, and I am looking forward to that. The simpler of the two web apps is an admin console exposing some APIs on a private LAN and the Try Clojure web app is a great starting point, as well as an example of a nicely laid out Noir application.

Since Clojure is such a concise language I find it easy to read through, understand, evaluate, and use projects. Since I am still learning Clojure (I have just used Clojure for about 6 months of paid work over the last couple of years) the time spent reading a lot of available code to find useful stuff is very well spent because reading good code with an open repl is a great way to learn new idioms.

Writing a simple SQL data source for the free LGPL version of SmartGWT

While travelling back from a vacation I cleaned up some old experimental code for writing a fairly generic SmartGWT data source with the required server side support code. The commercial versions of SmartGWT have support for connecting client side grid and other components to server side databases. For the free version of SmartGWT you have to roll your own and in this post I'll show you a simple way to do this that should get you started. Copy the sample web app that is included in the free LGPL version of SmartGWT and make the modifications listed below.

I also set up a Github project that contains everything ready to run in IntelliJ.

The goal is to support defining client side grids connected to a database using a simple SQL statement to fetch the required data using a custom class SqlDS. I had to strangely format the following code snippets to get them to fit the content width for my blog:

    ListGrid listGrid = new ListGrid();
    listGrid.setDataSource(
      new SqlDS(
         "select title, content, uri from news where " +
         "content like '%Congress%'"));
    listGrid.setAutoFetchData(true);

The following datasource looks for the column names (i.e., "title", "content", and "uri") in the SQL query and creates fields in the constructed SqlDS instance with those column names. I also assume that there is a servlet defined to process the HTTP GET fetch at the bottom of the constructor:

package com.markwatson.client;

import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.data.DataSourceField;
import com.smartgwt.client.types.DSDataFormat;
import com.smartgwt.client.types.FieldType;

import java.util.Arrays;
import java.util.List;

public class SqlDS extends DataSource {
  public SqlDS(String sql) {
    setID(id);
    setDataFormat(DSDataFormat.JSON);

    List<String> tokens =
        Arrays.asList(sql.toLowerCase()
           .replaceAll(",", " ").split(" "));
    int index1 = tokens.indexOf("select");
    int index2 = tokens.indexOf("from");
    for (int i=index1+1; i<index2; i++) {
      if (tokens.get(i).length() > 0) {
         addField(new DataSourceField(tokens.get(i),
               FieldType.TEXT, tokens.get(i)));
      }
    }
    // should do a better job at UUENCODEing SQL:
    setDataURL("/news?query="+ sql.replaceAll(" ","20%"));
  }
}

The only thing left to do is write a servlet that processes web wervice requests like /news?query=... and returns JSON data with fields from the SQL query for each returned row for display in the list grid:

package com.markwatson.server;

import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;

public class DbRestServlet extends HttpServlet {
    @Override
    public void doGet(HttpServletRequest req,
         HttpServletResponse resp) throws IOException {
      PrintWriter out = resp.getWriter();
      try {
          // remove "query="
          String sql = req.getQueryString().substring(6); 
          int index = sql.indexOf("&");
          sql = sql.substring(0, index);
          out.println(
             DbUtils.doQuery(sql.replaceAll("20%", " ")));
      } catch (Exception ex) {
        ex.printStackTrace(System.err);
        out.println("[]");
      }
    }
}

The utility class DbUtils returns JSON data which is what the client side SqlDS DataSource class expects from the server:

package com.markwatson.server;

import org.codehaus.jackson.map.ObjectMapper;

import java.io.StringWriter;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class DbUtils {
  static String dbURL;
  static Connection dbCon;

  static {
    try {
      Class.forName("org.postgresql.Driver");
      // Define the data source for the driver
      dbURL = "jdbc:postgresql://localhost/test_database";
      dbCon = DriverManager.getConnection(
                     dbURL, "postgres", "password");
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

  public static String doQuery(String sql)
                               throws Exception {
    ObjectMapper mapper =
       new ObjectMapper(); // should cache and reuse this!
    List<Map<String, String>> ret =
        new ArrayList<Map<String, String>>();
    Statement statement = dbCon.createStatement();
    ResultSet rs = statement.executeQuery(
                            sql.replaceAll("20%", " "));
    java.sql.ResultSetMetaData meta = rs.getMetaData();
    int size = meta.getColumnCount();
    while (rs.next()) {
      Map<String, String> row =
         new HashMap<String, String>();
      for (int i = 1; i <= size; i++) {
        String column = meta.getColumnName(i);
        Object obj = rs.getObject(i);
        row.put(column, "" + obj);
      }
      ret.add(row);
    }
    StringWriter sw = new StringWriter();
    mapper.writeValue(sw, ret);
    return sw.toString();
  }
}

I had to add three JAR files to the SmartGWT sample project:

jackson-core-lgpl-1.8.1.jar
jackson-mapper-lgpl-1.8.1.jar
postgresql-9.0-801.jdbc4.jar

SmartGWT's DataSource abstraction is a real improvement over how I connect to databases in GWT apps where I tend to write a lot of small RPC services to fetch and save data as required. My simple DataSource subclass SqlDS does not support writing data back to the database from the client; it can either be extended or you can use a RPC service call to save edited data.

Annoyed by anti-MongoDB post on HN

I am not going to link to this article - no point in giving it more attention. The anonymous post claimed data loss and basic disaster using MongoDB. I call bullshit on this anonymous rant. Why was it posted anonymously?

I am sitting in an airport waiting to fly home right now: just finished extending a Java+MongoDB+GWT app and I am starting to do more work on a project using Clojure+Noir+MongoDB.

I do have a short checklist for using MongoDB:

• For each write operation I decide if I can use the default write and forget option or slightly slow down the write operation by checking CommandResult cr = db.getLastError(); - every write operation can be fine tuned based on the cost of losing data. I usually give up a little performance for data robustness unless data can be lost with minimal business cost.
• I usually use the journalling option.
• Use replica pairs or a slave.
• I favor using MongoDB for rapid prototyping and research.
• I use the right tool for each job. PostgreSQL, various RDF data stores, and sometimes Neo4J are also favorite data store tools.

Notes on converting an GWT + AppEngine web app using Objectify to a plain GWT + MongoDB web app

There has been a lot of noise in blog-space criticizing Google for the re-pricing of AppEngine services. I don't really agree with a lot of the complaints because it seems fair for Google to charge enough to make AppEngine a long term viable business.

That said, I have never done any customer work targeting the AppEngine platform because no one has requested it. (Although I have enthusiastically used AppEngine for some of my own projects and I have written several AppEngine and Wave specific articles.) I still host KnowledgeBooks.com on AppEngine.

I wrote a GWT + AppEngine app for my own use about a year ago, and since I always have at least one EC2 instance running for my own experiments and development work I decided to move my app. It turns out that converting my app is fairly easy using these steps:

• Copy my IntelliJ project, renaming it and removing AppEngine facets and libraires.
• Add the MongoDB Java required JARs
• I had all of my Objectify datastore operations in a single utility class on the server side - I converted this to use MongoDB

Sure, a complex application would take a while, but my app only has 6 model classes (all POJOs) so the whole process took less than 90 minutes.

Recent evaluations of web frameworks while on vacation

My wife Carol and I have been visiting family in Rhode Island this week and since our grandkids are in school on weekdays, I have had a lot of time to spend writing the fourth edition of my Java AI book and also catching up on reevaluating web frameworks.

Although my main skill sets are in data/text mining, general artificial intelligence work and Java server side development, I do find myself spending a lot of time also writing web applications. In the last few years, I have done a lot of work with Rails (and some Sinatra), GWT, and most recently with SmartGWT because one of my customers really liked SmartGWT's widgets. (Note: if you are in the San Jose area and want to work on a SmartGWT project with me, please email me!)

For my own use, because I have strong Java and Ruby skills, the combination of Rails, GWT, and SmartGWT works very well for me when I need to write a web app.

That said, I have spent time this week playing with Google's Closure Javascript tools and less time with ClojureScript that uses Google's Closure. Frankly, both Closure and ClojureScript look fantastic, but I have a personal bias against making Javascript development a career and although ClojureScript works around this issue by compiling a nice subset of Clojure to Javascript I am concerned that the market for developing with ClojureScript is probably small. If you do want to write Lisp code on the server and client side definitely spend a few evenings playing with ClojureScript because it may be a good fit for you. I have also recently had a good experience with Clojure and the Noir web framework.

Anyone know any SmartGWT and Java developers looking for a job?

A call out for some help: one of my favorite customers is looking for a SmartGWT and Java developer in San Jose area - anyone know anyone good + available?

Common Lisp example code for my Semantic Web book is now LGPL licensed

A few days ago I re-released the Java, JRuby, Clojure, and Scala example code for my JVM languages edition of my Semantic Web book under the LGPL.

I just did the same thing today for the Common Lisp edition of this book:

Github repository

Changed license from AGPLv3 to LGPLv3 for example code in my book "Practical Semantic Web and Linked Data Applications, Java, Scala, Clojure, and JRuby Edition"

Here is the github repository for the source code and all required libraries.

My open content web page" where you can download a free PDF for my book or follow the link to Lulu to buy a print version.

Enjoy!

Semantic Web, Web 3.0, and composable systems

I really enjoyed Steve Yegge's long post last week about the shortcomings of Google's architecture. Google provides great services that I use every day but building systems as Amazon does of composable web services (AWS) to build more complex products and services seems like a better approach.

I have been experimenting with Semantic Web (SW) technologies since reading Tim Berners-Lee, James Hendler, and Ora Lassila's 2001 Scientific American article. I have not often had customer interest in using Semantic Web technologies and I think that I am starting to understand why people miss the value-add:

Just as AWS provides composable web services SW helps information providers to provide structured and semantically meaningful data to customers and users who decide what information to fetch, as they need it. These consumers of SW data sources must have a much higher skill set to build automated systems compared to a user of the web who manually navigates around the web to find information that they need.

So I think that the issue becomes how can to make it relatively easy for system designers and software engineers to fetch and consume information. The easy answer is to point them to a good book on SPARQL and RDF data sources. A better answer is probably to provide examples using common programming languages, the "best" libraires for making SPARQL queries, and small sample applications tailored to the types of data that an information provider provides and what type of inferencing makes sense to discover implicit data that is not explicitly in the provider's data store.

I would describe the SW as building and using composable data sources that are defined in terms of ontology's that make it possible to merge data from different sources and to discover implicit data through inference/reasoning.

A letter to my friends and family: the death of American democracy: not dying, but already dead

Hello family and friends,

Democracy in our country is dead, but you would not know it from reading the highly censored corporate-owned and controlled "news"/propaganda media.

If you look to foreign news or youtube or the general Internet or talk to friends in foreign countries that have a free press, you will understand that is is not rank and file cops, but their supervisors commiting what I think can only be called illegal brutality against the "occupy" movement.

The high-ranking police do this because they are ordered by their puppet-masters to do so. There is a huge disparity between what the general public wants and what the corporate lackeys in Congress and the corporate lackey Obama (following in the ubber corporate lackey W.Bush's footsteps) do. As Warren Buffet said in a recent interview, the USA is now a plutocracy, and that is a shame. Good writeup on a writer's arrest: I enjoy Naomi Wolf's work - a writer with reasonable views.

Our founding fathers warned us about banks and control of currency, control by the rich, etc. taking control of our country, but the right-wing extremists have removed civics from school curriculums, used total control of the media to scare people into giving up their rights as American citizens, and acted against the common good.

One Republican agenda is to make it difficult for the poor, the elderly without transportation, and students to register to vote, and their anti American "values" disgust me. Anyone who has anything to do with keeping American citizens from voting by making registration more difficult is an asshole. These are people who don't understand what our country is supposed to represent, or don't care.

One last comment: the un-educated are most easily swayed by right-wing propaganda, which is why I believe that our educational system has been deconstructed by those on the far right politically.

It is time for people who have self-identified as republicans and conservatives to speak up against the un-American right wingers who corrupt our political system. Barry Goldwater had great conservative ideals which I largely agree with, but I bet he is rolling in his grave in disgust by the current bunch of "conservatives."

Appreciating Steve Jobs and the people taking part in "Occupy Wall Street"

First: my condolences to Steve Job's family and friends. He was an awesome guy who lived on his own terms and made the world a better place by doing things that he loved and was proud of.

I would also like to give a shout out of appreciation to the broad spectrum of Americans who are taking part in "Occupy Wall Street." They are facing state sponsored brutality: the elite class doesn't like the legal protests so they put pressure on the government and government influences police to do things that in their hearts they know are not right. I have been reading a lot of strong criticism of the police for their brutality in New York City against mostly peaceful American citizens exercising their first amendment rights - I personally try to not blame the police because I think it is more accurate to blame the people who control them. There are shocking videos on youtube of police brutality against US citizens in New York City during these protests and I thought about putting some links here, but these videos are violent and may be upsetting to many people.

The erosion of rights for American citizens is shocking even though I have been watching the same thing happening already in England so I have been expecting a similar collapse of basic American rights and values in favor of the powerful.