Monday, November 26, 2012

I am going to be mostly off the Internet for 3 weeks

I moderate comments because occasionally someone leaves some SPAM as a comment - so, there is usually just a short delay between the time readers post comments and when I moderate/publish them.

I will be on vacation, and for most of the time I will not have an Internet connection, so any comments left on my blog may not get moderated until the end of December when I get back home.

Best regards,
Mark

Saturday, November 24, 2012

Deep Learning

I worked in the field of artificial intelligence during the "AI winter" (a backlash against too optimistic predictions of achieving "real AI") and to this day I avoid getting too optimistic of huge short term gains in our field. That said, in the last several months a few things are stirring up my old optimism!

I have been enjoying Geoffrey Hinton's Coursera course Neural Networks for Machine Learning and I was pleased to see a front page New York Times article this morning on deep learning. Another really nice reference for deep learning is a very large PDF/viewgraph presentation by Richard Socher, Yoshua Bengio and Chris Manning.

Another very good resource is the Deep Learning Tutorial that provides the theory, math, and working Python example code.

Deep neural networks have many hidden layers and have traditionally been difficult to train.

In addition to very fast processors (graphics chipsets) a very neat engineering trick is pre-training weights in deep networks by stacking Restricted Boltzmann Machines, etc. After pre-training, weights can be fine-tuned using back propagation.

I haven't been this optimistic about (relatively) short term progress in AI since the early 1980s. Hoorah!

Saturday, November 10, 2012

"ClojureScript: Up and Running" book

I bought the Kindle edition of the book "ClojureScript: Up and Running" by Stuart Sierra and Luke VanderHart a few days ago. It is well written and a good way to ramp up on using ClojureScript for web client programming in Clojure instead of Javascript.

I have experimented with ClojureScript before, and now that two of the three Coursera classes I have been taking this fall are done, getting up to speed on ClojureScript has moved to the top of my side-project to-do list. (BTW, as I have written before, one of the classes I have taken this fall is the Functional Programming Principles in Scala, and that class is also of value to Clojure programmers who might not have a strong interest in Scala specifically, but have an interest into better understanding functional programming - the lectures for that class were especially enjoyable and useful.)

I have converted all three of my main web sites to Clojure + Noir in the last few months and this is a great combination (especially when used with a high level UI framework like Twitter Bootstrap or Dojo Mobile) for web applications. Since I am already happy with this web app development stack, why learn ClojureScript? Basically because I don't like to do a lot of coding in JavaScript. In the last few years I have enjoyed several projects done using either GWT or SmartGWT where both web client side and server side is done in Java, and I anticipate using ClojureScript, but only for projects with complex client side functionality.