Sunday, October 21, 2012

Clojure vs. Scala smackdown

Just kidding with the title of this post :-)

I believe in using the best tools for any given task, but this is not always possible when working with teams where most developers already know one programming language and/or framework. Also, as a consultant I usually favor using which ever tools are already used in the customer's organization.

All that said, I find that alternative JVM languages like Clojure, Scala, and JRuby are so much more effective for the projects that I work on that I have a strong preference to not use Java.

I find the decision when to use JRuby to generally be easy, using it on projects requiring fast development, web services, and as glue code for existing Java software. Increasingly though, I am viewing Scala and Clojure to be almost as agile as JRuby, with much better runtime performance.

For me, the tough decision is between Scala and Clojure. Taking Martin Odersky's Functional Programming with Scala class definitely affects my decision because I feel like I am learning best practices for Scala development and I am enthusiastic about using it for new projects. To balance that out I have bought and read four books on Clojure and I have used Clojure a lot in my work in the last several years and Clojure is a pleasure to use.

Although I have conflicted and undecided views on when to use Scala or Clojure, I do believe without a doubt that Rich Hickey and Martin Odersky have designed, implemented, and maintain two languages which blow "plain Java" away in developer productivity. For our industry, this is a problem because there are so many Java developers who are already trained using Java and Java frameworks, and there is a lot of inertia for not learning new languages, especially like Scala which has a steep learning curve.

11 comments:

Jeremy Huffman said...

I'm taking the same class and having similar thoughts though I do not have Clojure background.

Scala seems to have immense potential to me. In addition to the class I bought and read Martin's book and also started the SICP (which got me interested in Clojure as well). Anyway what really interest me about Scala is how much thought he has put into making language extensions possible, such that you can build a DSL or framework that would provide similar power or expressiveness to what Ruby frameworks do. An example of this is ScalaTest which can do almost everything that Rspec can do and arguably more aesthetically pleasing in some ways due to the method infix notation.

When I looked at the web/app development frameworks though, they don't give you nearly that much joyful syntax. In fact to me they seem hung up on things like replicating SQL in a DSL. I get that, but to me there should be another layer on top of it that could make the 80% as joyful as it is in Ruby.

When I look into writing some of this framework myself, the type system becomes incredibly tedious. So lots of potential, but for me yet it just doesn't seem finished enough to start building applications with that won't constantly annoy with me what is still an excessive amount of boilerplate and ceremony (compared to Ruby frameworks).

Clojure looks less bogged down to me in those respects though it is also a lot more foreign to me so far. You have so much more experience with Clojure though - so does Scala appeal to you because it can get a little better performance? Or are there other benefits that draw you to it?

Jeremy Huffman said...

I'm taking the same class and having similar thoughts though I do not have Clojure background.

Scala seems to have immense potential to me. In addition to the class I bought and read Martin's book and also started the SICP (which got me interested in Clojure as well). Anyway what really interest me about Scala is how much thought he has put into making language extensions possible, such that you can build a DSL or framework that would provide similar power or expressiveness to what Ruby frameworks do. An example of this is ScalaTest which can do almost everything that Rspec can do and arguably more aesthetically pleasing in some ways due to the method infix notation.

When I looked at the web/app development frameworks though, they don't give you nearly that much joyful syntax. In fact to me they seem hung up on things like replicating SQL in a DSL. I get that, but to me there should be another layer on top of it that could make the 80% as joyful as it is in Ruby.

When I look into writing some of this framework myself, the type system becomes incredibly tedious. So lots of potential, but for me yet it just doesn't seem finished enough to start building applications with that won't constantly annoy with me what is still an excessive amount of boilerplate and ceremony (compared to Ruby frameworks).

Clojure looks less bogged down to me in those respects though it is also a lot more foreign to me so far. You have so much more experience with Clojure though - so does Scala appeal to you because it can get a little better performance? Or are there other benefits that draw you to it?

Mark Watson, author and consultant said...

Hello Jeremy,

Scala, like Java, does take less memory that Clojure and is a little faster but that does not make much difference as far as I am concerned.

Scala has a lot of upside because it is extensible and Clojure has a lot of upside because it is an easy language to learn.

-Mark

Attila Magyar said...

Groovy is also more joyful then java. Unfortunately in the java world most people think that frameworks are much more important than the language. They don't realize that many framework features are just workaround for the limitations of the language.

Paul said...

"Scala has a lot of upside because it is extensible and Clojure has a lot of upside because it is an easy language to learn."

I would argue that Clojure is as extensible if not more so than Scala. The patterns that protocols enable, the strengths of a first-class macro system, the abilities enabled with reader literals, the many applications of the reader itself (and prn), and ClojureScript (as well as the general toolchain behind it) - all make the case for Clojure's extensibility, in ways that Scala struggles to match or can't match at all.

That's not a dig on Scala, I've successfully used the language on a few projects, but I don't think there's an argument to be made about Scala's extensibility over Clojure's.

Mark Watson, author and consultant said...

Attila: I evaluated Groovy and Grails for a customer project, but writing a 3 or 4 hour sample that we then looked over is mostly my experience. If it works for you, great!

Paul: I agree both on Clojure's extensibility and the nice ecosystem around Clojure (e.g., ClojureScript)

Dmitri Sotnikov said...

I found that for me Clojure wins on simplicity and consistency. While it looks more alien initially, once you learn the basics, you just reuse the same patterns everywhere. I find that the only things I look up for Clojure are specific functions from standard library.

On the other hand with Scala, I found myself looking up syntax and quirks constantly, and the mental overhead of reading and writing code was definitely higher.

I got my team interested in Clojure about a year ago, and we decided to see if it would work for us. I was the only person on the team with any FP experience. My team is composed of 5 people, myself included. The rest of the team were primarily experienced in Java.

First thing we realized is how little Clojure one needs to know to do real world tasks. We write web applications and majority of work involves getting some data from a database or a service, transforming it in some fashion, and displaying it to the user. This is incredibly trivial to do with Clojure, since it has a lot of powerful collection manipulation libraries.

One major concern was maintainability, since it's fairly easy to write very dense code. This turned out to not be a problem in practice. Because Clojure code is written as a tree, refactoring it is very easy. You can write your code to do what you need, then rip out nodes into separate functions until it looks clean.

Using the REPL is a huge advantage there as well in my opinion, as it encourages playing with code and testing it on the fly. You're much more likely to refactor something if you can test it on the spot without having to restart the whole application.

It also turns out that you need very little code to do things that would be very involved and tedious in Java. Having a small declarative code base is definitely a bonus when it comes to maintenance.

Currently, we're very happy with the results. We have Clojure applications in production and we even released an open source library which we're using to do our reporting, which can be found at https://github.com/yogthos/clj-pdf

Mark Watson, author and consultant said...

Thanks for sharing that Dmitri. The simplicity of Clojure is a win, for sure. The standard seq APIs are perhaps the best example of this. Rich is a practical guy, and I think that he optimized the language to make professional programmer's lives easier. That said, Scala is incredible also, even though it took me about three attempts to learn the language before it clicked for me.

Dmitri Sotnikov said...

They're both nice languages, and definitely have their respective pros and cons. In my opinion, the more FP languages start spilling over into the mainstream the better. After all, it's the mindset that counts and not skills in any particular language.

People who know one imperative language can pick up another fairly easily, since they already have a sense of how to structure code and what patterns to use.

The reason FP is considered difficult is due to the fact that you have to internalize new patterns and concepts, instead of simply learning some syntax sugar and the libraries. Both Clojure and Scala help people become comfortable in that regard.

Michael Campbell said...

> Taking Martin Odersky's Functional Programming with Scala class definitely affects my decision because I feel like I am learning best practices for Scala development

I am also taking the class, but I have a different view here. I feel I'm not learning any Scala best practices; indeed I don't feel I'm learning ANY practices. There are a couple examples of recursion, using an accumulator, and an enormous amount of time on the "substitution model", which, while interesting, is not terribly pragmatic for the course.

Week 4 ended with an assignment that many are saying you need week 5 lectures to do very well. Week 4 has caused the forums to be flooded with philosophical debates over what API methods can/should/should not/cannot be used, with NO direction from the course owners. I don't think many of us care what the answer to "should we use the List API" would be, but it's evident a lot of people want SOME answer.

The course, while very illustrative, is a bit too vague and handwavey for my tastes; tell us at least what direction you are trying to get us to go. It has taken on an autodidacticism that I find a bit weird.

That said, I'm learning despite all this, and maybe it's just an incongruity between their teaching style and my learning style, and that's fine; this is no indictment of the course or the people, just a totally different experience than you are getting.

Mark Watson, author and consultant said...

Thanks for the comments Michael. By best practices, I refer to emphasis on tests, understanding the substitution model, better understanding functional programming, enjoying the Scala IDE dynamic worksheets, etc., etc.

It is true that the course itself is not perfected like Andrew Ng's machine learning class, but I feel that I can learn a lot from Martin Odersky.

I have been initially frustrated by some of the programming homework, but after I worked through the problems I encountered, I felt like it was time well spent.