Thursday, November 18, 2010

Benchmarks: memory use can be as important as runtime for some applications

I often look to the benchmark game results. While I am pleased that Clojure is now included, I find the memory use (at least for these benchmark programs) to be disturbing (e.g., Clojure vs. Ruby 1.9 and Clojure vs. Java). When deploying to small servers, VPSs, small EC2 instances, etc. memory use can be critical.

2 comments:

Rich Hickey said...

I think one needs to distinguish between the amount of memory a programs requires, vs the amount of memory a program uses because it is allowed to. These benchmarks don't do that.

When running with the Java defaults, any program that uses a lot of dynamic memory over time (even short lived ephemeral objects) will seem to 'use' all the memory made available to it, even if its max working set is substantially smaller. This is simply good GC policy (you said I had X to work with, so this is the best collection schedule). The same programs could potentially run in far less memory if given a smaller -Xmx limit, and a larger program could run in the same memory.

Mark Watson, author and consultant said...

Thanks Rich. My company is (largely) a Clojure shop and I was concerned about memory use. I'll experiment with setting a -Xmx limit.