Richard Bucker

Benchmarks

Posted at — Dec 27, 2012

I have a new perspective on benchmarks. If you have two things that are supposed to be the same thing or provide the same function… if the benchmark is not the same then they are not the same.If they are really supposed to be nearly identical… say (a) implemented in python and (b) implemented in Ruby, then if you want to compare the benchmarks then you are also comparing their individual JITs, libraries, miscellaneous cruft, and coding style.As I read an article on MBassador and Gauva… It’s impossible for them to provide the exact same functionality or reliability… so what’s missing may be as important as the benchmark when making the selection.