Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Geoff Wozniak: Tinkering with jump tables (exploring-lisp.blogspot.com)
8 points by fogus on July 13, 2009 | hide | past | favorite | 4 comments


Unfortunately the numbers are rather misleading here, since the benchmark scaffolding he uses has very different performance characteristics on different implementations. He's passing the symbols naming the functions to test to time-tests, but funcalling symbols (instead of functions) is very expensive on sbcl. So it's pretty much a benchmark of the speed of symbol-function, not of different case implementations :-)


Surely that doesn't have any effect on the intra-implementation results. So, for instance, it still seems he's shown that case has O(1) performance on CLISP but some sort of (apparently quite severe) number-of-cases dependence on SBCL and ACL, whereas ncase and switch are pretty much O(1) on all three implementations.


When 95% of the runtime is spent in the benchmarking infrastructure rather than the benchmarked code, interpreting even the intra-implementation results is hard. Here's the results I get (benchmark, original, symbol-function lifted out of time-test). Times in milliseconds:

  CASE-10 240 12
  CASE-100 244 52
  CASE-1000 632 432

  NCASE-10 212 20
  NCASE-100 220 24

  SWITCH-10 276 80
  SWITCH-100 272 76
  SWITCH-1000 268 72
For example SWITCH looks a lot more reasonable for a small number of cases with the original benchmark than it does when the benchmarking overhead is reduced.

(Edit: try to reformat the table)


I'm assuming those are SBCL numbers, given who's posting them :-). So, it seems to me that these figures confirm that CASE gets expensive for large numbers of cases whereas SWITCH doesn't. (You're missing the NCASE-1000 entry. If it's also quite small then you're also confirming that NCASE doesn't get expensive for large numbers of cases.) Which I thought was the most interesting finding here. You've also confirmed that SWITCH is a bit more expensive than CASE and NCASE except when the former blows up.

However. Geoff Wozniak's results seem to show (1) CASE-100 being more expensive than CASE-10, where you find it costing the same; and (2) CASE-1000 being more expensive than CASE-10 by a much larger factor than you've found. The curious thing here is that a constant extra bogus overhead from an expensive SYMBOL-FUNCTION implementation ought (I'd have thought) to bias his results in the opposite direction to that. Are you and he using different versions of SBCL, or something?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: