Pitiful benchmark performance

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Pitiful benchmark performance

James Hague-3
Sean Hinde wrote:
>
> Now I know that benchmarks are not representative etc etc but they do
> encapsulate quite a few common idioms and there shouldn't be
> a good reason
> for Erlang to do so badly. Maybe the HIPE guys can use (are
> using?) these in
> their investigations into optimal GC schemes etc for R8?

Networking aside, some of these benchmarks are just plain bad.  The List
Operations benchmark involves accessing lists from both ends.  Naturally,
naive implementations of this are going to suffer in functional languages,
and languages that implement "lists" as arrays (like Perl) are going to do
better.  Some of the others involve calls to lists:append inside inner
loops.

While it would be nice for Erlang to have a better showing--Bagley's page
has gotten a lot of attention--I think these benchmarks are too trivial to
be of much value.  What *is* heartening though, is that Erlang still beats
Python in most cases :)

James


Reply | Threaded
Open this post in threaded view
|

Pitiful benchmark performance

Erik.Johansson
> Sean Hinde wrote:
> >
> > Now I know that benchmarks are not representative etc etc but they do
> > encapsulate quite a few common idioms and there shouldn't be
> > a good reason
> > for Erlang to do so badly. Maybe the HIPE guys can use (are
> > using?) these in
> > their investigations into optimal GC schemes etc for R8?

I think there are problems with the benchmark that casues the huge need of memory, and it is not certain that a bettter GC would make the problem go away. Still all benchmarks are interesting to us so we might steal some of them...

A nice thing though is that Erlang behaves quite well on some benchmarks, for example nested loops, where it is one of the fastest non-natively compiled systems. (And HiPE is more than 6 times faster than BEAM on this benchmark putting Erlang in the ballpark of the statically typed smlnj.)

/Erik