Simple perf testing using rpc:pmap

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Simple perf testing using rpc:pmap

Todd-2
Hi everyone,

Longtime lurker here. I decided to take the summer off and do something I've wanted to do for years: learn Erlang.

Newbie questions to follow...

Similar to the thread on 'distribution links utilizing resources'... I've been looking at Erlang in a couple of different ways, and here are some graphs (and code) to look at...

I started by using Erlang as an execution framework for comparing the performance of some common sorting algorithms:

Then I took one of the algorithms and looked at the performance of this 'embarrassingly parallel' problem as it was spread across the available processors on my laptop via 'rpc:pmap':

I was surprised a couple of things:
* It is ridiculously easy to use `rpc:pmap`
* However, the gains are mostly in adding a second or third processor, adding more processors yielded marginally better results, but really not worth the effort (IMHO)

I'd like to better understand the forces at work here:
Q: How to apply Amdahls Law here? I have enough data, I should be able to calculate the constant time factor...
Q: Why is the constant time factor so large? I expected much better overall performance when adding >3 processors...
Q: How does 'rpc:pmap' differ from the skel framework that Joe mentioned in this thread:
Q: Does anyone have a write-up on how to measure performance of distributed Erlang/BEAM apps on linux? What tools are you using, what stats are the most relevant, what things are missing that you metric yourself, etc? Obviously there's a _mountain_ of tools and things to look at here... I've been looking into sysstat (https://github.com/sysstat/sysstat).

One of the first things I did when analyzing the algorithms in Part 1 was to start writing metrics to ETS...but then I discovered that Erlang already has tools like cover, cprof, fprof... In fact, it seems that whenever you are looking for something you need, it's actually there, somewhere.

-Todd Greenwood

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions