R8B PRE-RELEASES

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

R8B PRE-RELEASES

Kostis Sagonas-3
Per Bergqvist asked:

 > Any plans to add this support ?

Yes.  The plan is to provide this support "really soon now",
meaning that we (the HiPE group) plan to do this probably
before the end of the year.

At this point, i.e., before the R8 release, we are concentrating
our efforts in making sure that HiPE is efficient and gets released
as bug-free as possible.

 > Unified heap and hipe are both two very good things that give
 > great performance boosts and I would hate to have to choose ...

Thanks!

Here is a more detailed version of our plan:

 As mentioned, we have quite good reasons to believe that the
 unified heap architecture is a better memory architecture than
 the current one.  However, we would like to be sure of this
 before committing to one of them (supporting both of them is
 a nightmare we would very much like to avoid).

   We urge the Erlang community to test *applications* on the
   unified heap architecture and report its findings either to us
   or directly in this list as e.g. Pascal Brisset did yesterday;
   (Pascal we were excited when we saw your mail - merci!).

   In the interest of the community, it is important to try to get
   data that show both the pros and cons of the two architectures.
   So please, if you have an interesting application that you have
   reasons to believe it will behave better in the one or the other
   architecture, bench it and preferably send its _kernel_ to us.

 Once we are finished with this benchmarking, we will choose one
 of the architectures as the supported one and we will make sure
 that HiPE runs on it.  This will happen before R9.


In the meantime, the choice (for me) is obvious:

 - configure with --enable-hipe and use this system for everyday use;

 - if you suspect that a specific program behaves better (or worse)
   with unified heap, or you are simply curious, try this architecture
   and if your findings are interesting, please do send them to us.


Kostis Sagonas (for the HiPE group).



Reply | Threaded
Open this post in threaded view
|

Unified heap (was: Re: R8B PRE-RELEASES)

Per Hedeland-4
I am also a believer in the unified heap (or rather any technique that
avoids memory copying)
both by gut feeling and experience.

I have always seen great performance improvements in my code with such
techniques (i.e. vee in
the good old days/dark ages and unified heap) and but for some strange
reason I seem to be
rather lonely with such findings. The majority of the community seem to
get less improvements.

It probably has to do very much with design patterns. I think that the
greatest thing with Erlang is that
the low cost processes allows designs where you don't have to mix
behaviours in processes.
Instead you have many, many small and simple independent but cooperating
processes.
The code become much cleaner easier to understand.
(I my opinion this really the core in why it is possible to build such
great systems with Erlang.)

To avoid getting performance boosts with unified heap you probably need
to avoid message
passing and have few complex processes in your application.

I think that nobody can argue unnecessary memory copying is of evil.
(It is a performance killer for every system, it takes time, trash caches
...)

What should be discussed is garbage collection.
The critic of schemes avoiding memory copying always seems to be that a
good gc
will be very very hard if not impossible.
Is that really true ? I am not the man to answer.
It is true that the current gc enables a smooth execution and it is quite
uncommon with
long time system stalls for gc.

Comments anyone ?

/Per




Reply | Threaded
Open this post in threaded view
|

Unified heap (was: Re: R8B PRE-RELEASES)

Francesco Mazzoli-2
When looking at the unified heap, one of my major worries is memory
fragmentation. Due to past experience (Jam 1998) after having run heavy duty
programs for a long period of time, I am scared it is bound to happen. Has
the Hipe team looked into this? Have you any theories or done any
measurements?

Regards,
Francesco

Per Bergqvist wrote:

> I am also a believer in the unified heap (or rather any technique that
> avoids memory copying)
> both by gut feeling and experience.
>
> I have always seen great performance improvements in my code with such
> techniques (i.e. vee in
> the good old days/dark ages and unified heap) and but for some strange
> reason I seem to be
> rather lonely with such findings. The majority of the community seem to
> get less improvements.
>
> It probably has to do very much with design patterns. I think that the
> greatest thing with Erlang is that
> the low cost processes allows designs where you don't have to mix
> behaviours in processes.
> Instead you have many, many small and simple independent but cooperating
> processes.
> The code become much cleaner easier to understand.
> (I my opinion this really the core in why it is possible to build such
> great systems with Erlang.)
>
> To avoid getting performance boosts with unified heap you probably need
> to avoid message
> passing and have few complex processes in your application.
>
> I think that nobody can argue unnecessary memory copying is of evil.
> (It is a performance killer for every system, it takes time, trash caches
> ...)
>
> What should be discussed is garbage collection.
> The critic of schemes avoiding memory copying always seems to be that a
> good gc
> will be very very hard if not impossible.
> Is that really true ? I am not the man to answer.
> It is true that the current gc enables a smooth execution and it is quite
> uncommon with
> long time system stalls for gc.
>
> Comments anyone ?
>
> /Per

--
http://www.erlang-consulting.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: francesco.vcf
Type: text/x-vcard
Size: 352 bytes
Desc: Card for Francesco Cesarini
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20011011/65afa2cb/attachment.vcf>

Reply | Threaded
Open this post in threaded view
|

Unified heap (was: Re: R8B PRE-RELEASES)

Robert Virding-4
In reply to this post by Per Hedeland-4
This discussion about unified heaps has made me happy.

I have been advocating unified heaps for the last 13 years and I have
always felt that the multiples heaps were a kludge to get reasonable
gc preformance, at least some of the time.  I have even done a few
Erlang implementations based on it, the results were promising but
more work was needed.  I could not get any support for the work,
however, so I was unable to keep up with Erlang development.

The really big boost was that message passing was _fast_!  _Always_!
You don't have to avoid many processes because you would be sending
large messages all the time.  Here you could send all the messages you
wanted to.  This did expose a misfeature in Erlang devlopment(*).

There are definitely problems in doing a good implementation.  The
HIPE groups claim that a stop-and-gc of the whole heap still gives
acceptable gc performance does not hold in real applications.
Especially if you use large ETS tables.  You have to be much more
cunning.  I had some incremental collectors with bounded time gc
breaks that that were truly non-disruptive, but there were problems.
I think that the final one based from 1995 on a modified version of a
gc scheme by Joe Armstrong and me was the way to go.  It was truly
scalable and handled things like collectable atom tables and (large)
constant data structures easily and efficiently.  I had an efficient
ETS extension which ran without any copying planned.

I don't think that it is possible to drop in a good, incremental gc
into a BEAM without any extra work.  There are many places where you
must "keep your tongue right in your mouth".  It is not difficult, but
you have to be aware.

Joes and my algorithm is described in IWMM'95 (International Workshop
on memory management).  If anybody is interested I'll explain my
extensions.

Anyway I hope the HIPE people (or someone else) can do more with this.

        Robert

(*) Erlang processes have slowly been getting heavier as more features
have been added to them.  My dream has always been to have extremely
light weight processes, in every respect, and then code new features
using extra processes.  It may not always be possible to do this.

P.S. I am not at Bluetail/Alteon/Nortel anymore, just using the mail
address.

Per Bergqvist <per> writes:

>I am also a believer in the unified heap (or rather any technique that
>avoids memory copying)
>both by gut feeling and experience.
>
>I have always seen great performance improvements in my code with such
>techniques (i.e. vee in
>the good old days/dark ages and unified heap) and but for some strange
>reason I seem to be
>rather lonely with such findings. The majority of the community seem to
>get less improvements.
>
>It probably has to do very much with design patterns. I think that the
>greatest thing with Erlang is that
>the low cost processes allows designs where you don't have to mix
>behaviours in processes.
>Instead you have many, many small and simple independent but cooperating
>processes.
>The code become much cleaner easier to understand.
>(I my opinion this really the core in why it is possible to build such
>great systems with Erlang.)
>>To avoid getting performance boosts with unified heap you probably need
>to avoid message
>passing and have few complex processes in your application.
>
>I think that nobody can argue unnecessary memory copying is of evil.
>(It is a performance killer for every system, it takes time, trash caches
>...)
>
>What should be discussed is garbage collection.
>The critic of schemes avoiding memory copying always seems to be that a
>good gc
>will be very very hard if not impossible.
>Is that really true ? I am not the man to answer.
>It is true that the current gc enables a smooth execution and it is quite
>uncommon with
>long time system stalls for gc.
>
>Comments anyone ?
>
>/Per
>


Reply | Threaded
Open this post in threaded view
|

Unified heap (was: Re: R8B PRE-RELEASES)

Ulf Wiger-4
On Sun, 21 Oct 2001 rv wrote:

>There are definitely problems in doing a good implementation.  The
>HIPE groups claim that a stop-and-gc of the whole heap still gives
>acceptable gc performance does not hold in real applications.
>Especially if you use large ETS tables.  You have to be much more
>cunning.

As a reference, in AXD 301 (a rather large application), it's not
at all uncommon with VM sizes of 80-100 MB, 3-400 processes, and
we many hundreds of ETS tables, some that grow to contain >
60,000 objects.

/Uffe
--
Ulf Wiger                                    tfn: +46  8 719 81 95
Senior System Architect                      mob: +46 70 519 81 95
Strategic Product & System Management    ATM Multiservice Networks
Data Backbone & Optical Services Division      Ericsson Telecom AB