Erlang Development Environment

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

James Hague-3
>I'd rather the root of this problem be solved; namely, that GS is slow
>under Windows.  Could the performance not be improved if it were
>implemented directly with Windows API calls (avoiding Tk completely)?

Probably, but it would be a huge effort.  Delphi is orders of magnitude
easier to use than raw Win32 calls.  My gut feeling is that there's not much
sense in bulking up Erlang with huge graphical modules.  Just calling
external programs is a nice way of dodging the problem and keeping focused.

> > a. There's no simple, efficient way to have large, constant
> data structures.
>
> OK, but I'd rather not there be some "special method" in Erlang for
> making large constant data structures, efficient.  An
> optimizer of some sort should see to that, behind the scenes.

I agree.  If constant data had it's own non-GCed memory pool, that would be
great.  But that's a big change to the runtime, I expect.  Oh, but it would
really open up possibilities, IMO.

James


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

Claes Wikström
On Tue, Mar 06, 2001 at 08:58:51AM -0600, James Hague wrote:

> > > a. There's no simple, efficient way to have large, constant
> > data structures.
> >
> > OK, but I'd rather not there be some "special method" in Erlang for
> > making large constant data structures, efficient.  An
> > optimizer of some sort should see to that, behind the scenes.
>
> I agree.  If constant data had it's own non-GCed memory pool, that would be
> great.  But that's a big change to the runtime, I expect.  Oh, but it would
> really open up possibilities, IMO.
>


This is the reason why I once added the ets tables to the runtime.
I wanted to keep very large amounts of data in RAM without exposing that
data to the gc. I guess the ets tables are more suited for many, but not
so large objects as opposed to few, but very large objects though.


/klacke


--
Claes Wikstrom                        -- Caps lock is nowhere and
Alteon WebSystems                     -- everything is under control          
http://www.bluetail.com/~klacke       --



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

Erik.Johansson
In reply to this post by James Hague-3
> I agree.  If constant data had it's own non-GCed memory pool, that would be
> great.  But that's a big change to the runtime, I expect.  Oh, but it would
> really open up possibilities, IMO.

Well, the change is not that big, it is a bit problematic with the current GC, but it is doable.
In fact, when you compile a function to native code with HiPE all constant data structures are stored in such a memory pool. (This feature might be added to BEAM at some point...)

/Erik
HiPE -- Comming soon to an OTP implementation near you.

Ex:
const() ->
  {a,somewhat,big,"constant"}.
loop(0) ->
  const();
loop(N) ->
  const(),
  loop(N-1).
test() ->
  loop(1000).

> timer:tc(jt,test,[]).
{1557,{a,somewhat,big,"constant"}}
> hipe:c(jt,[o2]).      
> timer:tc(jt,test,[]).
{152,{a,somewhat,big,"constant"}}



> hipe:compile({jt,const,0},[o2,pp_rtl]).  
{jt,const,0}() ->
;; Info: []
.DataSegment
.DL0: {a,somewhat,big,[99,111,110,115,116,97,110,116]}
.CodeSegment
L9:
    r39 <- %l6 add 208
    if (r39 lt %l7) then L1 (0.99) else L7
L7:
    ;; save_frame
    [%l6+0] <- %o7
    %l6 <- %l6 add 4
    jsr inc_stack_0 [c] () then L8
L8:
    ;; restore_frame
    %o7 <- [%l6+-4]
    ;; pop_frame
    %l6 <- %l6 sub 4
    goto L1
L1:
    %l5 <- %l5 sub 1 if lt then L3 (0.01) else L2
L2:
    v38 <- DL0
    v37 <- v38 add 2
    goto L5
L5:
    %o0 <= v37
    jmp %o7+8 (%o0)
L3:
    ;; save_frame
    [%l6+0] <- %o7
    %l6 <- %l6 add 4
    jsr suspend_0 [c] () then L6
L6:
    ;; restore_frame
    %o7 <- [%l6+-4]
    ;; pop_frame
    %l6 <- %l6 sub 4
    goto L2





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

Robert Virding-4
"Erik.Johansson" <Erik.Johansson> writes:
>> I agree.  If constant data had it's own non-GCed memory pool, that would be
>> great.  But that's a big change to the runtime, I expect.  Oh, but it would
>> really open up possibilities, IMO.
>
>Well, the change is not that big, it is a bit problematic with the current GC, but it is doable.
>In fact, when you compile a function to native code with HiPE all constant data structures are stored in such a memory pool. (This
> feature might be added to BEAM at some point...)

How do you garbage collect this memory pool?  I presume that the idea
is to NOT a structure from the pool into a process heap.  In that case
how do you know when to clear structures from the pool?  Just removing
them when the module is purged is not enough as they may still be
referenced from process heaps.

I am assuming here that you using the seperate process heap model of
the BEAM.  If you are using a single heap then the problem goes away.

    Robert


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

Happi

"Robert Virding" <rv> wrote:


> "Erik.Johansson" <Erik.Johansson> writes:
> >In fact, when you compile a function to native code with HiPE all
constant
> >data structures are stored in such a memory pool.
>
> How do you garbage collect this memory pool?  I presume that the idea
> is to NOT a structure from the pool into a process heap.  In that case
> how do you know when to clear structures from the pool?  Just removing
> them when the module is purged is not enough as they may still be
> referenced from process heaps.

Yes in ordinary execution the data in the constant pool is not copied to the
process heap.

When a module, with constant data, is purged the following is done:
 A flag is set in each process indicating that the next GC should be
special.
 This special GC kopies all live terms from the constant area to the process
heap.
 When all marked processes have done a special GC then the constant pool is
freed.
 (If new processes are created they do not need to do a special GC since
they can't
  get any references to the constnats in question. This assumes that
messages are copied.)


 This scheme is not yet fully implemented in HiPE ;)
 (There are a few parameters that can be tweeked like:
   * Copy all data from the constant pool or just data associated with the
purged module.
   * When to do the next GC.
   * Not copy constant data when doing a send. (This probably cost more than
one can gain.)
  )

> I am assuming here that you using the seperate process heap model of
> the BEAM.  If you are using a single heap then the problem goes away.

Yes, and we plan to experiment with a single heap.

/Erik




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

David Gould-2
In reply to this post by Erik.Johansson
On Tue, Mar 06, 2001 at 04:19:03PM +0100, Erik.Johansson wrote:

>
> Well, the change is not that big, it is a bit problematic with the current GC, but it is doable.
> In fact, when you compile a function to native code with HiPE all constant data structures are stored in such a memory pool. (This feature might be added to BEAM at some point...)
>
> /Erik
> HiPE -- Comming soon to an OTP implementation near you.
>
> Ex:
> const() ->
>   {a,somewhat,big,"constant"}.
> loop(0) ->
>   const();
> loop(N) ->
>   const(),
>   loop(N-1).
> test() ->
>   loop(1000).
>
> > timer:tc(jt,test,[]).
> {1557,{a,somewhat,big,"constant"}}
> > hipe:c(jt,[o2]).      
> > timer:tc(jt,test,[]).
> {152,{a,somewhat,big,"constant"}}

Oooooh Oooh Ooh, I can hardly wait. But, what is happening with the x86
version? I have not seen any updates on the hipe page for a while.

-dg
 

--
David Gould                davidg               510 536 1443
If simplicity worked, the world would be overrun with insects.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

Happi

"David Gould" <davidg> wrote:
> On Tue, Mar 06, 2001 at 04:19:03PM +0100, Erik.Johansson wrote:
> > HiPE -- Comming soon to an OTP implementation near you.
> Oooooh Oooh Ooh, I can hardly wait. But, what is happening with the x86
> version? I have not seen any updates on the hipe page for a while.

No, we are working full time on the system and have no time to update the
webpage ;)

Current status is (unofficially and approximately):
 HiPE is closing in on a final BETA version (for UltraSPARC) and will
 hopefully be released as an *unsupported* addition to OTP R8. (No promises
yet.)
 There will be some important aspects missing in this release such as
automatic
 purging of old native code.
 We are working on the X86 port in parallel and we hope it will be released
this summer.
 (Possibly with much luck there will be an alpha version ready for R8, but
don't bet on it.)

/Erik



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang Development Environment

David Gould-2
On Wed, Mar 07, 2001 at 10:20:37AM +0100, Erik Johansson wrote:
>
>  HiPE is closing in on a final BETA version (for UltraSPARC) and will
>  hopefully be released as an *unsupported* addition to OTP R8. (No promises
> yet.)
>
hands make light work. Or is it
>  There will be some important aspects missing in this release such as
> automatic
>  purging of old native code.
>  We are working on the X86 port in parallel and we hope it will be released
> this summer.
>  (Possibly with much luck there will be an alpha version ready for R8, but
> don't bet on it.)

I would really like to see _something_ for x86 as early as possible. One
of the issues I run into trying to introduce Erlang to the unwashed is
"well, it can't be fast enough, especially without a compiler, we need speed,
so will have to use Java", ok, the bit about Java was a joke. But to be able
to say, I am testing an early alpha of the compiler and "next year when we
need to deploy at large scale" it will be 10x faster" could go a long
way to getting a chance to use Erlang for some projects. It is not
that it really isn't fast enough now, but overcoming "I've never heard of it",
"and noone knows it", and "it might be too slow" is almost impossible so
it is hard to get a chance to succeed or fail.

When I say "_something_", it does not need to work. Nor to support all of
otp. Just enough to say, ok, fib() is 10x faster, and regexing through a
megabyte of text is 4x faster, or whatever would be great.

Thanks

-dg

--
David Gould                davidg               510 536 1443
If simplicity worked, the world would be overrun with insects.


Loading...