halfword emulator state

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

halfword emulator state

Paolo Negri
Dear list

I would like to assess the state of development and recommended usage
of the halfword emulator.

Apart from the initial announcement [1] and the fixes/improvements
introduced with R14B03 [2] there hasn't been much discussion about it.
As we're evaluating the use of the halfword emulator in production I
would like to ask questions.

- Is anyone already using the halfword emulator in production
environments, if yes, are what are the experiences, is it stable, what
are the effective gains, is there any measurable overhead?

- Can the halfword emulator be considered suitable and stable for
production usage as the distribution of which is part?

- [2] states: "Much more of internal memory structures have been made
able to use "high" memory and are no longer restricted to the 4Gb
limit that still applies for all process heap data."
  Is there any documentation explaining exactly which structures can
use the high memory and what instead is constrained to the "low"
memory? Where would the content of the constant pool reside?

- How could the containment of structures restricted to the "low"
memory be measured? How would someone monitor the memory usage of the
VM being able to prevent out of memory errors without restricting the
whole VM to use over 4Gb?

- What are the caveats - if any - for using nifs when using the
halfword emulator? Which aspects of the nifs should be tested in order
to consider safe to run them with the halfword emulator?

[1] http://www.erlang.org/download/otp_src_R14B02.readme
[2] http://www.erlang.org/download/otp_src_R14B03.readme

Thanks for releasing the halfword emulator and thanks in advance for
your answers.

Paolo
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: halfword emulator state

Richard A. O'Keefe-2

On 31/05/2011, at 8:20 AM, Paolo Negri wrote:
> I would like to assess the state of development and recommended usage
> of the halfword emulator.

Could I just make the point that "halfword" is confusing?
I thought it referred to 16-bit units of something.
It might be clearer if called the "small pointer emulator"
or something like that.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: halfword emulator state

Jeff Schultz
On Tue, May 31, 2011 at 03:38:57PM +1200, Richard O'Keefe wrote:
> On 31/05/2011, at 8:20 AM, Paolo Negri wrote:
> > I would like to assess the state of development and recommended usage
> > of the halfword emulator.

> Could I just make the point that "halfword" is confusing?
> I thought it referred to 16-bit units of something.
> It might be clearer if called the "small pointer emulator"
> or something like that.

Or "compact pointer emulator" or "compact address emulator."


    Jeff Schultz
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: halfword emulator state

OvermindDL1


On May 30, 2011 9:47 PM, "Jeff Schultz" <[hidden email]> wrote:
>
> On Tue, May 31, 2011 at 03:38:57PM +1200, Richard O'Keefe wrote:
> > On 31/05/2011, at 8:20 AM, Paolo Negri wrote:
> > > I would like to assess the state of development and recommended usage
> > > of the halfword emulator.
>
> > Could I just make the point that "halfword" is confusing?
> > I thought it referred to 16-bit units of something.
> > It might be clearer if called the "small pointer emulator"
> > or something like that.
>
> Or "compact pointer emulator" or "compact address emulator."
>
>
>    Jeff Schultz

Was it not only for 64bit, hence a halfword will only be 32bit, not 16bit?


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: halfword emulator state

Attila Rajmund Nohl
In reply to this post by Paolo Negri
2011/5/30, Paolo Negri <[hidden email]>:

> Dear list
>
> I would like to assess the state of development and recommended usage
> of the halfword emulator.
>
> Apart from the initial announcement [1] and the fixes/improvements
> introduced with R14B03 [2] there hasn't been much discussion about it.
> As we're evaluating the use of the halfword emulator in production I
> would like to ask questions.
>
> - Is anyone already using the halfword emulator in production
> environments, if yes, are what are the experiences, is it stable, what
> are the effective gains, is there any measurable overhead?

We've had some emulator crashes on R14B02 (since fixed in R14B03), so
I definitely not advise the halfword emulator in R14B02. Apart from
this bug it was quite stable. Unfortunately it turned out that some of
our Erlang processes use a lot more memory than we expected, so due to
the 4GB limit for process memory we couldn't decrease the number of
running Erlang VMs, that's why we stayed with the 32 bit emulator
currently. We'll have an other go with R14B03.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: halfword emulator state

Björn-Egil Dahlberg-2
In reply to this post by OvermindDL1
Perhaps the vanilla x86_64 beam should be called doubleword beam. And no halfword.

 It always struck me as strange that erlangs wordsize was dependent on what machine it was running on.

// Björn-Egil
Skickat från min iPhone

31 maj 2011 kl. 07:10 skrev OvermindDL1 <[hidden email]>:


On May 30, 2011 9:47 PM, "Jeff Schultz" <[hidden email]> wrote:
>
> On Tue, May 31, 2011 at 03:38:57PM +1200, Richard O'Keefe wrote:
> > On 31/05/2011, at 8:20 AM, Paolo Negri wrote:
> > > I would like to assess the state of development and recommended usage
> > > of the halfword emulator.
>
> > Could I just make the point that "halfword" is confusing?
> > I thought it referred to 16-bit units of something.
> > It might be clearer if called the "small pointer emulator"
> > or something like that.
>
> Or "compact pointer emulator" or "compact address emulator."
>
>
>    Jeff Schultz

Was it not only for 64bit, hence a halfword will only be 32bit, not 16bit?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: halfword emulator state

Attila Rajmund Nohl
Maybe because that's not Erlang word size, but machine word size :-)
What I've found strange is that erlang:process_info returns the heap
size in words but erlang:system_info shows the memory used by
processes in bytes.

2011/5/31, Björn-Egil Dahlberg <[hidden email]>:

> Perhaps the vanilla x86_64 beam should be called doubleword beam. And no
> halfword.
>
>  It always struck me as strange that erlangs wordsize was dependent on what
> machine it was running on.
>
> // Björn-Egil
> Skickat från min iPhone
>
> 31 maj 2011 kl. 07:10 skrev OvermindDL1 <[hidden email]>:
>
>
> On May 30, 2011 9:47 PM, "Jeff Schultz" <[hidden email]> wrote:
>>
>> On Tue, May 31, 2011 at 03:38:57PM +1200, Richard O'Keefe wrote:
>> > On 31/05/2011, at 8:20 AM, Paolo Negri wrote:
>> > > I would like to assess the state of development and recommended usage
>> > > of the halfword emulator.
>>
>> > Could I just make the point that "halfword" is confusing?
>> > I thought it referred to 16-bit units of something.
>> > It might be clearer if called the "small pointer emulator"
>> > or something like that.
>>
>> Or "compact pointer emulator" or "compact address emulator."
>>
>>
>>    Jeff Schultz
>
> Was it not only for 64bit, hence a halfword will only be 32bit, not 16bit?
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: halfword emulator state

Sverker Eriksson
In reply to this post by Paolo Negri
Paolo Negri wrote:
> [...]
> - [2] states: "Much more of internal memory structures have been made
> able to use "high" memory and are no longer restricted to the 4Gb
> limit that still applies for all process heap data."
>   Is there any documentation explaining exactly which structures can
> use the high memory and what instead is constrained to the "low"
> memory? Where would the content of the constant pool reside?
>
>  
Beam code and Erlang terms are stored in low memory.
Except:
* ETS table data
* Large binaries (>64 bytes)
* NIF resources

> - How could the containment of structures restricted to the "low"
> memory be measured? How would someone monitor the memory usage of the
> VM being able to prevent out of memory errors without restricting the
> whole VM to use over 4Gb?
>
>  
In halfword emulator from R14B03
* erlang:memory() shows a 'low' value counting all allocated low memory.
* erlang:system_info(allocator) shows a boolean value 'low' to indicate
if an allocator is using low memory or not.

> - What are the caveats - if any - for using nifs when using the
> halfword emulator? Which aspects of the nifs should be tested in order
> to consider safe to run them with the halfword emulator?
>
>  
It should be tested on a halfword emulator. A NIF library compiled
towards a regular 64-bit vm will fail to load on a halfword vm and vice
versa.
Don't break the API by assuming anything about the opaque types, such as
ERL_NIF_TERM.


/Sverker, Erlang/OTP Ericsson


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions