Deprecation and removal of the non-smp emulator

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Deprecation and removal of the non-smp emulator

Lukas Larsson-8
Hello!

At the moment it is possible to build three different production versions of the Erlang VM; the smp emulator, the threaded non-smp emulator and the non-threaded non-smp emulator.

The non-smp versions have been kept around since the introduction of the smp emulator as they can run on OS:s without thread support and also for certain applications they provide a small performance boost. However we now feel that they have outlived their usefulness and plan to remove them.

The main reason why we want to remove non-smp is because we do not want to spend time on implementing the dirty-scheduler feature for threaded non-smp, which is something we would have to do for the threaded non-smp emulator to remain production quality. The different variants also increase our test-scope by quite a lot, so reducing that is a very beneficial side effect.

The current plan is for the non-smp emulators to be deprecated and not built by default in OTP-20, and then removed completely in OTP-21.

If you have a very good reason for wanting to use the non-smp emulator, please do let us know as soon as possible so that we can consider whether we want to keep the non-smp version anyway.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Dmytro Lytovchenko
One point that I've heard a couple of times, is that single CPU emulator (not sure which one of them) is noticeably (like up to 20%) faster than SMP due to all the locks it doesn't have to maintain. So for some applications running say 8 of these will be better than a single SMP 8 core emulator. This is a casual opinion from the couch and I don't have the numbers to prove the point :)

2017-04-03 15:33 GMT+02:00 Lukas Larsson <[hidden email]>:
Hello!

At the moment it is possible to build three different production versions of the Erlang VM; the smp emulator, the threaded non-smp emulator and the non-threaded non-smp emulator.

The non-smp versions have been kept around since the introduction of the smp emulator as they can run on OS:s without thread support and also for certain applications they provide a small performance boost. However we now feel that they have outlived their usefulness and plan to remove them.

The main reason why we want to remove non-smp is because we do not want to spend time on implementing the dirty-scheduler feature for threaded non-smp, which is something we would have to do for the threaded non-smp emulator to remain production quality. The different variants also increase our test-scope by quite a lot, so reducing that is a very beneficial side effect.

The current plan is for the non-smp emulators to be deprecated and not built by default in OTP-20, and then removed completely in OTP-21.

If you have a very good reason for wanting to use the non-smp emulator, please do let us know as soon as possible so that we can consider whether we want to keep the non-smp version anyway.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Ameretat Reith
In reply to this post by Lukas Larsson-8
We used to run containerized non-SMP VMs and we were under that
impression it scales better, but I don't have numbers too. Since we
don't use dirty schedulers and software is more Erlang than Erlang
bounded to C, SMP looked unnecessary overkill for us, but If It's
planned that dirty scheduler spread to core applications -say crypto
module depend on it- I prefer non-SMP support dropped rather than my
software stuck in a random function.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Lukas Larsson-8
On Mon, Apr 3, 2017 at 4:49 PM, Ameretat Reith <[hidden email]> wrote:
 If It's planned that dirty scheduler spread to core applications -say crypto
module depend on it- I prefer non-SMP support dropped rather than my
software stuck in a random function.

The first and largest thing we are planning to do is to implement file I/O as nifs using dirty schedulers.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Tristan Sloughter-4
In reply to this post by Lukas Larsson-8
First use case I thought of when I heard this news was 'erlc' itself :). It disables smp by default, right?

This gives a few milliseconds speedup in startup time when booting the VM which is mostly a non-issue but nice for cli tools.

Because of tasks like common test, dialyzer and in the future if we add back parallel compile (for the most common case, only building a few files, it is much faster to just build sequentially) we can't use non-smp by default with rebar3. But I figured I'd throw this out there in case anyone was looking to speed up the boot time :)

--
  Tristan Sloughter
  "I am not a crackpot" - Abe Simpson



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Lukas Larsson-8
On Mon, Apr 3, 2017 at 6:34 PM, Tristan Sloughter <[hidden email]> wrote:
First use case I thought of when I heard this news was 'erlc' itself :). It disables smp by default, right?

erlc uses the following startup flags: "erl +A0 +sbtu -noinput -mode minimal -boot start_clean"

So, it uses the smp emulator if available.
 
This gives a few milliseconds speedup in startup time when booting the VM which is mostly a non-issue but nice for cli tools. 

If startup time is important, I think the time would be better spent optimizing the startup time of the smp-emulator, then to maintain the non-smp emulators just for it's startup speed.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Tristan Sloughter-4
Ah, ok, I thought the minimal mode meant no smp and there was a special flag for erlc to turn on smp if you wanted it.

But either way, I agree improving startup time of smp enabled vm is preferable.

--
  Tristan Sloughter
  "I am not a crackpot" - Abe Simpson


On Tue, Apr 4, 2017, at 12:55 AM, Lukas Larsson wrote:
On Mon, Apr 3, 2017 at 6:34 PM, Tristan Sloughter <[hidden email]> wrote:

First use case I thought of when I heard this news was 'erlc' itself :). It disables smp by default, right?

erlc uses the following startup flags: "erl +A0 +sbtu -noinput -mode minimal -boot start_clean"

So, it uses the smp emulator if available.
 

This gives a few milliseconds speedup in startup time when booting the VM which is mostly a non-issue but nice for cli tools. 

If startup time is important, I think the time would be better spent optimizing the startup time of the smp-emulator, then to maintain the non-smp emulators just for it's startup speed.

Lukas
_______________________________________________
erlang-questions mailing list


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Ryan Stewart
If a big selling point of Erlang is how easy it makes concurrency (that's how it seems to me), then spending a lot of effort on maintaining non-SMP support seems silly. If you're not running SMP, then you're not really doing what Erlang is excellent at, right? You could just as well be writing C, Java, Lisp, or name your own language that doesn't have a great, built-in concurrency model. I'm a relative noob, though, so what do I know?

On Tue, Apr 4, 2017 at 1:08 PM Tristan Sloughter <[hidden email]> wrote:
Ah, ok, I thought the minimal mode meant no smp and there was a special flag for erlc to turn on smp if you wanted it.

But either way, I agree improving startup time of smp enabled vm is preferable.

--
  Tristan Sloughter
  "I am not a crackpot" - Abe Simpson


On Tue, Apr 4, 2017, at 12:55 AM, Lukas Larsson wrote:
On Mon, Apr 3, 2017 at 6:34 PM, Tristan Sloughter <[hidden email]> wrote:

First use case I thought of when I heard this news was 'erlc' itself :). It disables smp by default, right?

erlc uses the following startup flags: "erl +A0 +sbtu -noinput -mode minimal -boot start_clean"

So, it uses the smp emulator if available.
 

This gives a few milliseconds speedup in startup time when booting the VM which is mostly a non-issue but nice for cli tools. 

If startup time is important, I think the time would be better spent optimizing the startup time of the smp-emulator, then to maintain the non-smp emulators just for it's startup speed.

Lukas
_______________________________________________
erlang-questions mailing list

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Roger Lipscombe-2
In reply to this post by Lukas Larsson-8
On 3 April 2017 at 14:33, Lukas Larsson <[hidden email]> wrote:
If you have a very good reason for wanting to use the non-smp emulator, please do let us know as soon as possible so that we can consider whether we want to keep the non-smp version anyway.

Question: the smp emulator will continue to be supported on single core machines, right?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Éric Pailleau
In reply to this post by Ryan Stewart

Hi,
Erlang can start a process on another node.
So Erlang can also be nice for a distributed single Cpu cluster of nodes.
In a sens Erlang is always multi Cpu, then.
Regards

"Envoyé depuis mon mobile " Eric



---- Ryan Stewart a écrit ----

If a big selling point of Erlang is how easy it makes concurrency (that's how it seems to me), then spending a lot of effort on maintaining non-SMP support seems silly. If you're not running SMP, then you're not really doing what Erlang is excellent at, right? You could just as well be writing C, Java, Lisp, or name your own language that doesn't have a great, built-in concurrency model. I'm a relative noob, though, so what do I know?

On Tue, Apr 4, 2017 at 1:08 PM Tristan Sloughter <[hidden email]> wrote:
Ah, ok, I thought the minimal mode meant no smp and there was a special flag for erlc to turn on smp if you wanted it.

But either way, I agree improving startup time of smp enabled vm is preferable.

--
  Tristan Sloughter
  "I am not a crackpot" - Abe Simpson


On Tue, Apr 4, 2017, at 12:55 AM, Lukas Larsson wrote:
On Mon, Apr 3, 2017 at 6:34 PM, Tristan Sloughter <[hidden email]> wrote:

First use case I thought of when I heard this news was 'erlc' itself :). It disables smp by default, right?

erlc uses the following startup flags: "erl +A0 +sbtu -noinput -mode minimal -boot start_clean"

So, it uses the smp emulator if available.
 

This gives a few milliseconds speedup in startup time when booting the VM which is mostly a non-issue but nice for cli tools. 

If startup time is important, I think the time would be better spent optimizing the startup time of the smp-emulator, then to maintain the non-smp emulators just for it's startup speed.

Lukas
_______________________________________________
erlang-questions mailing list

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Lukas Larsson-8
In reply to this post by Roger Lipscombe-2
On Wed, Apr 5, 2017 at 8:59 AM, Roger Lipscombe <[hidden email]> wrote:
On 3 April 2017 at 14:33, Lukas Larsson <[hidden email]> wrote:
If you have a very good reason for wanting to use the non-smp emulator, please do let us know as soon as possible so that we can consider whether we want to keep the non-smp version anyway.

Question: the smp emulator will continue to be supported on single core machines, right?

Yes, it will continue to be possible to run a 1 scheduler SMP emulator on a single core machine.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

John Doe
Not always. Only if kernel was built with smp support. Some cheap VPS providers build kernels of their one-cpu VPS without smp, for example 512M RAM GoDaddy VPS a year ago was built without smp.
I don't think this would be valid reason to continue support for non-smp emulator, but just fyi.

2017-04-05 10:33 GMT+03:00 Lukas Larsson <[hidden email]>:
On Wed, Apr 5, 2017 at 8:59 AM, Roger Lipscombe <[hidden email]> wrote:
On 3 April 2017 at 14:33, Lukas Larsson <[hidden email]> wrote:
If you have a very good reason for wanting to use the non-smp emulator, please do let us know as soon as possible so that we can consider whether we want to keep the non-smp version anyway.

Question: the smp emulator will continue to be supported on single core machines, right?

Yes, it will continue to be possible to run a 1 scheduler SMP emulator on a single core machine.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Deprecation and removal of the non-smp emulator

Johannes Weißl-2
In reply to this post by Lukas Larsson-8
Hello Lukas,

> If you have a very good reason for wanting to use the non-smp
> emulator, please do let us know as soon as possible so that we can
> consider whether we want to keep the non-smp version anyway.

We use the non-smp emulator on small ARM Linux boxes (single CPU).
I have tested the smp emulator over the last two weeks, here are my
observations:

- smp version increases VmSize by 10-35 MB, but VmRSS by only about 1 MB
- smp version is slower, but not critically (can still run our software)
- qemu-arm-static 2.8 has problems running the smp version

So we will stick to the non-smp version until OTP-21, especially because
of the broken qemu (used for test suite).

Johannes
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Loading...