Improved IO polling in erts

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Improved IO polling in erts

Lukas Larsson-8
Hello everyone,

I've just opened up a new PR at github containing a solution to the scalability problems that the current polling mechanism in erts has. You can read more about it in the PR here: https://github.com/erlang/otp/pull/1552.

Our internal measurements have shown great potential, but creating realistic benchmarks is very hard. I would therefore encourage as many as possible to test and see how the changes proposed effects your applications. Please come back with any findings/questions, be they good or bad. As this is an essential part of many applications, we want to make the new implementation as good as can be.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Improved IO polling in erts

Max Lapshin-2
hi.  perhaps it will help us to capture 1 gbit of udp without failure

On Tue, Aug 29, 2017 at 9:11 PM, Lukas Larsson <[hidden email]> wrote:
Hello everyone,

I've just opened up a new PR at github containing a solution to the scalability problems that the current polling mechanism in erts has. You can read more about it in the PR here: https://github.com/erlang/otp/pull/1552.

Our internal measurements have shown great potential, but creating realistic benchmarks is very hard. I would therefore encourage as many as possible to test and see how the changes proposed effects your applications. Please come back with any findings/questions, be they good or bad. As this is an essential part of many applications, we want to make the new implementation as good as can be.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Improved IO polling in erts

Lukas Larsson-8
In reply to this post by Lukas Larsson-8
I've pushed a number of bug fixes to the branch on github.

Again I would like to encourage you all to try the changes out and see if you can notice any differences in your system.

Lukas

On Tue, Aug 29, 2017 at 8:11 PM, Lukas Larsson <[hidden email]> wrote:
Hello everyone,

I've just opened up a new PR at github containing a solution to the scalability problems that the current polling mechanism in erts has. You can read more about it in the PR here: https://github.com/erlang/otp/pull/1552.

Our internal measurements have shown great potential, but creating realistic benchmarks is very hard. I would therefore encourage as many as possible to test and see how the changes proposed effects your applications. Please come back with any findings/questions, be they good or bad. As this is an essential part of many applications, we want to make the new implementation as good as can be.

Lukas


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Improved IO polling in erts

Ameretat Reith

This is amazing!

Here is result of simple UDP echo server. It spans 1000 processes for client and servers [1]. I ran it on Xeon E5-2620 with eight physical cores.

On baseline 20:

16:22:34.977 [notice] 1505303554: recv_pkts: 1793113 recv_size: 8965565 sent_pkts: 1793113 sent_size: 2639462336 recv: 7.172452 Mbit/s send: 2111.569869 Mbit/s
16:22:44.978 [notice] 1505303564: recv_pkts: 1801476 recv_size: 9007380 sent_pkts: 1801476 sent_size: 2651772672 recv: 7.205904 Mbit/s send: 2121.418138 Mbit/s
16:22:54.979 [notice] 1505303574: recv_pkts: 1799551 recv_size: 8997755 sent_pkts: 1799551 sent_size: 2648939072 recv: 7.198204 Mbit/s send: 2119.151258 Mbit/s

On poll-threads branch I get best results by two polling threads:

16:58:14.696 [notice] 1505305694: recv_pkts: 2835781 recv_size: 14178905 sent_pkts: 2835781 sent_size: <a href="tel:%28417%29%20426-9632" value="+14174269632" target="_blank">4174269632 recv: 11.343124 Mbit/s send: 3339.415706 Mbit/s
16:58:24.698 [notice] 1505305704: recv_pkts: 2836201 recv_size: 14181005 sent_pkts: 2836201 sent_size: <a href="tel:%28417%29%20488-7872" value="+14174887872" target="_blank">4174887872 recv: 11.344804 Mbit/s send: 3339.910298 Mbit/s
16:58:34.699 [notice] 1505305714: recv_pkts: 2833905 recv_size: 14169525 sent_pkts: 2833905 sent_size: 4171508160 recv: 11.335620 Mbit/s send: 3337.206528 Mbit/s

A %50 throughput increase for this small system! I expect more enhancements in bigger systems; those involving more message passings or using encryption.

1: https://github.com/reith/udpstress

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions