+secio emulator flag

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

+secio emulator flag

Mikael Karlsson-7
Hi,

I noticed that the secio emulator flag (scheduler eager check for io) default value has changed from false to true since ERTS 7.0. Could someone explain what this means in practice, I do not quite understand the line: "This, however, also implies that execution of outstanding I/O operation is not prioritized to the same extent as when false is passed."

Is there any special cases where you would prefer to still have it set to false.

/Mikael

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: +secio emulator flag

Lukas Larsson-8
Hello,

On Thu, Aug 31, 2017 at 10:07 AM, Mikael Karlsson <[hidden email]> wrote:
I noticed that the secio emulator flag (scheduler eager check for io) default value has changed from false to true since ERTS 7.0. Could someone explain what this means in practice, I do not quite understand the line: "This, however, also implies that execution of outstanding I/O operation is not prioritized to the same extent as when false is passed."

The eager check io flag instructs the emulator when it should check for new I/O events (internally called check_io)  in the fds that it is currently interested in.

Without secio, the schedulers will first finish with all work generated from the previous check_io. If a lot of I/O events where triggered by the check_io, ports may be temporarily prioritized higher than processes in order to be able to do the next check_io sooner.

With secio, the next check_io can be done before all events of the previous check_io are consumed. When this feature is enabled, the temporary higher prioritization of ports is removed. This is the change in priority that the documentation is trying to explain.
 
Is there any special cases where you would prefer to still have it set to false.

I see no reason to not have secio enabled.

Lukas

(sorry for the late reply, I meant to reply to this before but forgot)

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: +secio emulator flag

Michael Truog
On 09/11/2017 06:32 AM, Lukas Larsson wrote:
Hello,

On Thu, Aug 31, 2017 at 10:07 AM, Mikael Karlsson <[hidden email]> wrote:
I noticed that the secio emulator flag (scheduler eager check for io) default value has changed from false to true since ERTS 7.0. Could someone explain what this means in practice, I do not quite understand the line: "This, however, also implies that execution of outstanding I/O operation is not prioritized to the same extent as when false is passed."

The eager check io flag instructs the emulator when it should check for new I/O events (internally called check_io)  in the fds that it is currently interested in.

Without secio, the schedulers will first finish with all work generated from the previous check_io. If a lot of I/O events where triggered by the check_io, ports may be temporarily prioritized higher than processes in order to be able to do the next check_io sooner.

With secio, the next check_io can be done before all events of the previous check_io are consumed. When this feature is enabled, the temporary higher prioritization of ports is removed. This is the change in priority that the documentation is trying to explain.
 
Is there any special cases where you would prefer to still have it set to false.

I see no reason to not have secio enabled.

I have tested in the past to determine that secio can make Erlang source code performance poor, if it isn't utilizing an Erlang port type, though the testing was with Erlang/OTP 17.4 (https://github.com/CloudI/loadtests/raw/master/tests/http_req/loadtest/results_v1_4_0/201412_summary.pdf).

Best Regards,
Michael

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: +secio emulator flag

Mikael Karlsson-7
Thanks Lukas and Michael,

> (sorry for the late reply, I meant to reply to this before but forgot)
No problem, maybe your personal secio flag also was set to true and
something else came in between :-) .

The use case I have here is a single (tcp) connection with high load.
I think that there are other bottlenecks with this scenario since one
can only attach one scheduler/erlang process to this, if I understand
it right, but nevertheless would setting secio to false give the
attached erlang process more time to finish the work in this case?

Best Regards
Mikael


2017-09-11 22:09 GMT+02:00 Michael Truog <[hidden email]>:

> On 09/11/2017 06:32 AM, Lukas Larsson wrote:
>
> Hello,
>
> On Thu, Aug 31, 2017 at 10:07 AM, Mikael Karlsson <[hidden email]>
> wrote:
>>
>> I noticed that the secio emulator flag (scheduler eager check for io)
>> default value has changed from false to true since ERTS 7.0. Could someone
>> explain what this means in practice, I do not quite understand the line:
>> "This, however, also implies that execution of outstanding I/O operation is
>> not prioritized to the same extent as when false is passed."
>
>
> The eager check io flag instructs the emulator when it should check for new
> I/O events (internally called check_io)  in the fds that it is currently
> interested in.
>
> Without secio, the schedulers will first finish with all work generated from
> the previous check_io. If a lot of I/O events where triggered by the
> check_io, ports may be temporarily prioritized higher than processes in
> order to be able to do the next check_io sooner.
>
> With secio, the next check_io can be done before all events of the previous
> check_io are consumed. When this feature is enabled, the temporary higher
> prioritization of ports is removed. This is the change in priority that the
> documentation is trying to explain.
>
>>
>> Is there any special cases where you would prefer to still have it set to
>> false.
>
>
> I see no reason to not have secio enabled.
>
> I have tested in the past to determine that secio can make Erlang source
> code performance poor, if it isn't utilizing an Erlang port type, though the
> testing was with Erlang/OTP 17.4
> (https://github.com/CloudI/loadtests/raw/master/tests/http_req/loadtest/results_v1_4_0/201412_summary.pdf).
>
> Best Regards,
> Michael
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: +secio emulator flag

Michael Truog
On 09/12/2017 10:23 AM, Mikael Karlsson wrote:

> Thanks Lukas and Michael,
>
>> (sorry for the late reply, I meant to reply to this before but forgot)
> No problem, maybe your personal secio flag also was set to true and
> something else came in between :-) .
>
> The use case I have here is a single (tcp) connection with high load.
> I think that there are other bottlenecks with this scenario since one
> can only attach one scheduler/erlang process to this, if I understand
> it right, but nevertheless would setting secio to false give the
> attached erlang process more time to finish the work in this case?

My impression is that secio set to true will help if you really do have lots of throughput coming through some number of Erlang ports (sockets or other file descriptors), though it may reduce the performance of Erlang source code that doesn't need to use Erlang ports, so, heavy use of an Erlang process that benefits from spending as much time on a scheduler as possible, though that is based on older testing.  The default changed from false to true, so it appears that most use should benefit from keeping it set to true.  Ideally, you could loadtest with the setting set either way, to see which is best for your use-case.

Best Regards,
Michael


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: +secio emulator flag

Mikael Karlsson-7
2017-09-12 19:49 GMT+02:00 Michael Truog wrote:
> My impression is that secio set to true will help if you really do have lots
> of throughput coming through some number of Erlang ports (sockets or other
> file descriptors), though it may reduce the performance of Erlang source
> code that doesn't need to use Erlang ports, so, heavy use of an Erlang
> process that benefits from spending as much time on a scheduler as possible,
> though that is based on older testing.  The default changed from false to
> true, so it appears that most use should benefit from keeping it set to
> true.  Ideally, you could loadtest with the setting set either way, to see
> which is best for your use-case.

Thanks for making it clear.

Best Regards
Mikael
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions