Calling (equivalent of) erlang:send_after/3 from a NIF?

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Calling (equivalent of) erlang:send_after/3 from a NIF?

Roger Lipscombe-2
I'd like to start a timer in my NIF. It should send a message to an
Erlang process after a delay. There doesn't appear to be an
enif_send_after.

Is there any way to do this?

(Obligatory "yes I know that NIFs are bad, mmkay...").

Regards,
Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Nathaniel Waisbrot
Is the idea that your NIF would pause the system for some time and then send its message and unpause the system? Is the NIF doing work during this time? If you're looking for enif_send_after my first thought would be that you should just wrap your NIF in an Erlang process and you NIF tells that process how and when to wait. But maybe I'm not understanding what you want this NIF to do.


On Mon, Oct 29, 2018, at 12:48 PM, Roger Lipscombe wrote:
I'd like to start a timer in my NIF. It should send a message to an
Erlang process after a delay. There doesn't appear to be an
enif_send_after.

Is there any way to do this?

(Obligatory "yes I know that NIFs are bad, mmkay...").

Regards,
Roger.
_______________________________________________
erlang-questions mailing list


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Igor Clark
In reply to this post by Roger Lipscombe-2
I'd love to know a "proper" way of doing this too. I did something like it once by using signal(3) to set a callback to handle SIGALRM, scheduling the SIGALRM to get sent by the OS kernel with ualarm(3), and then sending the message to the appropriate pid in the callback. It had to keep static pointers to the ErlNifPid & message content (which I set with NIF "setter" functions) so that the callback could see them, and it used a fair bit of CPU when I used it with a repeating alarm at a short interval (like 500µs) to create a timed pulse, but I guess it wouldn't be a big problem if it was just an occasional attach/schedule/send/clear. But it did feel like I was doing something pretty wrong (ill-advised/evil) in the BEAM context.

> On 29 Oct 2018, at 16:48, Roger Lipscombe <[hidden email]> wrote:
>
> I'd like to start a timer in my NIF. It should send a message to an
> Erlang process after a delay. There doesn't appear to be an
> enif_send_after.
>
> Is there any way to do this?
>
> (Obligatory "yes I know that NIFs are bad, mmkay...").
>
> Regards,
> Roger.
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Roger Lipscombe-2
In reply to this post by Nathaniel Waisbrot
On 29 October 2018 at 19:38, Nathaniel Waisbrot <[hidden email]> wrote:
> Is the idea that your NIF would pause the system for some time and then send
> its message and unpause the system?

I've got a reactor-style thing going on inside my NIF, where I service
a large number of timers and sockets. We're hitting scalability issues
with our current implementation, so I'm evaluating alternatives.

Since this is already in a NIF, I'd like to experiment with replacing
the socket polling with Erlang's enif_select, because I suspect it'll
be more scalable -- I'd like to find out, anyway. The plan is that, on
receiving the message from enif_select, I can quickly dip back into
NIF-land and service the relevant socket.

Some of these sockets are actually libcurl-driven, and one of the
things that libcurl wants is for us to start a timer. I was hoping
that there'd be an equivalent of enif_select, but for timers (i.e.
enif_send_after). We also service a large number of non-curl timers,
and so I was hoping to use Erlang's timer implementation for these as
well.

I found that erlang:send_after calls directly into the send_after_3
BIF, which calls setup_bif_timer, which manages Erlang's timers (it
appears to use a pair of timer wheels for short and medium timeouts,
and a red-black tree for longer timeouts). It'd fit my use-case
perfectly if I could just use that directly from a NIF.

> If you're looking for enif_send_after my first thought would be that you
> should just wrap your NIF in an Erlang process and you NIF tells that
> process how and when to wait.

The timeouts could be as short as 1ms. It'd be way less overhead to
just make use of Erlang's existing timer implementation, rather than
enif_send to a process which would then call erlang:send_after.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Roger Lipscombe-2
In reply to this post by Igor Clark
On 29 October 2018 at 20:06, Igor Clark <[hidden email]> wrote:
> I'd love to know a "proper" way of doing this too. I did something like it once by using signal(3) to set a callback to handle SIGALRM, scheduling the SIGALRM to get sent by the OS kernel with ualarm(3), and then sending the message to the appropriate pid in the callback. It had to keep static pointers to the ErlNifPid & message content (which I set with NIF "setter" functions) so that the callback could see them, and it used a fair bit of CPU when I used it with a repeating alarm at a short interval (like 500µs) to create a timed pulse, but I guess it wouldn't be a big problem if it was just an occasional attach/schedule/send/clear. But it did feel like I was doing something pretty wrong (ill-advised/evil) in the BEAM context.

Yeah; I found timer_create and timer_start, which Linux implements,
and apparently supports 1ms granularity, but they use signals, and it
makes me uneasy to do signal-stuff inside the BEAM.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Lukas Larsson-8
In reply to this post by Roger Lipscombe-2


On Mon, Oct 29, 2018 at 5:48 PM Roger Lipscombe <[hidden email]> wrote:
I'd like to start a timer in my NIF. It should send a message to an
Erlang process after a delay. There doesn't appear to be an
enif_send_after.

Is there any way to do this?

No, there is no way to do this. One of the reasons this would be difficult to implement is because it is currently only possible to create timers on normal schedulers. So that functionality would have to be extended to be useable from dirty schedulers, which is no small task.

If I were to do what you are after, I would first see if it is fast enough to setup timers as the nif returns, using a monotonic timestamp to adjust them for the time it took for the nif to return.

Another idea would be to use timerfd_create and then do a enif_select on that. Or just roll your own thread, with a small enough number of counters it should be simple enough.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Roger Lipscombe-2
On 30 October 2018 at 09:37, Lukas Larsson <[hidden email]> wrote:
> No, there is no way to do this.

Pity.

> One of the reasons this would be difficult
> to implement is because it is currently only possible to create timers on
> normal schedulers. So that functionality would have to be extended to be
> useable from dirty schedulers, which is no small task.

I'd be using it from a background thread -- cf enif_send(NULL, ...) --
 rather than a dirty scheduler, but I imagine the same constraint
applies?

> If I were to do what you are after, I would first see if it is fast enough
> to setup timers as the nif returns, using a monotonic timestamp to adjust
> them for the time it took for the nif to return.

Because this would potentially be from a background, arbitrary, thread
-- as in: there's no well-defined place to return this from -- that's
not gonna fly, unfortunately.

> Another idea would be to use timerfd_create and then do a enif_select on
> that.

That involves a kernel-mode transition, which -- since
Spectre/Meltdown -- is something we're trying to avoid, particularly
since we're on VMs. I might try it for comparison, though.

> Or just roll your own thread, with a small enough number of counters
> it should be simple enough.

We're potentially looking at ~20-25K timers (not just curl-required
ones). I was hoping to leverage someone else's timer wheel
implementation :)

Thanks,
Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Lukas Larsson-8


On Tue, Oct 30, 2018 at 11:15 AM Roger Lipscombe <[hidden email]> wrote:
On 30 October 2018 at 09:37, Lukas Larsson <[hidden email]> wrote:
> One of the reasons this would be difficult
> to implement is because it is currently only possible to create timers on
> normal schedulers. So that functionality would have to be extended to be
> useable from dirty schedulers, which is no small task.

I'd be using it from a background thread -- cf enif_send(NULL, ...) --
 rather than a dirty scheduler, but I imagine the same constraint
applies?

yes.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Nathaniel Waisbrot
In reply to this post by Roger Lipscombe-2
On Mon, Oct 29, 2018, at 5:24 PM, Roger Lipscombe wrote:
On 29 October 2018 at 19:38, Nathaniel Waisbrot <[hidden email]> wrote:
Is the idea that your NIF would pause the system for some time and then send
its message and unpause the system?

I've got a reactor-style thing going on inside my NIF, where I service
a large number of timers and sockets. We're hitting scalability issues
with our current implementation, so I'm evaluating alternatives.

Since this is already in a NIF, I'd like to experiment with replacing
the socket polling with Erlang's enif_select, because I suspect it'll
be more scalable -- I'd like to find out, anyway. The plan is that, on
receiving the message from enif_select, I can quickly dip back into
NIF-land and service the relevant socket.

Some of these sockets are actually libcurl-driven, and one of the
things that libcurl wants is for us to start a timer. I was hoping
that there'd be an equivalent of enif_select, but for timers (i.e.
enif_send_after). We also service a large number of non-curl timers,
and so I was hoping to use Erlang's timer implementation for these as
well.


Maybe you could use a port driver for this. http://erlang.org/doc/man/erl_driver.html

I played a little bit with creating a port driver to run Perl in a parallel thread, but I ultimately found it more trouble than I was willing to deal with. (The port driver code is complex and the documentation is difficult)

I did not try creating a C node, but maybe that would actually have been easier; I assume that with a C node I wouldn't have been crashing my run-time as frequently.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Roger Lipscombe-2
On 30 October 2018 at 14:03, Nathaniel Waisbrot <[hidden email]> wrote:
> Maybe you could use a port driver for this.
> http://erlang.org/doc/man/erl_driver.html

The port driver API does appear to be slightly "richer" in the
specific timer/select area, but the API looks kinda complicated, and
we've already got a NIF, so...

> I did not try creating a C node, but maybe that would actually have been
> easier; I assume that with a C node I wouldn't have been crashing my
> run-time as frequently.

With a C node, I'd have to reimplement (or find) a suitable timer
implementation. We've got a timer/select implementation; it just
doesn't scale as well as I'd like, so I was hoping to use Erlang's
(hopefully battle-hardened and scalable) implementation. I've got a
couple more easy wins I can make to our existing implementation, but
then we're going to have to take a long, hard look at it.

As it is, our NIF is already running in its own Erlang node, so I'm
not *that* bothered about crashing the runtime[1], not that we *do*
crash the runtime (not since over a year ago, anyway).

[1] See, I said I was aware of the risks of NIFs ;-)
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Lukas Larsson-3


On Tue, Oct 30, 2018 at 4:04 PM Roger Lipscombe <[hidden email]> wrote:
On 30 October 2018 at 14:03, Nathaniel Waisbrot <[hidden email]> wrote:
> Maybe you could use a port driver for this.
> http://erlang.org/doc/man/erl_driver.html

The port driver API does appear to be slightly "richer" in the
specific timer/select area, but the API looks kinda complicated, and
we've already got a NIF, so...


"Most functions in this API are not thread-safe, that is, they cannot be called from arbitrary threads. Functions that are not documented as thread-safe can only be called from driver callbacks or function calls descending from a driver callback call."

So, you cannot use the driver timer functions in arbitrary threads either.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Calling (equivalent of) erlang:send_after/3 from a NIF?

Roger Lipscombe-2
On 30 October 2018 at 15:19, Lukas Larsson <[hidden email]> wrote:
> From the documentation: http://erlang.org/doc/man/erl_driver.html
>
> "Most functions in this API are not thread-safe, that is, they cannot be
> called from arbitrary threads. Functions that are not documented as
> thread-safe can only be called from driver callbacks or function calls
> descending from a driver callback call."
>
> So, you cannot use the driver timer functions in arbitrary threads either.

Plus there's that :-)

Thanks,
Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions