Sending message at a specific and accurate time

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Sending message at a specific and accurate time

Joe Armstrong-2
I want to send a UDP message at a specific time -

For each event I spawn a process


at_time_send(Host, Port, Time, Socket, Bin) ->
    %% io:format("at time ~p send",[Time]),
    spawn(fun() ->
                   send_message_at_time(Host, Port, Time, Socket, Bin)
                end).

The process sends a message to itself at the specified time

send_message_at_time(Host, Port, WantedTime, Socket, Bin) ->
    Tnow  = my_now(),
    Delta = trunc((WantedTime - Tnow)*1000),
    if
       Delta > 0 ->
          erlang:send_after(Delta, self(), ping),
          receive
             ping ->
               gen_udp:send(Socket, Host, Port, Bin)
          end;
       true ->
         void
      end.

my_now() ->
    %% seconds past epoch
    erlang:system_time()/1000000000.

I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent and
the time I want it to be sent - but I'd like it to be much more accurate
(sub ms if possible)

Is there a more accurate way to do this than in my code? Should I just
have one process for all events (not one per event) - can I mess with
process priorities or command line flag to achieve better timing
accuracy?


Cheers

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Jesper Louis Andersen-2

On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]> wrote:
I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent and
the time I want it to be sent - but I'd like it to be much more accurate
(sub ms if possible)

I would start by making measurements without the network component. When you would send the gen_udp message, you take a timestamp, and analyze the skew from the suggested skew. This lets you estimate the overhead of the rest of the system in isolation from your own code and the Erlang VM.

Intuitively, 4 to 9 milliseconds is much higher than what I would expect. But note that if you sleep for, say, 40ms, you will be awoken on the 41ms flank at the earliest. This is because you are usually "inside" a millisecond when you make the call so you start by taking the ceiling of that milli-second before you.

How much other work is your Erlang VM doing when you make these measurements? You are saying between 4 to 9 ms, which is variance suggesting the VM has lots of work to do at that moment. And of course such stuff will affect the running time. You can switch priority of your processes up to high, but this comes at the expense of other calculations if you can't finish your work quickly in the process with high priority.


--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
I tried a simpler program:

test() -> test(10, []).

test(0, L) ->
    L;
test(K, L) ->
    T1 = ms_time(),
    erlang:send_after(5000, self(), ping),
    receive
      ping ->
         T2 = ms_time(),
          test(K-1, [T2-T1|L])
    end.

ms_time() ->
    erlang:system_time() div 1000000.

Running this gives the following times

[5001,5001,5006,5006,5002,5003,5006,5002,5002,5006]

I'd expected 5000 or 5001

This is in an unloaded OS with an unloaded erlang. 6ms seems very long -
there are very few processes running and the system load is virtually zero.

I tried erl -snp disable and setting process_flag(priority, max) but the results
are pretty much the same.

Waiting for shorter times like 100 ms makes no difference - still
events with a 6 ms delay.

I want to use this for scheduling music events (controlling synths) and these
delays are far more than I expected.

/Joe



On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen
<[hidden email]> wrote:

>
> On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]> wrote:
>>
>> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent
>> and
>> the time I want it to be sent - but I'd like it to be much more accurate
>> (sub ms if possible)
>
>
> I would start by making measurements without the network component. When you
> would send the gen_udp message, you take a timestamp, and analyze the skew
> from the suggested skew. This lets you estimate the overhead of the rest of
> the system in isolation from your own code and the Erlang VM.
>
> Intuitively, 4 to 9 milliseconds is much higher than what I would expect.
> But note that if you sleep for, say, 40ms, you will be awoken on the 41ms
> flank at the earliest. This is because you are usually "inside" a
> millisecond when you make the call so you start by taking the ceiling of
> that milli-second before you.
>
> How much other work is your Erlang VM doing when you make these
> measurements? You are saying between 4 to 9 ms, which is variance suggesting
> the VM has lots of work to do at that moment. And of course such stuff will
> affect the running time. You can switch priority of your processes up to
> high, but this comes at the expense of other calculations if you can't
> finish your work quickly in the process with high priority.
>
>
> --
> J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Felix Gallo-2

What os and hardware are you seeing these results on?

On Feb 21, 2016 1:20 PM, "Joe Armstrong" <[hidden email]> wrote:
I tried a simpler program:

test() -> test(10, []).

test(0, L) ->
    L;
test(K, L) ->
    T1 = ms_time(),
    erlang:send_after(5000, self(), ping),
    receive
      ping ->
         T2 = ms_time(),
          test(K-1, [T2-T1|L])
    end.

ms_time() ->
    erlang:system_time() div 1000000.

Running this gives the following times

[5001,5001,5006,5006,5002,5003,5006,5002,5002,5006]

I'd expected 5000 or 5001

This is in an unloaded OS with an unloaded erlang. 6ms seems very long -
there are very few processes running and the system load is virtually zero.

I tried erl -snp disable and setting process_flag(priority, max) but the results
are pretty much the same.

Waiting for shorter times like 100 ms makes no difference - still
events with a 6 ms delay.

I want to use this for scheduling music events (controlling synths) and these
delays are far more than I expected.

/Joe



On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen
<[hidden email]> wrote:
>
> On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]> wrote:
>>
>> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent
>> and
>> the time I want it to be sent - but I'd like it to be much more accurate
>> (sub ms if possible)
>
>
> I would start by making measurements without the network component. When you
> would send the gen_udp message, you take a timestamp, and analyze the skew
> from the suggested skew. This lets you estimate the overhead of the rest of
> the system in isolation from your own code and the Erlang VM.
>
> Intuitively, 4 to 9 milliseconds is much higher than what I would expect.
> But note that if you sleep for, say, 40ms, you will be awoken on the 41ms
> flank at the earliest. This is because you are usually "inside" a
> millisecond when you make the call so you start by taking the ceiling of
> that milli-second before you.
>
> How much other work is your Erlang VM doing when you make these
> measurements? You are saying between 4 to 9 ms, which is variance suggesting
> the VM has lots of work to do at that moment. And of course such stuff will
> affect the running time. You can switch priority of your processes up to
> high, but this comes at the expense of other calculations if you can't
> finish your work quickly in the process with high priority.
>
>
> --
> J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
mac os-x El Capitan 10.11.2
Macbook retina pro
3.1 GHz core i7
16 GB ram

Can you run the above program locally and post back the results?



/Joe

On Sun, Feb 21, 2016 at 10:34 PM, Felix Gallo <[hidden email]> wrote:

> What os and hardware are you seeing these results on?
>
> On Feb 21, 2016 1:20 PM, "Joe Armstrong" <[hidden email]> wrote:
>>
>> I tried a simpler program:
>>
>> test() -> test(10, []).
>>
>> test(0, L) ->
>>     L;
>> test(K, L) ->
>>     T1 = ms_time(),
>>     erlang:send_after(5000, self(), ping),
>>     receive
>>       ping ->
>>          T2 = ms_time(),
>>           test(K-1, [T2-T1|L])
>>     end.
>>
>> ms_time() ->
>>     erlang:system_time() div 1000000.
>>
>> Running this gives the following times
>>
>> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006]
>>
>> I'd expected 5000 or 5001
>>
>> This is in an unloaded OS with an unloaded erlang. 6ms seems very long -
>> there are very few processes running and the system load is virtually
>> zero.
>>
>> I tried erl -snp disable and setting process_flag(priority, max) but the
>> results
>> are pretty much the same.
>>
>> Waiting for shorter times like 100 ms makes no difference - still
>> events with a 6 ms delay.
>>
>> I want to use this for scheduling music events (controlling synths) and
>> these
>> delays are far more than I expected.
>>
>> /Joe
>>
>>
>>
>> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen
>> <[hidden email]> wrote:
>> >
>> > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]> wrote:
>> >>
>> >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is
>> >> sent
>> >> and
>> >> the time I want it to be sent - but I'd like it to be much more
>> >> accurate
>> >> (sub ms if possible)
>> >
>> >
>> > I would start by making measurements without the network component. When
>> > you
>> > would send the gen_udp message, you take a timestamp, and analyze the
>> > skew
>> > from the suggested skew. This lets you estimate the overhead of the rest
>> > of
>> > the system in isolation from your own code and the Erlang VM.
>> >
>> > Intuitively, 4 to 9 milliseconds is much higher than what I would
>> > expect.
>> > But note that if you sleep for, say, 40ms, you will be awoken on the
>> > 41ms
>> > flank at the earliest. This is because you are usually "inside" a
>> > millisecond when you make the call so you start by taking the ceiling of
>> > that milli-second before you.
>> >
>> > How much other work is your Erlang VM doing when you make these
>> > measurements? You are saying between 4 to 9 ms, which is variance
>> > suggesting
>> > the VM has lots of work to do at that moment. And of course such stuff
>> > will
>> > affect the running time. You can switch priority of your processes up to
>> > high, but this comes at the expense of other calculations if you can't
>> > finish your work quickly in the process with high priority.
>> >
>> >
>> > --
>> > J.
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Jesper Louis Andersen-2
On a Archlinux machine, Core i7 Haswell:

6> z:test().
[5001,5001,5001,5001,5001,5001,5001,5001,5001,5001]

This is what I expect to see. I'm awoken on the right flank up to one millisecond after.

Immediate thought: this has to do with CPU frequency scaling and battery saving features of OSX. Since nothing is done for 5 seconds, the system throttles down and goes to sleep. When it is awoken again, you are enjoying some latency.

Enable dtrace and hunt :)


On Sun, Feb 21, 2016 at 10:40 PM, Joe Armstrong <[hidden email]> wrote:
mac os-x El Capitan 10.11.2
Macbook retina pro
3.1 GHz core i7
16 GB ram

Can you run the above program locally and post back the results?



/Joe

On Sun, Feb 21, 2016 at 10:34 PM, Felix Gallo <[hidden email]> wrote:
> What os and hardware are you seeing these results on?
>
> On Feb 21, 2016 1:20 PM, "Joe Armstrong" <[hidden email]> wrote:
>>
>> I tried a simpler program:
>>
>> test() -> test(10, []).
>>
>> test(0, L) ->
>>     L;
>> test(K, L) ->
>>     T1 = ms_time(),
>>     erlang:send_after(5000, self(), ping),
>>     receive
>>       ping ->
>>          T2 = ms_time(),
>>           test(K-1, [T2-T1|L])
>>     end.
>>
>> ms_time() ->
>>     erlang:system_time() div 1000000.
>>
>> Running this gives the following times
>>
>> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006]
>>
>> I'd expected 5000 or 5001
>>
>> This is in an unloaded OS with an unloaded erlang. 6ms seems very long -
>> there are very few processes running and the system load is virtually
>> zero.
>>
>> I tried erl -snp disable and setting process_flag(priority, max) but the
>> results
>> are pretty much the same.
>>
>> Waiting for shorter times like 100 ms makes no difference - still
>> events with a 6 ms delay.
>>
>> I want to use this for scheduling music events (controlling synths) and
>> these
>> delays are far more than I expected.
>>
>> /Joe
>>
>>
>>
>> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen
>> <[hidden email]> wrote:
>> >
>> > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]> wrote:
>> >>
>> >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is
>> >> sent
>> >> and
>> >> the time I want it to be sent - but I'd like it to be much more
>> >> accurate
>> >> (sub ms if possible)
>> >
>> >
>> > I would start by making measurements without the network component. When
>> > you
>> > would send the gen_udp message, you take a timestamp, and analyze the
>> > skew
>> > from the suggested skew. This lets you estimate the overhead of the rest
>> > of
>> > the system in isolation from your own code and the Erlang VM.
>> >
>> > Intuitively, 4 to 9 milliseconds is much higher than what I would
>> > expect.
>> > But note that if you sleep for, say, 40ms, you will be awoken on the
>> > 41ms
>> > flank at the earliest. This is because you are usually "inside" a
>> > millisecond when you make the call so you start by taking the ceiling of
>> > that milli-second before you.
>> >
>> > How much other work is your Erlang VM doing when you make these
>> > measurements? You are saying between 4 to 9 ms, which is variance
>> > suggesting
>> > the VM has lots of work to do at that moment. And of course such stuff
>> > will
>> > affect the running time. You can switch priority of your processes up to
>> > high, but this comes at the expense of other calculations if you can't
>> > finish your work quickly in the process with high priority.
>> >
>> >
>> > --
>> > J.
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions



--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
On Sun, Feb 21, 2016 at 10:56 PM, Jesper Louis Andersen
<[hidden email]> wrote:

> On a Archlinux machine, Core i7 Haswell:
>
> 6> z:test().
> [5001,5001,5001,5001,5001,5001,5001,5001,5001,5001]
>
> This is what I expect to see. I'm awoken on the right flank up to one
> millisecond after.
>
> Immediate thought: this has to do with CPU frequency scaling and battery
> saving features of OSX. Since nothing is done for 5 seconds, the system
> throttles down and goes to sleep. When it is awoken again, you are enjoying
> some latency.

It was the same with 100 ms delays

>
> Enable dtrace and hunt :)
>
>
> On Sun, Feb 21, 2016 at 10:40 PM, Joe Armstrong <[hidden email]> wrote:
>>
>> mac os-x El Capitan 10.11.2
>> Macbook retina pro
>> 3.1 GHz core i7
>> 16 GB ram
>>
>> Can you run the above program locally and post back the results?
>>
>>
>>
>> /Joe
>>
>> On Sun, Feb 21, 2016 at 10:34 PM, Felix Gallo <[hidden email]>
>> wrote:
>> > What os and hardware are you seeing these results on?
>> >
>> > On Feb 21, 2016 1:20 PM, "Joe Armstrong" <[hidden email]> wrote:
>> >>
>> >> I tried a simpler program:
>> >>
>> >> test() -> test(10, []).
>> >>
>> >> test(0, L) ->
>> >>     L;
>> >> test(K, L) ->
>> >>     T1 = ms_time(),
>> >>     erlang:send_after(5000, self(), ping),
>> >>     receive
>> >>       ping ->
>> >>          T2 = ms_time(),
>> >>           test(K-1, [T2-T1|L])
>> >>     end.
>> >>
>> >> ms_time() ->
>> >>     erlang:system_time() div 1000000.
>> >>
>> >> Running this gives the following times
>> >>
>> >> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006]
>> >>
>> >> I'd expected 5000 or 5001
>> >>
>> >> This is in an unloaded OS with an unloaded erlang. 6ms seems very long
>> >> -
>> >> there are very few processes running and the system load is virtually
>> >> zero.
>> >>
>> >> I tried erl -snp disable and setting process_flag(priority, max) but
>> >> the
>> >> results
>> >> are pretty much the same.
>> >>
>> >> Waiting for shorter times like 100 ms makes no difference - still
>> >> events with a 6 ms delay.
>> >>
>> >> I want to use this for scheduling music events (controlling synths) and
>> >> these
>> >> delays are far more than I expected.
>> >>
>> >> /Joe
>> >>
>> >>
>> >>
>> >> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen
>> >> <[hidden email]> wrote:
>> >> >
>> >> > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]>
>> >> > wrote:
>> >> >>
>> >> >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is
>> >> >> sent
>> >> >> and
>> >> >> the time I want it to be sent - but I'd like it to be much more
>> >> >> accurate
>> >> >> (sub ms if possible)
>> >> >
>> >> >
>> >> > I would start by making measurements without the network component.
>> >> > When
>> >> > you
>> >> > would send the gen_udp message, you take a timestamp, and analyze the
>> >> > skew
>> >> > from the suggested skew. This lets you estimate the overhead of the
>> >> > rest
>> >> > of
>> >> > the system in isolation from your own code and the Erlang VM.
>> >> >
>> >> > Intuitively, 4 to 9 milliseconds is much higher than what I would
>> >> > expect.
>> >> > But note that if you sleep for, say, 40ms, you will be awoken on the
>> >> > 41ms
>> >> > flank at the earliest. This is because you are usually "inside" a
>> >> > millisecond when you make the call so you start by taking the ceiling
>> >> > of
>> >> > that milli-second before you.
>> >> >
>> >> > How much other work is your Erlang VM doing when you make these
>> >> > measurements? You are saying between 4 to 9 ms, which is variance
>> >> > suggesting
>> >> > the VM has lots of work to do at that moment. And of course such
>> >> > stuff
>> >> > will
>> >> > affect the running time. You can switch priority of your processes up
>> >> > to
>> >> > high, but this comes at the expense of other calculations if you
>> >> > can't
>> >> > finish your work quickly in the process with high priority.
>> >> >
>> >> >
>> >> > --
>> >> > J.
>> >> _______________________________________________
>> >> erlang-questions mailing list
>> >> [hidden email]
>> >> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
>
> --
> J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

vicbaz
In reply to this post by Joe Armstrong-2
Hello,

Try +K flag.

$ erl
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2]
[async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.2.1 (abort with ^G)
1> c(t).
{ok,t}
2> t:test().
[5003,5006,5002,5003,5006,5002,5003,5006,5001,5005]

$ erl +K true
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2]
[async-threads:10] [hipe] [kernel-poll:true]

Eshell V7.2.1 (abort with ^G)
1> c(t).
{ok,t}
2> t:test().
[5001,5001,5001,5001,5001,5001,5001,5001,5001,5001]

$ uname -a
Linux frontier 4.3.0-0.bpo.1-amd64 #1 SMP Debian 4.3.3-7~bpo8+1
(2016-01-19) x86_64 GNU/Linux

$ cat /proc/cpuinfo | grep 'model name'
model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz
model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz

On 22/02/16 00:20, Joe Armstrong wrote:

> I tried a simpler program:
>
> test() -> test(10, []).
>
> test(0, L) ->
>      L;
> test(K, L) ->
>      T1 = ms_time(),
>      erlang:send_after(5000, self(), ping),
>      receive
>        ping ->
>           T2 = ms_time(),
>            test(K-1, [T2-T1|L])
>      end.
>
> ms_time() ->
>      erlang:system_time() div 1000000.
>
> Running this gives the following times
>
> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006]
>
> I'd expected 5000 or 5001
>
> This is in an unloaded OS with an unloaded erlang. 6ms seems very long -
> there are very few processes running and the system load is virtually zero.
>
> I tried erl -snp disable and setting process_flag(priority, max) but the results
> are pretty much the same.
>
> Waiting for shorter times like 100 ms makes no difference - still
> events with a 6 ms delay.
>
> I want to use this for scheduling music events (controlling synths) and these
> delays are far more than I expected.
>
> /Joe
>
>
>
> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen
> <[hidden email]> wrote:
>>
>> On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]> wrote:
>>>
>>> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent
>>> and
>>> the time I want it to be sent - but I'd like it to be much more accurate
>>> (sub ms if possible)
>>
>>
>> I would start by making measurements without the network component. When you
>> would send the gen_udp message, you take a timestamp, and analyze the skew
>> from the suggested skew. This lets you estimate the overhead of the rest of
>> the system in isolation from your own code and the Erlang VM.
>>
>> Intuitively, 4 to 9 milliseconds is much higher than what I would expect.
>> But note that if you sleep for, say, 40ms, you will be awoken on the 41ms
>> flank at the earliest. This is because you are usually "inside" a
>> millisecond when you make the call so you start by taking the ceiling of
>> that milli-second before you.
>>
>> How much other work is your Erlang VM doing when you make these
>> measurements? You are saying between 4 to 9 ms, which is variance suggesting
>> the VM has lots of work to do at that moment. And of course such stuff will
>> affect the running time. You can switch priority of your processes up to
>> high, but this comes at the expense of other calculations if you can't
>> finish your work quickly in the process with high priority.
>>
>>
>> --
>> J.
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
I made a new test - here's the result with kernel poll (I've also changed the
units of time to microseconds - so we can see the sub-millisecond effect)

$ erl +K true
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4]
[async-threads:10] [hipe] [kernel-poll:true]
Eshell V7.2.1  (abort with ^G)
1> t:test().
[1006032,1006176,1005987,1006065,1000988,1001926,1005561,
 1001584,1000335,1001442]
2> t:test().
[1001963,1005706,1006087,1006171,1005994,1005992,1005985,
 1001037,1005900,1005268]

To test how fast we can send a message and receive it I changed

     erlang:send_after(1000, self(), ping),
to
     self() ! ping

and reran - now I get delays of the order of 2 microseconds
admittedly the process does not now get scheduled in and out

I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the
mac - if inter-processing scheduling and messaging had 6 millisecond
delays Erlang would be unusable.

The time unit in send_after is milliseconds - and it is true that
sending something
6 milliseconds after I wanted it is indeed "after" - but it would also
be true if the message
was sent a year after I asked for it.

I guess I could do a send_after 10ms before I want and then do a busy wait
but this would be crazy.

Here's the slightly modified program.


-module(t).
-compile(export_all).

test() ->
    process_flag(priority, max),
    test(10, []).

test(0, L) ->
    L;
test(K, L) ->
    T1 = microsecond_time(),
    %% erlang:send_after(1000, self(), ping),
    self() ! ping,
    receive
ping ->
   T2 = microsecond_time(),
   test(K-1, [T2-T1|L])
    end.


microsecond_time() ->
    erlang:system_time() div 1000.


/Joe

On Sun, Feb 21, 2016 at 11:29 PM,  <[hidden email]> wrote:

> Hello,
>
> Try +K flag.
>
> $ erl
> Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2] [async-threads:10]
> [hipe] [kernel-poll:false]
>
> Eshell V7.2.1 (abort with ^G)
> 1> c(t).
> {ok,t}
> 2> t:test().
> [5003,5006,5002,5003,5006,5002,5003,5006,5001,5005]
>
> $ erl +K true
> Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2] [async-threads:10]
> [hipe] [kernel-poll:true]
>
> Eshell V7.2.1 (abort with ^G)
> 1> c(t).
> {ok,t}
> 2> t:test().
> [5001,5001,5001,5001,5001,5001,5001,5001,5001,5001]
>
> $ uname -a
> Linux frontier 4.3.0-0.bpo.1-amd64 #1 SMP Debian 4.3.3-7~bpo8+1 (2016-01-19)
> x86_64 GNU/Linux
>
> $ cat /proc/cpuinfo | grep 'model name'
> model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz
> model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz
>
>
> On 22/02/16 00:20, Joe Armstrong wrote:
>>
>> I tried a simpler program:
>>
>> test() -> test(10, []).
>>
>> test(0, L) ->
>>      L;
>> test(K, L) ->
>>      T1 = ms_time(),
>>      erlang:send_after(5000, self(), ping),
>>      receive
>>        ping ->
>>           T2 = ms_time(),
>>            test(K-1, [T2-T1|L])
>>      end.
>>
>> ms_time() ->
>>      erlang:system_time() div 1000000.
>>
>> Running this gives the following times
>>
>> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006]
>>
>> I'd expected 5000 or 5001
>>
>> This is in an unloaded OS with an unloaded erlang. 6ms seems very long -
>> there are very few processes running and the system load is virtually
>> zero.
>>
>> I tried erl -snp disable and setting process_flag(priority, max) but the
>> results
>> are pretty much the same.
>>
>> Waiting for shorter times like 100 ms makes no difference - still
>> events with a 6 ms delay.
>>
>> I want to use this for scheduling music events (controlling synths) and
>> these
>> delays are far more than I expected.
>>
>> /Joe
>>
>>
>>
>> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen
>> <[hidden email]> wrote:
>>>
>>>
>>> On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong <[hidden email]> wrote:
>>>>
>>>>
>>>> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is
>>>> sent
>>>> and
>>>> the time I want it to be sent - but I'd like it to be much more accurate
>>>> (sub ms if possible)
>>>
>>>
>>>
>>> I would start by making measurements without the network component. When
>>> you
>>> would send the gen_udp message, you take a timestamp, and analyze the
>>> skew
>>> from the suggested skew. This lets you estimate the overhead of the rest
>>> of
>>> the system in isolation from your own code and the Erlang VM.
>>>
>>> Intuitively, 4 to 9 milliseconds is much higher than what I would expect.
>>> But note that if you sleep for, say, 40ms, you will be awoken on the 41ms
>>> flank at the earliest. This is because you are usually "inside" a
>>> millisecond when you make the call so you start by taking the ceiling of
>>> that milli-second before you.
>>>
>>> How much other work is your Erlang VM doing when you make these
>>> measurements? You are saying between 4 to 9 ms, which is variance
>>> suggesting
>>> the VM has lots of work to do at that moment. And of course such stuff
>>> will
>>> affect the running time. You can switch priority of your processes up to
>>> high, but this comes at the expense of other calculations if you can't
>>> finish your work quickly in the process with high priority.
>>>
>>>
>>> --
>>> J.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Max Lapshin-2
Your problem is very painful for us.

There is such a task in video streaming: send udp packets with very precise timing.  It is required for old hardware and non-IP media like air/cable streaming.

As far as I remember, we could reach required precision for IP clients, but cannot achive microsecond precision for non-IP clients.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Jesper Louis Andersen-2
In reply to this post by Joe Armstrong-2

On Mon, Feb 22, 2016 at 8:30 AM, Joe Armstrong <[hidden email]> wrote:
Here's the slightly modified program.

I *have* to nitpick this since I worry people will misuse the new 18.0 timing API. Changes to the program:

* Use erlang:monotonic_time() which gives you native resolution by default according to the precision of your systems clock (older devices will give you micro_seconds, newer systems nano_seconds). Also, by using monotonic_time we avoid time warping, which you may get with system_time.

* Request the timestamps in native format, operate on the native format and finally convert down to the desired output format. This avoids rounding errors.

I get the same bad results on OSX, but not on Linux. It would be interesting to analyze why it takes so long to wake up on OSX after having slept for 1 second. It definitely looks like a problem with being woken up and having to do work different from the scheduler loop, but I have yet to investigate why that happens.

-module(t).
-compile(export_all).

test() ->
    process_flag(priority, max),
    test(10, []).

test(0, L) ->
    L;
test(K, L) ->
    T1 = erlang:monotonic_time(),
    erlang:send_after(1000, self(), ping),
    receive
      ping ->
        T2 = erlang:monotonic_time(),
        test(K-1, [erlang:convert_time_unit(T2-T1, native, micro_seconds) | L ])
    end.




--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
On Mon, Feb 22, 2016 at 12:04 PM, Jesper Louis Andersen
<[hidden email]> wrote:

>
> On Mon, Feb 22, 2016 at 8:30 AM, Joe Armstrong <[hidden email]> wrote:
>>
>> Here's the slightly modified program.
>
>
> I *have* to nitpick this since I worry people will misuse the new 18.0
> timing API. Changes to the program:
>
> * Use erlang:monotonic_time() which gives you native resolution by default
> according to the precision of your systems clock (older devices will give
> you micro_seconds, newer systems nano_seconds). Also, by using
> monotonic_time we avoid time warping, which you may get with system_time.
>
> * Request the timestamps in native format, operate on the native format and
> finally convert down to the desired output format. This avoids rounding
> errors.
>
> I get the same bad results on OSX, but not on Linux. It would be interesting
> to analyze why it takes so long to wake up on OSX after having slept for 1
> second. It definitely looks like a problem with being woken up and having to
> do work different from the scheduler loop, but I have yet to investigate why
> that happens.

I get the same effect irrespective of the sleep interval if I sleep for 100 ms
then the wake-up still can be still 6ms later than the requested time ...

Is the timer code in linux and OS-X different, or do they call the
same libraries
and the libraries behave differently?

I have no idea

/Joe



>
> -module(t).
> -compile(export_all).
>
> test() ->
>     process_flag(priority, max),
>     test(10, []).
>
> test(0, L) ->
>     L;
> test(K, L) ->
>     T1 = erlang:monotonic_time(),
>     erlang:send_after(1000, self(), ping),
>     receive
>       ping ->
>         T2 = erlang:monotonic_time(),
>         test(K-1, [erlang:convert_time_unit(T2-T1, native, micro_seconds) |
> L ])
>     end.
>
>
>
>
> --
> J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Ola Bäckström
Do you get different result with power cable plugged in compared to not plugged in?

As Jesper mentioned earlier does OS X have some power saving stuff, it is called "timer coalescing".  
See for instance http://arstechnica.com/apple/2013/06/how-os-x-mavericks-works-its-power-saving-magic/
There seems to be ways to disable it, http://www.timoliver.com.au/2014/01/25/disabling-timer-coalescing-in-os-x-mavericks/
, but that is not something I've tried.

I thought this was "os X"-only stuff, but now when I google I see that linux seems to have coalescing since many years too. If it is in general enabled I don't know.

/Ola
-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Joe Armstrong
Sent: den 22 februari 2016 12:52
To: Jesper Louis Andersen <[hidden email]>
Cc: Erlang <[hidden email]>
Subject: Re: [erlang-questions] Sending message at a specific and accurate time

On Mon, Feb 22, 2016 at 12:04 PM, Jesper Louis Andersen <[hidden email]> wrote:

>
> On Mon, Feb 22, 2016 at 8:30 AM, Joe Armstrong <[hidden email]> wrote:
>>
>> Here's the slightly modified program.
>
>
> I *have* to nitpick this since I worry people will misuse the new 18.0
> timing API. Changes to the program:
>
> * Use erlang:monotonic_time() which gives you native resolution by
> default according to the precision of your systems clock (older
> devices will give you micro_seconds, newer systems nano_seconds).
> Also, by using monotonic_time we avoid time warping, which you may get with system_time.
>
> * Request the timestamps in native format, operate on the native
> format and finally convert down to the desired output format. This
> avoids rounding errors.
>
> I get the same bad results on OSX, but not on Linux. It would be
> interesting to analyze why it takes so long to wake up on OSX after
> having slept for 1 second. It definitely looks like a problem with
> being woken up and having to do work different from the scheduler
> loop, but I have yet to investigate why that happens.

I get the same effect irrespective of the sleep interval if I sleep for 100 ms then the wake-up still can be still 6ms later than the requested time ...

Is the timer code in linux and OS-X different, or do they call the same libraries and the libraries behave differently?

I have no idea

/Joe



>
> -module(t).
> -compile(export_all).
>
> test() ->
>     process_flag(priority, max),
>     test(10, []).
>
> test(0, L) ->
>     L;
> test(K, L) ->
>     T1 = erlang:monotonic_time(),
>     erlang:send_after(1000, self(), ping),
>     receive
>       ping ->
>         T2 = erlang:monotonic_time(),
>         test(K-1, [erlang:convert_time_unit(T2-T1, native,
> micro_seconds) | L ])
>     end.
>
>
>
>
> --
> J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Jesper Louis Andersen-2
In reply to this post by Joe Armstrong-2

On Mon, Feb 22, 2016 at 12:51 PM, Joe Armstrong <[hidden email]> wrote:
I get the same effect irrespective of the sleep interval if I sleep for 100 ms
then the wake-up still can be still 6ms later than the requested time ...

Sleeping for 100ms is ages in modern computers, so it would not surprise me one bit that you ended up going to sleep to save power in that time frame.

The Erlang VM has a compatibility layer plugging into the underlying operating system. You would be surprised over how different they are and what they support at the lowest level. And what the "correct means of operation" is on system A might be a really bad idea on system B.

The right way to figure out what is happening is dtrace(1) and plugging into the timer wheel to figure out what is happening at that level. Either you are getting the message too late from the kernel, or you end up doing something weird inside Erlang as a VM.

--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Jesper Louis Andersen-2
In reply to this post by Ola Bäckström

On Mon, Feb 22, 2016 at 1:16 PM, Ola Bäckström <[hidden email]> wrote:
Do you get different result with power cable plugged in compared to not plugged in?

I controlled for that by trying it with and without the cable plugged. It isn't affecting the latencies.

Any kind of dynamic timing system shouldn't really affect this either. If anything it ought to improve the precision.


--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Tony Rogvall-2
> L = fun Loop() -> receive after 1 ->  Loop()  end end.
> spawn_link(L).

That is a process doing minimal sleep, but still sleeps ( I think )

Adding this little ”helper” process changes the timing on my mac to the better.

From 3-4 ms latency to to around 1 or less.

Power cable did not affect the latency for me either.

/Tony

> On 22 feb 2016, at 13:19, Jesper Louis Andersen <[hidden email]> wrote:
>
>
> On Mon, Feb 22, 2016 at 1:16 PM, Ola Bäckström <[hidden email]> wrote:
> Do you get different result with power cable plugged in compared to not plugged in?
>
> I controlled for that by trying it with and without the cable plugged. It isn't affecting the latencies.
>
> Any kind of dynamic timing system shouldn't really affect this either. If anything it ought to improve the precision.
>
>
> --
> J.
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
On Mon, Feb 22, 2016 at 3:31 PM, Tony Rogvall <[hidden email]> wrote:
>> L = fun Loop() -> receive after 1 ->  Loop()  end end.
>> spawn_link(L).

Busy sleep :-) < I thought busy waiting was frowned upon>

Actually after 10 works as well.

So a work-around is to set the timer 10 ms *before* I want and run a busy sleep
loop -

So I keep having to thumping the damn thing to make it doesn't fall asleep.

I've only tested on an unloaded system - perhaps if the system was loaded
the timings would be better?

Thanks for the tip.

/Joe

>
> That is a process doing minimal sleep, but still sleeps ( I think )
>
> Adding this little ”helper” process changes the timing on my mac to the better.
>
> From 3-4 ms latency to to around 1 or less.
>
> Power cable did not affect the latency for me either.
>
> /Tony
>
>> On 22 feb 2016, at 13:19, Jesper Louis Andersen <[hidden email]> wrote:
>>
>>
>> On Mon, Feb 22, 2016 at 1:16 PM, Ola Bäckström <[hidden email]> wrote:
>> Do you get different result with power cable plugged in compared to not plugged in?
>>
>> I controlled for that by trying it with and without the cable plugged. It isn't affecting the latencies.
>>
>> Any kind of dynamic timing system shouldn't really affect this either. If anything it ought to improve the precision.
>>
>>
>> --
>> J.
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Rickard Green-2
In reply to this post by Joe Armstrong-2
> I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the
> mac - if inter-processing scheduling and messaging had 6 millisecond
> delays Erlang would be unusable.
>

When schedulers run out of work they eventually need to sleep. A
timeout is set up in order to service next timeout. On macosx we will
sleep in select(). The timeout is set on microsecond level, but on
macosx the thread is often woken milliseconds after the time we
requested. When the thread wakeup is delayed like this, timers handled
by the thread will also be delayed. I verified the delayed timeout in
select() using a simple c-program, and the figures corresponds to what
can be seen in your send_after() test. That is, it is not the timer
implementation in the vm that delays the timers.

> I guess I could do a send_after 10ms before I want and then do a busy wait
> but this would be crazy.
>

If the primitive used for waiting has this kind of delay, a busy wait
is more or less what is left. The vm could of course perform this busy
wait for you, but my guess is that this busy wait wouldn't be received
well by the users. It is however an option.

Regards,
Rickard
--
Rickard Green, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
On Mon, Feb 22, 2016 at 4:55 PM, Rickard Green <[hidden email]> wrote:
>> I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the
>> mac - if inter-processing scheduling and messaging had 6 millisecond
>> delays Erlang would be unusable.
>>
>
> When schedulers run out of work they eventually need to sleep. A
> timeout is set up in order to service next timeout. On macosx we will
> sleep in select().

> The timeout is set on microsecond level, but on
> macosx the thread is often woken milliseconds after the time we
> requested.

UUgh -- that's horrible - seems daft setting a microsecond timer
if it refuses to obey - (

There might be a crazy work around - on the mac core audio
can send midi events at very precise times - so if you subscribe to these
 you'll get a callback up at a very precise time - so you could use the midi
scheduler to schedule future events.

(Actually I ran into this problem when generating audio events, so it's not
surprising)

Midi is essentially "play this note" at time <t>, which is not that
different from
"send this message" at time <t> - and core audio supports this at a very low OS
level.

The audio timers are designed to take priority over other timers and are
be very accurate ...

A quick google search turned up several posts that pursued this idea

/Joe

> When the thread wakeup is delayed like this, timers handled
> by the thread will also be delayed. I verified the delayed timeout in
> select() using a simple c-program, and the figures corresponds to what
> can be seen in your send_after() test. That is, it is not the timer
> implementation in the vm that delays the timers.
>
>> I guess I could do a send_after 10ms before I want and then do a busy wait
>> but this would be crazy.
>>
>
> If the primitive used for waiting has this kind of delay, a busy wait
> is more or less what is left. The vm could of course perform this busy
> wait for you, but my guess is that this busy wait wouldn't be received
> well by the users. It is however an option.
>
> Regards,
> Rickard
> --
> Rickard Green, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Sending message at a specific and accurate time

Joe Armstrong-2
In reply to this post by Rickard Green-2
On Mon, Feb 22, 2016 at 4:55 PM, Rickard Green <[hidden email]> wrote:

>> I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the
>> mac - if inter-processing scheduling and messaging had 6 millisecond
>> delays Erlang would be unusable.
>>
>
> When schedulers run out of work they eventually need to sleep. A
> timeout is set up in order to service next timeout. On macosx we will
> sleep in select(). The timeout is set on microsecond level, but on
> macosx the thread is often woken milliseconds after the time we
> requested. When the thread wakeup is delayed like this, timers handled
> by the thread will also be delayed. I verified the delayed timeout in
> select() using a simple c-program, and the figures corresponds to what
> can be seen in your send_after() test. That is, it is not the timer
> implementation in the vm that delays the timers.
>
>> I guess I could do a send_after 10ms before I want and then do a busy wait
>> but this would be crazy.
>>
>
> If the primitive used for waiting has this kind of delay, a busy wait
> is more or less what is left. The vm could of course perform this busy
> wait for you, but my guess is that this busy wait wouldn't be received
> well by the users. It is however an option.

You'd need two primitives

    send_after  (as today) and send_after_precise(Effort, Time, ...)

where Effort is high, low, ... (or something)

   Actually I think this is a key primitive in Erlang - far more
important than maps
and module names - the "essence" of Erlang has to do with messaging -
this is where Erlang should (and does) excel - the rest (maps, etc.)
is just syntax.

Sending a message at a precise time is an underlying mechanism upon
which a large number of things can be built.

Why is Erlang great? - because you can have thousands of parallel processes
and send and receive message quickly. The Zen of erlang involves fast
process spawning, fast message sending and receiving and control over
time.

Personally I would put a lot of effort into getting this right -

Remember it's all about messaging, all the rest is syntax

Cheers

/Joe




>
> Regards,
> Rickard
> --
> Rickard Green, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
12