ets:update_counter/3 optimal settings

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

ets:update_counter/3 optimal settings

Frank Muller
Hi Everyone,

I've some troubles to understand the optimal settings for this little simple problem:

Two Erlang processes A and B.

1. Process A randomly increments a counter during its lifetime using "ets:update_counter/3"

2. Process B reads the counter every X seconds and reset it to 0.

I decided to use these options when creating the table:

{write_concurrency, true}, %% A and B
{read_concurrency, false}  %% only B

Are these settings correct and more importantly "optimal" (please elaborate your answer)?

/Frank

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Alex S.
Sounds right to me, with the caveat that read + reset to zero can lose some data if you are preempted between those two calls. My usual approach is read and decrement by -N, where N is the value I just read.
9 июня 2017 г., в 8:44, Frank Muller <[hidden email]> написал(а):

Hi Everyone,

I've some troubles to understand the optimal settings for this little simple problem:

Two Erlang processes A and B.

1. Process A randomly increments a counter during its lifetime using "ets:update_counter/3"

2. Process B reads the counter every X seconds and reset it to 0.

I decided to use these options when creating the table:

{write_concurrency, true}, %% A and B
{read_concurrency, false}  %% only B

Are these settings correct and more importantly "optimal" (please elaborate your answer)?

/Frank
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Frank Muller
Hi Alex,

Thanks for the tip.
Anything else I should be aware of?

/Frank

Le ven. 9 juin 2017 à 08:08, Alex S. <[hidden email]> a écrit :
Sounds right to me, with the caveat that read + reset to zero can lose some data if you are preempted between those two calls. My usual approach is read and decrement by -N, where N is the value I just read.
9 июня 2017 г., в 8:44, Frank Muller <[hidden email]> написал(а):

Hi Everyone,

I've some troubles to understand the optimal settings for this little simple problem:

Two Erlang processes A and B.

1. Process A randomly increments a counter during its lifetime using "ets:update_counter/3"

2. Process B reads the counter every X seconds and reset it to 0.

I decided to use these options when creating the table:

{write_concurrency, true}, %% A and B
{read_concurrency, false}  %% only B

Are these settings correct and more importantly "optimal" (please elaborate your answer)?

/Frank
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Jesper Louis Andersen-2
In reply to this post by Frank Muller
On Fri, Jun 9, 2017 at 7:45 AM Frank Muller <[hidden email]> wrote:
Hi Everyone,

I've some troubles to understand the optimal settings for this little simple problem:

Two Erlang processes A and B.


If you need the speed, then measure. If you only have two processes, chances are that you can't measure any kind of difference really, because a single RX mutex would be enough for you to handle both processes. The read_concurrency and write_concurrency options tend to kick in at a certain core count and two processes can at most invoke 2 cores, and even then the chance is that they won't run at the same time in which case most of the points here are moot.

Rule of thumb: a read or write to ETS takes less than 1us. So you can often run about a million of them per second.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Frank Muller
Thanks Jesper!

/Frank

Le ven. 9 juin 2017 à 09:56, Jesper Louis Andersen <[hidden email]> a écrit :
On Fri, Jun 9, 2017 at 7:45 AM Frank Muller <[hidden email]> wrote:
Hi Everyone,

I've some troubles to understand the optimal settings for this little simple problem:

Two Erlang processes A and B.


If you need the speed, then measure. If you only have two processes, chances are that you can't measure any kind of difference really, because a single RX mutex would be enough for you to handle both processes. The read_concurrency and write_concurrency options tend to kick in at a certain core count and two processes can at most invoke 2 cores, and even then the chance is that they won't run at the same time in which case most of the points here are moot.

Rule of thumb: a read or write to ETS takes less than 1us. So you can often run about a million of them per second.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Tony Rogvall-2
In reply to this post by Alex S.

> On 9 jun 2017, at 08:08, Alex S. <[hidden email]> wrote:
>
> Sounds right to me, with the caveat that read + reset to zero can lose some data if you are preempted between those two calls. My usual approach is read and decrement by -N, where N is the value I just read.

When counter is none negative this can be done atomically like:

hd(ets:update_counter(Tab, Key, [{Pos,0},{Pos,0,0,0}])).

/Tony

>> 9 июня 2017 г., в 8:44, Frank Muller <[hidden email]> написал(а):
>>
>> Hi Everyone,
>>
>> I've some troubles to understand the optimal settings for this little simple problem:
>>
>> Two Erlang processes A and B.
>>
>> 1. Process A randomly increments a counter during its lifetime using "ets:update_counter/3"
>>
>> 2. Process B reads the counter every X seconds and reset it to 0.
>>
>> I decided to use these options when creating the table:
>>
>> {write_concurrency, true}, %% A and B
>> {read_concurrency, false}  %% only B
>>
>> Are these settings correct and more importantly "optimal" (please elaborate your answer)?
>>
>> /Frank
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Frank Muller
That's what I've implemented at the first place Tony. 
But I like Alex's idea too.

/Frank


Le ven. 9 juin 2017 à 12:33, Tony Rogvall <[hidden email]> a écrit :

> On 9 jun 2017, at 08:08, Alex S. <[hidden email]> wrote:
>
> Sounds right to me, with the caveat that read + reset to zero can lose some data if you are preempted between those two calls. My usual approach is read and decrement by -N, where N is the value I just read.

When counter is none negative this can be done atomically like:

hd(ets:update_counter(Tab, Key, [{Pos,0},{Pos,0,0,0}])).

/Tony

>> 9 июня 2017 г., в 8:44, Frank Muller <[hidden email]> написал(а):
>>
>> Hi Everyone,
>>
>> I've some troubles to understand the optimal settings for this little simple problem:
>>
>> Two Erlang processes A and B.
>>
>> 1. Process A randomly increments a counter during its lifetime using "ets:update_counter/3"
>>
>> 2. Process B reads the counter every X seconds and reset it to 0.
>>
>> I decided to use these options when creating the table:
>>
>> {write_concurrency, true}, %% A and B
>> {read_concurrency, false}  %% only B
>>
>> Are these settings correct and more importantly "optimal" (please elaborate your answer)?
>>
>> /Frank
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Max Lapshin-2
In reply to this post by Alex S.
Sometimes may help something like:


ets:update_counter(Table, Key, [{#record.field, 0}, {#record.field, -10000000000, 0, 0}]).


On Fri, Jun 9, 2017 at 9:08 AM, Alex S. <[hidden email]> wrote:
Sounds right to me, with the caveat that read + reset to zero can lose some data if you are preempted between those two calls. My usual approach is read and decrement by -N, where N is the value I just read.
9 июня 2017 г., в 8:44, Frank Muller <[hidden email]> написал(а):

Hi Everyone,

I've some troubles to understand the optimal settings for this little simple problem:

Two Erlang processes A and B.

1. Process A randomly increments a counter during its lifetime using "ets:update_counter/3"

2. Process B reads the counter every X seconds and reset it to 0.

I decided to use these options when creating the table:

{write_concurrency, true}, %% A and B
{read_concurrency, false}  %% only B

Are these settings correct and more importantly "optimal" (please elaborate your answer)?

/Frank
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Frank Muller
Max, in which case you can use this trick ?

/Frank

Le ven. 9 juin 2017 à 15:47, Max Lapshin <[hidden email]> a écrit :
Sometimes may help something like:


ets:update_counter(Table, Key, [{#record.field, 0}, {#record.field, -10000000000, 0, 0}]).


On Fri, Jun 9, 2017 at 9:08 AM, Alex S. <[hidden email]> wrote:
Sounds right to me, with the caveat that read + reset to zero can lose some data if you are preempted between those two calls. My usual approach is read and decrement by -N, where N is the value I just read.
9 июня 2017 г., в 8:44, Frank Muller <[hidden email]> написал(а):

Hi Everyone,

I've some troubles to understand the optimal settings for this little simple problem:

Two Erlang processes A and B.

1. Process A randomly increments a counter during its lifetime using "ets:update_counter/3"

2. Process B reads the counter every X seconds and reset it to 0.

I decided to use these options when creating the table:

{write_concurrency, true}, %% A and B
{read_concurrency, false}  %% only B

Are these settings correct and more importantly "optimal" (please elaborate your answer)?

/Frank
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Max Lapshin-2
If you are 100% sure that you can give such negative decrement that will be always bigger than you value.

Not very good way to program, but you can get atomic "read-and-reset" counter that can be ok for example for counters like network bytes.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Frank Muller
Ok, thanks a lot!

/Frank

Le ven. 9 juin 2017 à 15:55, Max Lapshin <[hidden email]> a écrit :
If you are 100% sure that you can give such negative decrement that will be always bigger than you value.

Not very good way to program, but you can get atomic "read-and-reset" counter that can be ok for example for counters like network bytes.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ets:update_counter/3 optimal settings

Sverker Eriksson-4
In reply to this post by Frank Muller
Table option {write_concurrency, true} can only reduce lock contention
when you got concurrent write operations toward *different* keys
within the same table.

Write operations towards one specific table object is always serialized
with locking.


/Sverker, Erlang/OTP


On 06/09/2017 07:44 AM, Frank Muller wrote:
Hi Everyone,

I've some troubles to understand the optimal settings for this little
simple problem:

Two Erlang processes A and B.

1. Process A randomly increments a counter during its lifetime using "ets
:update_counter/3"

2. Process B reads the counter every X seconds and reset it to 0.

I decided to use these options when creating the table:

{write_concurrency, true}, %% A and B
{read_concurrency, false}  %% only B

Are these settings correct and more importantly "optimal" (please elaborate
your answer)?

/Frank



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions