Cross OTP version compare-and-swap semantic on ETS

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Cross OTP version compare-and-swap semantic on ETS

Benoit Chesneau-2
Hi all,

I’m looking on the best way (in term of concurrency and response speed) to provide a equivalent of compare-and-swap semantic on differs Erlang version with ETS.

For now what I do is using `select_replace/2` on OTP 20 and for other versions I’m using a pen_server to serialise the writes on this values, making this gen_server a bottle neck. Which is fine for a small list of keys to update. I didn’t find any other solution to do it right now. But maybe someone has an idea?

Also something I’m currently missing is the possibility do the dame for delete. `select_delete` isn’t atomic right?


Benoît
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Cross OTP version compare-and-swap semantic on ETS

Chandru-4
On 12 October 2017 at 09:01, Benoit Chesneau <[hidden email]> wrote:
Hi all,

I’m looking on the best way (in term of concurrency and response speed) to provide a equivalent of compare-and-swap semantic on differs Erlang version with ETS. 

For now what I do is using `select_replace/2` on OTP 20 and for other versions I’m using a pen_server to serialise the writes on this values, making this gen_server a bottle neck. Which is fine for a small list of keys to update. I didn’t find any other solution to do it right now. But maybe someone has an idea?

The only thing I can think of is to partition the key space and use multiple ETS tables (with a dedicated process for each).

Also something I’m currently missing is the possibility do the dame for delete. `select_delete` isn’t atomic right?

Not sure what you mean by 'atomic' here. I expect ETS will delete whatever entries have matched this specified pattern at that point in time. In the context of ETS operations, the consideration of whether it is 'atomic' or not makes sense in the 'insert', 'replace' or 'update' scenario.

cheers,
Chandru



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Cross OTP version compare-and-swap semantic on ETS

Max Lapshin-2
You should remember, that when you have a gen_server as a bottleneck, you can inspect his queue.

When you move this bottleneck to raw access to ets, you get locks and they are very hard to introspect.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Cross OTP version compare-and-swap semantic on ETS

Benoit Chesneau-2
In reply to this post by Chandru-4
Thanks for the answer :)

On 12 Oct 2017, at 10:17, Chandru <[hidden email]> wrote:

On 12 October 2017 at 09:01, Benoit Chesneau <[hidden email]> wrote:
Hi all,

I’m looking on the best way (in term of concurrency and response speed) to provide a equivalent of compare-and-swap semantic on differs Erlang version with ETS. 

For now what I do is using `select_replace/2` on OTP 20 and for other versions I’m using a pen_server to serialise the writes on this values, making this gen_server a bottle neck. Which is fine for a small list of keys to update. I didn’t find any other solution to do it right now. But maybe someone has an idea?

The only thing I can think of is to partition the key space and use multiple ETS tables (with a dedicated process for each).

True. I can do that indeed … That may be the easiest thing to do in the long term. 

For now I I did this simple lib: https://gitlab.com/barrel-db/lab/ets_cas with limited test coverage. 


Also something I’m currently missing is the possibility do the dame for delete. `select_delete` isn’t atomic right?

Not sure what you mean by 'atomic' here. I expect ETS will delete whatever entries have matched this specified pattern at that point in time. In the context of ETS operations, the consideration of whether it is 'atomic' or not makes sense in the 'insert', 'replace' or 'update' scenario.


I’m not sure what’s the difference between a delete and a replace there with the current ETS librarie 20.1 . By atomic I mean atomic at the key level, ie 2 concurrent delete  will be serialisable which seems to be the case for `select_replace` .

Benoit.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Cross OTP version compare-and-swap semantic on ETS

Benoit Chesneau-2
In reply to this post by Max Lapshin-2


> On 12 Oct 2017, at 11:32, Max Lapshin <[hidden email]> wrote:
>
> You should remember, that when you have a gen_server as a bottleneck, you can inspect his queue.
>
> When you move this bottleneck to raw access to ets, you get locks and they are very hard to introspect.

I hear  point IMO the question is probably more to know until when it becomes a bottleneck ETS offers a simple way to maintain a fast concurrent access to data across processes. I believe it will takes time before achieving any bottleneck in term of concurrent accesses (do we have benches for it?) . But I like your conservative approach also. I need to think a little about it.

Benoit

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Cross OTP version compare-and-swap semantic on ETS

Sverker Eriksson-4
In reply to this post by Benoit Chesneau-2



On 10/12/2017 01:15 PM, Benoit Chesneau wrote:
Thanks for the answer :)

On 12 Oct 2017, at 10:17, Chandru <[hidden email]> wrote:

On 12 October 2017 at 09:01, Benoit Chesneau <[hidden email]> wrote:
Hi all,

I’m looking on the best way (in term of concurrency and response speed) to provide a equivalent of compare-and-swap semantic on differs Erlang version with ETS. 

For now what I do is using `select_replace/2` on OTP 20 and for other versions I’m using a pen_server to serialise the writes on this values, making this gen_server a bottle neck. Which is fine for a small list of keys to update. I didn’t find any other solution to do it right now. But maybe someone has an idea?

The only thing I can think of is to partition the key space and use multiple ETS tables (with a dedicated process for each).

True. I can do that indeed … That may be the easiest thing to do in the long term. 

For now I I did this simple lib: https://gitlab.com/barrel-db/lab/ets_cas with limited test coverage. 


Also something I’m currently missing is the possibility do the dame for delete. `select_delete` isn’t atomic right?

Not sure what you mean by 'atomic' here. I expect ETS will delete whatever entries have matched this specified pattern at that point in time. In the context of ETS operations, the consideration of whether it is 'atomic' or not makes sense in the 'insert', 'replace' or 'update' scenario.


I’m not sure what’s the difference between a delete and a replace there with the current ETS librarie 20.1 . By atomic I mean atomic at the key level, ie 2 concurrent delete  will be serialisable which seems to be the case for `select_replace` .



All ets:select* functions are atomic per table object.
That is, each match-and-* operation is atomic. The iteration is not.


/Sverker

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions