ETS’s performances and limits?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

ETS’s performances and limits?

Frank Muller
Hi,

Is ETS capable of handling 1 million key/value (or more)? My keys are 36B and all my values are local PIDs. My ETS is mostly read-only!!!

What’s the largest ETS one has to deal with?
Is there any limitations or point when the ETS’s perf degrades?

I’m on CentOS7, Erlang 20.x & 21.x with 256GB of RAM.

/Frank 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ETS’s performances and limits?

Anthony Molinaro-4
Hi,

The simple answer is yes, but there’s some complications under scale.

I’ve got a few ordered_sets with ~12 million entries in them which seem to work reasonably well under normal loads, but they tend to show abnormal performance under heavier concurrency.

This is a known limitation which AFAIK has not been addressed yet (https://www.youtube.com/watch?v=40shUKSdh1A) in the implementation of ordered_sets (I’d love to be told differently, but looking at the code it does not appear to have been changed in some time).

I don’t believe that sets have the same issue.  In all likelihood things will work fine for your use case, and you can test it out easily enough.

HTH,

-Anthony

On Sep 1, 2018, at 1:53 PM, Frank Muller <[hidden email]> wrote:

Hi,

Is ETS capable of handling 1 million key/value (or more)? My keys are 36B and all my values are local PIDs. My ETS is mostly read-only!!!

What’s the largest ETS one has to deal with?
Is there any limitations or point when the ETS’s perf degrades?

I’m on CentOS7, Erlang 20.x & 21.x with 256GB of RAM.

/Frank 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ETS’s performances and limits?

Valentin Micic-5

On 01 Sep 2018, at 11:09 PM, ANTHONY MOLINARO wrote:

>
> I’ve got a few ordered_sets with ~12 million entries in them which seem to work reasonably well under normal loads, but they tend to show abnormal performance under heavier concurrency.
>

There is a good chance that performance degradation you are referring to may *not* be related to a "heavier concurrency" as much as it may be caused by a rate of inserts into a table of type "ordered_set". The manual indicates that ordered_set tables do behave differently "in come situations" (which I read as "insert operations when a table gets big").

Out of interest, could you please quantify "heavier concurrency" in terms of a number of processes running and interacting with the table?

My experience is with an ETS table of type "set", hosting more than 190 million entries, and a steady insert rate (which may be updating existing, but also adding new entries) of 200 entries per second, with concurrent lookups from more than a hundred short-lived processes at any point in time (well, rather, most of the time).

Apart from requiring some time to load all that data from disk to the ETS memory (System Start-Up), and needing some time to release all the memory upon ETS table destruction (System Shutdown), the performance is quite consistent.

Kind regards

V/

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ETS’s performances and limits?

Mikael Pettersson-5
In reply to this post by Frank Muller
On Sat, Sep 1, 2018 at 10:53 PM, Frank Muller
<[hidden email]> wrote:
> Hi,
>
> Is ETS capable of handling 1 million key/value (or more)? My keys are 36B
> and all my values are local PIDs. My ETS is mostly read-only!!!

Yes, that's tiny.  Using integers as keys would be best.

> What’s the largest ETS one has to deal with?

You're only limited by available RAM.  We routinely handle tables with
several 100s of million records in each, and we have many such tables
in each node.

> Is there any limitations or point when the ETS’s perf degrades?

ETS tables with complex keys can degrade drastically under high
insertion rates, but are Ok when mostly read with with occasional (say
<< 1k/s) insertions.

> I’m on CentOS7, Erlang 20.x & 21.x with 256GB of RAM.

You may want to disable transparent hugepages.  There's also a default
limit to the number of individual memory mappings per Linux process
that you might
run into with larger nodes, we did and it was a nightmare because it
caused random unexpected VM deaths with no diagnostics of the real
culprit.  (Solution:
bump the kernel's max_map_count, but be advised the limit is there
because some memory map operations in the kernel don't scale well to
large address spaces).

/Mikael
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ETS’s performances and limits?

Sverker Eriksson-5
In reply to this post by Valentin Micic-5
On sön, 2018-09-02 at 01:29 +0200, Valentin Micic wrote:
> There is a good chance that performance degradation you are referring to may
> *not* be related to a "heavier concurrency" as much as it may be caused by a
> rate of inserts into a table of type "ordered_set". The manual indicates that
> ordered_set tables do behave differently "in come situations" (which I read as
> "insert operations when a table gets big").
>

The paragraph containing "different behavior in some situations" is about the
semantic differences for ordered_set due to keys compared with arithmetic ==
instead of matching =:= as the other table types.


Performance wise ordered_set use AVL trees with O(log N) for
lookup/insert/delete while the others use linear hashing with O(1).

ordered_set does also not implement option 'write_concurrency'. Every mutating
operation on an ordered_set will seize exclusive write lock on the entire table.
This will most probably improve as we are expecting a very exciting pull request
in a near future implementing 'write_concurrency' for ordered_set.


/Sverker, Erlang/OTP @ Ericsson

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ETS’s performances and limits?

Frank Muller
I’m only interested in ETS tables of type “set” and their performances/limits. 

/Frank

On sön, 2018-09-02 at 01:29 +0200, Valentin Micic wrote:
> There is a good chance that performance degradation you are referring to may
> *not* be related to a "heavier concurrency" as much as it may be caused by a
> rate of inserts into a table of type "ordered_set". The manual indicates that
> ordered_set tables do behave differently "in come situations" (which I read as
> "insert operations when a table gets big").
>

The paragraph containing "different behavior in some situations" is about the
semantic differences for ordered_set due to keys compared with arithmetic ==
instead of matching =:= as the other table types.


Performance wise ordered_set use AVL trees with O(log N) for
lookup/insert/delete while the others use linear hashing with O(1).

ordered_set does also not implement option 'write_concurrency'. Every mutating
operation on an ordered_set will seize exclusive write lock on the entire table.
This will most probably improve as we are expecting a very exciting pull request
in a near future implementing 'write_concurrency' for ordered_set.


/Sverker, Erlang/OTP @ Ericsson

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ETS’s performances and limits?

Chandru-4
You may find this paper interesting.


regards,
Chandru


On Tue, 4 Sep 2018 at 22:52, Frank Muller <[hidden email]> wrote:
I’m only interested in ETS tables of type “set” and their performances/limits. 

/Frank

On sön, 2018-09-02 at 01:29 +0200, Valentin Micic wrote:
> There is a good chance that performance degradation you are referring to may
> *not* be related to a "heavier concurrency" as much as it may be caused by a
> rate of inserts into a table of type "ordered_set". The manual indicates that
> ordered_set tables do behave differently "in come situations" (which I read as
> "insert operations when a table gets big").
>

The paragraph containing "different behavior in some situations" is about the
semantic differences for ordered_set due to keys compared with arithmetic ==
instead of matching =:= as the other table types.


Performance wise ordered_set use AVL trees with O(log N) for
lookup/insert/delete while the others use linear hashing with O(1).

ordered_set does also not implement option 'write_concurrency'. Every mutating
operation on an ordered_set will seize exclusive write lock on the entire table.
This will most probably improve as we are expecting a very exciting pull request
in a near future implementing 'write_concurrency' for ordered_set.


/Sverker, Erlang/OTP @ Ericsson

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions