efficient way to make some resources available as precompiled beams

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

efficient way to make some resources available as precompiled beams

Benoit Chesneau-2
Hi all,

I'm wondering what Is most efficient when you want to make some resource available as precompiled beams. 

Say for example, you have a a resource available as CSV and there is a need for a lookup on this list based on one of column. 

I am for now transforming this list in a list of matching functions (lookup(X1) ->. V1; ... lookup(XN) -> VN.). Ie you have a CSV (simplified version):

    Key1; V1
    Key2; V2
    ...
    KeyN; VN  

which is transformed in the following erlang file, later compiled:

    lookup(Key1) -> V1;
    lookup(Key2) -> V2;
    ...
    lookup(KeyN) -> VN.


But with the new optimisations in Erlang 20/21 sharing literals I am wondering if it wouldn't be more efficient to use a maps there?

So the generated erlang file would be:

    -define(to_lookup_map, #{ Key1 => V1;
                                               Key2 => V2;
                                               ...
                                               KeyN => VN }).

     lookup(Key) -> maps:get(Key, ?to_lookup_map).


A maps would also have some benefit in compressing a little the info I guess.

Thoughts?  Beside that maybe having that ressource in a memory mapped file would be better? I believe that beams are fully loaded in RAM, aren't they?

Benoit


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: efficient way to make some resources available as precompiled beams

Lukas Larsson-8


On Mon, Aug 20, 2018 at 11:31 AM Benoit Chesneau <[hidden email]> wrote:
But with the new optimisations in Erlang 20/21 sharing literals I am wondering if it wouldn't be more efficient to use a maps there?

That optimization should not effect the performance of the lookups at all. It will change how much it will cost to send the values once you have done the lookup, but that should be the same with both approaches.
 

So the generated erlang file would be:

    -define(to_lookup_map, #{ Key1 => V1;
                                               Key2 => V2;
                                               ...
                                               KeyN => VN }).

     lookup(Key) -> maps:get(Key, ?to_lookup_map).


A maps would also have some benefit in compressing a little the info I guess.

Thoughts?  Beside that maybe having that ressource in a memory mapped file would be better? I believe that beams are fully loaded in RAM, aren't they?


The difference in between the two approaches will depend on what the keys are. If they are strings (list or binary) then I doubt that the maps code will be able to beat compiled beam code. If they are integers, then maybe maps will stand a chance, unless the compiler is able to build a jump table of the integers.

There are bound to be cases where the maps approach is faster, but I can't think of any right now.

As always, measure measure measure.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions