Store maps in ets/dets?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Max Lapshin
Maybe I have missed something and failed to google this info, but are there
any plans to store maps in ets?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140410/65d07e94/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Jesper Louis Andersen
The semantics of maps is that of a persistent data type. ETS is ephemeral.
So the fit is not good. Furthermore, maps are living int he process heap
and thus, they are automatically garbage collected. ETS is manually
collected in contrast. So the fit is even worse.

Maps currently has no order (based on maps:map/2 and maps:fold/3), which
means they can be implemented as HAMTs (like in Clojure) once they grow
large. This has very good runtime behaviour, while keeping the process
semantics intact. The other option is to give maps order, but then we can't
use digital searching techniques anymore and must resort to balanced search
trees, e.g., AVL or RB trees and friends.

The current representation is very memory efficient however. It will
probably be very good for small maps with a keyset that doesn't change.


On Thu, Apr 10, 2014 at 10:41 AM, Max Lapshin <max.lapshin> wrote:

> Maybe I have missed something and failed to google this info, but are
> there any plans to store maps in ets?
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions
> http://erlang.org/mailman/listinfo/erlang-questions
>
>


--
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140410/0f255d9b/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Michael Loftis
On Thu, Apr 10, 2014 at 2:09 AM, Jesper Louis Andersen
<jesper.louis.andersen> wrote:
> The semantics of maps is that of a persistent data type. ETS is ephemeral.
> So the fit is not good. Furthermore, maps are living int he process heap and
> thus, they are automatically garbage collected. ETS is manually collected in
> contrast. So the fit is even worse.
>

I think you might mean the opposite here....that maps is ephemeral and
ETS is persistent (or can be persistent using DETS anyway)

> Maps currently has no order (based on maps:map/2 and maps:fold/3), which
> means they can be implemented as HAMTs (like in Clojure) once they grow
> large. This has very good runtime behaviour, while keeping the process
> semantics intact. The other option is to give maps order, but then we can't
> use digital searching techniques anymore and must resort to balanced search
> trees, e.g., AVL or RB trees and friends.
>
> The current representation is very memory efficient however. It will
> probably be very good for small maps with a keyset that doesn't change.
>
>
> On Thu, Apr 10, 2014 at 10:41 AM, Max Lapshin <max.lapshin> wrote:
>>
>> Maybe I have missed something and failed to google this info, but are
>> there any plans to store maps in ets?
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
>
> --
> J.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions
> http://erlang.org/mailman/listinfo/erlang-questions
>



--

"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Björn-Egil Dahlberg-3
On 2014-04-10 18:14, Michael Loftis wrote:
> On Thu, Apr 10, 2014 at 2:09 AM, Jesper Louis Andersen
> <jesper.louis.andersen> wrote:
>> The semantics of maps is that of a persistent data type. ETS is ephemeral.
>> So the fit is not good. Furthermore, maps are living int he process heap and
>> thus, they are automatically garbage collected. ETS is manually collected in
>> contrast. So the fit is even worse.
>>
> I think you might mean the opposite here....that maps is ephemeral and
> ETS is persistent (or can be persistent using DETS anyway)

I don't like the word ephemeral here =)

Persistent is used all over to describe Maps as immutable. Though if you
have a database background the meaning of "persistent" is of course
another. Describing ETS as persistent is stretching it a bit far.

Maps are copied on update (atleast the values are copied) i.e. Maps are
immutable and in some circles this is equivalent to persistent.

// Bj?rn-Egil

>
>> Maps currently has no order (based on maps:map/2 and maps:fold/3), which
>> means they can be implemented as HAMTs (like in Clojure) once they grow
>> large. This has very good runtime behaviour, while keeping the process
>> semantics intact. The other option is to give maps order, but then we can't
>> use digital searching techniques anymore and must resort to balanced search
>> trees, e.g., AVL or RB trees and friends.
>>
>> The current representation is very memory efficient however. It will
>> probably be very good for small maps with a keyset that doesn't change.
>>
>>
>> On Thu, Apr 10, 2014 at 10:41 AM, Max Lapshin <max.lapshin> wrote:
>>> Maybe I have missed something and failed to google this info, but are
>>> there any plans to store maps in ets?
>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>
>>
>> --
>> J.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Jesper Louis Andersen
In reply to this post by Michael Loftis
On Thu, Apr 10, 2014 at 6:14 PM, Michael Loftis <mloftis> wrote:

> I think you might mean the opposite here....that maps is ephemeral and
> ETS is persistent (or can be persistent using DETS anyway)
>

Actually not. Two things to address though:

Max might have asked if you can store maps in ets:

5> ets:new(foo, [named_table]).
foo
6> ets:insert(foo, {#{a => 3, b => 4}}).
true
7> ets:match_object(foo, '_').
[{#{a => 3,b => 4}}]
8> ets:insert(foo, #{'not' => possible}).
** exception error: bad argument
     in function  ets:insert/2
        called as ets:insert(foo,#{'not' => possible})

and the answer is that you can, but the topmost entry must be a tuple as
the example shows.

The other thing is about persistent vs ephemeral data structures.

http://en.wikipedia.org/wiki/Persistent_data_structure

Maps are persistent, but ETS in general is not. It is ephemeral according
to the wikipedia definition.




--
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140410/e2b2c40d/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Michael Loftis
In reply to this post by Björn-Egil Dahlberg-3
On Thu, Apr 10, 2014 at 9:33 AM, Bj?rn-Egil Dahlberg <egil> wrote:

> On 2014-04-10 18:14, Michael Loftis wrote:
>> I think you might mean the opposite here....that maps is ephemeral and
>> ETS is persistent (or can be persistent using DETS anyway)
>
>
> I don't like the word ephemeral here =)
>
> Persistent is used all over to describe Maps as immutable. Though if you
> have a database background the meaning of "persistent" is of course another.
> Describing ETS as persistent is stretching it a bit far.
>
> Maps are copied on update (atleast the values are copied) i.e. Maps are
> immutable and in some circles this is equivalent to persistent.

Immutable is definitely the word...ephemeral definitely isn't the best
fit - and they're most definitely not persistent if they're
in-process.

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Max Lapshin
I'm asking exactly about

 ets:insert(web_sessions, #{session_id => <<"1234">>,bytes => 0}).

ets:update_counter(web_sessions, <<"1234">>, {bytes, 150})
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140411/25e41e17/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Matthias Lang-2
In reply to this post by Michael Loftis

> On Thu, Apr 10, 2014 at 9:33 AM, Bj?rn-Egil Dahlberg <egil> wrote:

> > Persistent is used all over to describe Maps as immutable. Though if you
> > have a database background the meaning of "persistent" is of course another.
...

On Thursday, April 10, Michael Loftis wrote:

> and they're most definitely not persistent if they're in-process.

'persistent' means (at least) two different things. Here are the wikipedia
pages on those two different things:

  http://en.wikipedia.org/wiki/Persistence_%28computer_science%29
  http://en.wikipedia.org/wiki/Persistent_data_structure

When you write "[maps] are most definitely not persistent", I think
you're using the first definition. Bj?rn-Egil indicated that he was
using the second.

Matt

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Robert Virding-6
In reply to this post by Max Lapshin
Then it is a Jesper pointed out: yes, you can store maps in an ETS table,
but at the top-most structure must be a tuple. So you will not be able to
write as you have done.

Robert



On 11 April 2014 06:31, Max Lapshin <max.lapshin> wrote:

> I'm asking exactly about
>
>  ets:insert(web_sessions, #{session_id => <<"1234">>,bytes => 0}).
>
> ets:update_counter(web_sessions, <<"1234">>, {bytes, 150})
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140414/544919b8/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Store maps in ets/dets?

Daniel Goertzen-4
In reply to this post by Max Lapshin
Direct map support in ets, dets, and mnesia would indeed be nice.  I am
currently using a number of mnesia tables to pack maps into tuples with the
key copied into the first tuple spot:


#{key => 123, value1 => abc, value2 => 3.14, ...}

... becomes...

{123, #{key => 123, value1 => abc, value2 => 3.14, ...}}

Dan.



On Thu, Apr 10, 2014 at 11:31 PM, Max Lapshin <max.lapshin> wrote:

> I'm asking exactly about
>
>  ets:insert(web_sessions, #{session_id => <<"1234">>,bytes => 0}).
>
> ets:update_counter(web_sessions, <<"1234">>, {bytes, 150})
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140414/01342e60/attachment.html>