lists module functions for maps?

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

lists module functions for maps?

Vance Shipley
Should we have a corollary to lists:keyfind/3 which works on lists of map()?

     keyfind(Key, MapList) -> Map | false

... and the rest:

     keydelete(Key, MapList1) -> MapList2
     keymap(Fun, Key, MapList1) -> MapList2
     keymember(Key, MapList) -> boolean()
     keymerge(Key, MapList1, MapList2) -> MapList3
     keyreplace(Key, MapList1, NewMap) -> MapList2
     keysort(Key, MapList1) -> MapList2
     keystore(Key, MapList1, NewMap) -> MapList2
     keytake(Key, MapList1) -> {value, Map, MapList2} | false


--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Karl Velicka
I'm afraid I can't see the equivalence in your example - I see a list of tuples as a list of values, whereas a list of maps is a list of containers. In my mind, a more apt comparison would be a list of maps vs a list of lists of tuples, and functions operating on the latter would strike me as an odd addition to stdlib.

I'd be curious to know what your usecase for these functions is.

Some of them can be trivially implemented:

keyfind(Key, MapList) -> hd(lists:filter(fun(Map) -> maps:is_key(Key, Map) end, MapList).

(The example implementation there is inefficient but I hope you get the idea)

Some others, like keysort(Key, MapList1) -> MapList2 don't quite make sense to me as the semantics are unclear.


I think elaborating on the use cases would help your proposal.

All the best,
Karl

On Mon, 23 Sep 2019 at 09:00, Vance Shipley <[hidden email]> wrote:
Should we have a corollary to lists:keyfind/3 which works on lists of map()?

     keyfind(Key, MapList) -> Map | false

... and the rest:

     keydelete(Key, MapList1) -> MapList2
     keymap(Fun, Key, MapList1) -> MapList2
     keymember(Key, MapList) -> boolean()
     keymerge(Key, MapList1, MapList2) -> MapList3
     keyreplace(Key, MapList1, NewMap) -> MapList2
     keysort(Key, MapList1) -> MapList2
     keystore(Key, MapList1, NewMap) -> MapList2
     keytake(Key, MapList1) -> {value, Map, MapList2} | false


--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Brujo Benavides-3
There is some historical context at play here, Karl.
Records (which are, in many occasions, the predecessors of maps) are tuples and therefore it’s not uncommon to find code that uses the functions listed by Vance to work with them. For instance…

find_user(UserName, Users) ->
lists:keyfind(UserName, #user.username, Users).

If we move from records to maps as our means to implement users, then we need something like…

find_user(UserName, Users) ->
case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, Users) of
[] -> false;
[User|_] -> User
end.

It would be nice if we don’t need to do that and instead can do…

find_user(UserName, Users) ->
lists:keyfind_for_maps(UserName, username, Users).

Hope this helps :)

On 23 Sep 2019, at 08:07, Karl Velicka <[hidden email]> wrote:

I'm afraid I can't see the equivalence in your example - I see a list of tuples as a list of values, whereas a list of maps is a list of containers. In my mind, a more apt comparison would be a list of maps vs a list of lists of tuples, and functions operating on the latter would strike me as an odd addition to stdlib.

I'd be curious to know what your usecase for these functions is.

Some of them can be trivially implemented:

keyfind(Key, MapList) -> hd(lists:filter(fun(Map) -> maps:is_key(Key, Map) end, MapList).

(The example implementation there is inefficient but I hope you get the idea)

Some others, like keysort(Key, MapList1) -> MapList2 don't quite make sense to me as the semantics are unclear.


I think elaborating on the use cases would help your proposal.

All the best,
Karl

On Mon, 23 Sep 2019 at 09:00, Vance Shipley <[hidden email]> wrote:
Should we have a corollary to lists:keyfind/3 which works on lists of map()?

     keyfind(Key, MapList) -> Map | false

... and the rest:

     keydelete(Key, MapList1) -> MapList2
     keymap(Fun, Key, MapList1) -> MapList2
     keymember(Key, MapList) -> boolean()
     keymerge(Key, MapList1, MapList2) -> MapList3
     keyreplace(Key, MapList1, NewMap) -> MapList2
     keysort(Key, MapList1) -> MapList2
     keystore(Key, MapList1, NewMap) -> MapList2
     keytake(Key, MapList1) -> {value, Map, MapList2} | false


--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Karl Velicka
Ah, of course - that makes sense. I would still like to argue that maps are a good substitute for only some kinds of records and I see both data structures as more orthogonal than predecessor-successor but this isn't the right place for it and I don't want to derail the discussion (plus, I'm sure I'd find myself out of my depth in that discussion rather quickly!).

Thanks for pointing out the obvious that I have somehow missed.

On Mon, 23 Sep 2019 at 14:15, Brujo Benavides <[hidden email]> wrote:
There is some historical context at play here, Karl.
Records (which are, in many occasions, the predecessors of maps) are tuples and therefore it’s not uncommon to find code that uses the functions listed by Vance to work with them. For instance…

find_user(UserName, Users) ->
lists:keyfind(UserName, #user.username, Users).

If we move from records to maps as our means to implement users, then we need something like…

find_user(UserName, Users) ->
case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, Users) of
[] -> false;
[User|_] -> User
end.

It would be nice if we don’t need to do that and instead can do…

find_user(UserName, Users) ->
lists:keyfind_for_maps(UserName, username, Users).

Hope this helps :)

On 23 Sep 2019, at 08:07, Karl Velicka <[hidden email]> wrote:

I'm afraid I can't see the equivalence in your example - I see a list of tuples as a list of values, whereas a list of maps is a list of containers. In my mind, a more apt comparison would be a list of maps vs a list of lists of tuples, and functions operating on the latter would strike me as an odd addition to stdlib.

I'd be curious to know what your usecase for these functions is.

Some of them can be trivially implemented:

keyfind(Key, MapList) -> hd(lists:filter(fun(Map) -> maps:is_key(Key, Map) end, MapList).

(The example implementation there is inefficient but I hope you get the idea)

Some others, like keysort(Key, MapList1) -> MapList2 don't quite make sense to me as the semantics are unclear.


I think elaborating on the use cases would help your proposal.

All the best,
Karl

On Mon, 23 Sep 2019 at 09:00, Vance Shipley <[hidden email]> wrote:
Should we have a corollary to lists:keyfind/3 which works on lists of map()?

     keyfind(Key, MapList) -> Map | false

... and the rest:

     keydelete(Key, MapList1) -> MapList2
     keymap(Fun, Key, MapList1) -> MapList2
     keymember(Key, MapList) -> boolean()
     keymerge(Key, MapList1, MapList2) -> MapList3
     keyreplace(Key, MapList1, NewMap) -> MapList2
     keysort(Key, MapList1) -> MapList2
     keystore(Key, MapList1, NewMap) -> MapList2
     keytake(Key, MapList1) -> {value, Map, MapList2} | false


--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Richard O'Keefe
In reply to this post by Vance Shipley
Keep the function names the same but put them in another module, say 'maps:'.

On Mon, 23 Sep 2019 at 18:01, Vance Shipley <[hidden email]> wrote:

>
> Should we have a corollary to lists:keyfind/3 which works on lists of map()?
>
>      keyfind(Key, MapList) -> Map | false
>
> ... and the rest:
>
>      keydelete(Key, MapList1) -> MapList2
>      keymap(Fun, Key, MapList1) -> MapList2
>      keymember(Key, MapList) -> boolean()
>      keymerge(Key, MapList1, MapList2) -> MapList3
>      keyreplace(Key, MapList1, NewMap) -> MapList2
>      keysort(Key, MapList1) -> MapList2
>      keystore(Key, MapList1, NewMap) -> MapList2
>      keytake(Key, MapList1) -> {value, Map, MapList2} | false
>
>
> --
>      -Vance
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Andreas Schultz-3
In reply to this post by Brujo Benavides-3
Am Mo., 23. Sept. 2019 um 13:15 Uhr schrieb Brujo Benavides <[hidden email]>:
There is some historical context at play here, Karl.
Records (which are, in many occasions, the predecessors of maps) are tuples and therefore it’s not uncommon to find code that uses the functions listed by Vance to work with them. For instance…

find_user(UserName, Users) ->
lists:keyfind(UserName, #user.username, Users).

If we move from records to maps as our means to implement users, then we need something like…

find_user(UserName, Users) ->
case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, Users) of
[] -> false;
[User|_] -> User
end.
 
It would be nice if we don’t need to do that and instead can do…

find_user(UserName, Users) ->
lists:keyfind_for_maps(UserName, username, Users).

Your example doesn't make sense to me. Why don't you simply use a map with the username as key?

Maps are a Key-Value stores. If you need secondary indices, you can always maintain additional maps for them. Non-unique secondary keys can be implemented by using a maps as value. Or use an ets where you can match on additional fields.
Adding a linear search like lists:keyfind to maps make no sense in my opinion.

Talking about ets, adding a match like API to maps might make sense. If only to make it possible to replace ets with maps without having to change to much code.

Regards
Andreas
 
[...]

--

Andreas Schultz



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Karolis Petrauskas-2
You can use a match spec to filter the list (http://erlang.org/doc/man/ets.html#match_spec_run-2).
Pattern matching works well with lists of maps.
That's in the case, if you need to keep your data in a list instead of map (eg. by username).

Karolis

On Mon, Sep 23, 2019 at 2:55 PM Andreas Schultz <[hidden email]> wrote:
Am Mo., 23. Sept. 2019 um 13:15 Uhr schrieb Brujo Benavides <[hidden email]>:
There is some historical context at play here, Karl.
Records (which are, in many occasions, the predecessors of maps) are tuples and therefore it’s not uncommon to find code that uses the functions listed by Vance to work with them. For instance…

find_user(UserName, Users) ->
lists:keyfind(UserName, #user.username, Users).

If we move from records to maps as our means to implement users, then we need something like…

find_user(UserName, Users) ->
case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, Users) of
[] -> false;
[User|_] -> User
end.
 
It would be nice if we don’t need to do that and instead can do…

find_user(UserName, Users) ->
lists:keyfind_for_maps(UserName, username, Users).

Your example doesn't make sense to me. Why don't you simply use a map with the username as key?

Maps are a Key-Value stores. If you need secondary indices, you can always maintain additional maps for them. Non-unique secondary keys can be implemented by using a maps as value. Or use an ets where you can match on additional fields.
Adding a linear search like lists:keyfind to maps make no sense in my opinion.

Talking about ets, adding a match like API to maps might make sense. If only to make it possible to replace ets with maps without having to change to much code.

Regards
Andreas
 
[...]

--

Andreas Schultz


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Vance Shipley
In reply to this post by Richard O'Keefe
On Mon, Sep 23, 2019 at 7:45 PM Richard O'Keefe <[hidden email]> wrote:
> Keep the function names the same but put them in another module, say 'maps:'.

I would argue that as functions which operate on lists they belong in
the lists module.

--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Vance Shipley
In reply to this post by Karl Velicka
On Mon, Sep 23, 2019 at 7:07 PM Karl Velicka <[hidden email]> wrote:
> I'm afraid I can't see the equivalence in your example

Think small maps and big lists.

> I'd be curious to know what your usecase for these functions is.

As Brujo suggests many use cases for lists:keyfind/3 et. al. deal with
lists of records. This is a very common pattern in our organization.
As we use maps more we naturally want to have the simple efficient
(BIF) implementation in the lists module.

One example is with JSON where we have an object which contains an
array of objects which we want to filter, sort, get, take, etc..

     1> #{id := Id, name := Name, characteristic := Chars} = zj:decode(Body),
     1> length(Chars).
     100
     2> hd(Chars).
     #{id = 42, name = foo, type = bar, value = "hello"}.

Sure I can always write simple funs to handle these but we have
collected common patterns of operations on lists into the lists module
and made it very fast. It seems quite obvious to add support for the
new type of term which may be in those lists.


--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Richard O'Keefe
In reply to this post by Vance Shipley
I would argue that the focus of these functions is the maps, not the
lists they happen to be inside.

Perhaps the whole thing is inside out.  Perhaps what's really needed
is something that says
 - here is a list of maps and a key
 - give me a map of lists and a list
 If {M,R} = maps:transpose(Ms, K)
 then R is the filter of Ms by the condition "does not have key K"
 and M is a map from V to the filter of Ms by the condition "has V as
value for key K".
Only do the linear scan once.

On Tue, 24 Sep 2019 at 00:18, Vance Shipley <[hidden email]> wrote:

>
> On Mon, Sep 23, 2019 at 7:45 PM Richard O'Keefe <[hidden email]> wrote:
> > Keep the function names the same but put them in another module, say 'maps:'.
>
> I would argue that as functions which operate on lists they belong in
> the lists module.
>
> --
>      -Vance
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Vance Shipley
On Tue, Sep 24, 2019 at 10:48 AM Richard O'Keefe <[hidden email]> wrote:
> I would argue that the focus of these functions is the maps, not the lists they happen to be inside.

In my mind this is all about lists. Think small maps and big lists.
That the terms are maps is a detail I have to deal with in processing
my lists. I just want the lists module to be orthogonal. If we aren't
going to move all the lists:key* functions to a 'records' module
(please don't) let's simply add support for the new term type which
"replaces" records.

Your example is certainly all about maps but it doesn't help my use cases.

> Perhaps the whole thing is inside out.  Perhaps what's really needed
> is something that says
>  - here is a list of maps and a key
>  - give me a map of lists and a list
>  If {M,R} = maps:transpose(Ms, K)
>  then R is the filter of Ms by the condition "does not have key K"
>  and M is a map from V to the filter of Ms by the condition "has V as
> value for key K".
> Only do the linear scan once.


--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Dmitry Belyaev-3
If it's all about lists, you've got all you need: lists:search/2 or as previously mentioned lists:dropwhile.

To me presence of lists:key... functions is a serious legacy which is justified only because there are tons of code using it. I'm glad we don't have all of the 'sofs' functions in 'lists' module.
Of course it would be nice to have faster versions than handcrafted functions, but I'd like them to be introduced in a separate module, e.g. list_of_maps.

On 24 September 2019 3:32:55 pm AEST, Vance Shipley <[hidden email]> wrote:
On Tue, Sep 24, 2019 at 10:48 AM Richard O'Keefe <[hidden email]> wrote:
I would argue that the focus of these functions is the maps, not the lists they happen to be inside.

In my mind this is all about lists. Think small maps and big lists.
That the terms are maps is a detail I have to deal with in processing
my lists. I just want the lists module to be orthogonal. If we aren't
going to move all the lists:key* functions to a 'records' module
(please don't) let's simply add support for the new term type which
"replaces" records.

Your example is certainly all about maps but it doesn't help my use cases.

Perhaps the whole thing is inside out. Perhaps what's really needed
is something that says
- here is a list of maps and a key
- give me a map of lists and a list
If {M,R} = maps:transpose(Ms, K)
then R is the filter of Ms by the condition "does not have key K"
and M is a map from V to the filter of Ms by the condition "has V as
value for key K".
Only do the linear scan once.


--
Kind regards,
Dmitry Belyaev
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Karl Velicka
I agree with Dmitry - if the pattern is common enough then a new module for dealing with such structures seems like the best way forward. Invisible to those that don't need it but available and performant to those that do.

Karl

On Tue, 24 Sep 2019, 15:11 Dmitry Belyaev, <[hidden email]> wrote:
If it's all about lists, you've got all you need: lists:search/2 or as previously mentioned lists:dropwhile.

To me presence of lists:key... functions is a serious legacy which is justified only because there are tons of code using it. I'm glad we don't have all of the 'sofs' functions in 'lists' module.
Of course it would be nice to have faster versions than handcrafted functions, but I'd like them to be introduced in a separate module, e.g. list_of_maps.

On 24 September 2019 3:32:55 pm AEST, Vance Shipley <[hidden email]> wrote:
On Tue, Sep 24, 2019 at 10:48 AM Richard O'Keefe <[hidden email]> wrote:
I would argue that the focus of these functions is the maps, not the lists they happen to be inside.

In my mind this is all about lists. Think small maps and big lists.
That the terms are maps is a detail I have to deal with in processing
my lists. I just want the lists module to be orthogonal. If we aren't
going to move all the lists:key* functions to a 'records' module
(please don't) let's simply add support for the new term type which
"replaces" records.

Your example is certainly all about maps but it doesn't help my use cases.

Perhaps the whole thing is inside out. Perhaps what's really needed
is something that says
- here is a list of maps and a key
- give me a map of lists and a list
If {M,R} = maps:transpose(Ms, K)
then R is the filter of Ms by the condition "does not have key K"
and M is a map from V to the filter of Ms by the condition "has V as
value for key K".
Only do the linear scan once.


--
Kind regards,
Dmitry Belyaev
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

Vance Shipley
In reply to this post by Dmitry Belyaev-3
On Tue, Sep 24, 2019, 8:11 PM Dmitry Belyaev <[hidden email]> wrote:
To me presence of lists:key... functions is a serious legacy which is justified only because there are tons of code using it.

That is an argument I could accept, if not agree 

Of course it would be nice to have faster versions than handcrafted functions, but I'd like them to be introduced in a separate module, e.g. list_of_maps.

I would withdraw my suggestion at that point.
Not because your approach is wrong, just because I don't want to make this a bigger issue than the simple question I raised.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: lists module functions for maps?

zxq9-2
In reply to this post by Dmitry Belyaev-3
On 2019/09/24 21:10, Dmitry Belyaev wrote:
> To me presence of lists:key... functions is a serious legacy which is
> justified only because there are tons of code using it.

Lists of records are used all over the place in new code as well and the
lists:key*/N functions are the standard way of dealing with them. Some
shops have simply forgotten that maps are NOT a replacement for records.

The only strange thing about those functions is that they are in the
lists module instead of some module written specifically for
manipulating lists of records.

Lists of maps fall into the same category. It is easy to see how lists
of maps would manifest many places when dealing with data from external
systems (JSON, XML, Python dicts, YAML, etc.). This probably doesn't
belong in the lists module, but something available somewhere would
reduce the noisiness of the code needed to be written for consumers of
that kind of data.

I was stuck at home today sick, so wrote this as a starting
point/substitute/copypasta for anyone who might find it useful:
https://gitlab.com/zxq9/lom/blob/master/src/lom.erl

lom.erl exports
[new/0, store/3, take/3, replace/3, delete/3, find/3, member/3,
keymap/3, sort/2, merge/3]

And as Vance noted earlier, is a very natural thing to pair with a JSON
decoder. The typespec doesn't match Vance's proposal in every case but
only because in some cases you must provide both the key's name and its
value.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions