Best choice for complex data structure

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Best choice for complex data structure

Donald Steven
What, in your experience, is the best choice (for  set, get, update,
etc.) for a complex data structure that's nested three or four levels deep?
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Best choice for complex data structure

zxq9-2
On 2019年1月23日水曜日 21時52分28秒 JST Donald Steven wrote:
> What, in your experience, is the best choice (for  set, get, update,
> etc.) for a complex data structure that's nested three or four levels deep?

Hi, Donald.

It depends entirely on:
- What kind of data it is
- What the program is doing with the data

Could you provide a little more context?
Explain your problem. Describe the context. Then people can give good answers.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Best choice for complex data structure

Dmitry Kolesnikov-2
I found that lenses gives reasonable good abstraction to deal with deeply nested structures.

Best Regards,
Dmitry


> On 24 Jan 2019, at 12.19, [hidden email] wrote:
>
> On 2019年1月23日水曜日 21時52分28秒 JST Donald Steven wrote:
>> What, in your experience, is the best choice (for  set, get, update,
>> etc.) for a complex data structure that's nested three or four levels deep?
>
> Hi, Donald.
>
> It depends entirely on:
> - What kind of data it is
> - What the program is doing with the data
>
> Could you provide a little more context?
> Explain your problem. Describe the context. Then people can give good answers.
>
> -Craig
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Best choice for complex data structure

zxq9-2
On 2019年1月24日木曜日 12時43分52秒 JST you wrote:
> I found that lenses gives reasonable good abstraction to deal with deeply nested structures.

That said, I find that very often things like lenses are "chindougu": Unuseless inventions.
They solve problems that would never have existed had more thought been given to the actual shape of data within the program in question.

That's not always true, but quite often it is.

Come to think of it, much of the most heavily hyped efforts in software are chindougu.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Best choice for complex data structure

Donald Steven

I'm trying to figure out how to shape the (musical) data.  In everyday language, a musical idea has an architecture of formal nested units and gestures made up of one or more events which have sounds and silences.  In Rust, the latter part of the above would yield a statement like:

scoreIdea.gesture[iGesture-1].event[iEvent-1].sounds += 1

which is manageable.  But when I think about accessing data nested many levels deep in Erlang, I'm kinda flummoxed. Instinctively, I don't think transliterating something like this to records (or even maps?) is the way to go?

All suggestions greatly appreciated.

On 1/24/2019 6:11 AM, [hidden email] wrote:
On 2019年1月24日木曜日 12時43分52秒 JST you wrote:
I found that lenses gives reasonable good abstraction to deal with deeply nested structures.
That said, I find that very often things like lenses are "chindougu": Unuseless inventions.
They solve problems that would never have existed had more thought been given to the actual shape of data within the program in question.

That's not always true, but quite often it is.

Come to think of it, much of the most heavily hyped efforts in software are chindougu.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Best choice for complex data structure

zxq9-2
On 2019年1月24日木曜日 6時37分03秒 JST you wrote:

> I'm trying to figure out how to shape the (musical) data.  In everyday
> language, a musical idea has an architecture of formal nested units and
> gestures made up of one or more events which have sounds and silences.  
> In Rust, the latter part of the above would yield a statement like:
>
> scoreIdea.gesture[iGesture-1].event[iEvent-1].sounds +=1
>
> which is manageable.  But when I think about accessing data nested many
> levels deep in Erlang, I'm kinda flummoxed. Instinctively, I don't think
> transliterating something like this to records (or even maps?) is the
> way to go?
>
> All suggestions greatly appreciated.

This kind of context is extremely useful to know.

We have some canonical examples of musical data and quite a lot of it is not nested, but rather heavily attributed (includes a lot of meta, but the data itself is flat).

But that is how notations work for musical players (MIDI being a common example).
If you want to capture musical architecture, though, that is something a bit different...
I don't think we've done very well coming up with notations for musical structure in writing, much less computer data.

Overlays and segments for mixes is the best I can think of -- where you store a MIDI sequence along with some meta that identifies it by a useful set of categories (sort of like "tags" on blog posts) and whatever other sort of data might be useful, and then pick from a library of MIDI overlays or segments to insert into an ongoing composition ("flourish X overlays the melody here, segment Y provides a bridge there", etc.).

For maintaining a library of selectable inserts and references you could use simple lists of tuples as the canonical store, and then build reference indexes on them as maps.

CanonicalData :: [{segment, Title :: string(), Authors :: [string()], Date :: date(), Tags :: [string()], MIDI_Data :: binary()}]

TagIndex :: #{Tag := [Title]}

AuthorIndex :: #{Author := [Title]}

etc.

Keeping in mind, of course, that some data element (or combination) will have to constitute a unique key for the data.

I suppose the key question here would be whether you expect this data to be useful for rendering the music in a player, displaying visually in a composition explorer (which could possibly export combined snippets in a flatter, playable format), or interfacing with an existing music tool (Casio/Yamaha/Roland/etc. MIDI interfaces, sheet music PDF renderers, etc.). Any idea on that?

Scores can themselves be compositions of these elements, actually:

Score :: [{InsertTime, Title, SegmentStart, SegmentStop, MIDI_Data, RepeatMeta}]

So that you can move the pieces around in chunks and use the InsertTime as the offset from 0 (in partial beats, milliseconds, whatever meter you want to use) to determine when it first occurs in the piece, the Title to reference which MIDI data should be pulled from the library to include, the SegmentStart as the start of the segment within the MIDI you want to use, and the SegmentStop as the final part of the MIDI you want to use, and RepeatMeta as an optional value (usually something like RepeatMeta :: none | {forever | Count, BreakDelay | BeatIndex}) that can indicate whether a particular segment repeats, how much the delay between occurrences should be (or what beat index within the next opening measure after prior completion it should recur, etc.).

Blah blah...

Anyway, this all depends on what you want the tool to do. Musical scoring in software can get pretty complicated (or sometimes be really simple) depending on what you hope to get out of it.

Hopefully the wandering blather above gives you some ideas.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Best choice for complex data structure

zxq9-2
On 2019年1月24日木曜日 11時08分09秒 JST you wrote:

> Hi Craig,
>
> Thanks so much for sharing this.  You're very generous.
>
> I have a fully functional MIDI module with a tested and effective set of
> low level IO.  I'm working now at the meta level (as a composer),
> looking to translate some impressionistic higher level ideas through
> intermediary functions to MIDI output.
>
> I'm also just learning Erlang, so I'm sure some of my questions are very
> naive.
>
> What is:
>
> Score :: [{InsertTime, Title, SegmentStart, SegmentStop, MIDI_Data,
> RepeatMeta}]
>
> Is Score a variable -- or a type that is a list of tuples?

Nice! Welcome to Erlang!
Once your eyes adjust I think you'll thoroughly enjoy it around here. :-D

And yes, you are spot on, `Score` above is a type specification.
You'll find this helpful:
http://erlang.org/doc/reference_manual/typespec.html

It is the required notation when talking to Dialyzer (obviously) but also useful when communicating amongst Erlangers.
I gave quite a few type specification examples in the original message, so that link will help you decipher them.

Everything capitalized is a possible type (a type variable that I left undefined -- use your imagination about what type of values might be useful in those places) and everything that is lower-case is an actual definite type. Parens after a lower-case string are a defined type, an atom with no parens is just the atom. A pipe (like `|`) is an "or".

To get more specific...


  Score :: [{InsertTime, Title, SegmentStart, SegmentStop, MIDI_Data, RepeatMeta}]
    where InsertTime   :: time_unit(),
          Title        :: string(),
          SegmentStart :: time_unit(),
          SegmentStop  :: time_unit(),
          MIDI_Data    :: binary(),
          RepeatMeta   :: none | repeat_meta().

  -type time_unit() :: non_neg_integer() % an integer value in milliseconds representing absolute time
                     | {time_signature(), Beats :: non_neg_integer()}.


The above isn't 100% proper type notation (the way Score is defined looks like a -spec, not a -type, but it is a useful notation amongst erlangers, anyway), and I never defined what `repeat_meta()` should be, but should convey some general idea what I intended.

The point in expressing things this way is that a list of MIDI segments (which already carry their own musical data, relative to themselves) can be included as data into a higher-level program with the addition of metadata that is meaningful to that higher-level program, and the resulting list of MIDI segment points (some composed locally, some referenced from a library -- the origin not being important) can be synthesized by a program to produce a new MIDI output readable by a musical notation output program that reads MIDI, or by a player.

That may or may not be what you're looking for (maybe you want to write a musical synthesis program that focuses specifically on musical architecture, in which case your real focus is the segment meta and arrangements among the segments, not the actual MIDI data), but hopefully this gives you some ideas for thinking of data as more than just nested data or OOP-style "objects".

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Best choice for complex data structure

Joe Armstrong-2
Musical representation is *very* complex - I've also written several
libraries to manipulate midi
and turn note representations into SVG - had you thought about basing
your representation on musicXML - sooner or later you might want to
talk to a program that speaks musicXML.

I have drivers for coreaudio that run on a mac that interface erlang
to any midi device
connected to a mac - if anybody is interested I can make these available.

Cheers

/Joe


On Thu, Jan 24, 2019 at 7:01 PM <[hidden email]> wrote:

>
> On 2019年1月24日木曜日 11時08分09秒 JST you wrote:
> > Hi Craig,
> >
> > Thanks so much for sharing this.  You're very generous.
> >
> > I have a fully functional MIDI module with a tested and effective set of
> > low level IO.  I'm working now at the meta level (as a composer),
> > looking to translate some impressionistic higher level ideas through
> > intermediary functions to MIDI output.
> >
> > I'm also just learning Erlang, so I'm sure some of my questions are very
> > naive.
> >
> > What is:
> >
> > Score :: [{InsertTime, Title, SegmentStart, SegmentStop, MIDI_Data,
> > RepeatMeta}]
> >
> > Is Score a variable -- or a type that is a list of tuples?
>
> Nice! Welcome to Erlang!
> Once your eyes adjust I think you'll thoroughly enjoy it around here. :-D
>
> And yes, you are spot on, `Score` above is a type specification.
> You'll find this helpful:
> http://erlang.org/doc/reference_manual/typespec.html
>
> It is the required notation when talking to Dialyzer (obviously) but also useful when communicating amongst Erlangers.
> I gave quite a few type specification examples in the original message, so that link will help you decipher them.
>
> Everything capitalized is a possible type (a type variable that I left undefined -- use your imagination about what type of values might be useful in those places) and everything that is lower-case is an actual definite type. Parens after a lower-case string are a defined type, an atom with no parens is just the atom. A pipe (like `|`) is an "or".
>
> To get more specific...
>
>
>   Score :: [{InsertTime, Title, SegmentStart, SegmentStop, MIDI_Data, RepeatMeta}]
>     where InsertTime   :: time_unit(),
>           Title        :: string(),
>           SegmentStart :: time_unit(),
>           SegmentStop  :: time_unit(),
>           MIDI_Data    :: binary(),
>           RepeatMeta   :: none | repeat_meta().
>
>   -type time_unit() :: non_neg_integer() % an integer value in milliseconds representing absolute time
>                      | {time_signature(), Beats :: non_neg_integer()}.
>
>
> The above isn't 100% proper type notation (the way Score is defined looks like a -spec, not a -type, but it is a useful notation amongst erlangers, anyway), and I never defined what `repeat_meta()` should be, but should convey some general idea what I intended.
>
> The point in expressing things this way is that a list of MIDI segments (which already carry their own musical data, relative to themselves) can be included as data into a higher-level program with the addition of metadata that is meaningful to that higher-level program, and the resulting list of MIDI segment points (some composed locally, some referenced from a library -- the origin not being important) can be synthesized by a program to produce a new MIDI output readable by a musical notation output program that reads MIDI, or by a player.
>
> That may or may not be what you're looking for (maybe you want to write a musical synthesis program that focuses specifically on musical architecture, in which case your real focus is the segment meta and arrangements among the segments, not the actual MIDI data), but hopefully this gives you some ideas for thinking of data as more than just nested data or OOP-style "objects".
>
> -Craig
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions