Spurious '100' in array:new

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Spurious '100' in array:new

Donald Steven
When I declare a new fixed-size array (of 19 values) within a map, with the statement:

Sound = #{

    contourTag =>  SndContour   = array:new(19, {default,{0, unspecified, 0}}),
    ...

},

and print it out using io:format and ~p, I get:

Sound: #{contourTag => {array,19,0,{0,unspecified,0},100},

---

Where is the '100' coming from?

Thanks.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Spurious '100' in array:new

Fred Youhanaie-2
That's part of the internal structure of the array module, the data structure is opaque and should not be relied upon.

If you need to access/print the data contents of the array use array:to_list(Array).

If you're curious about the internal structure of the array, you're best option is to read the source code itself. You can also experiment with creating arrays of various sizes and assigning values to
some elements. Try sizes of 9, 19, 119, etc.

Cheers,
Fred

On 26/01/2019 16:10, Donald Steven wrote:

> When I declare a new fixed-size array (of 19 values) within a map, with the statement:
>
> Sound = #{
>
>      contourTag =>  SndContour   = array:new(19, {default,{0, unspecified, 0}}),
>      ...
>
> },
>
> and print it out using io:format and ~p, I get:
>
> Sound: #{contourTag => {array,19,0,{0,unspecified,0},100},
>
> ---
>
> Where is the '100' coming from?
>
> Thanks.
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Spurious '100' in array:new

Luke Bakken-2
In reply to this post by Donald Steven
Hi Donald -

I have never used the array data type, but I assume that the 100 is an
internal implementation detail.

https://github.com/erlang/otp/blob/OTP-21.2.4/lib/stdlib/src/array.erl#L151-L160

Notice that the last element in the array record is for "elements",
which are defined immediately above the record in the source code.
non_neg_integer() can be the sole element in your array, and is
determined by this code:

https://github.com/erlang/otp/blob/OTP-21.2.4/lib/stdlib/src/array.erl#L286-L298

In your case, the ?extend macro is evaluated and the result of 10 *
(NODESIZE * LEAFSIZE) is stored in the elements field of the record.

At least, I think this is the correct reading of this code :-)
Thanks,
Luke

On Sat, Jan 26, 2019 at 8:11 AM Donald Steven <[hidden email]> wrote:

>
> When I declare a new fixed-size array (of 19 values) within a map, with the statement:
>
> Sound = #{
>
>     contourTag =>  SndContour   = array:new(19, {default,{0, unspecified, 0}}),
>     ...
>
> },
>
> and print it out using io:format and ~p, I get:
>
> Sound: #{contourTag => {array,19,0,{0,unspecified,0},100},
>
> ---
>
> Where is the '100' coming from?
>
> Thanks.
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Spurious '100' in array:new

Luke Bakken-2
I left an extra 10 in my explanation, it should have read:

In your case, the ?extend macro is evaluated and the result of
(NODESIZE * LEAFSIZE) is stored in the elements field of the record.

On Mon, Jan 28, 2019 at 7:36 AM Luke Bakken <[hidden email]> wrote:

>
> Hi Donald -
>
> I have never used the array data type, but I assume that the 100 is an
> internal implementation detail.
>
> https://github.com/erlang/otp/blob/OTP-21.2.4/lib/stdlib/src/array.erl#L151-L160
>
> Notice that the last element in the array record is for "elements",
> which are defined immediately above the record in the source code.
> non_neg_integer() can be the sole element in your array, and is
> determined by this code:
>
> https://github.com/erlang/otp/blob/OTP-21.2.4/lib/stdlib/src/array.erl#L286-L298
>
> In your case, the ?extend macro is evaluated and the result of 10 *
> (NODESIZE * LEAFSIZE) is stored in the elements field of the record.
>
> At least, I think this is the correct reading of this code :-)
> Thanks,
> Luke
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Spurious '100' in array:new

Jesper Louis Andersen-2
In reply to this post by Fred Youhanaie-2
On Sat, Jan 26, 2019 at 6:29 PM Fred Youhanaie <[hidden email]> wrote:
That's part of the internal structure of the array module, the data structure is opaque and should not be relied upon.


For full reference, you can glean on the internal structure:

Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]

Eshell V10.2.1  (abort with ^G)
1> rr(array).
[array]
2> array:new(19, {default, {0, unspecified, 0}}).
#array{size = 19,max = 0,
       default = {0,unspecified,0},
       elements = 100}
3> array:new(9, {default, {0, unspecified, 0}}).
#array{size = 9,max = 0,
       default = {0,unspecified,0},
       elements = 10}

Here is what is happening: In Erlang, an Array is a very flat tree with a branching factor of 10. It is stored in `elements`. But instead of storing a tree, you may store an integer encoding how much more space is allowed in this part of the tree when it is expanded. You have created a tree of size 19, which means it needs two expansions of size 10, or 100---though not all of those expansions will be used, as it is extra padding. If you had created an array of size 101, it would have been 1000, and so on. It is always the next log10 ceiling value. In the example with size 9, only one expansion is needed in which case we have a `10` in the elements.

You might ask why it isn't represented as a traditional array. This is because of persistence in data structures: older versions of the tree can be kept alive, so you cannot overwrite an older array. It is useful in a number of situations, the most important one being that you can know the state of a crashing process before it crashed, and what event led to said crash.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Spurious '100' in array:new

Donald Steven
Thanks Jesper, that's most helpful to know.  Is there a particular way to print it in io:format that will *not* show the 'extra stuff'. (I used ~p)

Best,

Don

On 1/28/2019 11.01 AM, Jesper Louis Andersen wrote:
On Sat, Jan 26, 2019 at 6:29 PM Fred Youhanaie <[hidden email]> wrote:
That's part of the internal structure of the array module, the data structure is opaque and should not be relied upon.


For full reference, you can glean on the internal structure:

Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]

Eshell V10.2.1  (abort with ^G)
1> rr(array).
[array]
2> array:new(19, {default, {0, unspecified, 0}}).
#array{size = 19,max = 0,
       default = {0,unspecified,0},
       elements = 100}
3> array:new(9, {default, {0, unspecified, 0}}).
#array{size = 9,max = 0,
       default = {0,unspecified,0},
       elements = 10}

Here is what is happening: In Erlang, an Array is a very flat tree with a branching factor of 10. It is stored in `elements`. But instead of storing a tree, you may store an integer encoding how much more space is allowed in this part of the tree when it is expanded. You have created a tree of size 19, which means it needs two expansions of size 10, or 100---though not all of those expansions will be used, as it is extra padding. If you had created an array of size 101, it would have been 1000, and so on. It is always the next log10 ceiling value. In the example with size 9, only one expansion is needed in which case we have a `10` in the elements.

You might ask why it isn't represented as a traditional array. This is because of persistence in data structures: older versions of the tree can be kept alive, so you cannot overwrite an older array. It is useful in a number of situations, the most important one being that you can know the state of a crashing process before it crashed, and what event led to said crash.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Spurious '100' in array:new

Fred Youhanaie-2
Don

If you mean using a single ~p to print the whole map with the array inside it, then you'll end up with the extra stuff too. io:format cannot extract the data part of the array automatically.

However, you can use the array:to_list/1 function to explicitly extract the data part of the array, and then create a "printable" map using maps:update/3, i.e.

Map_with_array = ...

% extract the array data as a list
Array_list = array:to_list( maps:get(arrayKey, Map_with_array) ).

% create the printable map, by replacing the array structure with the list equivalent
Map_printable = maps:update( arrayKey, Array_list, Map_with_array ).

% print the whole map with a single ~p
io:format("The map: ~p.~n", [Map_printable]).

Do bear in mind that array:to_list/1 will give you a long list with 19 x {0,unspecified,0} elements for your initial array!

Cheers,
Fred

On 28/01/2019 20:26, Donald Steven wrote:

> Thanks Jesper, that's most helpful to know.  Is there a particular way to print it in io:format that will *not* show the 'extra stuff'. (I used ~p)
>
> Best,
>
> Don
>
> On 1/28/2019 11.01 AM, Jesper Louis Andersen wrote:
>> On Sat, Jan 26, 2019 at 6:29 PM Fred Youhanaie <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     That's part of the internal structure of the array module, the data structure is opaque and should not be relied upon.
>>
>>
>> For full reference, you can glean on the internal structure:
>>
>> Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]
>>
>> Eshell V10.2.1  (abort with ^G)
>> 1> rr(array).
>> [array]
>> 2> array:new(19, {default, {0, unspecified, 0}}).
>> #array{size = 19,max = 0,
>>        default = {0,unspecified,0},
>>        elements = 100}
>> 3> array:new(9, {default, {0, unspecified, 0}}).
>> #array{size = 9,max = 0,
>>        default = {0,unspecified,0},
>>        elements = 10}
>>
>> Here is what is happening: In Erlang, an Array is a very flat tree with a branching factor of 10. It is stored in `elements`. But instead of storing a tree, you may store an integer encoding how
>> much more space is allowed in this part of the tree when it is expanded. You have created a tree of size 19, which means it needs two expansions of size 10, or 100---though not all of those
>> expansions will be used, as it is extra padding. If you had created an array of size 101, it would have been 1000, and so on. It is always the next log10 ceiling value. In the example with size 9,
>> only one expansion is needed in which case we have a `10` in the elements.
>>
>> You might ask why it isn't represented as a traditional array. This is because of persistence in data structures: older versions of the tree can be kept alive, so you cannot overwrite an older
>> array. It is useful in a number of situations, the most important one being that you can know the state of a crashing process before it crashed, and what event led to said crash.
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions