Understanding etp-heapdump output in gdb

[Roger Lipscombe-2]

I'm attempting to figure out what binaries a particular process is
holding onto. I have a core dump; I have a (seemingly) working gdb
installation.

I've found the Process I care about:

    (gdb) set $p = (Process *)0x7f6be3d3f560
    (gdb) etp-heapdump $p

When I run etp-heapdump, I get a boatload of output.

It starts with:

% heapdump (22077):
 0x7f6b93bb8028: | H:   5-tuple | A:0x0003aa4b | B:0x93bb8072 |
A:0x0002abcb | C:0x93bb8090 | A:0x001a09cb | H:   2-tuple |
<0.1805.000>
 0x7f6b93bb8068: | B:0x93bb80a2 | header     3 |   0x93bb8078 |
0x93bb8080 |   0x93bb8088 | B:0x93bb80ba | C:0x93bb80d8 | header     2
 0x7f6b93bb80a8: |   0x93bb80a8 |   0x93bb80b0 | header     3 |
0x93bb80c0 |   0x93bb80c8 |   0x93bb80d0 | B:0x93bb80ea |    [] (NIL)
 0x7f6b93bb80e8: | header     3 |   0x93bb80f0 |   0x93bb80f8 |
0x93bb8100 | H:   2-tuple |           ok | <0.20777.000> | H:
3-tuple

How do I interpret this output?
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Lukas Larsson-8]

Hello,

On Mon, Dec 18, 2017 at 8:30 PM, Roger Lipscombe <[hidden email]> wrote:

I'm attempting to figure out what binaries a particular process is
holding onto. I have a core dump; I have a (seemingly) working gdb
installation.

If it is off-heap binaries that you are interested in I would use etp-offheapdump to get information from the process.

(gdb) etp-offheapdump $p->off_heap.first

[#Fun<0xa1e7c178,0xa20313a0,0x6,0>,

 #Fun<0xa1e7c2e0,0xa2031360,0x1,0>,

 #Fun<0xa1e7c268,0xa2031268,0x2,0>,

 #RefcBinary<0x5,0xa2031140,0xb53c0f60,0xb53c0f78,0>,

 #RefcBinary<0x3,0xa20310d0,0xb53c3468,0xb53c3480,0>,

 #Fun<0xb5b11900,0xa20310a0,0x2,0x3>,

 #Fun<0xb5b12350,0xa2031070,0x1,0>,

 #Fun<0xb5b11f18,0xa2030d08,0x1,0>,

 #RefcBinary<0x1,0xa2030c30,0xb53c1ac0,0xb53c1ad8,0>,

 #Fun<0xb5b11900,0xa2030cd8,0x2,0x3>,

 #Fun<0xb5b12350,0xa2030ca8,0x1,0>,

 #Fun<0xb5b11f18,0xa2048608,0x1,0>,

 #RefcBinary<0x1f,0xa20485a0,0xa1c405f0,0xa1c40608,0>,

 #Fun<0xb5b11900,0xa2048668,0x2,0x3>,

 #Fun<0xb5b12350,0xa2048638,0x1,0>,

 #Fun<0xb5b11f18,0,0x1,0>].

The second to last value of the #RefcBinary should be a char * to the binary data.

I've found the Process I care about:

    (gdb) set $p = (Process *)0x7f6be3d3f560
    (gdb) etp-heapdump $p

When I run etp-heapdump, I get a boatload of output.

It starts with:

% heapdump (22077):
 0x7f6b93bb8028: | H:   5-tuple | A:0x0003aa4b | B:0x93bb8072 |
A:0x0002abcb | C:0x93bb8090 | A:0x001a09cb | H:   2-tuple |
<0.1805.000>
 0x7f6b93bb8068: | B:0x93bb80a2 | header     3 |   0x93bb8078 |
0x93bb8080 |   0x93bb8088 | B:0x93bb80ba | C:0x93bb80d8 | header     2
 0x7f6b93bb80a8: |   0x93bb80a8 |   0x93bb80b0 | header     3 |
0x93bb80c0 |   0x93bb80c8 |   0x93bb80d0 | B:0x93bb80ea |    [] (NIL)
 0x7f6b93bb80e8: | header     3 |   0x93bb80f0 |   0x93bb80f8 |
0x93bb8100 | H:   2-tuple |           ok | <0.20777.000> | H:
3-tuple

How do I interpret this output?

You have to have a good understanding of how the erlang term format works internally in order to understand what etp-heapdump prints. I believe that https://github.com/happi/theBeamBook has a section describing how it works.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Roger Lipscombe-2]

Thanks. Are those values relative to something? I'm seeing:

[#RefcBinary<0x10,0x17079e08,0x39700560,0x39700578,0>,
 #RefcBinary<0x10,0x17079d68,0x39448c70,0x39448c88,0>,
...etc.

Those values look a little bit short for a 64-bit BEAM, or have I just
got holes in my core dump? Can you *have* holes in a core dump on
Linux?

On 19 December 2017 at 13:33, Lukas Larsson <[hidden email]> wrote:

> Hello,
>
> On Mon, Dec 18, 2017 at 8:30 PM, Roger Lipscombe <[hidden email]>
> wrote:
>>
>> I'm attempting to figure out what binaries a particular process is
>> holding onto. I have a core dump; I have a (seemingly) working gdb
>> installation.
>
>
> If it is off-heap binaries that you are interested in I would use
> etp-offheapdump to get information from the process.
>
> (gdb) etp-offheapdump $p->off_heap.first
> [#Fun<0xa1e7c178,0xa20313a0,0x6,0>,
>  #Fun<0xa1e7c2e0,0xa2031360,0x1,0>,
>  #Fun<0xa1e7c268,0xa2031268,0x2,0>,
>  #RefcBinary<0x5,0xa2031140,0xb53c0f60,0xb53c0f78,0>,
>  #RefcBinary<0x3,0xa20310d0,0xb53c3468,0xb53c3480,0>,
>  #Fun<0xb5b11900,0xa20310a0,0x2,0x3>,
>  #Fun<0xb5b12350,0xa2031070,0x1,0>,
>  #Fun<0xb5b11f18,0xa2030d08,0x1,0>,
>  #RefcBinary<0x1,0xa2030c30,0xb53c1ac0,0xb53c1ad8,0>,
>  #Fun<0xb5b11900,0xa2030cd8,0x2,0x3>,
>  #Fun<0xb5b12350,0xa2030ca8,0x1,0>,
>  #Fun<0xb5b11f18,0xa2048608,0x1,0>,
>  #RefcBinary<0x1f,0xa20485a0,0xa1c405f0,0xa1c40608,0>,
>  #Fun<0xb5b11900,0xa2048668,0x2,0x3>,
>  #Fun<0xb5b12350,0xa2048638,0x1,0>,
>  #Fun<0xb5b11f18,0,0x1,0>].
>
> The second to last value of the #RefcBinary should be a char * to the binary
> data.
>
>>
>> I've found the Process I care about:
>>
>>     (gdb) set $p = (Process *)0x7f6be3d3f560
>>     (gdb) etp-heapdump $p
>>
>> When I run etp-heapdump, I get a boatload of output.
>>
>> It starts with:
>>
>> % heapdump (22077):
>>  0x7f6b93bb8028: | H:   5-tuple | A:0x0003aa4b | B:0x93bb8072 |
>> A:0x0002abcb | C:0x93bb8090 | A:0x001a09cb | H:   2-tuple |
>> <0.1805.000>
>>  0x7f6b93bb8068: | B:0x93bb80a2 | header     3 |   0x93bb8078 |
>> 0x93bb8080 |   0x93bb8088 | B:0x93bb80ba | C:0x93bb80d8 | header     2
>>  0x7f6b93bb80a8: |   0x93bb80a8 |   0x93bb80b0 | header     3 |
>> 0x93bb80c0 |   0x93bb80c8 |   0x93bb80d0 | B:0x93bb80ea |    [] (NIL)
>>  0x7f6b93bb80e8: | header     3 |   0x93bb80f0 |   0x93bb80f8 |
>> 0x93bb8100 | H:   2-tuple |           ok | <0.20777.000> | H:
>> 3-tuple
>>
>> How do I interpret this output?
>
>
> You have to have a good understanding of how the erlang term format works
> internally in order to understand what etp-heapdump prints. I believe that
> https://github.com/happi/theBeamBook has a section describing how it works.
>
> Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Lukas Larsson-8]

On Tue, Dec 19, 2017 at 3:53 PM, Roger Lipscombe <[hidden email]> wrote:

Thanks. Are those values relative to something? I'm seeing:

[#RefcBinary<0x10,0x17079e08,0x39700560,0x39700578,0>,
 #RefcBinary<0x10,0x17079d68,0x39448c70,0x39448c88,0>,
...etc.

Those values look a little bit short for a 64-bit BEAM, or have I just
got holes in my core dump? Can you *have* holes in a core dump on
Linux?

They seem to be truncated on 64-bit. Open etp-commands, look for function called etp-boxed-immediate-1, and change any "%#x" to "%p" and it should print the entire pointer.

PR's to make etp work better in 32/64 bits are welcome :)

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Roger Lipscombe-2]

That fixed it. I also had to set $etp_max_depth to something ludicrous.

I'll look into doing a PR against master.

Thanks,
Roger.

On 19 December 2017 at 15:28, Lukas Larsson <[hidden email]> wrote:

>
>
> On Tue, Dec 19, 2017 at 3:53 PM, Roger Lipscombe <[hidden email]>
> wrote:
>>
>> Thanks. Are those values relative to something? I'm seeing:
>>
>> [#RefcBinary<0x10,0x17079e08,0x39700560,0x39700578,0>,
>>  #RefcBinary<0x10,0x17079d68,0x39448c70,0x39448c88,0>,
>> ...etc.
>>
>> Those values look a little bit short for a 64-bit BEAM, or have I just
>> got holes in my core dump? Can you *have* holes in a core dump on
>> Linux?
>
>
> They seem to be truncated on 64-bit. Open etp-commands, look for function
> called etp-boxed-immediate-1, and change any "%#x" to "%p" and it should
> print the entire pointer.
>
> PR's to make etp work better in 32/64 bits are welcome :)
>
> Lukas
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions