log_mf_h and rb do not handle terms over 65k

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

log_mf_h and rb do not handle terms over 65k

Garret Smith-2
I was logging a couple very large terms using
error_logger:info_report and log_mf_h during application start,
and was always unable to read most of the first log file
generated.  This was mystifying me for a while until I finally
traced the source.

log_mf_h and rb use a 2-byte length indicator for the term they
are about to read/write, therefore they cannot handle terms
over 65,536 bytes in length.  See handle_event/2 in log_mf_h
and read_report/1 in rb

Increasing the length field would break backwards compatibility,
but maybe it would be prudent to truncate the term to the max
length or replace it with a "term to big" message so that the rest
of the log file is not corrupted?  When this happens in my
application, the rest of the log file is unreadable.

Has anyone else experienced this or know of a workaround
(other than the obvious of checking the term size in my
application code)?

Thanks,
Garret Smith

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: log_mf_h and rb do not handle terms over 65k

Jesper Louis Andersen-2
On Mon, May 17, 2010 at 8:15 PM, Garret Smith <[hidden email]> wrote:

> I was logging a couple very large terms using
> error_logger:info_report and log_mf_h during application start,
> and was always unable to read most of the first log file
> generated.  This was mystifying me for a while until I finally
> traced the source.
>
> log_mf_h and rb use a 2-byte length indicator for the term they
> are about to read/write, therefore they cannot handle terms
> over 65,536 bytes in length.  See handle_event/2 in log_mf_h
> and read_report/1 in rb

Yes, etorrent has the same problem for some of its processes. I
reported this little problem a couple of years ago, and even did the
backwards-incompatible patch against R11b-5 to fix it:

http://www.erlang.org/cgi-bin/ezmlm-cgi/3/187

--
J.

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: log_mf_h and rb do not handle terms over 65k

Garret Smith-2
Would the PTB be more willing to accept a backwards compatible patch that
either truncates the term or replaces it with a "Term too big" message?

I would be happy to create one if there is a good chance of it being accepted.

-Garret Smith

On Tue, May 18, 2010 at 3:39 AM, Jesper Louis Andersen
<[hidden email]> wrote:

> On Mon, May 17, 2010 at 8:15 PM, Garret Smith <[hidden email]> wrote:
>> I was logging a couple very large terms using
>> error_logger:info_report and log_mf_h during application start,
>> and was always unable to read most of the first log file
>> generated.  This was mystifying me for a while until I finally
>> traced the source.
>>
>> log_mf_h and rb use a 2-byte length indicator for the term they
>> are about to read/write, therefore they cannot handle terms
>> over 65,536 bytes in length.  See handle_event/2 in log_mf_h
>> and read_report/1 in rb
>
> Yes, etorrent has the same problem for some of its processes. I
> reported this little problem a couple of years ago, and even did the
> backwards-incompatible patch against R11b-5 to fix it:
>
> http://www.erlang.org/cgi-bin/ezmlm-cgi/3/187
>
> --
> J.
>

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: log_mf_h and rb do not handle terms over 65k

Jesper Louis Andersen-2
On Tue, May 18, 2010 at 4:53 PM, Garret Smith <[hidden email]> wrote:
> Would the PTB be more willing to accept a backwards compatible patch that
> either truncates the term or replaces it with a "Term too big" message?

I would definitely recommend replacing it with a term-too-big message.
As it stands at the moment, the bug means that the rest of your log
becomes unusable because it can't be read correctly. That is a log
worse than missing out on a single crash report.


--
J.

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: log_mf_h and rb do not handle terms over 65k

Kenneth Lundin
In reply to this post by Garret Smith-2
Yes, a backwards compatible patch which truncates the term or replaces
it with "term too big"
has very good chances to be accepted. Provided of course that it is submitted
according to the instructions on Github.

/Kenneth, Erlang/OTP Ericsson

On Tue, May 18, 2010 at 4:53 PM, Garret Smith <[hidden email]> wrote:

> Would the PTB be more willing to accept a backwards compatible patch that
> either truncates the term or replaces it with a "Term too big" message?
>
> I would be happy to create one if there is a good chance of it being accepted.
>
> -Garret Smith
>
> On Tue, May 18, 2010 at 3:39 AM, Jesper Louis Andersen
> <[hidden email]> wrote:
>> On Mon, May 17, 2010 at 8:15 PM, Garret Smith <[hidden email]> wrote:
>>> I was logging a couple very large terms using
>>> error_logger:info_report and log_mf_h during application start,
>>> and was always unable to read most of the first log file
>>> generated.  This was mystifying me for a while until I finally
>>> traced the source.
>>>
>>> log_mf_h and rb use a 2-byte length indicator for the term they
>>> are about to read/write, therefore they cannot handle terms
>>> over 65,536 bytes in length.  See handle_event/2 in log_mf_h
>>> and read_report/1 in rb
>>
>> Yes, etorrent has the same problem for some of its processes. I
>> reported this little problem a couple of years ago, and even did the
>> backwards-incompatible patch against R11b-5 to fix it:
>>
>> http://www.erlang.org/cgi-bin/ezmlm-cgi/3/187
>>
>> --
>> J.
>>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:[hidden email]
>
>

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]