Should we deprecate or modernize log_mf_h and rb?

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Should we deprecate or modernize log_mf_h and rb?

Siri Hansen
Hello list!

log_mf_h is an error_logger event handler which logs events to disk and does log rotation. rb (report browser) is the tool for reading and formatting the logs. The implementation of both modules is quite old and outdated. The fact that there is a size field for the events of only 16 bits, which we haven't got any complaints for, has made us believe that there might not be many users of these tools. I'm writing to the list to find out if this is correct... 

Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools?

Kind Regards
/siri@otp

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Martin Bjorklund
Hi,

We are not using log_mf_h, but instead we're using disk_log_h (see
http://erlang.org/pipermail/erlang-patches/2012-January/002583.html)

We use this handler to get log rotation, but we log all error logger
messages in clear text (this should IMO be the default).

If you're interested in adding disk_log_h to OTP (search for
disk_log_h in disk_log.erl...), the latest version
(updated as recently as 2006 :) can be found here:
https://github.com/richcarl/otp/blob/disk_log_h/lib/kernel/src/disk_log_h.erl

I support the deprecation initiative!


/martin



Siri Hansen <[hidden email]> wrote:

> Hello list!
>
> log_mf_h is an error_logger event handler which logs events to disk and
> does log rotation. rb (report browser) is the tool for reading and
> formatting the logs. The implementation of both modules is quite old and
> outdated. The fact that there is a size field for the events of only 16
> bits, which we haven't got any complaints for, has made us believe that
> there might not be many users of these tools. I'm writing to the list to
> find out if this is correct...
>
> Are you using log_mf_h (e.g. by setting environment variables
> 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or
> the binary logging (or both) that you are really after? Have you considered
> alternative tools?
>
> Kind Regards
> /siri@otp
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Richard Carlsson-3
Thanks for pointing out that old mail, Martin! These days though, it's probably worthwhile to rethink/rewrite these handlers from scratch rather than import disk_log_h into OTP as it is.


        /Richard

2016-04-01 13:08 GMT+02:00 Martin Bjorklund <[hidden email]>:
Hi,

We are not using log_mf_h, but instead we're using disk_log_h (see
http://erlang.org/pipermail/erlang-patches/2012-January/002583.html)

We use this handler to get log rotation, but we log all error logger
messages in clear text (this should IMO be the default).

If you're interested in adding disk_log_h to OTP (search for
disk_log_h in disk_log.erl...), the latest version
(updated as recently as 2006 :) can be found here:
https://github.com/richcarl/otp/blob/disk_log_h/lib/kernel/src/disk_log_h.erl

I support the deprecation initiative!


/martin



Siri Hansen <[hidden email]> wrote:
> Hello list!
>
> log_mf_h is an error_logger event handler which logs events to disk and
> does log rotation. rb (report browser) is the tool for reading and
> formatting the logs. The implementation of both modules is quite old and
> outdated. The fact that there is a size field for the events of only 16
> bits, which we haven't got any complaints for, has made us believe that
> there might not be many users of these tools. I'm writing to the list to
> find out if this is correct...
>
> Are you using log_mf_h (e.g. by setting environment variables
> 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or
> the binary logging (or both) that you are really after? Have you considered
> alternative tools?
>
> Kind Regards
> /siri@otp
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Robert Raschke
In reply to this post by Siri Hansen

I really like this "old" error logger, it's *easily* one of the fastest around actually, since it can leave formatting to read time! I've hacked it to truncate overlong entries. The rb application is also easily hackable to suit your needs. I have a version that looks up formatting functions based on a bit of metadata in the logged "messages".

The major boon of being so blindingly fast, is that you do not have to set log levels at logging time. Instead you choose which log level you want to see when investigating!

Unfortunately, straight logged binary data and formatting it at read time appears to be majorly out of fashion :-(

Oh well.

If you want to keep it, I think it would be worth adding the truncation bit, or make it accept any size log message.

Cheers,
Robby

On Apr 1, 2016 12:44 PM, "Siri Hansen" <[hidden email]> wrote:
Hello list!

log_mf_h is an error_logger event handler which logs events to disk and does log rotation. rb (report browser) is the tool for reading and formatting the logs. The implementation of both modules is quite old and outdated. The fact that there is a size field for the events of only 16 bits, which we haven't got any complaints for, has made us believe that there might not be many users of these tools. I'm writing to the list to find out if this is correct... 

Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools?

Kind Regards
/siri@otp

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Loïc Hoguin-3
In reply to this post by Siri Hansen
Hello,

I didn't know we had that. It sounds like a hidden gem in OTP, and one I
would say worth keeping/improving upon; but not necessarily as part of
OTP. If all I had to do was add an application as dependency to benefit
from this, I would most certainly do so.

On 04/01/2016 12:43 PM, Siri Hansen wrote:

> Hello list!
>
> log_mf_h is an error_logger event handler which logs events to disk and
> does log rotation. rb (report browser) is the tool for reading and
> formatting the logs. The implementation of both modules is quite old and
> outdated. The fact that there is a size field for the events of only 16
> bits, which we haven't got any complaints for, has made us believe that
> there might not be many users of these tools. I'm writing to the list to
> find out if this is correct...
>
> Are you using log_mf_h (e.g. by setting environment variables
> 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation
> or the binary logging (or both) that you are really after? Have you
> considered alternative tools?
>
> Kind Regards
> /siri@otp
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

--
Loïc Hoguin
http://ninenines.eu
Author of The Erlanger Playbook,
A book about software development using Erlang
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

PAILLEAU Eric
In reply to this post by Robert Raschke

Hi,
I integrated rb log search in observer since several versions , and me too, like this fast error logger.
I think rewriting it, or enhance it would be nice.
The main issue was to write a pseudo io server to catch rb output.
I would be very sad if it is deprecated and vanish. Dear Otp team, save rb and log_mf_h !

"Envoyé depuis mon mobile " Eric



---- Robert Raschke a écrit ----

I really like this "old" error logger, it's *easily* one of the fastest around actually, since it can leave formatting to read time! I've hacked it to truncate overlong entries. The rb application is also easily hackable to suit your needs. I have a version that looks up formatting functions based on a bit of metadata in the logged "messages".

The major boon of being so blindingly fast, is that you do not have to set log levels at logging time. Instead you choose which log level you want to see when investigating!

Unfortunately, straight logged binary data and formatting it at read time appears to be majorly out of fashion :-(

Oh well.

If you want to keep it, I think it would be worth adding the truncation bit, or make it accept any size log message.

Cheers,
Robby

On Apr 1, 2016 12:44 PM, "Siri Hansen" <[hidden email]> wrote:
Hello list!

log_mf_h is an error_logger event handler which logs events to disk and does log rotation. rb (report browser) is the tool for reading and formatting the logs. The implementation of both modules is quite old and outdated. The fact that there is a size field for the events of only 16 bits, which we haven't got any complaints for, has made us believe that there might not be many users of these tools. I'm writing to the list to find out if this is correct... 

Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools?

Kind Regards
/siri@otp

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Mark Allen-3
In reply to this post by Siri Hansen
I support deprecation, for what its worth. 


On Friday, April 1, 2016 5:44 AM, Siri Hansen <[hidden email]> wrote:


Hello list!

log_mf_h is an error_logger event handler which logs events to disk and does log rotation. rb (report browser) is the tool for reading and formatting the logs. The implementation of both modules is quite old and outdated. The fact that there is a size field for the events of only 16 bits, which we haven't got any complaints for, has made us believe that there might not be many users of these tools. I'm writing to the list to find out if this is correct... 

Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools?

Kind Regards
/siri@otp

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Vance Shipley
In reply to this post by Robert Raschke
On Apr 1, 2016 12:44 PM, "Siri Hansen" <[hidden email]> wrote:
> Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb?

Yes, this is part of my default configuration for embedded systems.

> If so, why - is it the log rotation or the binary logging (or both) that you are really after?

Both.

> Have you considered alternative tools?

I've considered application specific versions of this scheme.

On Sat, Apr 2, 2016 at 1:23 AM, Robert Raschke <[hidden email]> wrote:
> Unfortunately, straight logged binary data and formatting it at read time appears to be majorly out of fashion :-(

Stupidly so IMHO.  I often see systems with a mission of moving
massive amounts of message traffic logging data orders of magnitude
larger than the size of the actual message.  Doing so naively results
in the servers spending much more time logging than doing the work
they're intended to do.  Logging is important and at this rate it
costs real money to accomplish so it deserves to be engineered
sensibly.  My rational is that where a gen_fsm is spawned for each
transaction it should simply do it's work and before terminating send
it's State variable term, usually a record, directly to a log handler
(e.g. disk_log).  To have the worker for traffic doing string
processing for an human to possibly read a long time later makes no
sense.  Log binary data to disk and format just the data requested, at
request time, formatted for the requestor.

--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Robert Virding
Would it be possible to split out the binary data so it could be used as an option with the rotating files? I don't know if this would be interesting as such but you colud maybe still use rb to view the logs.

Another benefit of doing this would be to break out the formatting of the log messages outside of the erlang system. This would give all the speed benefits and still provide text logs for those who want it. It would also make it easier to provide different text formats.

Robert


On 2 April 2016 at 09:05, Vance Shipley <[hidden email]> wrote:
On Apr 1, 2016 12:44 PM, "Siri Hansen" <[hidden email]> wrote:
> Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb?

Yes, this is part of my default configuration for embedded systems.

> If so, why - is it the log rotation or the binary logging (or both) that you are really after?

Both.

> Have you considered alternative tools?

I've considered application specific versions of this scheme.

On Sat, Apr 2, 2016 at 1:23 AM, Robert Raschke <[hidden email]> wrote:
> Unfortunately, straight logged binary data and formatting it at read time appears to be majorly out of fashion :-(

Stupidly so IMHO.  I often see systems with a mission of moving
massive amounts of message traffic logging data orders of magnitude
larger than the size of the actual message.  Doing so naively results
in the servers spending much more time logging than doing the work
they're intended to do.  Logging is important and at this rate it
costs real money to accomplish so it deserves to be engineered
sensibly.  My rational is that where a gen_fsm is spawned for each
transaction it should simply do it's work and before terminating send
it's State variable term, usually a record, directly to a log handler
(e.g. disk_log).  To have the worker for traffic doing string
processing for an human to possibly read a long time later makes no
sense.  Log binary data to disk and format just the data requested, at
request time, formatted for the requestor.

--
     -Vance
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Jesper Louis Andersen-2
In reply to this post by Siri Hansen
I think this is an excellent idea, as it will revive a zombie thread from before 2010:


even with Kenneth/OTP making an apperance :)

The reason I like log_mf_h and rb is because I think it is a very good way of doing log information, especially in space constrained environments where a ring-buffer is good. I would have used this a lot were it not for the fact that it doesn't log anything above 64k in size, but then kills the whole log with that number afterwards. So I drifted toward lager instead. But had this been able to handle that situation gracefully, I'd definitely have used it in many situations. I would have fixed it long ago, were it not for the fact that it takes some effort to actually do correctly, and my 4-byte non-backwards-compatible hack worked back in the day.

The linux world is moving to binary logging in journald, but I'm not entirely sure I like it since garbled logs in binary are harder to save. On the other hand, given a binary log and an index, searching through logs is *much* faster. There is also an old project of mine which uses machine learning to autoclassify errors into groups, which is based on log_mf_h:


these kind of things are far harder to do on textual logs, where you have no erlang term structure readily available.

The other question though: should this be part of OTP? I think it is an excellent library to have *outside* OTP because it should only rely on an error_logger inside OTP. That way, we can evolve the library in another development cycle than the OTP system, which is good. One problem with OTP currently is that adding functionality has a latency of roughly 0.75 years on average. You write the feature, but it is only available 0.75 years later. This is far too slow for most people. But were the things separate, it might be easier to release an intermediate version between major releases.


On Fri, Apr 1, 2016 at 12:43 PM, Siri Hansen <[hidden email]> wrote:
Hello list!

log_mf_h is an error_logger event handler which logs events to disk and does log rotation. rb (report browser) is the tool for reading and formatting the logs. The implementation of both modules is quite old and outdated. The fact that there is a size field for the events of only 16 bits, which we haven't got any complaints for, has made us believe that there might not be many users of these tools. I'm writing to the list to find out if this is correct... 

Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools?

Kind Regards
/siri@otp

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Adam Rutkowski-3
In reply to this post by Siri Hansen
On Fri, Apr 1, 2016, at 12:43, Siri Hansen wrote:
 
Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools?
 
 
Hi,
 
Report browser was one of the first "wow factors" for me, I was shipping for enterprise sysops people and they absolutely loved it. I doubt what I've worked on will be ever upgraded from R14 though. I support Loic's idea to keep it, but extract it from the OTP.
 
/A.
 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Matthias Lang
In reply to this post by Siri Hansen
On 01. April 2016, Siri Hansen wrote:

> Are you using log_mf_h

In 2001, we used it and then abandoned it for two main reasons:

  1. There was at least one bug in about R7 which made binary_to_term()
     crash the entire VM for some input. This made _reading_ logs after
     a power failure dangerous.

  2. We had very limited space (less than 1 MB) for logging in earlier
     generations of our hardware. text logging + trunc_io (now in
     lager) + gzip gave us more log-per-MB than binary logging
     (without gzip; AFAIK it's not an option).

The logging system we ended up using is completely home-made.

Matthias
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Stefan Grundmann
In reply to this post by Siri Hansen
Hi
 
log_mf_h is used in appliance systems we build at $work because it
implements binary logging and log rotation and because it is
part of OTP.
We also have projects where lager is used but only because it
was pulled as a dependency of $other_stuff and we did not yet spend the time
to replace it.
 
sg
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Tim Stewart-2
Hello,

I use log_mf_h and rb every day, and I am quite fond of them.

My systems are usually configured with both lager and log_mf_h.  SASL + log_mf_h pick up all of the "core" OTP-based reports (info, warning, error, crash, supervisor, progress) and lager is used by our applications for application-level logging.  Since lager also picks up reports and messages created by error_logger, our support team can monitor only a single *line-based* log file to find problems.

Then, when we get actual crashes I turn to report browser for nicely formatted reports.  In my opinion, rb's formatted reports are easier to read than lager's crash.log.  I have error_logger configured to use only lager and log_mf_h handlers so that we are relatively protected from a high error_logger volume.

And, as Éric Pailleau mentioned, log_mf_h is the backend for Observer's log support.  I also use this feature.

I would be sad to see log_mf_h disappear.  I would *love* to see more GUI support for reading and searching through these logs (additional Observer support, web UI, etc.)

Anyway, I hope this helps.

-TimS



On Tue, Apr 5, 2016 at 4:49 AM, Stefan Grundmann <[hidden email]> wrote:
Hi

log_mf_h is used in appliance systems we build at $work because it
implements binary logging and log rotation and because it is
part of OTP.
We also have projects where lager is used but only because it
was pulled as a dependency of $other_stuff and we did not yet spend the time
to replace it.

sg
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



--
Tim Stewart
[hidden email]
+1 (404) 993-6492


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Siri Hansen
Thanks for the feedback, everyone!

We have now decided not to remove log_mf_h and rb. Instead there will be some work done to modernize these tools.

Kind Regards
/siri

2016-04-07 0:03 GMT+02:00 Tim Stewart <[hidden email]>:
Hello,

I use log_mf_h and rb every day, and I am quite fond of them.

My systems are usually configured with both lager and log_mf_h.  SASL + log_mf_h pick up all of the "core" OTP-based reports (info, warning, error, crash, supervisor, progress) and lager is used by our applications for application-level logging.  Since lager also picks up reports and messages created by error_logger, our support team can monitor only a single *line-based* log file to find problems.

Then, when we get actual crashes I turn to report browser for nicely formatted reports.  In my opinion, rb's formatted reports are easier to read than lager's crash.log.  I have error_logger configured to use only lager and log_mf_h handlers so that we are relatively protected from a high error_logger volume.

And, as Éric Pailleau mentioned, log_mf_h is the backend for Observer's log support.  I also use this feature.

I would be sad to see log_mf_h disappear.  I would *love* to see more GUI support for reading and searching through these logs (additional Observer support, web UI, etc.)

Anyway, I hope this helps.

-TimS



On Tue, Apr 5, 2016 at 4:49 AM, Stefan Grundmann <[hidden email]> wrote:
Hi

log_mf_h is used in appliance systems we build at $work because it
implements binary logging and log rotation and because it is
part of OTP.
We also have projects where lager is used but only because it
was pulled as a dependency of $other_stuff and we did not yet spend the time
to replace it.

sg
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



--
Tim Stewart
[hidden email]
<a href="tel:%2B1%20%28404%29%20993-6492" value="+14049936492" target="_blank">+1 (404) 993-6492


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Garret Smith-2
On Fri, Apr 8, 2016 at 5:36 AM, Siri Hansen <[hidden email]> wrote:
> Thanks for the feedback, everyone!
>
> We have now decided not to remove log_mf_h and rb. Instead there will be
> some work done to modernize these tools.
>

+1

I stopped using it a while back, but would probably resume if it were
modernized.
Intriguing idea: log_mf_h as a lager sink?

> Kind Regards
> /siri
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Andrew Berman
+1 on modernizing and the lager sink

On Fri, Apr 8, 2016, 8:41 AM Garret Smith <[hidden email]> wrote:
On Fri, Apr 8, 2016 at 5:36 AM, Siri Hansen <[hidden email]> wrote:
> Thanks for the feedback, everyone!
>
> We have now decided not to remove log_mf_h and rb. Instead there will be
> some work done to modernize these tools.
>

+1

I stopped using it a while back, but would probably resume if it were
modernized.
Intriguing idea: log_mf_h as a lager sink?

> Kind Regards
> /siri
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Richard Carlsson-3
In reply to this post by Richard Carlsson-3
If anyone is interested (hello Martin and Siri!), we have slightly modernized our rotating cleartext logger handler to keep it compatible with recent OTP versions, and split it up into three separate modules, one as api and handler-supervisor, and one for the formatting of messages. Available here, as a diff on top of the old version if anyone still wants that:

Den fre 1 apr. 2016 kl 13:34 skrev Richard Carlsson <[hidden email]>:
Thanks for pointing out that old mail, Martin! These days though, it's probably worthwhile to rethink/rewrite these handlers from scratch rather than import disk_log_h into OTP as it is.


        /Richard

2016-04-01 13:08 GMT+02:00 Martin Bjorklund <[hidden email]>:
Hi,

We are not using log_mf_h, but instead we're using disk_log_h (see
http://erlang.org/pipermail/erlang-patches/2012-January/002583.html)

We use this handler to get log rotation, but we log all error logger
messages in clear text (this should IMO be the default).

If you're interested in adding disk_log_h to OTP (search for
disk_log_h in disk_log.erl...), the latest version
(updated as recently as 2006 :) can be found here:
https://github.com/richcarl/otp/blob/disk_log_h/lib/kernel/src/disk_log_h.erl

I support the deprecation initiative!


/martin



Siri Hansen <[hidden email]> wrote:
> Hello list!
>
> log_mf_h is an error_logger event handler which logs events to disk and
> does log rotation. rb (report browser) is the tool for reading and
> formatting the logs. The implementation of both modules is quite old and
> outdated. The fact that there is a size field for the events of only 16
> bits, which we haven't got any complaints for, has made us believe that
> there might not be many users of these tools. I'm writing to the list to
> find out if this is correct...
>
> Are you using log_mf_h (e.g. by setting environment variables
> 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or
> the binary logging (or both) that you are really after? Have you considered
> alternative tools?
>
> Kind Regards
> /siri@otp
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Andrew Thompson-2
In reply to this post by Siri Hansen
On Fri, Apr 08, 2016 at 02:36:37PM +0200, Siri Hansen wrote:
> Thanks for the feedback, everyone!
>
> We have now decided not to remove log_mf_h and rb. Instead there will be
> some work done to modernize these tools.

The big problem we had with these at Basho (RIP) was that customers had
no idea how the log files worked because it's very different to
traditional UNIX logging. Usually you have things like app.log,
app.log.0, ... app.log.N and the number indicates their age. log_mf has
a weird index file where the active log file is encoded as a decimal
value (not ASCII, IIRC). This is very confusing for everyone who is not
an Erlang user (maybe even for them judging by this thread).

The main thing I'd like to see improve is the ergonomics. Make it work
more like a sysadmin would expect, have the newest log file be .0 and
dump the index. Every time you overflow move the files around to reflect
that.

Andrew
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Should we deprecate or modernize log_mf_h and rb?

Richard Carlsson-3
Yeah, the main thing the rotation mechanism is missing  in our handler as well is renumbering of existing files; it also simply increments the number as it goes, and you have to sort on timestamps to see which one is the latest. Hence, I can't recommend it as is unless you don't mind that sort of bother. On the other hand, clear text logs are nice. Somehow, there's always been something more important to do than improve the log numbering, but that's how it is.

        /Richard


Den fre 15 feb. 2019 kl 22:09 skrev Andrew Thompson <[hidden email]>:
On Fri, Apr 08, 2016 at 02:36:37PM +0200, Siri Hansen wrote:
> Thanks for the feedback, everyone!
>
> We have now decided not to remove log_mf_h and rb. Instead there will be
> some work done to modernize these tools.

The big problem we had with these at Basho (RIP) was that customers had
no idea how the log files worked because it's very different to
traditional UNIX logging. Usually you have things like app.log,
app.log.0, ... app.log.N and the number indicates their age. log_mf has
a weird index file where the active log file is encoded as a decimal
value (not ASCII, IIRC). This is very confusing for everyone who is not
an Erlang user (maybe even for them judging by this thread).

The main thing I'd like to see improve is the ergonomics. Make it work
more like a sysadmin would expect, have the newest log file be .0 and
dump the index. Every time you overflow move the files around to reflect
that.

Andrew
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions