os_mon & alarm_handler in R10B-10

[Serge Aleynikov]

Hi,

I've been experimenting with the reworked os_mon in R10B-10, and
encountered the following issue.

The documentation encourages to replace the default alarm handler with
something more sophisticated.  For that reason I created a custom
handler - lama_alarm_h (LAMA app in jungerl), which uses
gen_event:swap_sup_handler/3.

I initiate that handler prior to starting OS_MON, and then start OS_MON.

In the latest release R10B-10, OS_MON calls alarm_handler:get_alarms/0
upon startup.

This causes the 'alarm_handler' event manager issue a call in the
alarm_handler.erl module.  However, since that handler was replaced by a
custom alarm handler, the gen_event's call fails with
{error, bad_module}.

gen_event always dispatches a call/3 to a specific handler module passed
as a parameter, e.g.:

-----[alarm_handler.erl (line: 60)]-----
get_alarms() ->
     gen_event:call(alarm_handler, alarm_handler, get_alarms).
----------------------------------------

Yet, if the alarm_handler handler was swapped by another module, the
gen_event:call will report an error, therefore crashing OS_MON.

One way to resolve this problem would be to introduce another exported
function in gen_event:

gen_event:call(EventMgrRef, Request) -> Result

Can the OTP team suggest some other workaround?

Serge

--
Serge Aleynikov
R&D Telecom, IDT Corp.
Tel: [phone]
Fax: [phone]
[hidden email]

[Serge Aleynikov]

For now I used the following patch to take care of this issue, but I
would be curious to hear the opinion of the OTP staff.

Regards,

Serge

--- alarm_handler.erl.orig      Fri Mar 24 20:08:18 2006
+++ alarm_handler.erl   Fri Mar 24 20:19:15 2006
@@ -58,7 +58,12 @@
  %% Returns: [{AlarmId, AlarmDesc}]
  %%-----------------------------------------------------------------
  get_alarms() ->
-    gen_event:call(alarm_handler, alarm_handler, get_alarms).
+    case gen_event:which_handlers(alarm_handler) of
+    [M | _] ->
+        gen_event:call(alarm_handler, M, get_alarms);
+    [] ->
+        []
+    end.

  add_alarm_handler(Module) when atom(Module) ->
      gen_event:add_handler(alarm_handler, Module, []).


Serge Aleynikov wrote:

> Hi,
>
> I've been experimenting with the reworked os_mon in R10B-10, and
> encountered the following issue.
>
> The documentation encourages to replace the default alarm handler with
> something more sophisticated.  For that reason I created a custom
> handler - lama_alarm_h (LAMA app in jungerl), which uses
> gen_event:swap_sup_handler/3.
>
> I initiate that handler prior to starting OS_MON, and then start OS_MON.
>
> In the latest release R10B-10, OS_MON calls alarm_handler:get_alarms/0
> upon startup.
>
> This causes the 'alarm_handler' event manager issue a call in the
> alarm_handler.erl module.  However, since that handler was replaced by a
> custom alarm handler, the gen_event's call fails with
> {error, bad_module}.
>
> gen_event always dispatches a call/3 to a specific handler module passed
> as a parameter, e.g.:
>
> -----[alarm_handler.erl (line: 60)]-----
> get_alarms() ->
>     gen_event:call(alarm_handler, alarm_handler, get_alarms).
> ----------------------------------------
>
> Yet, if the alarm_handler handler was swapped by another module, the
> gen_event:call will report an error, therefore crashing OS_MON.
>
> One way to resolve this problem would be to introduce another exported
> function in gen_event:
>
> gen_event:call(EventMgrRef, Request) -> Result
>
> Can the OTP team suggest some other workaround?
>
> Serge
>

--
Serge Aleynikov
R&D Telecom, IDT Corp.
Tel: [phone]
Fax: [phone]
[hidden email]

[Nick Linker]

Good day.

I was playing with Erlang for calculation over large numbers and
discovered some issues:
1. math:log fails with "badarg" for big integers.
2. It is impossible to know how many digits in an integer _efficiently_
(i.e. with O(1) or O(log(n)), where n - number of digits).
length(number_to_list(N)) appears to have O(n).
3. Let me define the following function:
    fib(0) -> 0;
    fib(1) -> 1;
    fib(N) -> fib(N-1) + fib(N-2).
I have failed to convert this function _without_ put/get to be able to
compute even fib(100) within reasonable period of time (I guess I did it
wrong so that tail recursion was not here). Is there a way to compute
fib(N), N>=100 without side effects?

Thank you in advance,
Best regards,
Linker Nick.

[Kostis Sagonas]

Nick Linter wrote:
 > I was playing with Erlang for calculation over large numbers and
 > discovered some issues:
 > 1. math:log fails with "badarg" for big integers.

An example would have helped us understand better what the issue is.
Currently, I get:

Eshell V5.5  (abort with ^G)
1> math:log(43466557686938914862637500386755014010958388901725051132915256476112292920052539720295234060457458057800732025086130975998716977051839168242483814062805283311821051327273518050882075662659534523370463746326528).
480.407

 > 3. Let me define the following function:
 >   fib(0) -> 0;
 >   fib(1) -> 1;
 >   fib(N) -> fib(N-1) + fib(N-2).
 > I have failed to convert this function _without_ put/get to be able to
 > compute even fib(100) within reasonable period of time (I guess I did it
 > wrong so that tail recursion was not here). Is there a way to compute
 > fib(N), N>=100 without side effects?

Yes. Try:

-module(fib).
-export([fib/1]).
-import(math, [pow/3, sqrt/1]).

fib(N) ->
  trunc((1/sqrt(5)) * (pow(((1+sqrt(5))/2),N) - pow(((1-sqrt(5))/2),N))).

Kostis

PS. The performance you are experiencing in your version of fib/1
    has nothing to do with tail recursion...

[Javier París Fernández]

El Sábado, 25 de Marzo de 2006 08:55, Nick Linker escribió:

Hi,

> Good day.
> 3. Let me define the following function:
>     fib(0) -> 0;
>     fib(1) -> 1;
>     fib(N) -> fib(N-1) + fib(N-2).
> I have failed to convert this function _without_ put/get to be able to
> compute even fib(100) within reasonable period of time (I guess I did it
> wrong so that tail recursion was not here). Is there a way to compute
> fib(N), N>=100 without side effects?

fibaux(_N2,N1,N,N) -> N1;
fibaux(N2,N1,C,N) ->
        fibaux(N1,N1+N2,C+1,N).

fib(0) -> 0;
fib(N) ->
        fibaux(0,1,0,N).

However, as Kostis said, this has more to do with having two recursive calls
each time than with it being tail recursion or not. If you try to see how it
evaluated, the number of recursive calls expands exponentially.

Regards.

       

[Richard O'Keefe]

Nick Linker <[hidden email]> wrote:
        1. math:log fails with "badarg" for big integers.

math:log/1 was designed to work on floating-point arguments;
I suspect that it's the integer->float conversion that is failing here.

        2. It is impossible to know how many digits in an integer _efficiently_
        (i.e. with O(1) or O(log(n)), where n - number of digits).
        length(number_to_list(N)) appears to have O(n).

Anything else you can do with a number has to be at least O(n),
so *in the context of a real use* of bignum arithmetic, why does
measuring the size have to be O(lg n) rather than O(n)?

        3. Let me define the following function:
            fib(0) -> 0;
            fib(1) -> 1;
            fib(N) -> fib(N-1) + fib(N-2).

This is a SPECTACULARLY INEFFICIENT way to compute fibonacci numbers
in *any* programming language.

        I have failed to convert this function _without_ put/get to be able to
        compute even fib(100) within reasonable period of time (I guess I did it
        wrong so that tail recursion was not here).

It's not a tail recursion issue.  It is simply that the computation you
have specified does about 2**N function calls for argument N (*regardless*
of language).  So when you demand that fib(100) be computed that way,
you are demanding about 1,267,650,600,228,229,401,496,703,205,376
(10**30) function calls.  Assuming a machine that could do 10**10
function calls per second, that's 10**20 seconds, or about
three million million years.

        Is there a way to compute fib(N), N>=100 without side effects?
       
Yes, and it's easy.  In fact, there are two ways.  The simple obvious
tail recursion does O(N) integer additions.  The less obvious way treats
it as a matrix exponentiation problem and does O(lg N) 2x2 integer
matrix multiplies.

    fib(0) -> 1;
    fib(N) when N > 0 -> fib(N, 1, 1).

    fib(1, X, _) -> X;
    fib(N, X, Y) -> fib(N-1, X+Y, X).

This and the matrix exponentiation version generalise in fairly obvious
ways to recurrences of any order.

[Nick Linker]

Javier París Fernández wrote:

I made my version, but after posting the question :-)

fib(0) -> 0;
fib(1) -> 1;
fib(N) ->
    fib(N, 0, 1).
 
fib(I, Pr, Cu) when I =< 1 ->
    Cu;
fib(I, Pr, Cu) ->
    fib(I-1, Cu, Pr + Cu).

Thank you for your answer nonetheless.

However, as Kostis said, this has more to do with having two recursive calls 
each time than with it being tail recursion or not. If you try to see how it 
evaluated, the number of recursive calls expands exponentially.

Regards.

Best regards,
Linker Nick.

[David Hopwood]

Richard A. O'Keefe wrote:
> Nick Linker <[hidden email]> wrote:
> 3. Let me define the following function:
>    fib(0) -> 0;
>    fib(1) -> 1;
>    fib(N) -> fib(N-1) + fib(N-2).
>
> This is a SPECTACULARLY INEFFICIENT way to compute fibonacci numbers
> in *any* programming language.

Yes. See the logarithmic time algorithm at
<http://en.wikipedia.org/wiki/Fibonacci_number#Computation>.

> It's not a tail recursion issue.  It is simply that the computation you
> have specified does about 2**N function calls for argument N (*regardless*
> of language).

To be terribly pedantic, it is *possible* for a language to do automatic
memoization of side-effect-free functions. OTOH, without some annotations to
say which functions are worth memoizing (and to stop the memo table from
growing without bound), that can very often be a pessimization.

--
David Hopwood <[hidden email]>

[Nick Linker]

Kostis Sagonas wrote:

Nick Linter wrote:

An example would have helped us understand better what the issue is.
Currently, I get:

Eshell V5.5  (abort with ^G)
1> math:log(...).
480.407

Well, I get the following result:

43> math:log10(test:fib(1476)).
308.116
44> math:log10(test:fib(1477)).
 
=ERROR REPORT==== 27-Mar-2006::10:57:47 ===
Error in process <0.181.0> with exit value: {badarith,[{math,log10,[16#00012D269
C3FA9D767CB55B0DDF8E6A2DE7B1D967FF8D0BE61EB16ACD02D1A53C95A370ABD95285998D226919
D95DCA54298D92C348C27E635E1690E7858060F0DC14E885F0217413C55A1F820D6EB051F87C7C73
818AC23E4A9A00A2072C08C6697A2FAD66FC7DEBEEB7A5F582D7639A431B9C99CEC6315A9ED1C652
A81A6B59A39]},{erl_eval,do_apply,5},{shell,exprs,6},{shell,eval_loop,3}]}
 
** exited: {badarith,[{math,log10,
                            [211475298697902185255785861961179135570552502746803
25217495622655863402432394766663713782393252439761186467156621190833026337742520
45520741882086869936691237540043402509431087092122991804222930097654049305082429
75773774612140021599477983006713536106549441161323499077298115887067363710153036
315849480388057657]},
                      {erl_eval,do_apply,5},
                      {shell,exprs,6},
                      {shell,eval_loop,3}]} **

My system is Windows XP, Erlang R10B.

fib(N) ->
  trunc((1/sqrt(5)) * (pow(((1+sqrt(5))/2),N) - pow(((1-sqrt(5))/2),N))).
  

Good solution :-)
Now I also have different idea without using recursion. It is based on the following equation
[F_n  ]   [1  1]   [F_n-1]
[     ] = [    ] * [     ]
[F_n-1]   [1  0]   [F_n-2]
And we just have to calculate k-th power of the matrix [[1,1],[1,0]]. It is possible to do within O(log(k)).

PS. The performance you are experiencing in your version of fib/1
    has nothing to do with tail recursion...

Yes, exponential number of recursive calls... Thank you.

I'm sorry, but most interesting question (2nd, about number of digits in an integer) remains unanswered. But I guess it is very rare problem with numbers, so if there is no answer, I will understand.

Best regards,
Linker Nick.

[Nick Linker]

Richard A. O'Keefe wrote:

> 2. It is impossible to know how many digits in an integer _efficiently_
> (i.e. with O(1) or O(log(n)), where n - number of digits).
> length(number_to_list(N)) appears to have O(n).
>
>Anything else you can do with a number has to be at least O(n),
>so *in the context of a real use* of bignum arithmetic, why does
>measuring the size have to be O(lg n) rather than O(n)?
>  
>
I did exactly this: N is a number, n - is the number of digits =
log(N).  I have a big number N with n digits, and I am searching a way
of computing n without enumerating all the digits.

>This is a SPECTACULARLY INEFFICIENT way to compute fibonacci numbers
>in *any* programming language.
>  
>

> Is there a way to compute fib(N), N>=100 without side effects?
>
>Yes, and it's easy.  In fact, there are two ways.  The simple obvious
>tail recursion does O(N) integer additions.  The less obvious way treats
>it as a matrix exponentiation problem and does O(lg N) 2x2 integer
>matrix multiplies.
>
>    fib(0) -> 1;
>    fib(N) when N > 0 -> fib(N, 1, 1).
>
>    fib(1, X, _) -> X;
>    fib(N, X, Y) -> fib(N-1, X+Y, X).
>
>This and the matrix exponentiation version generalise in fairly obvious
>ways to recurrences of any order.
>

Unfortunately right after asking the question to the mailing list I got
the same idea. (Unfortunately = because it is too late to get the
question back). But thank you for the comprehensive explanation of my
problem with recursion.

Best regards,
Linker Nick.

[Tony Rogvall]

>  Good solution :-)
> Now I also have different idea without using recursion. It is based  
> on the following equation
> [F_n  ]   [1  1]   [F_n-1]
> [     ] = [    ] * [     ]
> [F_n-1]   [1  0]   [F_n-2]
> And we just have to calculate k-th power of the matrix [[1,1],
> [1,0]]. It is possible to do within O(log(k)).
>

Here is my contrib (written many years ago, it may work :-)

ffib(N) ->
     {UN1,_,_,_} = lucas_numbers(1,-1,N-1),
     UN1.

lucas_numbers(P, Q, M) ->
     m_mult(exp({P,-Q,1,0}, M), {1,P,0,2}).

exp(A, N) when tuple(A), size(A)==4, is_integer(N), N > 0 ->
     m_exp(A, N, {1,0,0,1}).

m_exp(A, 1, P) ->
     m_mult(A,P);
m_exp(A, N, P) when ?even(N) ->
     m_exp(m_sqr(A), N div 2, P);
m_exp(A, N, P) ->
     m_exp(m_sqr(A), N div 2, m_mult(A, P)).

m_mult({A11,A12,A21,A22}, {B11,B12,B21,B22}) ->
     { A11*B11 + A12*B21, A11*B12 + A12*B22,
       A21*B11 + A22*B21, A21*B12 + A22*B22 }.

m_sqr({A,B,C,D}) ->
     BC = B*C,
     A_D = A + D,
     { A*A+BC, B*A_D, C*A_D, D*D + BC }.


/Tony

[Gunilla Arendt]

It's a bug in os_mon, it shouldn't use get_alarms().
Thanks for the heads up.

Regards, Gunilla


Serge Aleynikov wrote:

> For now I used the following patch to take care of this issue, but I
> would be curious to hear the opinion of the OTP staff.
>
> Regards,
>
> Serge
>
> --- alarm_handler.erl.orig      Fri Mar 24 20:08:18 2006
> +++ alarm_handler.erl   Fri Mar 24 20:19:15 2006
> @@ -58,7 +58,12 @@
>  %% Returns: [{AlarmId, AlarmDesc}]
>  %%-----------------------------------------------------------------
>  get_alarms() ->
> -    gen_event:call(alarm_handler, alarm_handler, get_alarms).
> +    case gen_event:which_handlers(alarm_handler) of
> +    [M | _] ->
> +        gen_event:call(alarm_handler, M, get_alarms);
> +    [] ->
> +        []
> +    end.
>
>  add_alarm_handler(Module) when atom(Module) ->
>      gen_event:add_handler(alarm_handler, Module, []).
>
>
> Serge Aleynikov wrote:
>> Hi,
>>
>> I've been experimenting with the reworked os_mon in R10B-10, and
>> encountered the following issue.
>>
>> The documentation encourages to replace the default alarm handler with
>> something more sophisticated.  For that reason I created a custom
>> handler - lama_alarm_h (LAMA app in jungerl), which uses
>> gen_event:swap_sup_handler/3.
>>
>> I initiate that handler prior to starting OS_MON, and then start OS_MON.
>>
>> In the latest release R10B-10, OS_MON calls alarm_handler:get_alarms/0
>> upon startup.
>>
>> This causes the 'alarm_handler' event manager issue a call in the
>> alarm_handler.erl module.  However, since that handler was replaced by
>> a custom alarm handler, the gen_event's call fails with
>> {error, bad_module}.
>>
>> gen_event always dispatches a call/3 to a specific handler module
>> passed as a parameter, e.g.:
>>
>> -----[alarm_handler.erl (line: 60)]-----
>> get_alarms() ->
>>     gen_event:call(alarm_handler, alarm_handler, get_alarms).
>> ----------------------------------------
>>
>> Yet, if the alarm_handler handler was swapped by another module, the
>> gen_event:call will report an error, therefore crashing OS_MON.
>>
>> One way to resolve this problem would be to introduce another exported
>> function in gen_event:
>>
>> gen_event:call(EventMgrRef, Request) -> Result
>>
>> Can the OTP team suggest some other workaround?
>>
>> Serge
>>
>

[Joe Armstrong (TN/EAB)]

   Hello Nick,

 

    1)  - can I compute fib(N) efficiently, and,

    2)  - how many (decimal) digits are there in fib(N) 

 

The answer to 1) is yes

 

    Your original version runs in O (2 ^0.69 N) time  -

    The improved version is O(N) but you can do better than this and do this in O(ln N)

     (see the section marked double iteration in http://en.wikipedia.org/wiki/Fibonacci_number_program)

    

   2) is much more difficult - you could or course just compute fib(N) then take the log -

but this is cheating - so can you compute the number of decomal digits  in fib(N) without

actually going to the trouble of computing fib(N) - now this might be easy but it certainly is not

obvious...

 

/Joe

 

 

 

 

---

From: [hidden email] [mailto:[hidden email]] On Behalf Of Nick Linker
Sent: den 27 mars 2006 05:27
To: [hidden email]
Cc: Erlang Questions
Subject: Re: Newbie questions

Javier París Fernández wrote:

I made my version, but after posting the question :-)

fib(0) -> 0;
fib(1) -> 1;
fib(N) ->
    fib(N, 0, 1).
 
fib(I, Pr, Cu) when I =< 1 ->
    Cu;
fib(I, Pr, Cu) ->
    fib(I-1, Cu, Pr + Cu).

Thank you for your answer nonetheless.

However, as Kostis said, this has more to do with having two recursive calls 
each time than with it being tail recursion or not. If you try to see how it 
evaluated, the number of recursive calls expands exponentially.

Regards.

Best regards,
Linker Nick.

[Serge Aleynikov]

Gunilla,

I believe there might be another bug in SNMP revealed by my experiments
with OS_MON & OTP_MIBS.  If mnesia is started *after* the snmp agent,
and the snmp agent has the mibs parameter set, an attempt to initialize
mib OIDs using instrumentation functions with the 'new' operation (such
as otp_mib:erl_node_table(new)), leads to an ignored exception that
ideally should prevent the SNMP agent from starting.

Release file:
=============
{release, {"dripdb", "1.0"}, {erts, "5.4.13"},
   [
     {kernel  , "2.10.13"},
     {stdlib  , "1.13.12"},
     {sasl    , "2.1.1"},
     {lama    , "1.0"},
     {otp_mibs, "1.0.4"},
     {os_mon  , "2.0"},
     {snmp    , "4.7.1"},
     {mnesia  , "4.2.5"}
   ]
}.

Config file:
============

%%------------ SNMP agent configuration ----------------------
   {snmp,
      [{agent,
         [{config, [{dir, "etc/snmp/"},
                    {force_load, true}
                   ]},
          {db_dir, "var/snmp_db/"},
          {mibs,   ["mibs/priv/OTP-MIB",
                    "mibs/priv/OTP-OS-MON-MIB"]}
         ]
       }
      ]
   }

This is a trace of the error which hides the fact that there was a
problem with creation of the 'erlNodeAlloc' table:

(<0.126.0>) call
snmpa_mib_data:call_instrumentation({me,[1,3,6,1,4,1,193,19,3,1,2,1,1,1],
     table_entry,
     erlNodeEntry,
     undefined,
     'not-accessible',
     {otp_mib,erl_node_table,[]},
     false,
     [{table_entry_with_sequence,'ErlNodeEntry'}],
     undefined,
     undefined},new)
(<0.126.0>) returned from snmpa_mib_data:call_instrumentation/2 ->
   {'EXIT',{aborted,{node_not_running,drpdb@devstorm01}}}

Therefore all the SNMP manager's calls to OIDs inside 'erlNodeTable' or
'applTable' tables fail.

I can provide additional details if needed, if the information here is
not sufficient.  I believe the proper action to do would be not to
absorb the error in the call_instrumentation function when the Operation
is 'new'.  I am providing the snippet of code where that exception is
currently ignored:

snmpa_mib_data.erl(line 1319):
==============================
call_instrumentation(#me{entrytype = variable, mfa={M,F,A}}, Operation) ->
     ?vtrace("call instrumentation with"
            "~n   entrytype: variable"
            "~n   MFA:       {~p,~p,~p}"
            "~n   Operation: ~p",
            [M,F,A,Operation]),
     catch apply(M, F, [Operation | A]);
...


Regards,

Serge


Gunilla Arendt wrote:

> It's a bug in os_mon, it shouldn't use get_alarms().
> Thanks for the heads up.
>
> Regards, Gunilla
>
>
> Serge Aleynikov wrote:
>
>> For now I used the following patch to take care of this issue, but I
>> would be curious to hear the opinion of the OTP staff.
>>
>> Regards,
>>
>> Serge
>>
>> --- alarm_handler.erl.orig      Fri Mar 24 20:08:18 2006
>> +++ alarm_handler.erl   Fri Mar 24 20:19:15 2006
>> @@ -58,7 +58,12 @@
>>  %% Returns: [{AlarmId, AlarmDesc}]
>>  %%-----------------------------------------------------------------
>>  get_alarms() ->
>> -    gen_event:call(alarm_handler, alarm_handler, get_alarms).
>> +    case gen_event:which_handlers(alarm_handler) of
>> +    [M | _] ->
>> +        gen_event:call(alarm_handler, M, get_alarms);
>> +    [] ->
>> +        []
>> +    end.
>>
>>  add_alarm_handler(Module) when atom(Module) ->
>>      gen_event:add_handler(alarm_handler, Module, []).
>>
>>
>> Serge Aleynikov wrote:
>>
>>> Hi,
>>>
>>> I've been experimenting with the reworked os_mon in R10B-10, and
>>> encountered the following issue.
>>>
>>> The documentation encourages to replace the default alarm handler
>>> with something more sophisticated.  For that reason I created a
>>> custom handler - lama_alarm_h (LAMA app in jungerl), which uses
>>> gen_event:swap_sup_handler/3.
>>>
>>> I initiate that handler prior to starting OS_MON, and then start OS_MON.
>>>
>>> In the latest release R10B-10, OS_MON calls
>>> alarm_handler:get_alarms/0 upon startup.
>>>
>>> This causes the 'alarm_handler' event manager issue a call in the
>>> alarm_handler.erl module.  However, since that handler was replaced
>>> by a custom alarm handler, the gen_event's call fails with
>>> {error, bad_module}.
>>>
>>> gen_event always dispatches a call/3 to a specific handler module
>>> passed as a parameter, e.g.:
>>>
>>> -----[alarm_handler.erl (line: 60)]-----
>>> get_alarms() ->
>>>     gen_event:call(alarm_handler, alarm_handler, get_alarms).
>>> ----------------------------------------
>>>
>>> Yet, if the alarm_handler handler was swapped by another module, the
>>> gen_event:call will report an error, therefore crashing OS_MON.
>>>
>>> One way to resolve this problem would be to introduce another
>>> exported function in gen_event:
>>>
>>> gen_event:call(EventMgrRef, Request) -> Result
>>>
>>> Can the OTP team suggest some other workaround?
>>>
>>> Serge
>>>
>>
>
>

--
Serge Aleynikov
R&D Telecom, IDT Corp.
Tel: [phone]
Fax: [phone]
[hidden email]

[Micael Karlberg]

Hi,

The new (and delete) function is an optional one. Also
there is no defined return value for this function.
Therefor it's not worth the effort to try to figure if
the result is ok or not.

/BMK

Serge Aleynikov wrote:

> Gunilla,
>
> I believe there might be another bug in SNMP revealed by my experiments
> with OS_MON & OTP_MIBS.  If mnesia is started *after* the snmp agent,
> and the snmp agent has the mibs parameter set, an attempt to initialize
> mib OIDs using instrumentation functions with the 'new' operation (such
> as otp_mib:erl_node_table(new)), leads to an ignored exception that
> ideally should prevent the SNMP agent from starting.
>
> Release file:
> =============
> {release, {"dripdb", "1.0"}, {erts, "5.4.13"},
>   [
>     {kernel  , "2.10.13"},
>     {stdlib  , "1.13.12"},
>     {sasl    , "2.1.1"},
>     {lama    , "1.0"},
>     {otp_mibs, "1.0.4"},
>     {os_mon  , "2.0"},
>     {snmp    , "4.7.1"},
>     {mnesia  , "4.2.5"}
>   ]
> }.
>
> Config file:
> ============
>
> %%------------ SNMP agent configuration ----------------------
>   {snmp,
>      [{agent,
>         [{config, [{dir, "etc/snmp/"},
>                    {force_load, true}
>                   ]},
>          {db_dir, "var/snmp_db/"},
>          {mibs,   ["mibs/priv/OTP-MIB",
>                    "mibs/priv/OTP-OS-MON-MIB"]}
>         ]
>       }
>      ]
>   }
>
> This is a trace of the error which hides the fact that there was a
> problem with creation of the 'erlNodeAlloc' table:
>
> (<0.126.0>) call
> snmpa_mib_data:call_instrumentation({me,[1,3,6,1,4,1,193,19,3,1,2,1,1,1],
>     table_entry,
>     erlNodeEntry,
>     undefined,
>     'not-accessible',
>     {otp_mib,erl_node_table,[]},
>     false,
>     [{table_entry_with_sequence,'ErlNodeEntry'}],
>     undefined,
>     undefined},new)
> (<0.126.0>) returned from snmpa_mib_data:call_instrumentation/2 ->
>   {'EXIT',{aborted,{node_not_running,drpdb@devstorm01}}}
>
> Therefore all the SNMP manager's calls to OIDs inside 'erlNodeTable' or
> 'applTable' tables fail.
>
> I can provide additional details if needed, if the information here is
> not sufficient.  I believe the proper action to do would be not to
> absorb the error in the call_instrumentation function when the Operation
> is 'new'.  I am providing the snippet of code where that exception is
> currently ignored:
>
> snmpa_mib_data.erl(line 1319):
> ==============================
> call_instrumentation(#me{entrytype = variable, mfa={M,F,A}}, Operation) ->
>     ?vtrace("call instrumentation with"
>         "~n   entrytype: variable"
>         "~n   MFA:       {~p,~p,~p}"
>         "~n   Operation: ~p",
>         [M,F,A,Operation]),
>     catch apply(M, F, [Operation | A]);
> ...
>
>
> Regards,
>
> Serge
>
>
> Gunilla Arendt wrote:
>
>> It's a bug in os_mon, it shouldn't use get_alarms().
>> Thanks for the heads up.
>>
>> Regards, Gunilla
>>
>>
>> Serge Aleynikov wrote:
>>
>>> For now I used the following patch to take care of this issue, but I
>>> would be curious to hear the opinion of the OTP staff.
>>>
>>> Regards,
>>>
>>> Serge
>>>
>>> --- alarm_handler.erl.orig      Fri Mar 24 20:08:18 2006
>>> +++ alarm_handler.erl   Fri Mar 24 20:19:15 2006
>>> @@ -58,7 +58,12 @@
>>>  %% Returns: [{AlarmId, AlarmDesc}]
>>>  %%-----------------------------------------------------------------
>>>  get_alarms() ->
>>> -    gen_event:call(alarm_handler, alarm_handler, get_alarms).
>>> +    case gen_event:which_handlers(alarm_handler) of
>>> +    [M | _] ->
>>> +        gen_event:call(alarm_handler, M, get_alarms);
>>> +    [] ->
>>> +        []
>>> +    end.
>>>
>>>  add_alarm_handler(Module) when atom(Module) ->
>>>      gen_event:add_handler(alarm_handler, Module, []).
>>>
>>>
>>> Serge Aleynikov wrote:
>>>
>>>> Hi,
>>>>
>>>> I've been experimenting with the reworked os_mon in R10B-10, and
>>>> encountered the following issue.
>>>>
>>>> The documentation encourages to replace the default alarm handler
>>>> with something more sophisticated.  For that reason I created a
>>>> custom handler - lama_alarm_h (LAMA app in jungerl), which uses
>>>> gen_event:swap_sup_handler/3.
>>>>
>>>> I initiate that handler prior to starting OS_MON, and then start
>>>> OS_MON.
>>>>
>>>> In the latest release R10B-10, OS_MON calls
>>>> alarm_handler:get_alarms/0 upon startup.
>>>>
>>>> This causes the 'alarm_handler' event manager issue a call in the
>>>> alarm_handler.erl module.  However, since that handler was replaced
>>>> by a custom alarm handler, the gen_event's call fails with
>>>> {error, bad_module}.
>>>>
>>>> gen_event always dispatches a call/3 to a specific handler module
>>>> passed as a parameter, e.g.:
>>>>
>>>> -----[alarm_handler.erl (line: 60)]-----
>>>> get_alarms() ->
>>>>     gen_event:call(alarm_handler, alarm_handler, get_alarms).
>>>> ----------------------------------------
>>>>
>>>> Yet, if the alarm_handler handler was swapped by another module, the
>>>> gen_event:call will report an error, therefore crashing OS_MON.
>>>>
>>>> One way to resolve this problem would be to introduce another
>>>> exported function in gen_event:
>>>>
>>>> gen_event:call(EventMgrRef, Request) -> Result
>>>>
>>>> Can the OTP team suggest some other workaround?
>>>>
>>>> Serge
>>>>
>>>
>>
>>
>

[Serge Aleynikov]

Micael,

It is not directly apparent that these functions are optional for SNMP
tables (especially for the OS_MON and OTP_MIBS applications, which
happended to expect the data to be stored in mnesia tables).  If by
'optional' you mean that it's up to the programmer to decide whether to
implement them or not, it's one thing, but if the programmer decides to
implement them, then, IMHO,  errors in these functions should not be
ignored.

The docs doesn't say anything about ignoring exceptions raised in the
new/delete functions:

http://www.erlang.org/doc/doc-5.4.13/lib/snmp-4.7.1/doc/html/snmp_instr_functions.html#9

   9.1.1 New / Delete Operations

   ...

   For tables:

   table_access(new [, ExtraArg1, ...])
   table_access(delete [, ExtraArg1, ...])

   These functions are called for each object in an MIB when the MIB is
   unloaded or loaded, respectively.


Moreover depending on the startup order of the snmp & mnesia apps listed
in the release file, the functionality of os_mon and otp_mibs will be
different as was illustrated in my former email.  I suggest that this
either needs to be documented, or better fixed by escalating the raised
exception.

So, if calling os_mon_mib:load(snmp_master_agent) or
otp_mib:load(snmp_master_agent) with snmp applicaion started but without
mnesia running, the functions don't fail, and if mnesia is started right
after these calls, this makes it pretty difficult to figure out why SNMP
manager is reporting errors in SNMP queries, as all applications seem to
be running as expected on the SNMP agent node.

Regards,

Serge

Micael Karlberg wrote:

> Hi,
>
> The new (and delete) function is an optional one. Also
> there is no defined return value for this function.
> Therefor it's not worth the effort to try to figure if
> the result is ok or not.
>
> /BMK
>
> Serge Aleynikov wrote:
>
>> Gunilla,
>>
>> I believe there might be another bug in SNMP revealed by my
>> experiments with OS_MON & OTP_MIBS.  If mnesia is started *after* the
>> snmp agent, and the snmp agent has the mibs parameter set, an attempt
>> to initialize mib OIDs using instrumentation functions with the 'new'
>> operation (such as otp_mib:erl_node_table(new)), leads to an ignored
>> exception that ideally should prevent the SNMP agent from starting.
>>
>> Release file:
>> =============
>> {release, {"dripdb", "1.0"}, {erts, "5.4.13"},
>>   [
>>     {kernel  , "2.10.13"},
>>     {stdlib  , "1.13.12"},
>>     {sasl    , "2.1.1"},
>>     {lama    , "1.0"},
>>     {otp_mibs, "1.0.4"},
>>     {os_mon  , "2.0"},
>>     {snmp    , "4.7.1"},
>>     {mnesia  , "4.2.5"}
>>   ]
>> }.
>>
>> Config file:
>> ============
>>
>> %%------------ SNMP agent configuration ----------------------
>>   {snmp,
>>      [{agent,
>>         [{config, [{dir, "etc/snmp/"},
>>                    {force_load, true}
>>                   ]},
>>          {db_dir, "var/snmp_db/"},
>>          {mibs,   ["mibs/priv/OTP-MIB",
>>                    "mibs/priv/OTP-OS-MON-MIB"]}
>>         ]
>>       }
>>      ]
>>   }
>>
>> This is a trace of the error which hides the fact that there was a
>> problem with creation of the 'erlNodeAlloc' table:
>>
>> (<0.126.0>) call
>> snmpa_mib_data:call_instrumentation({me,[1,3,6,1,4,1,193,19,3,1,2,1,1,1],
>>     table_entry,
>>     erlNodeEntry,
>>     undefined,
>>     'not-accessible',
>>     {otp_mib,erl_node_table,[]},
>>     false,
>>     [{table_entry_with_sequence,'ErlNodeEntry'}],
>>     undefined,
>>     undefined},new)
>> (<0.126.0>) returned from snmpa_mib_data:call_instrumentation/2 ->
>>   {'EXIT',{aborted,{node_not_running,drpdb@devstorm01}}}
>>
>> Therefore all the SNMP manager's calls to OIDs inside 'erlNodeTable'
>> or 'applTable' tables fail.
>>
>> I can provide additional details if needed, if the information here is
>> not sufficient.  I believe the proper action to do would be not to
>> absorb the error in the call_instrumentation function when the
>> Operation is 'new'.  I am providing the snippet of code where that
>> exception is currently ignored:
>>
>> snmpa_mib_data.erl(line 1319):
>> ==============================
>> call_instrumentation(#me{entrytype = variable, mfa={M,F,A}},
>> Operation) ->
>>     ?vtrace("call instrumentation with"
>>         "~n   entrytype: variable"
>>         "~n   MFA:       {~p,~p,~p}"
>>         "~n   Operation: ~p",
>>         [M,F,A,Operation]),
>>     catch apply(M, F, [Operation | A]);
>> ...
>>
>>
>> Regards,
>>
>> Serge
>>
>>
>> Gunilla Arendt wrote:
>>
>>> It's a bug in os_mon, it shouldn't use get_alarms().
>>> Thanks for the heads up.
>>>
>>> Regards, Gunilla
>>>
>>>
>>> Serge Aleynikov wrote:
>>>
>>>> For now I used the following patch to take care of this issue, but I
>>>> would be curious to hear the opinion of the OTP staff.
>>>>
>>>> Regards,
>>>>
>>>> Serge
>>>>
>>>> --- alarm_handler.erl.orig      Fri Mar 24 20:08:18 2006
>>>> +++ alarm_handler.erl   Fri Mar 24 20:19:15 2006
>>>> @@ -58,7 +58,12 @@
>>>>  %% Returns: [{AlarmId, AlarmDesc}]
>>>>  %%-----------------------------------------------------------------
>>>>  get_alarms() ->
>>>> -    gen_event:call(alarm_handler, alarm_handler, get_alarms).
>>>> +    case gen_event:which_handlers(alarm_handler) of
>>>> +    [M | _] ->
>>>> +        gen_event:call(alarm_handler, M, get_alarms);
>>>> +    [] ->
>>>> +        []
>>>> +    end.
>>>>
>>>>  add_alarm_handler(Module) when atom(Module) ->
>>>>      gen_event:add_handler(alarm_handler, Module, []).
>>>>
>>>>
>>>> Serge Aleynikov wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've been experimenting with the reworked os_mon in R10B-10, and
>>>>> encountered the following issue.
>>>>>
>>>>> The documentation encourages to replace the default alarm handler
>>>>> with something more sophisticated.  For that reason I created a
>>>>> custom handler - lama_alarm_h (LAMA app in jungerl), which uses
>>>>> gen_event:swap_sup_handler/3.
>>>>>
>>>>> I initiate that handler prior to starting OS_MON, and then start
>>>>> OS_MON.
>>>>>
>>>>> In the latest release R10B-10, OS_MON calls
>>>>> alarm_handler:get_alarms/0 upon startup.
>>>>>
>>>>> This causes the 'alarm_handler' event manager issue a call in the
>>>>> alarm_handler.erl module.  However, since that handler was replaced
>>>>> by a custom alarm handler, the gen_event's call fails with
>>>>> {error, bad_module}.
>>>>>
>>>>> gen_event always dispatches a call/3 to a specific handler module
>>>>> passed as a parameter, e.g.:
>>>>>
>>>>> -----[alarm_handler.erl (line: 60)]-----
>>>>> get_alarms() ->
>>>>>     gen_event:call(alarm_handler, alarm_handler, get_alarms).
>>>>> ----------------------------------------
>>>>>
>>>>> Yet, if the alarm_handler handler was swapped by another module,
>>>>> the gen_event:call will report an error, therefore crashing OS_MON.
>>>>>
>>>>> One way to resolve this problem would be to introduce another
>>>>> exported function in gen_event:
>>>>>
>>>>> gen_event:call(EventMgrRef, Request) -> Result
>>>>>
>>>>> Can the OTP team suggest some other workaround?
>>>>>
>>>>> Serge

[Serge Aleynikov]

Hi Micael,

What keeps bothering me is the fact that if you examine otp_mibs
application file, it doesn't mention the need for mnesia:

{application, otp_mibs,
  [..., {applications, [kernel, stdlib, snmp]}, ... ]}.

Consequently, release files such as the following:

{release, ...,
   [
     {kernel  , "2.10.13"},
     {stdlib  , "1.13.12"},
     {otp_mibs, "1.0.4"},
     {mnesia  , "4.2.5"},   % Mnesia is started in this case
     {snmp    , "4.7.1"}
   ]
}.

{release, ...,
   [
     {kernel  , "2.10.13"},
     {stdlib  , "1.13.12"},
     {otp_mibs, "1.0.4"},
     {snmp    , "4.7.1"}
   ]
}.

together with the application's config file containing:

   {snmp, [{agent, [..., {mibs, ["mibs/priv/OTP-MIB"]}, ...]}]}

will start by successfully loading OTP-MIB into the SNMP agent's data
store.  Nonetheless, the later definition of the release file will lead
to run-time errors in SNMP manager's queries to OIDs inside the OTP-MIB.

While I still think that SNMP agent shouldn't allow to load a mib if a
'new' instrumentation function is defined for an OID, and the 'new'
instrumentation call raises an exception, the OTP-MIB's dependency on
mnesia should at least be documented to avoid confusion.

Regards,

Serge



Micael Karlberg wrote:

> Hi,
>
> Yeah, I got what you where after. But what I meant was that since
> the new (and delete) function(s) are optional, exceptions is to
> be expected! And since I don't know how instrumentation functions
> are implemented, I cannot know exactly what exceptions I can
> expect (function clause, case clause, ...). This together with
> the fact that these functions have no defined reply, makes it
> simply not worth the effort to try to "handle" the reply.
>
> Regards,
>     /BMK
>
> Serge Aleynikov wrote:
>
>> Micael Karlberg wrote:
>>
>>> Hi,
>>>
>>> The new (and delete) function is an optional one. Also
>>> there is no defined return value for this function.
>>> Therefor it's not worth the effort to try to figure if
>>> the result is ok or not.
>>
>>
>> BTW, I once again reread you email and realized that I should've
>> stated  this proposal in my former email to make my point more apparent:
>>
>> _= (catch call_instrumentation(..., new)).
>>
>> change to:
>>
>> _ = call_instrumentation(..., new).
>>
>> The result still doesn't need to be examined, but I believe, the
>> exception should not be ignored.
>>
>> Serge
>>
>>
>

[Nick Linker]

Tony Rogvall wrote:

>>  Good solution :-)
>> Now I also have different idea without using recursion. It is based  
>> on the following equation
>> [F_n  ]   [1  1]   [F_n-1]
>> [     ] = [    ] * [     ]
>> [F_n-1]   [1  0]   [F_n-2]
>> And we just have to calculate k-th power of the matrix [[1,1],
>> [1,0]]. It is possible to do within O(log(k)).
>>
>
> Here is my contrib (written many years ago, it may work :-)
>
> ffib(N) ->
>     {UN1,_,_,_} = lucas_numbers(1,-1,N-1),
>     UN1.
>
> lucas_numbers(P, Q, M) ->
>     m_mult(exp({P,-Q,1,0}, M), {1,P,0,2}).
>
> exp(A, N) when tuple(A), size(A)==4, is_integer(N), N > 0 ->
>     m_exp(A, N, {1,0,0,1}).
>
> m_exp(A, 1, P) ->
>     m_mult(A,P);
> m_exp(A, N, P) when ?even(N) ->
>     m_exp(m_sqr(A), N div 2, P);
> m_exp(A, N, P) ->
>     m_exp(m_sqr(A), N div 2, m_mult(A, P)).
>
> m_mult({A11,A12,A21,A22}, {B11,B12,B21,B22}) ->
>     { A11*B11 + A12*B21, A11*B12 + A12*B22,
>       A21*B11 + A22*B21, A21*B12 + A22*B22 }.
>
> m_sqr({A,B,C,D}) ->
>     BC = B*C,
>     A_D = A + D,
>     { A*A+BC, B*A_D, C*A_D, D*D + BC }.
>
>
> /Tony

Cool. I guess the problem of computing Fibonacci is finally closed. Thanks.

Best regards,
Linker Nick.

[Nick Linker]

Joe Armstrong (AL/EAB) wrote:

>     2)  - how many (decimal) digits are there in fib(N)
>    
>    2) is much more difficult - you could or course just compute fib(N)
> then take the log -
> but this is cheating - so can you compute the number of decomal
> digits  in fib(N) without
> actually going to the trouble of computing fib(N) - now this might be
> easy but it certainly is not
> obvious...
>  
> /Joe

I'm sorry, but I meant quite different question: given a big number N,
how to compute the number of digits?

There is obvious solution: length(integer_to_list(N)), but it is not
very efficient. I wanted to have a bit faster solution... Ok, nevermind.

Best regards,
Linker Nick.

[Raimo Niskanen]

Regarding your problem about determining the number of decimal
digits in a number, I just came to think of a simple enough
brute force O(log(N)) or rather O(NN) where NN is the
number of digits in the number:

    digits(N) when is_integer(N), N >= 0 -> digits(N, 1, 0).

    digits(N, M, D) when M > N -> D;
    digits(N, M, D) -> digits(N, M*10, D+1).

Or did I misunderstand what you ment by O(log(N)),
was N the number of decimal digits in the number?



[hidden email] (Nick Linker) writes:

> Kostis Sagonas wrote:
>
> >Nick Linter wrote:
> >
> >An example would have helped us understand better what the issue is.
> >Currently, I get:
> >
> >Eshell V5.5  (abort with ^G)
> >1> math:log(...).
> >480.407
> >
> Well, I get the following result:
>
>     43> math:log10(test:fib(1476)).
>     308.116
>     44> math:log10(test:fib(1477)).
>        =ERROR REPORT==== 27-Mar-2006::10:57:47 ===
>     Error in process <0.181.0> with exit value:
>     {badarith,[{math,log10,[16#00012D269
>     C3FA9D767CB55B0DDF8E6A2DE7B1D967FF8D0BE61EB16ACD02D1A53C95A370ABD95285998D226919
>
>     D95DCA54298D92C348C27E635E1690E7858060F0DC14E885F0217413C55A1F820D6EB051F87C7C73
>
>     818AC23E4A9A00A2072C08C6697A2FAD66FC7DEBEEB7A5F582D7639A431B9C99CEC6315A9ED1C652
>
>     A81A6B59A39]},{erl_eval,do_apply,5},{shell,exprs,6},{shell,eval_loop,3}]}
>
>        ** exited: {badarith,[{math,log10,
>                                  [211475298697902185255785861961179135570552502746803
>     25217495622655863402432394766663713782393252439761186467156621190833026337742520
>
>     45520741882086869936691237540043402509431087092122991804222930097654049305082429
>
>     75773774612140021599477983006713536106549441161323499077298115887067363710153036
>
>     315849480388057657]},
>                           {erl_eval,do_apply,5},
>                           {shell,exprs,6},
>                           {shell,eval_loop,3}]} **
>
> My system is Windows XP, Erlang R10B.
>
> >fib(N) ->
> >  trunc((1/sqrt(5)) * (pow(((1+sqrt(5))/2),N) - pow(((1-sqrt(5))/2),N))).
> >
> Good solution :-)
> Now I also have different idea without using recursion. It is based on
> the following equation
> [F_n  ]   [1  1]   [F_n-1]
> [     ] = [    ] * [     ]
> [F_n-1]   [1  0]   [F_n-2]
> And we just have to calculate k-th power of the matrix
> [[1,1],[1,0]]. It is possible to do within O(log(k)).
>
> >PS. The performance you are experiencing in your version of fib/1
> >    has nothing to do with tail recursion...
> >
> Yes, exponential number of recursive calls... Thank you.
>
> I'm sorry, but most interesting question (2nd, about number of digits
> in an integer) remains unanswered. But I guess it is very rare problem
> with numbers, so if there is no answer, I will understand.
>
> Best regards,
> Linker Nick.

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB