Erlang random number generator weak?

[Adam Kelly-2]

Hi,

While writing some test code, I noticed some extremely odd behaviour
from the random module.  Given a pretty standard parallel map
function, and remembering that erlang:now() never returns the same
value twice, I'd expect this code to produce random looking output.

62> pmap:pmap(fun(_) -> {A, B, C} = erlang:now(), random:seed(A, B,
C), random:uniform(1000) end, lists:seq(1,10)).
[31,32,33,34,34,35,36,37,38,39]

Throwing away the first ten numbers gives a sequence with a definite
pattern.  (It increases and wraps around.)

63> pmap:pmap(fun(_) -> {A, B, C} = erlang:now(), random:seed(A, B,
C), [random:uniform(1000) || X <- lists:seq(1, 10)],
random:uniform(1000) end, lists:seq(1,10)).
[724,993,935,204,354,451,652,853,951,152]

It works just as well with a single process map, but then I'd have
been able to initialise the random seed once and would never have
noticed.

64> lists:map(fun(_) -> {A, B, C} = erlang:now(), random:seed(A, B,
C), random:uniform(1000) end, lists:seq(1,10)).
[288,289,290,291,292,292,293,294,294,295]

So, is it an accepted fact that two processes started at almost the
same time will produce highly correlated random sequences?

Adam.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[ERLANG-3]

Hi List,

Sometimes,  I've to start my erlang applications with "-detached"  
option.
Suppose that an application needs a directory somewhere to be able to  
run.
If this directory is missing, nothing is reported in the shell du to  
the "-detached" option.

So my question is related to how to report any error when using "-
detached"?

My erlang apps are usually started from shell scripts. It'll be very  
helpful if the erlang app can report
errors with an "exit" code status different from 0. The shell script  
will use this exit code to throw
the error to a higher level.

I'm not interested to use the error logger to solve this problem.

Regards
Y.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[mats cronqvist-5]

"Adam Kelly" <[hidden email]> writes:


> So, is it an accepted fact that two processes started at almost the
> same time will produce highly correlated random sequences?

  not afaik.

  but i think it is (possibly un-)common knowledge that given very
  similar seeds, two sequences will start out similarly.

  the mess below shows the 10:th element of 10 sequences with similar
  seeds. the sequences diverges pretty rapidly.

  mats

1> L10=lists:seq(1,10).
2> [(fun()->random:seed(1,2,N), hd(lists:reverse([random:uniform(1000)||_<-L10])) end)() || N <- L10].

 [96,449,802,154,507,859,212,565,917,270]

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Michael McDaniel-4]

 If I use -detached and want to start from scripts (on Linux),
 I use something like...

------------
 export LOGS=/some/path/to/logs

 if ! $(test -d ${LOGS}) ; then
   echo "directory ${LOGS} does not exist"
   exit
 fi

 ...
------------

 to check for directories or files (run 'man test' for all the flags).

 I do not know about MS Windows.


~Michael


On Wed, Nov 12, 2008 at 09:02:58PM +0100, ERLANG wrote:

> Hi List,
>
> Sometimes,  I've to start my erlang applications with "-detached"  
> option.
> Suppose that an application needs a directory somewhere to be able to  
> run.
> If this directory is missing, nothing is reported in the shell du to  
> the "-detached" option.
>
> So my question is related to how to report any error when using "-
> detached"?
>
> My erlang apps are usually started from shell scripts. It'll be very  
> helpful if the erlang app can report
> errors with an "exit" code status different from 0. The shell script  
> will use this exit code to throw
> the error to a higher level.
>
> I'm not interested to use the error logger to solve this problem.
>
> Regards
> Y.
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-questions

--
Michael McDaniel
Portland, Oregon, USA
http://autosys.us

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Bob Ippolito]

We use the prng from the crypto module, the built-in random module is weak.

On Wed, Nov 12, 2008 at 9:04 PM, mats cronqvist <[hidden email]> wrote:

> "Adam Kelly" <[hidden email]> writes:
>
>
>> So, is it an accepted fact that two processes started at almost the
>> same time will produce highly correlated random sequences?
>
>  not afaik.
>
>  but i think it is (possibly un-)common knowledge that given very
>  similar seeds, two sequences will start out similarly.
>
>  the mess below shows the 10:th element of 10 sequences with similar
>  seeds. the sequences diverges pretty rapidly.
>
>  mats
>
> 1> L10=lists:seq(1,10).
> 2> [(fun()->random:seed(1,2,N), hd(lists:reverse([random:uniform(1000)||_<-L10])) end)() || N <- L10].
>
>  [96,449,802,154,507,859,212,565,917,270]
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Robert Virding]

What you are running into is that the random module is erlang process based, it keeps a separate seed in each process which uses it. So when you start a number of processes using erlang:now as seed then they will get the sequence from erlang:now as first number. The subsequent numbers will diverge though.

Note that the number sequence from random is in fact quite good, it uses a good algorithm. *BUT* it is deterministic if you know one number/seed, so while it is perfectly ok for simulation and such, it is *NOT* safe to use for cryptographic purposes!

Robert

2008/11/12 Adam Kelly <[hidden email]>

Hi,

While writing some test code, I noticed some extremely odd behaviour
from the random module.  Given a pretty standard parallel map
function, and remembering that erlang:now() never returns the same
value twice, I'd expect this code to produce random looking output.

62> pmap:pmap(fun(_) -> {A, B, C} = erlang:now(), random:seed(A, B,
C), random:uniform(1000) end, lists:seq(1,10)).
[31,32,33,34,34,35,36,37,38,39]

Throwing away the first ten numbers gives a sequence with a definite
pattern.  (It increases and wraps around.)

63> pmap:pmap(fun(_) -> {A, B, C} = erlang:now(), random:seed(A, B,
C), [random:uniform(1000) || X <- lists:seq(1, 10)],
random:uniform(1000) end, lists:seq(1,10)).
[724,993,935,204,354,451,652,853,951,152]

It works just as well with a single process map, but then I'd have
been able to initialise the random seed once and would never have
noticed.

64> lists:map(fun(_) -> {A, B, C} = erlang:now(), random:seed(A, B,
C), random:uniform(1000) end, lists:seq(1,10)).
[288,289,290,291,292,292,293,294,294,295]

So, is it an accepted fact that two processes started at almost the
same time will produce highly correlated random sequences?

Adam.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[ERLANG-3]

Hi Michael,

> If I use -detached and want to start from scripts (on Linux),
> I use something like...
>
> ------------
> export LOGS=/some/path/to/logs
>
> if ! $(test -d ${LOGS}) ; then
>   echo "directory ${LOGS} does not exist"
>   exit
> fi
>
> ...
> ------------
>

Maybe I wasn't clear enough. The directory  was just an example ;-)
Suppose your application failed to start for some reason. The reason
here isn't important at all.


> to check for directories or files (run 'man test' for all the flags).
> I do not know about MS Windows.

Anyway, thanks.

Regards
Y.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Michael McDaniel-4]

On Wed, Nov 12, 2008 at 10:52:05PM +0100, ERLANG wrote:

> Hi Michael,
>
>> If I use -detached and want to start from scripts (on Linux),
>> I use something like...
>>
>> ------------
>> export LOGS=/some/path/to/logs
>>
>> if ! $(test -d ${LOGS}) ; then
>>   echo "directory ${LOGS} does not exist"
>>   exit
>> fi
>>
>> ...
>> ------------
>>
>
> Maybe I wasn't clear enough. The directory  was just an example ;-)
> Suppose your application failed to start for some reason. The reason
> here isn't important at all.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 then I use application:start(sasl) in my program, and then use rb for
 looking at all the reports (from an erl shell)

~Michael

>
>
>> to check for directories or files (run 'man test' for all the flags).
>> I do not know about MS Windows.
>
> Anyway, thanks.
>
> Regards
> Y.

--
Michael McDaniel
Portland, Oregon, USA
http://autosys.us

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[ERLANG-3]

Hi Michael,

> then I use application:start(sasl) in my program, and then use rb for
> looking at all the reports (from an erl shell)
>

That correct and my apps log everything too. But this will force you  
to use "rb" to check the logs for errors (warnings or watherver).
This is what I want to avoid.

The main idea here is to be able to return an exit code different from  
0 to notify
the shell script that erlang app failed to load for some reason.


Regards
Y.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Claes Wikstrom-2]

ERLANG wrote:

> Hi Michael,
>
>> then I use application:start(sasl) in my program, and then use rb for
>> looking at all the reports (from an erl shell)
>>
>
> That correct and my apps log everything too. But this will force you  
> to use "rb" to check the logs for errors (warnings or watherver).
> This is what I want to avoid.
>
> The main idea here is to be able to return an exit code different from  
> 0 to notify
> the shell script that erlang app failed to load for some reason.

This is indeed a real problem. If daemon startup fails, you typically
want to indicate that through a call to exit(error_code) so that the
invoker can check $?


A tail-f we solved this by hacking the emulator startup a bit. I know
we sent those patches to otp, but I'm not sure if they were ever
incorporated.

You can also implement some sort of status checker in your program.
I a call that connects to the daemon and reports someting, thus startup
could be:

erl -detached ${flags}
erl -s mymod check_status
if [ ! $? = 0 ]; then
    echo "failed to start bla bla"
    exit 1
fi


/klacke
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[ERLANG-3]

Hi Claes,


> This is indeed a real problem. If daemon startup fails, you typically
> want to indicate that through a call to exit(error_code) so that the
> invoker can check $?
>

I'l try that as I didn't know that the "error_code" will be returned  
even
with "-detached" option.

> A tail-f we solved this by hacking the emulator startup a bit. I know
> we sent those patches to otp, but I'm not sure if they were ever
> incorporated.
>
> You can also implement some sort of status checker in your program.
> I a call that connects to the daemon and reports someting, thus  
> startup
> could be:
>
> erl -detached ${flags}
> erl -s mymod check_status
> if [ ! $? = 0 ]; then
>   echo "failed to start bla bla"
>   exit 1
> fi
>

This is excatly what I decided to use but it looks a bit ugly ;-)

Thanks for yout hint ;-)

Regards
Ferret.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[ERLANG-3]

HI Claes,

> This is indeed a real problem. If daemon startup fails, you typically
> want to indicate that through a call to exit(error_code) so that the
> invoker can check $?
>

Hmmm. Calling "exit(error_code)" isn't reported to the invoker.
So $? always equals to 0.

Is this an expected behaviour of calling "exit(error_code)" when using  
"-detached" flag.

I've checked that on both Linux and OSX Leopard with RB12-5.

Regards
Ferret
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Nicolas Niclausse]

ERLANG ecrivait le 14.11.2008 14:46:

> HI Claes,
>
>> This is indeed a real problem. If daemon startup fails, you typically
>> want to indicate that through a call to exit(error_code) so that the
>> invoker can check $?
>>
>
> Hmmm. Calling "exit(error_code)" isn't reported to the invoker.
> So $? always equals to 0.
>
> Is this an expected behaviour of calling "exit(error_code)" when using  
> "-detached" flag.
>

 halt(error_code) works.

--
Nicolas
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[ERLANG-3]

Hi Nicolas,

>>> This is indeed a real problem. If daemon startup fails, you  
>>> typically
>>> want to indicate that through a call to exit(error_code) so that the
>>> invoker can check $?
>>>
>>
>> Hmmm. Calling "exit(error_code)" isn't reported to the invoker.
>> So $? always equals to 0.
>>
>> Is this an expected behaviour of calling "exit(error_code)" when  
>> using
>> "-detached" flag.
>>
>
> halt(error_code) works.

No, It doesn't.

Regards
Ferret
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Nicolas Niclausse]

ERLANG ecrivait le 14.11.2008 15:32:

> Hi Nicolas,
>
>>>> This is indeed a real problem. If daemon startup fails, you typically
>>>> want to indicate that through a call to exit(error_code) so that the
>>>> invoker can check $?
>>>>
>>>
>>> Hmmm. Calling "exit(error_code)" isn't reported to the invoker.
>>> So $? always equals to 0.
>>>
>>> Is this an expected behaviour of calling "exit(error_code)" when using
>>> "-detached" flag.
>>>
>>
>> halt(error_code) works.
>
> No, It doesn't.

ok, maybe i misunderstand the question; for the "status checker", halt can
be useful to report a specific error code to the calling process:

>erl
Erlang (BEAM) emulator version 5.6.1 [source] [async-threads:0] [hipe]
[kernel-poll:false]

Eshell V5.6.1  (abort with ^G)
1> halt(4).
>echo $?
4

--
Nicolas
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Claes Wikstrom-2]

ERLANG wrote:

> Hi Nicolas,
>
>>>> This is indeed a real problem. If daemon startup fails, you  
>>>> typically
>>>> want to indicate that through a call to exit(error_code) so that the
>>>> invoker can check $?
>>>>
>>> Hmmm. Calling "exit(error_code)" isn't reported to the invoker.
>>> So $? always equals to 0.
>>>
>>> Is this an expected behaviour of calling "exit(error_code)" when  
>>> using
>>> "-detached" flag.
>>>
>> halt(error_code) works.
>
> No, It doesn't.
>

Yes it does,

[klacke@krom]~ > erl
Erlang (BEAM) emulator version 5.6.5 [source] [smp:2] [async-threads:0] [kernel-poll:false]

Eshell V5.6.5  (abort with ^G)
1> halt(77).
[klacke@krom]~ > echo $?
77


But that's not the point. What's needed is to fork(), let the new code run
for a while and check config/whatever, if all is well it calls foo:bar(ok) and the
invoker eventually gets a status code 0, whereas if the forked process (beam)
calls foo:bar({error, Code}) the invoker of erl -detached2 ... gets Code
in the UNIX shell


/klacke

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Steven Grady]

On Nov 12, 2008, at 1:40 PM, Robert Virding wrote:

> Note that the number sequence from random is in fact quite good, it  
> uses a good algorithm. *BUT* it is deterministic if you know one  
> number/seed, so while it is perfectly ok for simulation and such, it  
> is *NOT* safe to use for cryptographic purposes!

Actually, as Bob Ippolito pointed out, it's not good.

The suggested method for seeding the random # generator is with the  
output of now/0.  But it doesn't do that great a job.  For instance:

9> random:seed(1227,148109,510934).
{10546,1383,21000}
10> random:uniform().
0.7772113669872482
11> random:uniform().
0.940382527663111
12> random:uniform().
0.9099172311315766
13> random:uniform().
0.4748388276068418
14> random:seed(1227,148109,450288).
{10546,1383,21000}
15> random:uniform().
0.7772113669872482
16> random:uniform().
0.940382527663111
17> random:uniform().
0.9099172311315766
18> random:uniform().
0.4748388276068418

Note that lines 9 and 14 use _different_ seeds, as might come out of  
now/0. But the random generator still generates the same values.

Instead, use crypto.  Bonus: no seeding required.

        Steven




_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Hynek Vychodil]

No, it isn't.

$ erl -eval 'halt(77).'
Erlang (BEAM) emulator version 5.6.3 [source] [smp:2] [async-threads:0] [hipe] [kernel-poll:false]

$ echo $?
77
$ erl -detached -eval 'halt(77).'
$ echo $?
0

On Tue, Nov 18, 2008 at 8:47 PM, Claes Wikstrom <[hidden email]> wrote:

ERLANG wrote:
> Hi Nicolas,
>
>>>> This is indeed a real problem. If daemon startup fails, you
>>>> typically
>>>> want to indicate that through a call to exit(error_code) so that the
>>>> invoker can check $?
>>>>
>>> Hmmm. Calling "exit(error_code)" isn't reported to the invoker.
>>> So $? always equals to 0.
>>>
>>> Is this an expected behaviour of calling "exit(error_code)" when
>>> using
>>> "-detached" flag.
>>>
>> halt(error_code) works.
>
> No, It doesn't.
>

Yes it does,

[klacke@krom]~ > erl
Erlang (BEAM) emulator version 5.6.5 [source] [smp:2] [async-threads:0] [kernel-poll:false]

Eshell V5.6.5  (abort with ^G)
1> halt(77).
[klacke@krom]~ > echo $?
77


But that's not the point. What's needed is to fork(), let the new code run
for a while and check config/whatever, if all is well it calls foo:bar(ok) and the
invoker eventually gets a status code 0, whereas if the forked process (beam)
calls foo:bar({error, Code}) the invoker of erl -detached2 ... gets Code
in the UNIX shell


/klacke

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

--
--Hynek (Pichi) Vychodil

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions