Erlang/OTP 21.0-rc1 (Release Candidate)

[Henrik Nord X]

OTP 21 Release Candidate 1

This is the first of two planned release candidates before the OTP 21
release. The intention whit this release is that you as users try it
and give us feedback if something does not work as expected. Could be a
bug,an unexpected incompatibility, a significant change of
characteristics in a negative direction, etc. 

Erlang/OTP 21 is a new major release with new features, improvements as
well as incompatibilities.

Potential Incompatibilities

 * All Corba applications are now moved from the OTP repository
    * A new Corba repository will be created https://github.com/erlang
 * New applications ftp and tftp, moved from inets
 * ssl no longer supports 3_DES cipher suites or RSA-key exchange
cipher suites by default
 * erlang:monitor on a primitive node (erl_interface, jinterface, etc)
will no longer fail with badarg exception. Instead a monitor will be
created, but it will only supervise the connection to the node.

 Highlights

 Erts:

    * Enhanced IO scalability
    * Support for usage of distribution controller processes for
      alternative transports, routing etc
    * compact instructions on 64bit systems for code below 4GB 20% less
      memory for loaded code
    * Rewrite of the efile-driver with NIFs and "Dirty schedulers"
      resulting in faster file operations
    * non-smp VM removed
    * link and monitor optimized for scalability
    * os:getenv/putenv now work on thread-safe emulation. No longer in
      sync with libc getenv(3). Manual synchronization will be needed.

Compiler:
    * Misc compiler optimizations including contributions from the
      Elixir team resulting in 10% improvements in benchmarks
    * "Tuple calls" have been removed from the run-time system.
    * Code such as f({ok, Val}) -> {ok, Val} is now automatically
      rewritten to f({ok, Val} = Tuple) -> Tuple. this reduces code
      size, execution time, and removed GC pressure.
    * More information in stacktrace from a number of operators
    * erlang:get_stacktrace/0 deprecated to be replaced with try ...
      catch C:R:Stacktrace -> ...
    * Creation of small maps with literal keys optimized.

Security:
    * DTLS support in SSL
    * Enhanced support for distribution over TLS
    * "unsecure" ciphers removed from defaults in SSL and SSH.
    * A new option value defined to facilitate implementing exec
      servers. Old option kept for compatibility, but now gives errors
      on stderror.

Standard libraries:
    * New API for logging, logger
    * New uri_string module for parsing URIs according to "The
      standard"
    * New function lists:search(list,fun/1) -> {ok, Value} | false
    * Changed default behaviour of .erlang loading. escript, erlc,
      dialyzer and typer no longer load an .erlang at all.

For more details see
http://erlang.org/download/otp_src_21.0-rc1.readme

Pre built versions for Windows can be fetched here:
http://erlang.org/download/otp_win32_21.0-rc1.exe
http://erlang.org/download/otp_win64_21.0-rc1.exe

Online documentation can be browsed here:
http://erlang.org/documentation/doc-10.0-rc1/doc/

The Erlang/OTP source can also be found at GitHub on the official
Erlang repository,
https://github.com/erlang/otp with tag OTP-21.0-rc1


Thank you for all your contributions!
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Frans Schneider-2]

Is the fun/1 really the second argument in the call lists:search/2?

lists:search(list,fun/1) -> {ok, Value} | false

Doesn't feel consistent with the other functions in the list module
which have the fun/1 as the first argument.

Cheers,

Frans

On 05/02/2018 01:37 PM, Henrik Nord X wrote:
> lists:search(list,fun/1) -> {ok, Value} | false
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Dmitry Klionsky-2]

No, it's fine.

https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1402-L1413


On 05/02/2018 04:06 PM, Frans Schneider wrote:

> Is the fun/1 really the second argument in the call lists:search/2?
>
> lists:search(list,fun/1) -> {ok, Value} | false
>
> Doesn't feel consistent with the other functions in the list module
> which have the fun/1 as the first argument.
>
> Cheers,
>
> Frans
>
> On 05/02/2018 01:37 PM, Henrik Nord X wrote:
>> lists:search(list,fun/1) -> {ok, Value} | false
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

--
BR,
Dmitry

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Alexander Petrovsky-2]

Totally agree with Frans Schneider, the arguments order in different modules in stdlib differs from one to another. And I think will be better reorder args in lists:search.

For me much more close approach suggested by Chris Okasaki, short summary from SO (https://stackoverflow.com/questions/5863128/ordering-of-parameters-to-make-use-of-currying):

For languages that support currying and partial-application easily, there is one compelling series of arguments, originally from Chris Okasaki:

• Put the data structure as the last argument

Why? You can then compose operations on the data nicely. E.g. insert 1 $ insert 2 $ insert 3 $ s. This also helps for functions on state.
Standard libraries such as "containers" follow this convention.
Alternate arguments are sometimes given to put the data structure first, so it can be closed over, yielding functions on a static structure (e.g. lookup) that are a bit more concise. However, the broad consensus seems to be that this is less of a win, especially since it pushes you towards heavily parenthesized code.

• Put the most varying argument last

For recursive functions, it is common to put the argument that varies the most (e.g. an accumulator) as the last argument, while the argument that varies the least (e.g. a function argument) at the start. This composes well with the data structure last style.

So, the

lists:search(fun/1, list) -> {ok, Value} | false

 will be more appropriate then

lists:search(list, fun/1) -> {ok, Value} | false

 

ср, 2 мая 2018 г. в 16:10, Dmitry Klionsky <[hidden email]>:

No, it's fine.

https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1402-L1413


On 05/02/2018 04:06 PM, Frans Schneider wrote:
> Is the fun/1 really the second argument in the call lists:search/2?
>
> lists:search(list,fun/1) -> {ok, Value} | false
>
> Doesn't feel consistent with the other functions in the list module
> which have the fun/1 as the first argument.
>
> Cheers,
>
> Frans
>
> On 05/02/2018 01:37 PM, Henrik Nord X wrote:
>> lists:search(list,fun/1) -> {ok, Value} | false
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

--
BR,
Dmitry

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Lukas Larsson-8]

The argument order in the email is a typo, which is what Dmitry tried to point out.

http://erlang.org/documentation/doc-10.0-rc1/lib/stdlib-3.5/doc/html/lists.html#search-2

It is lists:search(Pred, List).

On Wed, 2 May 2018, 21:12 Alexander Petrovsky, <[hidden email]> wrote:

Totally agree with Frans Schneider, the arguments order in different modules in stdlib differs from one to another. And I think will be better reorder args in lists:search.

For me much more close approach suggested by Chris Okasaki, short summary from SO (https://stackoverflow.com/questions/5863128/ordering-of-parameters-to-make-use-of-currying):

For languages that support currying and partial-application easily, there is one compelling series of arguments, originally from Chris Okasaki:

• Put the data structure as the last argument

Why? You can then compose operations on the data nicely. E.g. insert 1 $ insert 2 $ insert 3 $ s. This also helps for functions on state.
Standard libraries such as "containers" follow this convention.
Alternate arguments are sometimes given to put the data structure first, so it can be closed over, yielding functions on a static structure (e.g. lookup) that are a bit more concise. However, the broad consensus seems to be that this is less of a win, especially since it pushes you towards heavily parenthesized code.

• Put the most varying argument last

For recursive functions, it is common to put the argument that varies the most (e.g. an accumulator) as the last argument, while the argument that varies the least (e.g. a function argument) at the start. This composes well with the data structure last style.

So, the

lists:search(fun/1, list) -> {ok, Value} | false

 will be more appropriate then

lists:search(list, fun/1) -> {ok, Value} | false

 

ср, 2 мая 2018 г. в 16:10, Dmitry Klionsky <[hidden email]>:

No, it's fine.

https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1402-L1413


On 05/02/2018 04:06 PM, Frans Schneider wrote:
> Is the fun/1 really the second argument in the call lists:search/2?
>
> lists:search(list,fun/1) -> {ok, Value} | false
>
> Doesn't feel consistent with the other functions in the list module
> which have the fun/1 as the first argument.
>
> Cheers,
>
> Frans
>
> On 05/02/2018 01:37 PM, Henrik Nord X wrote:
>> lists:search(list,fun/1) -> {ok, Value} | false
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

--
BR,
Dmitry

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Svilen Ivanov]

In Kernel User's Guide new logger API chapter
"2.6 Example: implement a handler"
callback function 'removing_handler' is documented to accept single
parameter logger:handler_id(),
but in the example for handler that prints to file it takes two
parameters (Id and Config).

The current implementation seems to call this function with single
parameter HandlerId, but that makes not possible to cleanly close the
file descriptor as shown in the example.

So, removing_handler/1 or removing_handler/2 ?

BR, Svilen

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Loïc Hoguin-3]

Hello!

Just a quick heads up, having a couple issues:

* It doesn't compile on FreeBSD. I can get you more details later but
"kerl build git https://github.com/erlang/otp OTP-21.0-rc1 OTP-21.0-rc1"
might be a quicker alternative.

* SSL is broken. See [1] for example. I can see the same thing happening
on 5 different Linux distributions (with different OpenSSL versions) and
on OSX. A quick try in the shell is not much better:

1> ssl:start().
ok
2> ssl:connect("google.com", 80, []).
{error,{tls_alert,"bad record mac"}}

Keep up the good work!

[1]
https://builds.ninenines.eu/logs/ranch/78/ubuntu/ct_run.ct_ranch@ubuntu0.2018-05-03_11.12.51/ninenines.ranch.acceptor_SUITE.logs/run.2018-05-03_11.12.52/acceptor_suite.ssl_echo.html

On 05/02/2018 01:37 PM, Henrik Nord X wrote:

> OTP 21 Release Candidate 1
>
> This is the first of two planned release candidates before the OTP 21
> release. The intention whit this release is that you as users try it
> and give us feedback if something does not work as expected. Could be a
> bug,an unexpected incompatibility, a significant change of
> characteristics in a negative direction, etc.
>
> Erlang/OTP 21 is a new major release with new features, improvements as
> well as incompatibilities.
>
> Potential Incompatibilities
>
>   * All Corba applications are now moved from the OTP repository
>      * A new Corba repository will be created https://github.com/erlang
>   * New applications ftp and tftp, moved from inets
>   * ssl no longer supports 3_DES cipher suites or RSA-key exchange
> cipher suites by default
>   * erlang:monitor on a primitive node (erl_interface, jinterface, etc)
> will no longer fail with badarg exception. Instead a monitor will be
> created, but it will only supervise the connection to the node.
>
>   Highlights
>
>   Erts:
>
>      * Enhanced IO scalability
>      * Support for usage of distribution controller processes for
>        alternative transports, routing etc
>      * compact instructions on 64bit systems for code below 4GB 20% less
>        memory for loaded code
>      * Rewrite of the efile-driver with NIFs and "Dirty schedulers"
>        resulting in faster file operations
>      * non-smp VM removed
>      * link and monitor optimized for scalability
>      * os:getenv/putenv now work on thread-safe emulation. No longer in
>        sync with libc getenv(3). Manual synchronization will be needed.
>
> Compiler:
>      * Misc compiler optimizations including contributions from the
>        Elixir team resulting in 10% improvements in benchmarks
>      * "Tuple calls" have been removed from the run-time system.
>      * Code such as f({ok, Val}) -> {ok, Val} is now automatically
>        rewritten to f({ok, Val} = Tuple) -> Tuple. this reduces code
>        size, execution time, and removed GC pressure.
>      * More information in stacktrace from a number of operators
>      * erlang:get_stacktrace/0 deprecated to be replaced with try ...
>        catch C:R:Stacktrace -> ...
>      * Creation of small maps with literal keys optimized.
>
> Security:
>      * DTLS support in SSL
>      * Enhanced support for distribution over TLS
>      * "unsecure" ciphers removed from defaults in SSL and SSH.
>      * A new option value defined to facilitate implementing exec
>        servers. Old option kept for compatibility, but now gives errors
>        on stderror.
>
> Standard libraries:
>      * New API for logging, logger
>      * New uri_string module for parsing URIs according to "The
>        standard"
>      * New function lists:search(list,fun/1) -> {ok, Value} | false
>      * Changed default behaviour of .erlang loading. escript, erlc,
>        dialyzer and typer no longer load an .erlang at all.
>
> For more details see
> http://erlang.org/download/otp_src_21.0-rc1.readme
>
> Pre built versions for Windows can be fetched here:
> http://erlang.org/download/otp_win32_21.0-rc1.exe
> http://erlang.org/download/otp_win64_21.0-rc1.exe
>
> Online documentation can be browsed here:
> http://erlang.org/documentation/doc-10.0-rc1/doc/
>
> The Erlang/OTP source can also be found at GitHub on the official
> Erlang repository,
> https://github.com/erlang/otp with tag OTP-21.0-rc1
>
>
> Thank you for all your contributions!
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Loïc Hoguin-3]

On 05/03/2018 01:54 PM, Loïc Hoguin wrote:

> Hello!
>
> Just a quick heads up, having a couple issues:
>
> * It doesn't compile on FreeBSD. I can get you more details later but
> "kerl build git https://github.com/erlang/otp OTP-21.0-rc1 OTP-21.0-rc1"
> might be a quicker alternative.
>
> * SSL is broken. See [1] for example. I can see the same thing happening
> on 5 different Linux distributions (with different OpenSSL versions) and
> on OSX. A quick try in the shell is not much better:
>
> 1> ssl:start().
> ok
> 2> ssl:connect("google.com", 80, []).
> {error,{tls_alert,"bad record mac"}}

That one was a brain fart. This works when the port is correct.

> Keep up the good work!
>
> [1]
> https://builds.ninenines.eu/logs/ranch/78/ubuntu/ct_run.ct_ranch@ubuntu0.2018-05-03_11.12.51/ninenines.ranch.acceptor_SUITE.logs/run.2018-05-03_11.12.52/acceptor_suite.ssl_echo.html 
>
>
> On 05/02/2018 01:37 PM, Henrik Nord X wrote:
>> OTP 21 Release Candidate 1
>>
>> This is the first of two planned release candidates before the OTP 21
>> release. The intention whit this release is that you as users try it
>> and give us feedback if something does not work as expected. Could be a
>> bug,an unexpected incompatibility, a significant change of
>> characteristics in a negative direction, etc.
>>
>> Erlang/OTP 21 is a new major release with new features, improvements as
>> well as incompatibilities.
>>
>> Potential Incompatibilities
>>
>>   * All Corba applications are now moved from the OTP repository
>>      * A new Corba repository will be created https://github.com/erlang
>>   * New applications ftp and tftp, moved from inets
>>   * ssl no longer supports 3_DES cipher suites or RSA-key exchange
>> cipher suites by default
>>   * erlang:monitor on a primitive node (erl_interface, jinterface, etc)
>> will no longer fail with badarg exception. Instead a monitor will be
>> created, but it will only supervise the connection to the node.
>>
>>   Highlights
>>
>>   Erts:
>>
>>      * Enhanced IO scalability
>>      * Support for usage of distribution controller processes for
>>        alternative transports, routing etc
>>      * compact instructions on 64bit systems for code below 4GB 20% less
>>        memory for loaded code
>>      * Rewrite of the efile-driver with NIFs and "Dirty schedulers"
>>        resulting in faster file operations
>>      * non-smp VM removed
>>      * link and monitor optimized for scalability
>>      * os:getenv/putenv now work on thread-safe emulation. No longer in
>>        sync with libc getenv(3). Manual synchronization will be needed.
>>
>> Compiler:
>>      * Misc compiler optimizations including contributions from the
>>        Elixir team resulting in 10% improvements in benchmarks
>>      * "Tuple calls" have been removed from the run-time system.
>>      * Code such as f({ok, Val}) -> {ok, Val} is now automatically
>>        rewritten to f({ok, Val} = Tuple) -> Tuple. this reduces code
>>        size, execution time, and removed GC pressure.
>>      * More information in stacktrace from a number of operators
>>      * erlang:get_stacktrace/0 deprecated to be replaced with try ...
>>        catch C:R:Stacktrace -> ...
>>      * Creation of small maps with literal keys optimized.
>>
>> Security:
>>      * DTLS support in SSL
>>      * Enhanced support for distribution over TLS
>>      * "unsecure" ciphers removed from defaults in SSL and SSH.
>>      * A new option value defined to facilitate implementing exec
>>        servers. Old option kept for compatibility, but now gives errors
>>        on stderror.
>>
>> Standard libraries:
>>      * New API for logging, logger
>>      * New uri_string module for parsing URIs according to "The
>>        standard"
>>      * New function lists:search(list,fun/1) -> {ok, Value} | false
>>      * Changed default behaviour of .erlang loading. escript, erlc,
>>        dialyzer and typer no longer load an .erlang at all.
>>
>> For more details see
>> http://erlang.org/download/otp_src_21.0-rc1.readme
>>
>> Pre built versions for Windows can be fetched here:
>> http://erlang.org/download/otp_win32_21.0-rc1.exe
>> http://erlang.org/download/otp_win64_21.0-rc1.exe
>>
>> Online documentation can be browsed here:
>> http://erlang.org/documentation/doc-10.0-rc1/doc/
>>
>> The Erlang/OTP source can also be found at GitHub on the official
>> Erlang repository,
>> https://github.com/erlang/otp with tag OTP-21.0-rc1
>>
>>
>> Thank you for all your contributions!
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Lukas Larsson-8]

Hello,

On Thu, May 3, 2018 at 1:54 PM, Loïc Hoguin <[hidden email]> wrote:

* It doesn't compile on FreeBSD. I can get you more details later but "kerl build git https://github.com/erlang/otp OTP-21.0-rc1 OTP-21.0-rc1" might be a quicker alternative.

Strange, works fine on FreeBSD 10.4 and 11.1 in our tests.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Max Lapshin-2]

Not sure about new logging:

$ erl

Erlang/OTP 21 [RELEASE CANDIDATE 1] [erts-10.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Eshell V10.0  (abort with ^G)

1> whereis(error_logger).

undefined

2> 

From http://erlang.org/documentation/doc-10.0-rc1/lib/kernel-6.0/doc/html/error_logger.html  I do not see that error_logger is no more with us.

Is it a existing process or it is completely abandoned?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Lukas Larsson-8]

On Thu, May 3, 2018 at 3:07 PM, Max Lapshin <[hidden email]> wrote:

Not sure about new logging:

$ erl

Erlang/OTP 21 [RELEASE CANDIDATE 1] [erts-10.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Eshell V10.0  (abort with ^G)

1> whereis(error_logger).

undefined

2> 

From http://erlang.org/documentation/doc-10.0-rc1/lib/kernel-6.0/doc/html/error_logger.html  I do not see that error_logger is no more with us.

Is it a existing process or it is completely abandoned?

 

The error_logger event handler is started lazily when the first error_logger:add_report_handler call is done. So the new logging framework does not use the error_logger process, but it is started if legacy code uses it. All calls to error_logger:*_report/msg are redirected to logger which then decides what to do with them.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Kostis Sagonas-2]

On 05/03/2018 03:29 PM, Lukas Larsson wrote:

>
>
> On Thu, May 3, 2018 at 3:07 PM, Max Lapshin <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Not sure about new logging:
>
>     ...
>
>      From
>     http://erlang.org/documentation/doc-10.0-rc1/lib/kernel-6.0/doc/html/error_logger.html
>     <http://erlang.org/documentation/doc-10.0-rc1/lib/kernel-6.0/doc/html/error_logger.html>
>     I do not see that error_logger is no more with us.
>
>     Is it a existing process or it is completely abandoned?
>
> The error_logger event handler is started lazily when the first
> error_logger:add_report_handler call is done. So the new logging
> framework does not use the error_logger process, but it is started if
> legacy code uses it. All calls to error_logger:*_report/msg are
> redirected to logger which then decides what to do with them.

Something has clearly changed in the behaviour of the error logger in
ways that I am not sure I completely understand.

I have a simple script to run the HiPE tests which looks as follows:
=====================================
#! /bin/sh

export ERL_TOP=$PWD
export PATH=$ERL_TOP/bin:$PATH

TDIR=/tmp/otp_tests

rm -rf $TDIR
./otp_build tests $TDIR
cd $TDIR
ct_run -dir hipe_test
=====================================

Till last Friday, this script was running fine, even on master.  Since
Friday when a lot of changes about the new 'logger' were merged, this
script produces tons of lines of the form:

   =ERROR REPORT==== 3-May-2018::15:38:47.765241 ===
   Error in process <0.1524.0> on node ct@MACHINE with exit value:
   {timeout_value,[{basic_exceptions,bad_guy,2,[]}]}

   =WARNING REPORT==== 3-May-2018::15:38:51.663313 ===
   Process <0.2680.0> on node 'ct@MACHINE' enabled access to the
emulator internal state.
   NOTE: This is an erts internal test feature and should *only* be used
by OTP test-suites.


How does one suppress these?

Kostis
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Lukas Larsson-3]

On Thu, May 3, 2018 at 3:41 PM, Kostis Sagonas <[hidden email]> wrote:

On 05/03/2018 03:29 PM, Lukas Larsson wrote:

On Thu, May 3, 2018 at 3:07 PM, Max Lapshin <[hidden email] <mailto:[hidden email]>> wrote:

    Not sure about new logging:

    ...

     From
    http://erlang.org/documentation/doc-10.0-rc1/lib/kernel-6.0/doc/html/error_logger.html
    <http://erlang.org/documentation/doc-10.0-rc1/lib/kernel-6.0/doc/html/error_logger.html>     I do not see that error_logger is no more with us.

    Is it a existing process or it is completely abandoned?

The error_logger event handler is started lazily when the first error_logger:add_report_handler call is done. So the new logging framework does not use the error_logger process, but it is started if legacy code uses it. All calls to error_logger:*_report/msg are redirected to logger which then decides what to do with them.

Something has clearly changed in the behaviour of the error logger in ways that I am not sure I completely understand.

I have a simple script to run the HiPE tests which looks as follows:
=====================================
#! /bin/sh

export ERL_TOP=$PWD
export PATH=$ERL_TOP/bin:$PATH

TDIR=/tmp/otp_tests

rm -rf $TDIR
./otp_build tests $TDIR
cd $TDIR
ct_run -dir hipe_test
=====================================

Till last Friday, this script was running fine, even on master.  Since Friday when a lot of changes about the new 'logger' were merged, this script produces tons of lines of the form:

  =ERROR REPORT==== 3-May-2018::15:38:47.765241 ===
  Error in process <0.1524.0> on node ct@MACHINE with exit value:
  {timeout_value,[{basic_exceptions,bad_guy,2,[]}]}

  =WARNING REPORT==== 3-May-2018::15:38:51.663313 ===
  Process <0.2680.0> on node 'ct@MACHINE' enabled access to the emulator internal state.
  NOTE: This is an erts internal test feature and should *only* be used by OTP test-suites.


How does one suppress these?

There seems to be some bug in the cth_log_redirect. You should be able to do "ct_run -kernel logger_dest silent -dir hipe_test" to work around it for now. We'll fix the bug.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Kostis Sagonas-2]

On 05/03/2018 03:53 PM, Lukas Larsson wrote:

>
>
> On Thu, May 3, 2018 at 3:41 PM, Kostis Sagonas <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Something has clearly changed in the behaviour of the error logger
>     in ways that I am not sure I completely understand.
>
>     I have a simple script to run the HiPE tests which looks as follows:
>     ... <SNIP>
>
>
>     How does one suppress these?
>
>
> There seems to be some bug in the cth_log_redirect. You should be able
> to do "ct_run -kernel logger_dest silent -dir hipe_test" to work around
> it for now. We'll fix the bug.

Yep, this works.

Thanks for the quick reply!

Kostis
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Max Lapshin-2]

Ok, I was using direct gen_event:   gen_event:add_sup_handler(error_logger, Module, Config)

On Thu, May 3, 2018 at 4:29 PM, Lukas Larsson <[hidden email]> wrote:

On Thu, May 3, 2018 at 3:07 PM, Max Lapshin <[hidden email]> wrote:

Not sure about new logging:

$ erl

Erlang/OTP 21 [RELEASE CANDIDATE 1] [erts-10.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Eshell V10.0  (abort with ^G)

1> whereis(error_logger).

undefined

2> 

From http://erlang.org/documentation/doc-10.0-rc1/lib/kernel-6.0/doc/html/error_logger.html  I do not see that error_logger is no more with us.

Is it a existing process or it is completely abandoned?

 

The error_logger event handler is started lazily when the first error_logger:add_report_handler call is done. So the new logging framework does not use the error_logger process, but it is started if legacy code uses it. All calls to error_logger:*_report/msg are redirected to logger which then decides what to do with them.

Lukas

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Loïc Hoguin-3]

Hello,

On 05/03/2018 01:54 PM, Loïc Hoguin wrote:
> * SSL is broken. See [1] for example. I can see the same thing happening
> on 5 different Linux distributions (with different OpenSSL versions) and
> on OSX. A quick try in the shell is not much better:

OK it's just a very misleading error message I think.

Switching my server's test keys from RSA to DSA fixes it so I think this
issue is caused by:

   OTP-14769    Application(s): ssl

                For security reasons RSA-key exchange cipher suites are
                no longer supported by default

Still, it probably should provide a more helpful error message than this:

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03
11:13:04.343 ***
=INFO REPORT==== 3-May-2018::11:13:04.342940 ===
TLS server: In state hello at tls_handshake.erl:130 generated SERVER
ALERT: Fatal - Handshake Failure - malformed_handshake_data

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03
11:13:04.348 ***
=INFO REPORT==== 3-May-2018::11:13:04.348265 ===
TLS client: In state hello received SERVER ALERT: Fatal - Handshake Failure

"malformed_handshake_data" sounds like the client would have sent a
malformed handshake, ie bad data, when the actual issue seems to be that
the certificate configured is no longer supported. The server generating
an alert about its own certificate doesn't sound quite right either.

That being said I do not really know the intent so I'm guessing a bit.
All I know for sure is that it's confusing.

Cheers,

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Ingela Andin]

Hi!

2018-05-03 18:08 GMT+02:00 Loïc Hoguin <[hidden email]>:

Hello,

On 05/03/2018 01:54 PM, Loïc Hoguin wrote:

* SSL is broken. See [1] for example. I can see the same thing happening on 5 different Linux distributions (with different OpenSSL versions) and on OSX. A quick try in the shell is not much better:

Thank you for shouting, that is what the release candidate is for. So we can catch the problems early!

 

OK it's just a very misleading error message I think.

Well that depends, this is not really an error message that you should get unless you have a buggy or malicious client. But of course now we might be getting it due to a bug and then

it could be misleading!

 

Switching my server's test keys from RSA to DSA fixes it so I think this issue is caused by:

  OTP-14769    Application(s): ssl

               For security reasons RSA-key exchange cipher suites are
               no longer supported by default

I do not really suspect this change. RSA-certificates are still supported. Just cipher suites using RSA encryption/decryption in the key exchange process are not supported.

When you switched to a DSA-certificate an other cipher suite was picked that did not expose the problem.  If there had been no common cipher suites you would have got another error.

 

Still, it probably should provide a more helpful error message than this:

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03 11:13:04.343 ***
=INFO REPORT==== 3-May-2018::11:13:04.342940 ===
TLS server: In state hello at tls_handshake.erl:130 generated SERVER ALERT: Fatal - Handshake Failure - malformed_handshake_data

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03 11:13:04.348 ***
=INFO REPORT==== 3-May-2018::11:13:04.348265 ===
TLS client: In state hello received SERVER ALERT: Fatal - Handshake Failure

"malformed_handshake_data" sounds like the client would have sent a malformed handshake, ie bad data, when the actual issue seems to be that the certificate configured is no longer supported. The server generating an alert about its own certificate doesn't sound quite right either.

That being said I do not really know the intent so I'm guessing a bit. All I know for sure is that it's confusing.

This error is consistent with one of the errors I am seeing in the nightly builds when running OpenSSL with only default parameters so I suspect something is off in combination

version negotiation and cipher suite selection checks. I am looking in to it!

Regards Ingela Erlang/OTP Team

 

Cheers,

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Heinz N. Gies]

I’m having quite some trouble on the SSL front too. I’m giving it a try to move the riak_core code to R21 and the ssl test [1] fails.

If I use the original keys (in the repo under test/site1/2-…) I get a handshake failure.

Using those keys I get an handshake failure too:

openssl req -x509 -newkey rsa:4096 -keyout test/site1-key.pem -out test/site1-cert.pem -days 3650 -nodes -subj '/CN=US'

openssl req -x509 -newkey rsa:4096 -keyout test/site2-key.pem -out test/site2-cert.pem -days 3650 -nodes -subj '/CN=US’

DSA keys

openssl dsaparam -out dsaparams-site1.pem 1024

openssl dsaparam -out dsaparams-site2.pem 1024

openssl req -x509 -newkey dsa:dsaparams-site1.pem -keyout test/site1-key.pem -out test/site1-cert.pem -days 3650 -nodes -subj '/CN=US'

openssl req -x509 -newkey dsa:dsaparams-site2.pem -keyout test/site2-key.pem -out test/site2-cert.pem -days 3650 -nodes -subj '/CN=US’

Fail with:

{badmatch,

     {error,

         {options,

             {keyfile,"test/site1-key.pem",

                 {error,

                     {asn1,

                         {{wrong_tag,

                              {{expected,16},

                               {got,2,

                                   {2,

                                    <<74,130,213,43,78,73,39,24,206,62,159,

                                      168,30,65,230,24,14,31,209,192>>}}}},

                          [{'OTP-PUB-KEY',match_tags,2,

                               [{file,"OTP-PUB-KEY.erl"},{line,20535}]},

                           {'OTP-PUB-KEY',dec_DSAPrivateKey,2,

                               [{file,"OTP-PUB-KEY.erl"},{line,1789}]},

                           {'OTP-PUB-KEY',decode,2,

                               [{file,"OTP-PUB-KEY.erl"},{line,1103}]},

                           {public_key,der_decode,2,

                               [{file,"public_key.erl"},{line,248}]},

                           {ssl_config,init_private_key,5,

                               [{file,"ssl_config.erl"},{line,114}]},

                           {ssl_config,init,2,

                               [{file,"ssl_config.erl"},{line,38}]},

                           {ssl_connection,ssl_config,4,

                               [{file,"ssl_connection.erl"},{line,571}]},

                           {tls_connection,init,1,

                               [{file,"tls_connection.erl"},

                                {line,116}]}]}}}}}}}

EC keys fail with a handshake failure too

openssl ecparam -out ecparams-site1.pem -name prime256v1

openssl ecparam -out ecparams-site2.pem -name prime256v1

openssl req -x509 -newkey ec:ecparams-site1.pem -keyout test/site1-key.pem -out test/site1-cert.pem -days 3650 -nodes -subj '/CN=US'

openssl req -x509 -newkey ec:ecparams-site2.pem -keyout test/site2-key.pem -out test/site2-cert.pem -days 3650 -nodes -subj '/CN=US'

https://github.com/Kyorai/riak_core/blob/develop/src/riak_core_tcp_mon.erl#L450

On 4. May 2018, at 09:32, Ingela Andin <[hidden email]> wrote:

Hi!

2018-05-03 18:08 GMT+02:00 Loïc Hoguin <[hidden email]>:

Hello,

On 05/03/2018 01:54 PM, Loïc Hoguin wrote:

* SSL is broken. See [1] for example. I can see the same thing happening on 5 different Linux distributions (with different OpenSSL versions) and on OSX. A quick try in the shell is not much better:

Thank you for shouting, that is what the release candidate is for. So we can catch the problems early!

 

OK it's just a very misleading error message I think.

Well that depends, this is not really an error message that you should get unless you have a buggy or malicious client. But of course now we might be getting it due to a bug and then

it could be misleading!

 

Switching my server's test keys from RSA to DSA fixes it so I think this issue is caused by:

  OTP-14769    Application(s): ssl

               For security reasons RSA-key exchange cipher suites are
               no longer supported by default

I do not really suspect this change. RSA-certificates are still supported. Just cipher suites using RSA encryption/decryption in the key exchange process are not supported.

When you switched to a DSA-certificate an other cipher suite was picked that did not expose the problem.  If there had been no common cipher suites you would have got another error.

 

Still, it probably should provide a more helpful error message than this:

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03 11:13:04.343 ***
=INFO REPORT==== 3-May-2018::11:13:04.342940 ===
TLS server: In state hello at tls_handshake.erl:130 generated SERVER ALERT: Fatal - Handshake Failure - malformed_handshake_data

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03 11:13:04.348 ***
=INFO REPORT==== 3-May-2018::11:13:04.348265 ===
TLS client: In state hello received SERVER ALERT: Fatal - Handshake Failure

"malformed_handshake_data" sounds like the client would have sent a malformed handshake, ie bad data, when the actual issue seems to be that the certificate configured is no longer supported. The server generating an alert about its own certificate doesn't sound quite right either.

That being said I do not really know the intent so I'm guessing a bit. All I know for sure is that it's confusing.

This error is consistent with one of the errors I am seeing in the nightly builds when running OpenSSL with only default parameters so I suspect something is off in combination

version negotiation and cipher suite selection checks. I am looking in to it!

Regards Ingela Erlang/OTP Team 

 

Cheers,

-- 
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Roger Lipscombe-2]

On 4 May 2018 at 08:32, Ingela Andin <[hidden email]> wrote:
> This error is consistent with one of the errors I am seeing in the nightly
> builds when running OpenSSL with only default parameters so I suspect
> something is off in combination
> version negotiation and cipher suite selection checks. I am looking in to
> it!

I'm seeing the same, if it helps to reproduce. I generated my certificates with:

#!/bin/sh

# Create the CA key and certificate.
openssl genrsa -out ca.key 2048
openssl req -new -x509 -nodes -key ca.key -days 3653 -out ca.pem -subj
"/CN=Test CA"

# Create the server key and CSR.
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -subj "/CN=localhost"

# Sign the CSR with the CA key.
serial=$(date +"%s")
openssl x509 -req -days 3563 -CA ca.pem -CAserial $serial
-CAcreateserial -CAkey ca.key -in server.csr -out server.pem
rm $serial

I tested with:

% Server
{ok, _} = application:ensure_all_started(ssl).
Port = 11002.
LOpts = [{certfile, "server.pem"}, {keyfile, "server.key"}].
{ok, LSock} = ssl:listen(Port, LOpts).
{ok, CSock} = ssl:transport_accept(LSock).
ok = ssl:ssl_accept(CSock).

% Client
{ok, _} = application:ensure_all_started(ssl).
Port = 11002.
COpts = [{verify, verify_peer}, {cacertfile, "ca.pem"}].
{ok, Sock} = ssl:connect("localhost", Port, COpts).

The server reports:

=INFO REPORT==== 4-May-2018::11:22:20.971130 ===
TLS server: In state hello at tls_handshake.erl:130 generated SERVER
ALERT: Fatal - Handshake Failure - malformed_handshake_data

** exception error: no match of right hand side value
{error,{tls_alert,"handshake failure"}}

The client reports:

=INFO REPORT==== 4-May-2018::11:22:20.981524 ===
TLS client: In state hello received SERVER ALERT: Fatal - Handshake Failure

** exception error: no match of right hand side value
{error,{tls_alert,"handshake failure"}}

The same code works fine with 20.3.1

Thanks,
Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Ingela Andin]

Hi!

I traced my problem to sslv2_hello compatibility. I will give you something to try later today!

Regards Ingela Erlang/OTP team

2018-05-04 10:05 GMT+02:00 Heinz N. Gies <[hidden email]>:

I’m having quite some trouble on the SSL front too. I’m giving it a try to move the riak_core code to R21 and the ssl test [1] fails.

If I use the original keys (in the repo under test/site1/2-…) I get a handshake failure.

Using those keys I get an handshake failure too:

openssl req -x509 -newkey rsa:4096 -keyout test/site1-key.pem -out test/site1-cert.pem -days 3650 -nodes -subj '/CN=US'

openssl req -x509 -newkey rsa:4096 -keyout test/site2-key.pem -out test/site2-cert.pem -days 3650 -nodes -subj '/CN=US’

DSA keys

openssl dsaparam -out dsaparams-site1.pem 1024

openssl dsaparam -out dsaparams-site2.pem 1024

openssl req -x509 -newkey dsa:dsaparams-site1.pem -keyout test/site1-key.pem -out test/site1-cert.pem -days 3650 -nodes -subj '/CN=US'

openssl req -x509 -newkey dsa:dsaparams-site2.pem -keyout test/site2-key.pem -out test/site2-cert.pem -days 3650 -nodes -subj '/CN=US’

Fail with:

{badmatch,

     {error,

         {options,

             {keyfile,"test/site1-key.pem",

                 {error,

                     {asn1,

                         {{wrong_tag,

                              {{expected,16},

                               {got,2,

                                   {2,

                                    <<74,130,213,43,78,73,39,24,206,62,159,

                                      168,30,65,230,24,14,31,209,192>>}}}},

                          [{'OTP-PUB-KEY',match_tags,2,

                               [{file,"OTP-PUB-KEY.erl"},{line,20535}]},

                           {'OTP-PUB-KEY',dec_DSAPrivateKey,2,

                               [{file,"OTP-PUB-KEY.erl"},{line,1789}]},

                           {'OTP-PUB-KEY',decode,2,

                               [{file,"OTP-PUB-KEY.erl"},{line,1103}]},

                           {public_key,der_decode,2,

                               [{file,"public_key.erl"},{line,248}]},

                           {ssl_config,init_private_key,5,

                               [{file,"ssl_config.erl"},{line,114}]},

                           {ssl_config,init,2,

                               [{file,"ssl_config.erl"},{line,38}]},

                           {ssl_connection,ssl_config,4,

                               [{file,"ssl_connection.erl"},{line,571}]},

                           {tls_connection,init,1,

                               [{file,"tls_connection.erl"},

                                {line,116}]}]}}}}}}}

EC keys fail with a handshake failure too

openssl ecparam -out ecparams-site1.pem -name prime256v1

openssl ecparam -out ecparams-site2.pem -name prime256v1

openssl req -x509 -newkey ec:ecparams-site1.pem -keyout test/site1-key.pem -out test/site1-cert.pem -days 3650 -nodes -subj '/CN=US'

openssl req -x509 -newkey ec:ecparams-site2.pem -keyout test/site2-key.pem -out test/site2-cert.pem -days 3650 -nodes -subj '/CN=US'

https://github.com/Kyorai/riak_core/blob/develop/src/riak_core_tcp_mon.erl#L450

On 4. May 2018, at 09:32, Ingela Andin <[hidden email]> wrote:

Hi!

2018-05-03 18:08 GMT+02:00 Loïc Hoguin <[hidden email]>:

Hello,

On 05/03/2018 01:54 PM, Loïc Hoguin wrote:

* SSL is broken. See [1] for example. I can see the same thing happening on 5 different Linux distributions (with different OpenSSL versions) and on OSX. A quick try in the shell is not much better:

Thank you for shouting, that is what the release candidate is for. So we can catch the problems early!

 

OK it's just a very misleading error message I think.

Well that depends, this is not really an error message that you should get unless you have a buggy or malicious client. But of course now we might be getting it due to a bug and then

it could be misleading!

 

Switching my server's test keys from RSA to DSA fixes it so I think this issue is caused by:

  OTP-14769    Application(s): ssl

               For security reasons RSA-key exchange cipher suites are
               no longer supported by default

I do not really suspect this change. RSA-certificates are still supported. Just cipher suites using RSA encryption/decryption in the key exchange process are not supported.

When you switched to a DSA-certificate an other cipher suite was picked that did not expose the problem.  If there had been no common cipher suites you would have got another error.

 

Still, it probably should provide a more helpful error message than this:

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03 11:13:04.343 ***
=INFO REPORT==== 3-May-2018::11:13:04.342940 ===
TLS server: In state hello at tls_handshake.erl:130 generated SERVER ALERT: Fatal - Handshake Failure - malformed_handshake_data

*** System report during acceptor_SUITE:ssl_echo/1 in ssl 2018-05-03 11:13:04.348 ***
=INFO REPORT==== 3-May-2018::11:13:04.348265 ===
TLS client: In state hello received SERVER ALERT: Fatal - Handshake Failure

"malformed_handshake_data" sounds like the client would have sent a malformed handshake, ie bad data, when the actual issue seems to be that the certificate configured is no longer supported. The server generating an alert about its own certificate doesn't sound quite right either.

That being said I do not really know the intent so I'm guessing a bit. All I know for sure is that it's confusing.

This error is consistent with one of the errors I am seeing in the nightly builds when running OpenSSL with only default parameters so I suspect something is off in combination

version negotiation and cipher suite selection checks. I am looking in to it!

Regards Ingela Erlang/OTP Team 

 

Cheers,

-- 
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions