Desperately looking for official Erlang tests results

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Desperately looking for official Erlang tests results

REIX, Tony
Hi,

In order to understand which Erlang tests and features do not work fine within our port of version 19.2 on AIX, we need some "official" Erlang tests results for current version (19.2.3).
However, searching on Erlang web-site, I was unable to find such tests results.

So, I have built and tested Erlang on 3 environments: AIX 6.1, Fedora25/PPC64, and Fedora25/x86_64.
I hope I've correctly ran the tests, but I'm not sure.

In short, using nearly the same procedure (use Fedora .spec file), but without HiPE-SMP-Threads for AIX, I've got:

                 OK  FAIL   SKIP   TOTAL
------------------------------------------
AIX 6.1     
10,838   204  2,991  14,033   (no HiPE-SMP-Threads !)
FC25/PPC64   12,376   193  3,608  16,177
FC25/x86_64 
12,407   288  3,475  16,170

I am really surprised by:
  - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all tests !!
  - how few failures I have on AIX, comparable to the amount of failures on FC25/x86_64.
        (However:   - when trying with SMP and/or HiPE, I had hangs, so I'm first building/testing without them
                            - we applied several AIX-specific patches)
 - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . (However: there are some more SKIPped tests: 133)

Since result files contains about 600,000 lines, it is very difficult for me to understand where/what are the issues.
I think we have understood that, in many cases, some environment is missing/failing and thus the test is skipped.

I'd like to get some help about:
  - checking that I'm doing the Erlang configuration, build, and tests correctly,
  - finding some "official" Erlang tests results on Fedora / x86_64
  - how to analyze these results and find the real issues.

I have attached to this email the index.html file produced on Fedora25/x86_64 :
        /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html

Thanks

Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

index.html (55K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Björn Gustavsson-4
Did you run Erlang from the git repository/source
tree or did you do "make release" and run it from
there?

A few tests cases will fail if you have not done
"make release". (One example is
xref_SUITE:md/1 in tools.)

I think that the best way for us to help you is
if you can share the names of the failed
test cases in the following test suites:

emulator_test
stdlib_test
kernel_test

We should be able to tell you which ones
that are most likely real bugs, and which
ones that fail because of envioronmental
issues.

/Björn

On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony <[hidden email]> wrote:

> Hi,
>
> In order to understand which Erlang tests and features do not work fine
> within our port of version 19.2 on AIX, we need some "official" Erlang tests
> results for current version (19.2.3).
> However, searching on Erlang web-site, I was unable to find such tests
> results.
>
> So, I have built and tested Erlang on 3 environments: AIX 6.1,
> Fedora25/PPC64, and Fedora25/x86_64.
> I hope I've correctly ran the tests, but I'm not sure.
>
> In short, using nearly the same procedure (use Fedora .spec file), but
> without HiPE-SMP-Threads for AIX, I've got:
>
>                  OK  FAIL   SKIP   TOTAL
> ------------------------------------------
> AIX 6.1      10,838   204  2,991  14,033   (no HiPE-SMP-Threads !)
> FC25/PPC64   12,376   193  3,608  16,177
> FC25/x86_64  12,407   288  3,475  16,170
>
> I am really surprised by:
>   - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all
> tests !!
>   - how few failures I have on AIX, comparable to the amount of failures on
> FC25/x86_64.
>         (However:   - when trying with SMP and/or HiPE, I had hangs, so I'm
> first building/testing without them
>                             - we applied several AIX-specific patches)
>  - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
> (However: there are some more SKIPped tests: 133)
>
> Since result files contains about 600,000 lines, it is very difficult for me
> to understand where/what are the issues.
> I think we have understood that, in many cases, some environment is
> missing/failing and thus the test is skipped.
>
> I'd like to get some help about:
>   - checking that I'm doing the Erlang configuration, build, and tests
> correctly,
>   - finding some "official" Erlang tests results on Fedora / x86_64
>   - how to analyze these results and find the real issues.
>
> I have attached to this email the index.html file produced on
> Fedora25/x86_64 :
>
> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>
> Thanks
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>



--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Loïc Hoguin-3
On 03/02/2017 01:37 PM, Björn Gustavsson wrote:
> Did you run Erlang from the git repository/source
> tree or did you do "make release" and run it from
> there?
>
> A few tests cases will fail if you have not done
> "make release". (One example is
> xref_SUITE:md/1 in tools.)

When should we do this exactly?

It is not mentioned here: https://github.com/erlang/otp/wiki/Running-tests

Cheers,

> I think that the best way for us to help you is
> if you can share the names of the failed
> test cases in the following test suites:
>
> emulator_test
> stdlib_test
> kernel_test
>
> We should be able to tell you which ones
> that are most likely real bugs, and which
> ones that fail because of envioronmental
> issues.
>
> /Björn
>
> On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony <[hidden email]> wrote:
>> Hi,
>>
>> In order to understand which Erlang tests and features do not work fine
>> within our port of version 19.2 on AIX, we need some "official" Erlang tests
>> results for current version (19.2.3).
>> However, searching on Erlang web-site, I was unable to find such tests
>> results.
>>
>> So, I have built and tested Erlang on 3 environments: AIX 6.1,
>> Fedora25/PPC64, and Fedora25/x86_64.
>> I hope I've correctly ran the tests, but I'm not sure.
>>
>> In short, using nearly the same procedure (use Fedora .spec file), but
>> without HiPE-SMP-Threads for AIX, I've got:
>>
>>                  OK  FAIL   SKIP   TOTAL
>> ------------------------------------------
>> AIX 6.1      10,838   204  2,991  14,033   (no HiPE-SMP-Threads !)
>> FC25/PPC64   12,376   193  3,608  16,177
>> FC25/x86_64  12,407   288  3,475  16,170
>>
>> I am really surprised by:
>>   - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all
>> tests !!
>>   - how few failures I have on AIX, comparable to the amount of failures on
>> FC25/x86_64.
>>         (However:   - when trying with SMP and/or HiPE, I had hangs, so I'm
>> first building/testing without them
>>                             - we applied several AIX-specific patches)
>>  - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
>> (However: there are some more SKIPped tests: 133)
>>
>> Since result files contains about 600,000 lines, it is very difficult for me
>> to understand where/what are the issues.
>> I think we have understood that, in many cases, some environment is
>> missing/failing and thus the test is skipped.
>>
>> I'd like to get some help about:
>>   - checking that I'm doing the Erlang configuration, build, and tests
>> correctly,
>>   - finding some "official" Erlang tests results on Fedora / x86_64
>>   - how to analyze these results and find the real issues.
>>
>> I have attached to this email the index.html file produced on
>> Fedora25/x86_64 :
>>
>> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>>
>> Thanks
>>
>> Cordialement,
>>
>> Tony Reix
>>
>> Bull - ATOS
>> IBM Coop Architect & Technical Leader
>> Office : +33 (0) 4 76 29 72 67
>> 1 rue de Provence - 38432 Échirolles - France
>> www.atos.net
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
>

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Björn Gustavsson-4
On Thu, Mar 2, 2017 at 2:09 PM, Loïc Hoguin <[hidden email]> wrote:
> On 03/02/2017 01:37 PM, Björn Gustavsson wrote:
>>
>> A few tests cases will fail if you have not done
>> "make release". (One example is
>> xref_SUITE:md/1 in tools.)
>
>
> When should we do this exactly?
>

After building Erlang/OTP, do "make release". Then you will
have to run the Install script. On my Linux system the
commands would be:

make release
cd release/x86_64-unknown-linux-gnu
./Install `pwd`

When running the tests, started the released Erlang/OTP
system like this:

$ERL_TOP/release/x86_64-unknown-linux-gnu/bin/erl

(Then do ts:install(), ts:run() etc as in the instruction.)

> It is not mentioned here: https://github.com/erlang/otp/wiki/Running-tests

It is not mentioned because there are only a few test cases
that must be run in an installed system.

/Björn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Hans Nilsson R (AL/EAB)
In reply to this post by REIX, Tony
Hi,

I looked at the [crypto, public_key, ssh, ssl] group of applications.
It does look very bad indeed, except for public_key.

So let's start from the bottom (i.e. crypto). I need the following:

   1) start your tested Erlang and call
         1> crypto:info_lib().
               -- result 1 ---
         2> crypto:supports().
               -- result 2 ---
         3>

      What are the "result *" you get?

   2) Mail me the crypto test results directory

..../ct_run.test_server@dorado-vm1.2017-03-01_21.34.20/tests.crypto_test.logs

/Hans

On 03/02/2017 12:23 PM, REIX, Tony wrote:

> Hi,
>
> In order to understand which Erlang tests and features do not work fine
> within our port of version 19.2 on AIX, we need some "official" Erlang
> tests results for current version (19.2.3).
> However, searching on Erlang web-site, I was unable to find such tests
> results.
>
> So, I have built and tested Erlang on 3 environments: AIX 6.1,
> Fedora25/PPC64, and Fedora25/x86_64.
> I hope I've correctly ran the tests, but I'm not sure.
>
> In short, using nearly the same procedure (use Fedora .spec file), but
> without HiPE-SMP-Threads for AIX, I've got:
>
>                  OK  FAIL   SKIP   TOTAL
> ------------------------------------------
> AIX 6.1      10,838   204  2,991  14,033   (no HiPE-SMP-Threads !)
> FC25/PPC64   12,376   193  3,608  16,177
> FC25/x86_64  12,407   288  3,475  16,170
>
> I am really surprised by:
>   - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of
> all tests !!
>   - how few failures I have on AIX, comparable to the amount of failures
> on FC25/x86_64.
>         (However:   - when trying with SMP and/or HiPE, I had hangs, so
> I'm first building/testing without them
>                             - we applied several AIX-specific patches)
>  - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
> (However: there are some more SKIPped tests: 133)
>
> Since result files contains about 600,000 lines, it is very difficult
> for me to understand where/what are the issues.
> I think we have understood that, in many cases, some environment is
> missing/failing and thus the test is skipped.
>
> I'd like to get some help about:
>   - checking that I'm doing the Erlang configuration, build, and tests
> correctly,
>   - finding some "official" Erlang tests results on Fedora / x86_64
>   - how to analyze these results and find the real issues.
>
> I have attached to this email the index.html file produced on
> Fedora25/x86_64 :
>        
> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>
> Thanks
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

REIX, Tony
In reply to this post by Björn Gustavsson-4
Hi Björn,

I got Erlang source from Fedora:
  # yumdownloader --source erlang
I got:
  erlang-19.2.3-1.fc25.src.rpm

It contains a .spec file, otp-OTP-19.2.3.tar.gz, and all other files (patches) needed for building Erlang.

And I built Erlang by means of:
  rpmbuild -ba erlang-19.2.3-1.fc25.spec

However, this .spec file does not build and launch the Erlang tests. I had to search Erlang documentation and find some commands.
Main ones are:
  make release_tests
  make tests
  $ERL_TOP/bin/erl -pa $TESTROOT/test_server -s ts install -s ts run all_tests -s init stop
This produced the results I provided in previous email.

Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : [hidden email] [[hidden email]] de la part de Björn Gustavsson [[hidden email]]
Envoyé : jeudi 2 mars 2017 13:37
À : REIX, Tony
Cc : [hidden email]; GIRARDET, JEAN
Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results

Did you run Erlang from the git repository/source
tree or did you do "make release" and run it from
there?

A few tests cases will fail if you have not done
"make release". (One example is
xref_SUITE:md/1 in tools.)

I think that the best way for us to help you is
if you can share the names of the failed
test cases in the following test suites:

emulator_test
stdlib_test
kernel_test

We should be able to tell you which ones
that are most likely real bugs, and which
ones that fail because of envioronmental
issues.

/Björn

On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony <[hidden email]> wrote:

> Hi,
>
> In order to understand which Erlang tests and features do not work fine
> within our port of version 19.2 on AIX, we need some "official" Erlang tests
> results for current version (19.2.3).
> However, searching on Erlang web-site, I was unable to find such tests
> results.
>
> So, I have built and tested Erlang on 3 environments: AIX 6.1,
> Fedora25/PPC64, and Fedora25/x86_64.
> I hope I've correctly ran the tests, but I'm not sure.
>
> In short, using nearly the same procedure (use Fedora .spec file), but
> without HiPE-SMP-Threads for AIX, I've got:
>
>                  OK  FAIL   SKIP   TOTAL
> ------------------------------------------
> AIX 6.1      10,838   204  2,991  14,033   (no HiPE-SMP-Threads !)
> FC25/PPC64   12,376   193  3,608  16,177
> FC25/x86_64  12,407   288  3,475  16,170
>
> I am really surprised by:
>   - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all
> tests !!
>   - how few failures I have on AIX, comparable to the amount of failures on
> FC25/x86_64.
>         (However:   - when trying with SMP and/or HiPE, I had hangs, so I'm
> first building/testing without them
>                             - we applied several AIX-specific patches)
>  - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
> (However: there are some more SKIPped tests: 133)
>
> Since result files contains about 600,000 lines, it is very difficult for me
> to understand where/what are the issues.
> I think we have understood that, in many cases, some environment is
> missing/failing and thus the test is skipped.
>
> I'd like to get some help about:
>   - checking that I'm doing the Erlang configuration, build, and tests
> correctly,
>   - finding some "official" Erlang tests results on Fedora / x86_64
>   - how to analyze these results and find the real issues.
>
> I have attached to this email the index.html file produced on
> Fedora25/x86_64 :
>
> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>
> Thanks
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>



--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

REIX, Tony
In reply to this post by Björn Gustavsson-4
Resending this email WITHOUT the attachment since it is too big to be sent without some approval.


Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France

De : REIX, Tony
Envoyé : jeudi 2 mars 2017 16:19
À : Björn Gustavsson
Cc : [hidden email]; GIRARDET, JEAN
Objet : RE:[erlang-questions] Desperately looking for official Erlang tests results

Hi Björn

Compressed test results file from FC25/x86_64 is attached to this email: /tmp/erlang-19.2.3-1.fc25.tests.sh.res2.xz .

About kernel_test, there are failed tests common to both FC25/x86_64 and AIX, and failed tests specific to each of the 2 OS.

1) On FC25/x86_64 :

# grep "Testing tests.kernel_test" *tests*res2 | grep FAILED
1> Testing tests.kernel_test: *** FAILED test case 125 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 157 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 158 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 160 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 161 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 241 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 244 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 259 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 261 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 262 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 294 of 750 *** x86_64 only
1> Testing tests.kernel_test: *** FAILED test case 307 of 750 *** x86_64 only
1> Testing tests.kernel_test: *** FAILED {gen_tcp_api_SUITE,init_per_suite} ***
1> Testing tests.kernel_test: *** FAILED test case 384 of 750 *** x86_64 only
1> Testing tests.kernel_test: *** FAILED test case 417 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 561 of 750 *** x86_64 only
1> Testing tests.kernel_test: *** FAILED test case 562 of 750 *** x86_64 only
1> Testing tests.kernel_test: *** FAILED test case 606 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 607 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 612 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 613 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 627 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 629 of 750 ***
1> Testing tests.kernel_test: *** FAILED test case 630 of 750 ***
1> Testing tests.kernel_test: *** FAILED {seq_trace_SUITE,init_per_suite} ***



Case 125:
disk_log_SUITE:open_overwrite failed on line 1865
Reason: {badmatch,{ok,n}}

157,158:
erl_boot_server_SUITE:start failed on line 83
Reason: {badmatch,{error,{badarg,["dorado-vm1","gandalf"]}}}

160:
erl_boot_server_SUITE:add failed on line 153
Reason: {badmatch,{error,nxdomain}}

161:
erl_boot_server_SUITE:delete failed on line 186
Reason: {badmatch,{error,nxdomain}}

241:
file_SUITE:test_directory failed on line 1376
Reason: {badmatch,read}

244:
file_SUITE:file_write_file_info failed on line 1524
Reason: {badmatch,ok}

259:
file_SUITE:e_delete failed on line 2109
Reason: {badmatch,ok}

261:
file_SUITE:e_make_dir failed on line 2240
Reason: {badmatch,ok}

262:
file_SUITE:e_del_dir failed on line 2287
Reason: {badmatch,ok}

294:
file_name_SUITE:make_icky_dir failed on line 642
Reason: {badmatch,{error,no_translation}}

307:
gen_sctp_SUITE:ok failed on line 544
Reason: function_clause

384:
gen_tcp_misc_SUITE:test_prio_fail failed on line 1871
Reason: {badmatch,ok}

417:
gen_udp_SUITE:implicit_inet6 failed on line 733
Reason: not_ok

561:
multi_load_SUITE:do_ensure_modules_loaded failed on line 282
Reason: {badmatch,false}

562:
multi_load_SUITE:make_and_load failed on line 330
Reason: {badmatch,ok}

606:
prim_file_SUITE:test_directory failed on line 801
Reason: {badmatch,read}

607:
prim_file_SUITE:test_directory failed on line 801
Reason: {badmatch,read}

612:
prim_file_SUITE:file_write_file_info failed on line 968
Reason: {badmatch,ok}

613:
prim_file_SUITE:file_write_file_info failed on line 968
Reason: {badmatch,ok}

627:
prim_file_SUITE:e_delete failed on line 1455
Reason: {badmatch,ok}

629:
prim_file_SUITE:e_make_dir failed on line 1591
Reason: {badmatch,ok}

630:
prim_file_SUITE:e_del_dir failed on line 1641
Reason: {badmatch,ok}



2) On AIX, I have these AIX-specific tests that fail:

1> Testing tests.kernel_test: *** FAILED test case 88 of 750 ****
code_SUITE:big_boot_embedded failed on line 1119
Reason: {badmatch,{error,timeout}}

1> Testing tests.kernel_test: *** FAILED test case 220 of 750 ***
file_SUITE:expect failed on line 217
Reason: function_clause

1> Testing tests.kernel_test: *** FAILED test case 284 of 750 ***
erlang:hd failed
Reason: badarg

1> Testing tests.kernel_test: *** FAILED test case 285 of 750 ***
rlang:hd failed
Reason: badarg

1> Testing tests.kernel_test: *** FAILED test case 499 of 750 ***
inet_SUITE:do_getif failed on line 952
Reason: {test_case_failed,no_HWAs}

1> Testing tests.kernel_test: *** FAILED test case 502 of 750 ***
inet_SUITE:getifaddrs failed on line 999
Reason: {test_case_failed,{should_have_hwaddr,{unix,aix}}}

1> Testing tests.kernel_test: *** FAILED test case 590 of 750 ***
prim_file_SUITE:expect failed on line 157
Reason: function_clause

1> Testing tests.kernel_test: *** FAILED test case 591 of 750 ***
prim_file_SUITE:expect failed on line 157
Reason: function_clause


Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : [hidden email] [[hidden email]] de la part de Björn Gustavsson [[hidden email]]
Envoyé : jeudi 2 mars 2017 13:37
À : REIX, Tony
Cc : [hidden email]; GIRARDET, JEAN
Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results

Did you run Erlang from the git repository/source
tree or did you do "make release" and run it from
there?

A few tests cases will fail if you have not done
"make release". (One example is
xref_SUITE:md/1 in tools.)

I think that the best way for us to help you is
if you can share the names of the failed
test cases in the following test suites:

emulator_test
stdlib_test
kernel_test

We should be able to tell you which ones
that are most likely real bugs, and which
ones that fail because of envioronmental
issues.

/Björn

On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony <[hidden email]> wrote:
> Hi,
>
> In order to understand which Erlang tests and features do not work fine
> within our port of version 19.2 on AIX, we need some "official" Erlang tests
> results for current version (19.2.3).
> However, searching on Erlang web-site, I was unable to find such tests
> results.
>
> So, I have built and tested Erlang on 3 environments: AIX 6.1,
> Fedora25/PPC64, and Fedora25/x86_64.
> I hope I've correctly ran the tests, but I'm not sure.
>
> In short, using nearly the same procedure (use Fedora .spec file), but
> without HiPE-SMP-Threads for AIX, I've got:
>
> OK FAIL SKIP TOTAL
> ------------------------------------------
> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !)
> FC25/PPC64 12,376 193 3,608 16,177
> FC25/x86_64 12,407 288 3,475 16,170
>
> I am really surprised by:
> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all
> tests !!
> - how few failures I have on AIX, comparable to the amount of failures on
> FC25/x86_64.
> (However: - when trying with SMP and/or HiPE, I had hangs, so I'm
> first building/testing without them
> - we applied several AIX-specific patches)
> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
> (However: there are some more SKIPped tests: 133)
>
> Since result files contains about 600,000 lines, it is very difficult for me
> to understand where/what are the issues.
> I think we have understood that, in many cases, some environment is
> missing/failing and thus the test is skipped.
>
> I'd like to get some help about:
> - checking that I'm doing the Erlang configuration, build, and tests
> correctly,
> - finding some "official" Erlang tests results on Fedora / x86_64
> - how to analyze these results and find the real issues.
>
> I have attached to this email the index.html file produced on
> Fedora25/x86_64 :
>
> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>
> Thanks
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>



--
Björn Gustavsson, Erlang/OTP, Ericsson AB

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

REIX, Tony
In reply to this post by Hans Nilsson R (AL/EAB)
Hi Hans,

On AIX, I have only 1 test failing:
                OK FAIL SKIP TOTAL
crypto_test: 111 1 3 115


However, on FC25/x86_64, I have 4 more:
                OK FAIL SKIP TOTAL
crypto_test: 107 5 3 115



1) AIX:

1> Testing tests.crypto_test: *** FAILED test case 97 of 115 ***

crypto:strong_rand_mpint failed on line 1417
Reason: low_entropy

# cd otp_src_19.2

# ./bin/erl
/opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec
Erlang/OTP 19 [erts-8.2] [source]

Eshell V8.2  (abort with ^G)
1> crypto:info_lib().
[{<<"OpenSSL">>,268443823,
  <<"OpenSSL 1.0.2j  26 Sep 2016">>}]
2> crypto:supports().
[{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
 {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
           aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
           aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
           blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
           aes_gcm]},
 {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}]
3>


2) FC25/x86_64:

1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} ***

1> Testing tests.crypto_test: *** FAILED test case 26 of 115 ***
crypto:ecdh_compute_key_nif failed
Reason: badarg

1> Testing tests.crypto_test: *** FAILED test case 27 of 115 ***
crypto:ec_key_generate failed
Reason: badarg

1> Testing tests.crypto_test: *** FAILED test case 97 of 115 ***          Common with AIX
crypto:strong_rand_mpint failed on line 1417
Reason: low_entropy

1> Testing tests.crypto_test: *** FAILED test case 109 of 115 ***
crypto:ec_key_generate failed
Reason: badarg

1> Testing tests.crypto_test: *** FAILED test case 115 of 115 ***
Error detected: {'EXIT',low_entropy}

# pwd
/root/rpmbuild/BUILD/otp-OTP-19.2.3

#./bin/erl
Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V8.2.2  (abort with ^G)
1> crypto:info_lib().
[{<<"OpenSSL">>,268443839,
  <<"OpenSSL 1.0.2k-fips  26 Jan 2017">>}]
2> crypto:supports().
[{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
 {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
           aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
           aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
           blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
           aes_gcm]},
 {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}]



Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : Hans Nilsson R [[hidden email]]
Envoyé : jeudi 2 mars 2017 15:39
À : [hidden email]; REIX, Tony
Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results

Hi,

I looked at the [crypto, public_key, ssh, ssl] group of applications.
It does look very bad indeed, except for public_key.

So let's start from the bottom (i.e. crypto). I need the following:

1) start your tested Erlang and call
1> crypto:info_lib().
-- result 1 ---
2> crypto:supports().
-- result 2 ---
3>

What are the "result *" you get?

2) Mail me the crypto test results directory

..../ct_run.test_server@dorado-vm1.2017-03-01_21.34.20/tests.crypto_test.logs

/Hans

On 03/02/2017 12:23 PM, REIX, Tony wrote:
> Hi,
>
> In order to understand which Erlang tests and features do not work fine
> within our port of version 19.2 on AIX, we need some "official" Erlang
> tests results for current version (19.2.3).
> However, searching on Erlang web-site, I was unable to find such tests
> results.
>
> So, I have built and tested Erlang on 3 environments: AIX 6.1,
> Fedora25/PPC64, and Fedora25/x86_64.
> I hope I've correctly ran the tests, but I'm not sure.
>
> In short, using nearly the same procedure (use Fedora .spec file), but
> without HiPE-SMP-Threads for AIX, I've got:
>
> OK FAIL SKIP TOTAL
> ------------------------------------------
> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !)
> FC25/PPC64 12,376 193 3,608 16,177
> FC25/x86_64 12,407 288 3,475 16,170
>
> I am really surprised by:
> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of
> all tests !!
> - how few failures I have on AIX, comparable to the amount of failures
> on FC25/x86_64.
> (However: - when trying with SMP and/or HiPE, I had hangs, so
> I'm first building/testing without them
> - we applied several AIX-specific patches)
> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
> (However: there are some more SKIPped tests: 133)
>
> Since result files contains about 600,000 lines, it is very difficult
> for me to understand where/what are the issues.
> I think we have understood that, in many cases, some environment is
> missing/failing and thus the test is skipped.
>
> I'd like to get some help about:
> - checking that I'm doing the Erlang configuration, build, and tests
> correctly,
> - finding some "official" Erlang tests results on Fedora / x86_64
> - how to analyze these results and find the real issues.
>
> I have attached to this email the index.html file produced on
> Fedora25/x86_64 :
>
> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>
> Thanks
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

Erlang.crypto_tests.FC25.x86_64.tar.xz (147K) Download Attachment
Erlang.crypto_tests.AIX.PPC64.tar.xz (145K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

REIX, Tony
Resending without the attachments since it is too big.
I'll sent the files to Hans directly.

Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France

De : REIX, Tony
Envoyé : jeudi 2 mars 2017 16:46
À : Hans Nilsson R; [hidden email]
Objet : RE:[erlang-questions] Desperately looking for official Erlang tests results

Hi Hans,

On AIX, I have only 1 test failing:
                OK FAIL SKIP TOTAL
crypto_test: 111 1 3 115


However, on FC25/x86_64, I have 4 more:
                OK FAIL SKIP TOTAL
crypto_test: 107 5 3 115



1) AIX:

1> Testing tests.crypto_test: *** FAILED test case 97 of 115 ***

crypto:strong_rand_mpint failed on line 1417
Reason: low_entropy

# cd otp_src_19.2

# ./bin/erl
/opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec
Erlang/OTP 19 [erts-8.2] [source]

Eshell V8.2  (abort with ^G)
1> crypto:info_lib().
[{<<"OpenSSL">>,268443823,
  <<"OpenSSL 1.0.2j  26 Sep 2016">>}]
2> crypto:supports().
[{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
 {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
           aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
           aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
           blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
           aes_gcm]},
 {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}]
3>


2) FC25/x86_64:

1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} ***

1> Testing tests.crypto_test: *** FAILED test case 26 of 115 ***
crypto:ecdh_compute_key_nif failed
Reason: badarg

1> Testing tests.crypto_test: *** FAILED test case 27 of 115 ***
crypto:ec_key_generate failed
Reason: badarg

1> Testing tests.crypto_test: *** FAILED test case 97 of 115 ***          Common with AIX
crypto:strong_rand_mpint failed on line 1417
Reason: low_entropy

1> Testing tests.crypto_test: *** FAILED test case 109 of 115 ***
crypto:ec_key_generate failed
Reason: badarg

1> Testing tests.crypto_test: *** FAILED test case 115 of 115 ***
Error detected: {'EXIT',low_entropy}

# pwd
/root/rpmbuild/BUILD/otp-OTP-19.2.3

#./bin/erl
Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V8.2.2  (abort with ^G)
1> crypto:info_lib().
[{<<"OpenSSL">>,268443839,
  <<"OpenSSL 1.0.2k-fips  26 Jan 2017">>}]
2> crypto:supports().
[{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
 {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
           aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
           aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
           blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
           aes_gcm]},
 {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}]



Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : Hans Nilsson R [[hidden email]]
Envoyé : jeudi 2 mars 2017 15:39
À : [hidden email]; REIX, Tony
Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results

Hi,

I looked at the [crypto, public_key, ssh, ssl] group of applications.
It does look very bad indeed, except for public_key.

So let's start from the bottom (i.e. crypto). I need the following:

1) start your tested Erlang and call
1> crypto:info_lib().
-- result 1 ---
2> crypto:supports().
-- result 2 ---
3>

What are the "result *" you get?

2) Mail me the crypto test results directory

..../ct_run.test_server@dorado-vm1.2017-03-01_21.34.20/tests.crypto_test.logs

/Hans

On 03/02/2017 12:23 PM, REIX, Tony wrote:
> Hi,
>
> In order to understand which Erlang tests and features do not work fine
> within our port of version 19.2 on AIX, we need some "official" Erlang
> tests results for current version (19.2.3).
> However, searching on Erlang web-site, I was unable to find such tests
> results.
>
> So, I have built and tested Erlang on 3 environments: AIX 6.1,
> Fedora25/PPC64, and Fedora25/x86_64.
> I hope I've correctly ran the tests, but I'm not sure.
>
> In short, using nearly the same procedure (use Fedora .spec file), but
> without HiPE-SMP-Threads for AIX, I've got:
>
> OK FAIL SKIP TOTAL
> ------------------------------------------
> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !)
> FC25/PPC64 12,376 193 3,608 16,177
> FC25/x86_64 12,407 288 3,475 16,170
>
> I am really surprised by:
> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of
> all tests !!
> - how few failures I have on AIX, comparable to the amount of failures
> on FC25/x86_64.
> (However: - when trying with SMP and/or HiPE, I had hangs, so
> I'm first building/testing without them
> - we applied several AIX-specific patches)
> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
> (However: there are some more SKIPped tests: 133)
>
> Since result files contains about 600,000 lines, it is very difficult
> for me to understand where/what are the issues.
> I think we have understood that, in many cases, some environment is
> missing/failing and thus the test is skipped.
>
> I'd like to get some help about:
> - checking that I'm doing the Erlang configuration, build, and tests
> correctly,
> - finding some "official" Erlang tests results on Fedora / x86_64
> - how to analyze these results and find the real issues.
>
> I have attached to this email the index.html file produced on
> Fedora25/x86_64 :
>
> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>
> Thanks
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Björn Gustavsson-4
In reply to this post by REIX, Tony
On Thu, Mar 2, 2017 at 4:26 PM, REIX, Tony <[hidden email]> wrote:
> Resending this email WITHOUT the attachment since it is too big to be sent
> without some approval.
>

It seems to have been approved now. I have received the other email.

I have taken a quick look at the log.

The major problem is that many test cases that call init_per_SUITE fail
when they attempt to compile C code during the running of the test suite,
especially when running emulator_tests. You should be able to
find more information from the generated html files. Starting
with index.html, you could click on test.emulator_test. On that page,
you could click on the links for the failed test cases. On the
page for a failed init_per_SUITE test case, you should be able
to see why the compilation failed. (Those failed test cases cause
all test in the same module to be skipped.)

On other thing so far: erl_boot_server_SUITE is not a good test
suite. It has some hard-coded assumptions about our
testing environment. It will fail outside our lab.

/Björn


--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

REIX, Tony
Hi Björn

Thanks for studying.

I had already looked at these error messages. Here is below what I got about test.emulator_test .

This is due to : ts_install_cth failed! exit:{{make_failed,make},[{ts_make,make,1,

I have this issue both on AIX and on FC25/x86_64.
I guess that this issue should NOT appear on FC25/x86_64. However, I have it 48 times.

On AIX, there are probably MANY reasons why this ts_install_cth failed.
However, on FC25./x86_64, I am VERY surprised that this fails.
However, the guy who ported Erlang on FC25/x86_64 said to me that he did not run the Erlang tests...
Since I found that their RPM .spec file was missing a Request: instruction for the C++ compiler (leading to errors at build), I guess that their .spec file may contain other issues.

However, I have NO idea about how I can find the root cause of this ts_install_cth failulre. I have already tried.


Tony


=== Config function: ct_framework:init_per_suite/1 (click for source code)

=== Group properties: [{suite,a_SUITE}]

=== Config value:

    [{tc_logfile,"/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/ct_run.test_server@dorado-vm1.2017-03-01_16.58.07/tests.emulator_test.logs/run.2017-03-01_16.58.23/ct_framework.init_per_suite.html"},
     {tc_group_properties,[{suite,a_SUITE}]},
     {tc_group_path,[]},
     {data_dir,"/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/emulator_test/a_SUITE_data/"},
     {priv_dir,"/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/ct_run.test_server@dorado-vm1.2017-03-01_16.58.07/tests.emulator_test.logs/run.2017-03-01_16.58.23/log_private/"}]

=== Current directory is "/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/ct_run.test_server@dorado-vm1.2017-03-01_16.58.07"

=== Started at 2017-03-01 16:58:31



Making "/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/emulator_test/a_SUITE_data/Makefile"
gcc -c -fPIC -D_THREAD_SAFE -D_REENTRANT -g -O2 -D_THREAD_SAFE -D_REENTRANT  -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/sys/unix -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/include -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/include/x86_64-unknown-linux-gnu -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_LIBM=1 -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_LINUX_TCP_H=1 -DHAVE_SANE_LINUX_TCP_H=1 -DHAVE_POLL_H=1 -DHAVE_STRERROR=1 -DHAVE_VSNPRINTF=1 -DHAVE_USLEEP=1 -DHAVE_LIBRESOLV=1 -DHAVE_RES_GETHOSTBYNAME=1 -DHAVE_FINITE=1 timer_driver.c


In file included from /root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam/erl_driver.h:40:0,


                 from timer_driver.c:6:


/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam/erl_drv_nif.h:76:34: erreur fatale : erl_int_sizes_config.h : No such file or directory


 #include "erl_int_sizes_config.h"


                                  ^


compilation terminée.


Makefile:13 : la recette pour la cible « timer_driver.so » a échouée


make: *** [timer_driver.so] Erreur 1



*** User 2017-03-01 16:58:31.290 ***
ts_install_cth failed! exit:{{make_failed,make},[{ts_make,make,1,
                                                  [{file,"ts_make.erl"},
                                                   {line,34}]},
                                                 {ts_lib,make_non_erlang_do,
                                                  2,
                                                  [{file,"ts_lib.erl"},
                                                   {line,357}]},
                                                 {ts_install_cth,
                                                  pre_init_per_suite,3,
                                                  [{file,"ts_install_cth.erl"},
                                                   {line,105}]},
                                                 {ct_hooks,catch_apply,4,
                                                  [{file,"ct_hooks.erl"},
                                                   {line,388}]},
                                                 {ct_hooks,call_generic,3,
                                                  [{file,"ct_hooks.erl"},
                                                   {line,191}]},
                                                 {ct_hooks,call,4,
                                                  [{file,"ct_hooks.erl"},
                                                   {line,240}]},
                                                 {ct_hooks,call,4,
                                                  [{file,"ct_hooks.erl"},
                                                   {line,243}]},
                                                 {ct_hooks,call,4,
                                                  [{file,"ct_hooks.erl"},
                                                   {line,227}]}]}


*** CT Error Notification 2017-03-01 16:58:31.291 ***
Error detected: ts_install_cth

Full error description and stacktrace




=== Ended at 2017-03-01 16:58:31
=== Location: {ct_framework,init_per_suite}
=== Reason: {ts_install_cth,
                     {exit,
                         {make_failed,make},
                         [{ts_make,make,1,[{file,"ts_make.erl"},{line,34}]},
                          {ts_lib,make_non_erlang_do,2,
                              [{file,"ts_lib.erl"},{line,357}]},
                          {ts_install_cth,pre_init_per_suite,3,
                              [{file,"ts_install_cth.erl"},{line,105}]},
                          {ct_hooks,catch_apply,4,
                              [{file,"ct_hooks.erl"},{line,388}]},
                          {ct_hooks,call_generic,3,
                              [{file,"ct_hooks.erl"},{line,191}]},
                          {ct_hooks,call,4,[{file,"ct_hooks.erl"},{line,240}]},
                          {ct_hooks,call,4,[{file,"ct_hooks.erl"},{line,243}]},
                          {ct_hooks,call,4,
                              [{file,"ct_hooks.erl"},{line,227}]}]}}
===
*** init_per_suite failed.
    Skipping all cases.


Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : [hidden email] [[hidden email]] de la part de Björn Gustavsson [[hidden email]]
Envoyé : jeudi 2 mars 2017 17:05
À : REIX, Tony
Cc : [hidden email]; GIRARDET, JEAN
Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results

On Thu, Mar 2, 2017 at 4:26 PM, REIX, Tony <[hidden email]> wrote:
> Resending this email WITHOUT the attachment since it is too big to be sent
> without some approval.
>

It seems to have been approved now. I have received the other email.

I have taken a quick look at the log.

The major problem is that many test cases that call init_per_SUITE fail
when they attempt to compile C code during the running of the test suite,
especially when running emulator_tests. You should be able to
find more information from the generated html files. Starting
with index.html, you could click on test.emulator_test. On that page,
you could click on the links for the failed test cases. On the
page for a failed init_per_SUITE test case, you should be able
to see why the compilation failed. (Those failed test cases cause
all test in the same module to be skipped.)

On other thing so far: erl_boot_server_SUITE is not a good test
suite. It has some hard-coded assumptions about our
testing environment. It will fail outside our lab.

/Björn


--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Hans Nilsson R (AL/EAB)
In reply to this post by REIX, Tony
Hi,

Thanks for the log files. I need another check by you.  What does the
function call

    crypto:ec_curves().

return on AIX.PPC64  and on .FC25.x86_64 ?

/Hans


On 03/02/2017 04:49 PM, REIX, Tony wrote:

> Resending without the attachments since it is too big.
> I'll sent the files to Hans directly.
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net <http://www.atos.net/>
> ------------------------------------------------------------------------
> *De :* REIX, Tony
> *Envoyé :* jeudi 2 mars 2017 16:46
> *À :* Hans Nilsson R; [hidden email]
> *Objet :* RE:[erlang-questions] Desperately looking for official Erlang
> tests results
>
> Hi Hans,
>
> On AIX, I have only 1 test failing:
>
> *OK FAIL SKIP TOTAL*
> crypto_test: 111 *1* 3 115
>
>
> However, on FC25/x86_64, I have 4 more:
>
> *OK FAIL SKIP TOTAL*
> crypto_test: 107 *5* 3 115
>
>
>
> *1) AIX:*
>
> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 ***
>
> crypto:strong_rand_mpint failed on line 1417
> Reason: low_entropy
>
> # cd otp_src_19.2
>
> # ./bin/erl
> /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec
> Erlang/OTP 19 [erts-8.2] [source]
>
> Eshell V8.2  (abort with ^G)
> 1> crypto:info_lib().
> [{<<"OpenSSL">>,268443823,
>   <<"OpenSSL 1.0.2j  26 Sep 2016">>}]
> 2> crypto:supports().
> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
>  {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
>            aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
>            aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
>            blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
>            aes_gcm]},
>  {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}]
> 3>
>
>
> *2) FC25/x86_64:*
>
> 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} ***
>
> 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 ***
> crypto:ecdh_compute_key_nif failed
> Reason: badarg
>
> 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 ***
> crypto:ec_key_generate failed
> Reason: badarg
>
> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115
> ***          Common with AIX
> crypto:strong_rand_mpint failed on line 1417
> Reason: low_entropy
>
> 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 ***
> crypto:ec_key_generate failed
> Reason: badarg
>
> 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 ***
> Error detected: {'EXIT',low_entropy}
>
> # pwd
> /root/rpmbuild/BUILD/otp-OTP-19.2.3
>
> #./bin/erl
> Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3]
> [async-threads:10] [hipe] [kernel-poll:false]
>
> Eshell V8.2.2  (abort with ^G)
> 1> crypto:info_lib().
> [{<<"OpenSSL">>,268443839,
>   <<"OpenSSL 1.0.2k-fips  26 Jan 2017">>}]
> 2> crypto:supports().
> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
>  {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
>            aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
>            aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
>            blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
>            aes_gcm]},
>  {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}]
>
>
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
> ________________________________________
> De : Hans Nilsson R [[hidden email]]
> Envoyé : jeudi 2 mars 2017 15:39
> À : [hidden email]; REIX, Tony
> Objet : Re: [erlang-questions] Desperately looking for official Erlang
> tests results
>
> Hi,
>
> I looked at the [crypto, public_key, ssh, ssl] group of applications.
> It does look very bad indeed, except for public_key.
>
> So let's start from the bottom (i.e. crypto). I need the following:
>
> 1) start your tested Erlang and call
> 1> crypto:info_lib().
> -- result 1 ---
> 2> crypto:supports().
> -- result 2 ---
> 3>
>
> What are the "result *" you get?
>
> 2) Mail me the crypto test results directory
>
> ..../ct_run.test_server@dorado-vm1.2017-03-01_21.34.20/tests.crypto_test.logs
>
> /Hans
>
> On 03/02/2017 12:23 PM, REIX, Tony wrote:
>> Hi,
>>
>> In order to understand which Erlang tests and features do not work fine
>> within our port of version 19.2 on AIX, we need some "official" Erlang
>> tests results for current version (19.2.3).
>> However, searching on Erlang web-site, I was unable to find such tests
>> results.
>>
>> So, I have built and tested Erlang on 3 environments: AIX 6.1,
>> Fedora25/PPC64, and Fedora25/x86_64.
>> I hope I've correctly ran the tests, but I'm not sure.
>>
>> In short, using nearly the same procedure (use Fedora .spec file), but
>> without HiPE-SMP-Threads for AIX, I've got:
>>
>> OK FAIL SKIP TOTAL
>> ------------------------------------------
>> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !)
>> FC25/PPC64 12,376 193 3,608 16,177
>> FC25/x86_64 12,407 288 3,475 16,170
>>
>> I am really surprised by:
>> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of
>> all tests !!
>> - how few failures I have on AIX, comparable to the amount of failures
>> on FC25/x86_64.
>> (However: - when trying with SMP and/or HiPE, I had hangs, so
>> I'm first building/testing without them
>> - we applied several AIX-specific patches)
>> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
>> (However: there are some more SKIPped tests: 133)
>>
>> Since result files contains about 600,000 lines, it is very difficult
>> for me to understand where/what are the issues.
>> I think we have understood that, in many cases, some environment is
>> missing/failing and thus the test is skipped.
>>
>> I'd like to get some help about:
>> - checking that I'm doing the Erlang configuration, build, and tests
>> correctly,
>> - finding some "official" Erlang tests results on Fedora / x86_64
>> - how to analyze these results and find the real issues.
>>
>> I have attached to this email the index.html file produced on
>> Fedora25/x86_64 :
>>
>> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>>
>> Thanks
>>
>> Cordialement,
>>
>> Tony Reix
>>
>> Bull - ATOS
>> IBM Coop Architect & Technical Leader
>> Office : +33 (0) 4 76 29 72 67
>> 1 rue de Provence - 38432 Échirolles - France
>> www.atos.net
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

REIX, Tony
Hi

Here are the outputs.


FC25/x86_64 :



# cd otp-OTP-19.2.3/
# ./bin/erl
Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V8.2.2 (abort with ^G)
1> crypto:ec_curves().
[secp112r1,secp112r2,secp128r1,secp128r2,secp160k1,
secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1,
secp256k1,secp256r1,secp384r1,secp521r1,prime192v1,
prime192v2,prime192v3,prime239v1,prime239v2,prime239v3,
prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1,
brainpoolP160t1|...]
2>


AIX:

# ./bin/erl
/opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec
Erlang/OTP 19 [erts-8.2] [source]

Eshell V8.2  (abort with ^G)
1> crypto:ec_curves().
[secp112r1,secp112r2,secp128r1,secp128r2,secp160k1,
 secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1,
 secp256k1,secp256r1,secp384r1,secp521r1,prime192v1,
 prime192v2,prime192v3,prime239v1,prime239v2,prime239v3,
 prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1,
 brainpoolP160t1|...]
2>



Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : Hans Nilsson R [[hidden email]]
Envoyé : jeudi 2 mars 2017 18:05
À : REIX, Tony; [hidden email]
Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results

Hi,

Thanks for the log files. I need another check by you. What does the
function call

crypto:ec_curves().

return on AIX.PPC64 and on .FC25.x86_64 ?

/Hans


On 03/02/2017 04:49 PM, REIX, Tony wrote:
> Resending without the attachments since it is too big.
> I'll sent the files to Hans directly.
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net <http://www.atos.net/>
> ------------------------------------------------------------------------
> *De :* REIX, Tony
> *Envoyé :* jeudi 2 mars 2017 16:46
> *À :* Hans Nilsson R; [hidden email]
> *Objet :* RE:[erlang-questions] Desperately looking for official Erlang
> tests results
>
> Hi Hans,
>
> On AIX, I have only 1 test failing:
>
> *OK FAIL SKIP TOTAL*
> crypto_test: 111 *1* 3 115
>
>
> However, on FC25/x86_64, I have 4 more:
>
> *OK FAIL SKIP TOTAL*
> crypto_test: 107 *5* 3 115
>
>
>
> *1) AIX:*
>
> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 ***
>
> crypto:strong_rand_mpint failed on line 1417
> Reason: low_entropy
>
> # cd otp_src_19.2
>
> # ./bin/erl
> /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec
> Erlang/OTP 19 [erts-8.2] [source]
>
> Eshell V8.2 (abort with ^G)
> 1> crypto:info_lib().
> [{<<"OpenSSL">>,268443823,
> <<"OpenSSL 1.0.2j 26 Sep 2016">>}]
> 2> crypto:supports().
> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
> {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
> aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
> aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
> blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
> aes_gcm]},
> {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}]
> 3>
>
>
> *2) FC25/x86_64:*
>
> 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} ***
>
> 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 ***
> crypto:ecdh_compute_key_nif failed
> Reason: badarg
>
> 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 ***
> crypto:ec_key_generate failed
> Reason: badarg
>
> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115
> *** Common with AIX
> crypto:strong_rand_mpint failed on line 1417
> Reason: low_entropy
>
> 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 ***
> crypto:ec_key_generate failed
> Reason: badarg
>
> 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 ***
> Error detected: {'EXIT',low_entropy}
>
> # pwd
> /root/rpmbuild/BUILD/otp-OTP-19.2.3
>
> #./bin/erl
> Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3]
> [async-threads:10] [hipe] [kernel-poll:false]
>
> Eshell V8.2.2 (abort with ^G)
> 1> crypto:info_lib().
> [{<<"OpenSSL">>,268443839,
> <<"OpenSSL 1.0.2k-fips 26 Jan 2017">>}]
> 2> crypto:supports().
> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
> {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
> aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
> aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
> blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
> aes_gcm]},
> {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}]
>
>
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
> ________________________________________
> De : Hans Nilsson R [[hidden email]]
> Envoyé : jeudi 2 mars 2017 15:39
> À : [hidden email]; REIX, Tony
> Objet : Re: [erlang-questions] Desperately looking for official Erlang
> tests results
>
> Hi,
>
> I looked at the [crypto, public_key, ssh, ssl] group of applications.
> It does look very bad indeed, except for public_key.
>
> So let's start from the bottom (i.e. crypto). I need the following:
>
> 1) start your tested Erlang and call
> 1> crypto:info_lib().
> -- result 1 ---
> 2> crypto:supports().
> -- result 2 ---
> 3>
>
> What are the "result *" you get?
>
> 2) Mail me the crypto test results directory
>
> ..../ct_run.test_server@dorado-vm1.2017-03-01_21.34.20/tests.crypto_test.logs
>
> /Hans
>
> On 03/02/2017 12:23 PM, REIX, Tony wrote:
>> Hi,
>>
>> In order to understand which Erlang tests and features do not work fine
>> within our port of version 19.2 on AIX, we need some "official" Erlang
>> tests results for current version (19.2.3).
>> However, searching on Erlang web-site, I was unable to find such tests
>> results.
>>
>> So, I have built and tested Erlang on 3 environments: AIX 6.1,
>> Fedora25/PPC64, and Fedora25/x86_64.
>> I hope I've correctly ran the tests, but I'm not sure.
>>
>> In short, using nearly the same procedure (use Fedora .spec file), but
>> without HiPE-SMP-Threads for AIX, I've got:
>>
>> OK FAIL SKIP TOTAL
>> ------------------------------------------
>> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !)
>> FC25/PPC64 12,376 193 3,608 16,177
>> FC25/x86_64 12,407 288 3,475 16,170
>>
>> I am really surprised by:
>> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of
>> all tests !!
>> - how few failures I have on AIX, comparable to the amount of failures
>> on FC25/x86_64.
>> (However: - when trying with SMP and/or HiPE, I had hangs, so
>> I'm first building/testing without them
>> - we applied several AIX-specific patches)
>> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
>> (However: there are some more SKIPped tests: 133)
>>
>> Since result files contains about 600,000 lines, it is very difficult
>> for me to understand where/what are the issues.
>> I think we have understood that, in many cases, some environment is
>> missing/failing and thus the test is skipped.
>>
>> I'd like to get some help about:
>> - checking that I'm doing the Erlang configuration, build, and tests
>> correctly,
>> - finding some "official" Erlang tests results on Fedora / x86_64
>> - how to analyze these results and find the real issues.
>>
>> I have attached to this email the index.html file produced on
>> Fedora25/x86_64 :
>>
>> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>>
>> Thanks
>>
>> Cordialement,
>>
>> Tony Reix
>>
>> Bull - ATOS
>> IBM Coop Architect & Technical Leader
>> Office : +33 (0) 4 76 29 72 67
>> 1 rue de Provence - 38432 Échirolles - France
>> www.atos.net
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>>


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Björn Gustavsson-4
In reply to this post by REIX, Tony
On Thu, Mar 2, 2017 at 5:33 PM, REIX, Tony <[hidden email]> wrote:

> Hi Björn
>
> Thanks for studying.
>
> I had already looked at these error messages. Here is below what I got about test.emulator_test .
>
> This is due to : ts_install_cth failed! exit:{{make_failed,make},[{ts_make,make,1,
>
> I have this issue both on AIX and on FC25/x86_64.
> I guess that this issue should NOT appear on FC25/x86_64. However, I have it 48 times.
>
> On AIX, there are probably MANY reasons why this ts_install_cth failed.
> However, on FC25./x86_64, I am VERY surprised that this fails.
> However, the guy who ported Erlang on FC25/x86_64 said to me that he did not run the Erlang tests...
> Since I found that their RPM .spec file was missing a Request: instruction for the C++ compiler (leading to errors at build), I guess that their .spec file may contain other issues.

Yes, it seems that not all necessary include files are included.
Without those files, it is not possible to compile drivers
or NIF libraries.

To avoid debugging an incorrect RPM .spec and failing test
cases at the same time, I suggest that you ignore the RPM
for now and use our git repository, at least on FC25.

Out wiki pages should contain enough information:

https://github.com/erlang/otp/wiki

If you would prefer to fix the RPM .spec, you must make sure that
the following files are copied to the same directory as erl_nif.h:

erl_fixed_size_int_types.h
erl_int_sizes_config.h
erl_memory_trace_parser.h

That is, in your case, the files should be in:

/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam

/Björn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Desperately looking for official Erlang tests results

Hans Nilsson R (AL/EAB)
In reply to this post by REIX, Tony
Thanks!  This just get weirder and weirder.  Your output from crypto:*
looks good, but the crypto test results made me expect other outputs...

The crypto app has a C-part called via NIF, and the C-part interfaces
libcrypto.so which in you case is OpenSSL 1.0.2j.  It is *very*
important to compile with the same version of .h files as the lib that
is linked later.  I don't say you have that error, but just as a reminder.

Since Björn found some strange things in the NIF/driver/emulator area, I
would suggest to fix that first.  If there then are any errors on
crypto+ssh+ssl+public_key when the more basic emulator parts are error
free, we could start investigating again.

-Hans

On 03/02/2017 06:30 PM, REIX, Tony wrote:

> Hi*
>
> *Here are the outputs.*
>
>
> FC25/x86_64 :*
>
>
> # cd otp-OTP-19.2.3/
> # ./bin/erl
> Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3]
> [async-threads:10] [hipe] [kernel-poll:false]
>
> Eshell V8.2.2 (abort with ^G)
> 1> crypto:ec_curves().
> [secp112r1,secp112r2,secp128r1,secp128r2,secp160k1,
> secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1,
> secp256k1,secp256r1,secp384r1,secp521r1,prime192v1,
> prime192v2,prime192v3,prime239v1,prime239v2,prime239v3,
> prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1,
> brainpoolP160t1|...]
> 2>
>
>
> *AIX:*
>
> # ./bin/erl
> /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec
> Erlang/OTP 19 [erts-8.2] [source]
>
> Eshell V8.2  (abort with ^G)
> 1> crypto:ec_curves().
> [secp112r1,secp112r2,secp128r1,secp128r2,secp160k1,
>  secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1,
>  secp256k1,secp256r1,secp384r1,secp521r1,prime192v1,
>  prime192v2,prime192v3,prime239v1,prime239v2,prime239v3,
>  prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1,
>  brainpoolP160t1|...]
> 2>
>
>
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
> ________________________________________
> De : Hans Nilsson R [[hidden email]]
> Envoyé : jeudi 2 mars 2017 18:05
> À : REIX, Tony; [hidden email]
> Objet : Re: [erlang-questions] Desperately looking for official Erlang
> tests results
>
> Hi,
>
> Thanks for the log files. I need another check by you. What does the
> function call
>
> crypto:ec_curves().
>
> return on AIX.PPC64 and on .FC25.x86_64 ?
>
> /Hans
>
>
> On 03/02/2017 04:49 PM, REIX, Tony wrote:
>> Resending without the attachments since it is too big.
>> I'll sent the files to Hans directly.
>>
>> Cordialement,
>>
>> Tony Reix
>>
>> Bull - ATOS
>> IBM Coop Architect & Technical Leader
>> Office : +33 (0) 4 76 29 72 67
>> 1 rue de Provence - 38432 Échirolles - France
>> www.atos.net <http://www.atos.net/>
>> ------------------------------------------------------------------------
>> *De :* REIX, Tony
>> *Envoyé :* jeudi 2 mars 2017 16:46
>> *À :* Hans Nilsson R; [hidden email]
>> *Objet :* RE:[erlang-questions] Desperately looking for official Erlang
>> tests results
>>
>> Hi Hans,
>>
>> On AIX, I have only 1 test failing:
>>
>> *OK FAIL SKIP TOTAL*
>> crypto_test: 111 *1* 3 115
>>
>>
>> However, on FC25/x86_64, I have 4 more:
>>
>> *OK FAIL SKIP TOTAL*
>> crypto_test: 107 *5* 3 115
>>
>>
>>
>> *1) AIX:*
>>
>> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 ***
>>
>> crypto:strong_rand_mpint failed on line 1417
>> Reason: low_entropy
>>
>> # cd otp_src_19.2
>>
>> # ./bin/erl
>> /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec
>> Erlang/OTP 19 [erts-8.2] [source]
>>
>> Eshell V8.2 (abort with ^G)
>> 1> crypto:info_lib().
>> [{<<"OpenSSL">>,268443823,
>> <<"OpenSSL 1.0.2j 26 Sep 2016">>}]
>> 2> crypto:supports().
>> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
>> {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
>> aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
>> aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
>> blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
>> aes_gcm]},
>> {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}]
>> 3>
>>
>>
>> *2) FC25/x86_64:*
>>
>> 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} ***
>>
>> 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 ***
>> crypto:ecdh_compute_key_nif failed
>> Reason: badarg
>>
>> 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 ***
>> crypto:ec_key_generate failed
>> Reason: badarg
>>
>> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115
>> *** Common with AIX
>> crypto:strong_rand_mpint failed on line 1417
>> Reason: low_entropy
>>
>> 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 ***
>> crypto:ec_key_generate failed
>> Reason: badarg
>>
>> 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 ***
>> Error detected: {'EXIT',low_entropy}
>>
>> # pwd
>> /root/rpmbuild/BUILD/otp-OTP-19.2.3
>>
>> #./bin/erl
>> Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3]
>> [async-threads:10] [hipe] [kernel-poll:false]
>>
>> Eshell V8.2.2 (abort with ^G)
>> 1> crypto:info_lib().
>> [{<<"OpenSSL">>,268443839,
>> <<"OpenSSL 1.0.2k-fips 26 Jan 2017">>}]
>> 2> crypto:supports().
>> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]},
>> {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc,
>> aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb,
>> aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc,
>> blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4,
>> aes_gcm]},
>> {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}]
>>
>>
>>
>> Cordialement,
>>
>> Tony Reix
>>
>> Bull - ATOS
>> IBM Coop Architect & Technical Leader
>> Office : +33 (0) 4 76 29 72 67
>> 1 rue de Provence - 38432 Échirolles - France
>> www.atos.net
>>
>> ________________________________________
>> De : Hans Nilsson R [[hidden email]]
>> Envoyé : jeudi 2 mars 2017 15:39
>> À : [hidden email]; REIX, Tony
>> Objet : Re: [erlang-questions] Desperately looking for official Erlang
>> tests results
>>
>> Hi,
>>
>> I looked at the [crypto, public_key, ssh, ssl] group of applications.
>> It does look very bad indeed, except for public_key.
>>
>> So let's start from the bottom (i.e. crypto). I need the following:
>>
>> 1) start your tested Erlang and call
>> 1> crypto:info_lib().
>> -- result 1 ---
>> 2> crypto:supports().
>> -- result 2 ---
>> 3>
>>
>> What are the "result *" you get?
>>
>> 2) Mail me the crypto test results directory
>>
>> ..../ct_run.test_server@dorado-vm1.2017-03-01_21.34.20/tests.crypto_test.logs
>>
>> /Hans
>>
>> On 03/02/2017 12:23 PM, REIX, Tony wrote:
>>> Hi,
>>>
>>> In order to understand which Erlang tests and features do not work fine
>>> within our port of version 19.2 on AIX, we need some "official" Erlang
>>> tests results for current version (19.2.3).
>>> However, searching on Erlang web-site, I was unable to find such tests
>>> results.
>>>
>>> So, I have built and tested Erlang on 3 environments: AIX 6.1,
>>> Fedora25/PPC64, and Fedora25/x86_64.
>>> I hope I've correctly ran the tests, but I'm not sure.
>>>
>>> In short, using nearly the same procedure (use Fedora .spec file), but
>>> without HiPE-SMP-Threads for AIX, I've got:
>>>
>>> OK FAIL SKIP TOTAL
>>> ------------------------------------------
>>> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !)
>>> FC25/PPC64 12,376 193 3,608 16,177
>>> FC25/x86_64 12,407 288 3,475 16,170
>>>
>>> I am really surprised by:
>>> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of
>>> all tests !!
>>> - how few failures I have on AIX, comparable to the amount of failures
>>> on FC25/x86_64.
>>> (However: - when trying with SMP and/or HiPE, I had hangs, so
>>> I'm first building/testing without them
>>> - we applied several AIX-specific patches)
>>> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
>>> (However: there are some more SKIPped tests: 133)
>>>
>>> Since result files contains about 600,000 lines, it is very difficult
>>> for me to understand where/what are the issues.
>>> I think we have understood that, in many cases, some environment is
>>> missing/failing and thus the test is skipped.
>>>
>>> I'd like to get some help about:
>>> - checking that I'm doing the Erlang configuration, build, and tests
>>> correctly,
>>> - finding some "official" Erlang tests results on Fedora / x86_64
>>> - how to analyze these results and find the real issues.
>>>
>>> I have attached to this email the index.html file produced on
>>> Fedora25/x86_64 :
>>>
>>> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>>>
>>> Thanks
>>>
>>> Cordialement,
>>>
>>> Tony Reix
>>>
>>> Bull - ATOS
>>> IBM Coop Architect & Technical Leader
>>> Office : +33 (0) 4 76 29 72 67
>>> 1 rue de Provence - 38432 Échirolles - France
>>> www.atos.net
>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> [hidden email]
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

smime.p7s (5K) Download Attachment