I found this feature from some of the merged PRs of Erlang/OTP 20 > Erlang/OTP 20.0, atoms and function can contain Unicode characters so I built the binary from git checked out source code (from revision efcb91bce) and play with it: it seems in escript interpreting mode it works, but compiling mode still has errors: ➸ cat ./hello.erl #!/usr/bin/env escript %% -*- coding: utf-8 -*- %%! +pc unicode -module(hello). %-mode(compile). -export([main/1, 'hello_юникод_世界'/0]). main(_) -> % io:format("~p~n", [{io:printable_range(), io:getopts()}]), % 'hello_'(), io:setopts([{encoding,unicode}]), % io:format("~p~n", [{io:printable_range(), io:getopts()}]), 'hello_юникод_世界'(), io:format("~tp~n", [{"Hello, 世界, юникод"}]). 'hello_юникод_世界'() -> io:format("~tp~n", [{'hello_юникод', <<"Hello, 世界; юникод"/utf8>>, "Hello, 世界; юникод"}]). ➸ erlang20 ./hello.erl {'hello_юникод',<<"Hello, 世界; юникод"/utf8>>,"Hello, 世界; юникод"} {"Hello, 世界, юникод"} If I uncomment the "-mode(compile)." line, it just runs into compilation error (same as below shell compilation errors) Or in the Erlang shell, trying to compile it just failed: (I have to remove (or comment out) the first "#!/usr/bin/env escript" line) ➸ erlang20 Erlang/OTP 20 [DEVELOPMENT] [erts-9.0] [source-efcb91bce] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] Eshell V9.0 (abort with ^G) 1> c(hello). hello.erl: internal error in beam_asm; crash reason: badarg in function list_to_binary/1 called as list_to_binary([[5,104,101,108,108,111], [4,109,97,105,110], [2,105,111], [7,115,101,116,111,112,116,115], [6,102,111,114,109,97,116], [15,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028], "\vmodule_info", [6,101,114,108,97,110,103], [15,103,101,116,95,109,111,100,117,108,101,95,105,110,102,111]]) in call from beam_asm:chunk/3 (beam_asm.erl, line 213) in call from beam_asm:build_file/8 (beam_asm.erl, line 97) in call from beam_asm:module/4 (beam_asm.erl, line 31) in call from compile:beam_asm/1 (compile.erl, line 1370) in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, line 329) in call from compile:fold_comp/3 (compile.erl, line 355) in call from compile:internal_comp/4 (compile.erl, line 339) error 2> I have looked up in many of the commit messages, still not seeing a good example of how to use unicode in function names? Could some developer behind this feature give more examples? _______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
Hi derek, I have sent a PR that improves the coverage around this feature: https://github.com/erlang/otp/pull/1439 The test currently passes on an old master reference. I will rebuild my development VM from master and follow up. On Thu, May 4, 2017 at 1:11 AM, derek <[hidden email]> wrote:
_______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
I have rebuilt Erlang from master and both the test and the code snippet you sent by e-mail works. In order to discard any encoding issue in the process, can you please push your code snippets to a repository? On Thu, May 4, 2017 at 8:04 AM, José Valim <[hidden email]> wrote:
_______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
On Thu, May 4, 2017 at 3:36 AM José Valim <[hidden email]> wrote:
_______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
I have ran the following on Erlang master, commit ffa80a4137: $ curl https://gist.githubusercontent.com/anonymous/2e8babd349167a49eb6f6d55ec0505c9/raw/cee893bbcd1e9ade6e145c7f66b8599ab9dd6959/hello.erl | tail -n +2 > hello.erl $ erlc hello.erl And everything compiles just fine. Hopefully someone else can give it a try and let us know if they can reproduce the issue or not. On Thu, May 4, 2017 at 11:23 PM, derek <[hidden email]> wrote:
_______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
No; I built this from current `master@9ac8bdb`, it seems nothing changed, either erlc or c() from erlang shell still doesn't compile it prompts error in line 213 of beam_asm.erl, but line 213 of this file doesn't have call to list_to_binary; as I read, it seems on #L245, the list_to_binary doesn't work with this list string `[16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44],` because this string has chars (>255), it needs unicode:characters_to_binary can work Erlang/OTP 20 [RELEASE CANDIDATE 1] [erts-9.0] [source-9ac8bdb] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] Eshell V9.0 (abort with ^G) 1> c(hello). hello.erl: internal error in beam_asm; crash reason: badarg in function list_to_binary/1 called as list_to_binary([[5,104,101,108,108,111], [4,109,97,105,110], [2,105,111], [7,115,101,116,111,112,116,115], [6,102,111,114,109,97,116], [16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44], "\vmodule_info", [6,101,114,108,97,110,103], [15,103,101,116,95,109,111,100,117,108,101,95,105,110,102,111]]) in call from beam_asm:chunk/3 (beam_asm.erl, line 213) in call from beam_asm:build_file/8 (beam_asm.erl, line 97) in call from beam_asm:module/4 (beam_asm.erl, line 31) in call from compile:beam_asm/1 (compile.erl, line 1370) in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, line 329) in call from compile:fold_comp/3 (compile.erl, line 355) in call from compile:internal_comp/4 (compile.erl, line 339) error 2> list_to_binary([16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44]). ** exception error: bad argument in function list_to_binary/1 called as list_to_binary([16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076, 95,19990,30028,44]) 3> unicode:characters_to_binary([16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44]). <<16,104,101,108,108,111,95,209,142,208,189,208,184,208, 186,208,190,208,180,95,228,184,150,231,149,140,44>> On Fri, May 5, 2017 at 5:35 AM José Valim <[hidden email]> wrote:
_______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
Good catch. I wonder why I can't trigger that branch though. Do you have by any chance any flag or option that is being set globally or that you pass to erlc that may affect compilation?
-- _______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
In reply to this post by books
The line numbers in the stacktrace don't make any sense.
For example, line 213 is a comment, not executable code in commit 9ac8bdb. Therefore I suspect that you somewhere have an older version of beam_asm.beam that is somehow found instead of the correct one. You can use beam_asm:module_info(md5) to compute the MD5 and compare it with my output below. You could also try code:which(beam_asm) and make sure that the path is correct. $ bin/erl +pc unicode Erlang/OTP 20 [RELEASE CANDIDATE 1] [erts-9.0] [source-9ac8bdb19f] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] Eshell V9.0 (abort with ^G) 1> c(hello). {ok,hello} 2> hello:main([]). {'hello_юникод',<<"Hello, 世界; юникод"/utf8>>,"Hello, 世界; юникод"} {"Hello, 世界, юникод"} ok 3> beam_asm:module_info(md5). <<159,89,125,255,14,214,31,228,207,196,247,228,180,227,92, 181>> /Björn On Sat, May 6, 2017 at 3:47 AM, derek <[hidden email]> wrote: > No; I built this from current `master@9ac8bdb`, it seems nothing changed, > either erlc or c() from erlang shell still doesn't compile > it prompts error in line 213 of beam_asm.erl, but line 213 of this file > doesn't have call to list_to_binary; as I read, it seems on #L245, the > list_to_binary doesn't work with this list string > `[16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44],` > because this string has chars (>255), it needs unicode:characters_to_binary > can work > https://github.com/erlang/otp/blob/master/lib/compiler/src/beam_asm.erl#L245 > > Erlang/OTP 20 [RELEASE CANDIDATE 1] [erts-9.0] [source-9ac8bdb] [64-bit] > [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V9.0 (abort with ^G) > 1> c(hello). > hello.erl: internal error in beam_asm; > crash reason: badarg > > in function list_to_binary/1 > called as list_to_binary([[5,104,101,108,108,111], > [4,109,97,105,110], > [2,105,111], > [7,115,101,116,111,112,116,115], > [6,102,111,114,109,97,116], > > [16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44], > "\vmodule_info", > [6,101,114,108,97,110,103], > [15,103,101,116,95,109,111,100,117,108,101,95,105,110,102,111]]) > in call from beam_asm:chunk/3 (beam_asm.erl, line 213) > in call from beam_asm:build_file/8 (beam_asm.erl, line 97) > in call from beam_asm:module/4 (beam_asm.erl, line 31) > in call from compile:beam_asm/1 (compile.erl, line 1370) > in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, line > 329) > in call from compile:fold_comp/3 (compile.erl, line 355) > in call from compile:internal_comp/4 (compile.erl, line 339) > error > 2> > list_to_binary([16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44]). > ** exception error: bad argument > in function list_to_binary/1 > called as > list_to_binary([16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076, > 95,19990,30028,44]) > 3> > unicode:characters_to_binary([16,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028,44]). > <<16,104,101,108,108,111,95,209,142,208,189,208,184,208, > 186,208,190,208,180,95,228,184,150,231,149,140,44>> > > > > On Fri, May 5, 2017 at 5:35 AM José Valim <[hidden email]> > wrote: >> >> I have ran the following on Erlang master, commit ffa80a4137: >> >> $ curl >> https://gist.githubusercontent.com/anonymous/2e8babd349167a49eb6f6d55ec0505c9/raw/cee893bbcd1e9ade6e145c7f66b8599ab9dd6959/hello.erl >> | tail -n +2 > hello.erl >> $ erlc hello.erl >> >> And everything compiles just fine. Hopefully someone else can give it a >> try and let us know if they can reproduce the issue or not. >> >> >> >> >> José Valim >> www.plataformatec.com.br >> Skype: jv.ptec >> Founder and Director of R&D >> >> On Thu, May 4, 2017 at 11:23 PM, derek <[hidden email]> wrote: >>> >>> like this gist? >>> https://gist.github.com/anonymous/2e8babd349167a49eb6f6d55ec0505c9 >>> >>> On Thu, May 4, 2017 at 3:36 AM José Valim >>> <[hidden email]> wrote: >>>> >>>> I have rebuilt Erlang from master and both the test and the code snippet >>>> you sent by e-mail works. In order to discard any encoding issue in the >>>> process, can you please push your code snippets to a repository? >>>> >>>> >>>> >>>> José Valim >>>> www.plataformatec.com.br >>>> Skype: jv.ptec >>>> Founder and Director of R&D >>>> >>>> On Thu, May 4, 2017 at 8:04 AM, José Valim >>>> <[hidden email]> wrote: >>>>> >>>>> Hi derek, >>>>> >>>>> I have sent a PR that improves the coverage around this feature: >>>>> https://github.com/erlang/otp/pull/1439 >>>>> >>>>> The test currently passes on an old master reference. I will rebuild my >>>>> development VM from master and follow up. >>>>> >>>>> >>>>> >>>>> José Valim >>>>> www.plataformatec.com.br >>>>> Skype: jv.ptec >>>>> Founder and Director of R&D >>>>> >>>>> On Thu, May 4, 2017 at 1:11 AM, derek <[hidden email]> wrote: >>>>>> >>>>>> >>>>>> I found this feature from some of the merged PRs of Erlang/OTP 20 >>>>>> >>>>>> >>>>>> https://github.com/erlang/otp/blob/master/lib/stdlib/doc/src/unicode_usage.xml#L66-L71 >>>>>> https://github.com/erlang/otp/pulls?q=unicode >>>>>> >>>>>> > Erlang/OTP 20.0, atoms and function can contain Unicode characters >>>>>> >>>>>> so I built the binary from git checked out source code (from revision >>>>>> efcb91bce) and play with it: it seems in escript interpreting mode it works, >>>>>> but compiling mode still has errors: >>>>>> >>>>>> ➸ cat ./hello.erl >>>>>> #!/usr/bin/env escript >>>>>> %% -*- coding: utf-8 -*- >>>>>> %%! +pc unicode >>>>>> >>>>>> -module(hello). >>>>>> %-mode(compile). >>>>>> -export([main/1, 'hello_юникод_世界'/0]). >>>>>> >>>>>> main(_) -> >>>>>> % io:format("~p~n", [{io:printable_range(), io:getopts()}]), >>>>>> % 'hello_'(), >>>>>> io:setopts([{encoding,unicode}]), >>>>>> % io:format("~p~n", [{io:printable_range(), io:getopts()}]), >>>>>> 'hello_юникод_世界'(), >>>>>> io:format("~tp~n", [{"Hello, 世界, юникод"}]). >>>>>> >>>>>> 'hello_юникод_世界'() -> >>>>>> io:format("~tp~n", [{'hello_юникод', <<"Hello, 世界; юникод"/utf8>>, >>>>>> "Hello, 世界; юникод"}]). >>>>>> >>>>>> ➸ erlang20 ./hello.erl >>>>>> {'hello_юникод',<<"Hello, 世界; юникод"/utf8>>,"Hello, 世界; юникод"} >>>>>> {"Hello, 世界, юникод"} >>>>>> >>>>>> If I uncomment the "-mode(compile)." line, it just runs into >>>>>> compilation error (same as below shell compilation errors) >>>>>> >>>>>> Or in the Erlang shell, trying to compile it just failed: (I have to >>>>>> remove (or comment out) the first "#!/usr/bin/env escript" line) >>>>>> >>>>>> ➸ erlang20 >>>>>> Erlang/OTP 20 [DEVELOPMENT] [erts-9.0] [source-efcb91bce] [64-bit] >>>>>> [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] >>>>>> >>>>>> Eshell V9.0 (abort with ^G) >>>>>> 1> c(hello). >>>>>> hello.erl: internal error in beam_asm; >>>>>> crash reason: badarg >>>>>> >>>>>> in function list_to_binary/1 >>>>>> called as list_to_binary([[5,104,101,108,108,111], >>>>>> [4,109,97,105,110], >>>>>> [2,105,111], >>>>>> [7,115,101,116,111,112,116,115], >>>>>> [6,102,111,114,109,97,116], >>>>>> >>>>>> [15,104,101,108,108,111,95,1102,1085,1080,1082,1086,1076,95,19990,30028], >>>>>> "\vmodule_info", >>>>>> [6,101,114,108,97,110,103], >>>>>> [15,103,101,116,95,109,111,100,117,108,101,95,105,110,102,111]]) >>>>>> in call from beam_asm:chunk/3 (beam_asm.erl, line 213) >>>>>> in call from beam_asm:build_file/8 (beam_asm.erl, line 97) >>>>>> in call from beam_asm:module/4 (beam_asm.erl, line 31) >>>>>> in call from compile:beam_asm/1 (compile.erl, line 1370) >>>>>> in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, >>>>>> line 329) >>>>>> in call from compile:fold_comp/3 (compile.erl, line 355) >>>>>> in call from compile:internal_comp/4 (compile.erl, line 339) >>>>>> error >>>>>> 2> >>>>>> >>>>>> I have looked up in many of the commit messages, still not seeing a >>>>>> good example of how to use unicode in function names? Could some developer >>>>>> behind this feature give more examples? >>>>>> >>>>>> https://github.com/erlang/otp/blame/master/lib/stdlib/doc/src/unicode_usage.xml#L66-L71 >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> [hidden email] >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>> >> > > _______________________________________________ > erlang-questions mailing list > [hidden email] > http://erlang.org/mailman/listinfo/erlang-questions > -- Björn Gustavsson, Erlang/OTP, Ericsson AB _______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
ok; so it seems because I have the ERL_LIBS pointing to a large directory of erlang libs that was taking precedence; if I run with `ERL_LIBS=./lib:$ERL_LIBS erl ...` then it works; thanks then here is a question about ERL_LIBS by the way: I am reading docs and code about current implementation, through #L85 "["."] ++ UserLibPaths ++ Paths;" it seems the code loading precedence is current dir "." >> all "*/ebin" from ERL_LIBS >> the "lib/*/ebin" from current ERL installation dir; I wonder should this be revised? to take ERL installation dir's "lib/*/ebin" precedence before extra ERL_LIBS ? or can we bring in another EXTRA_ERL_LIBS environment variable to be lookup after the "lib/*/ebin" from current ERL installation dir? On Fri, May 5, 2017 at 11:38 PM Björn Gustavsson <[hidden email]> wrote: The line numbers in the stacktrace don't make any sense. _______________________________________________ erlang-questions mailing list [hidden email] http://erlang.org/mailman/listinfo/erlang-questions |
Free forum by Nabble | Edit this page |