HiPE in OTP 22

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

HiPE in OTP 22

Kenneth Lundin-5
In OTP 22 HiPE (the native code generator) is not fully functional. 

The reasons for this are:
  1. We have introduced new beam instructions which the HiPE native code compiler does not handle.
  2. The new optimizations in the beam compiler create new combinations of instructions that HiPE does not handle correctly.
  3. Until now HiPE has been developed by Kostis Sagonas and his students at Uppsala University and Technial University of Athens. But now other projects take up all of their resources.
  4. The OTP team at Ericsson can't prioritize to address these issues either since 
    • there is no use of HiPE among our primary customers
    • the effort needed to gainthe competence to addess the problems above is quite high.

In OTP 22 we have therefore decided to do a workaround for HiPE (native code gen) in such a way that invokations of `erlc +native erlfile.erl` or `erlc +native erlfile.beam` will automatically fall back to producing a .beam file without native code if any of the new binary matching instructions are used. A warning will also be produced when this happens.

The new beam instructions are used for binary matching so it is mostly (or only) modules containing binary matching expressions which will fail the native code generation. Notably is that the native code generations that Dialyzer does on the fly will still work since the modules chosen do not contain binary matching. Native code generation of Dialyzer makes the analysis about 40% faster.

Building OTP with the configure option `--enable-native-libs` will still work in OTP 22 but a note about potential instability when running native code will be output, since we have noticed issues in our test when running these builds.

We have no idea about who are using HiPE and for what, but in order to keep HiPE in shape and as part of the OTP releases we need help in maintaining it.

Volunteers  are welcome

/Kenneth, Erlang/OTP Ericsson

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: HiPE in OTP 22

Dmytro Lytovchenko
My existing VM source knowledge and low level programming competence are probably close to the arcane knowledge required to make HiPE work.

Just registering my name on the waiting list, I am 100% busy at least until the Code Beam STO, possibly also a few weeks after.
Sometimes I do get paid time to assist the OTP team with whatever task is on the table, or HiPE might become a larger project, either my personal or between my employer and Ericsson's OTP team. Who knows.

On Fri, 22 Mar 2019 at 14:43, Kenneth Lundin <[hidden email]> wrote:
In OTP 22 HiPE (the native code generator) is not fully functional. 

The reasons for this are:
  1. We have introduced new beam instructions which the HiPE native code compiler does not handle.
  2. The new optimizations in the beam compiler create new combinations of instructions that HiPE does not handle correctly.
  3. Until now HiPE has been developed by Kostis Sagonas and his students at Uppsala University and Technial University of Athens. But now other projects take up all of their resources.
  4. The OTP team at Ericsson can't prioritize to address these issues either since 
    • there is no use of HiPE among our primary customers
    • the effort needed to gainthe competence to addess the problems above is quite high.

In OTP 22 we have therefore decided to do a workaround for HiPE (native code gen) in such a way that invokations of `erlc +native erlfile.erl` or `erlc +native erlfile.beam` will automatically fall back to producing a .beam file without native code if any of the new binary matching instructions are used. A warning will also be produced when this happens.

The new beam instructions are used for binary matching so it is mostly (or only) modules containing binary matching expressions which will fail the native code generation. Notably is that the native code generations that Dialyzer does on the fly will still work since the modules chosen do not contain binary matching. Native code generation of Dialyzer makes the analysis about 40% faster.

Building OTP with the configure option `--enable-native-libs` will still work in OTP 22 but a note about potential instability when running native code will be output, since we have noticed issues in our test when running these builds.

We have no idea about who are using HiPE and for what, but in order to keep HiPE in shape and as part of the OTP releases we need help in maintaining it.

Volunteers  are welcome

/Kenneth, Erlang/OTP Ericsson
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions