Customizing Beam Build

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Customizing Beam Build

Sargun Dhillon
I've customized my OTP build to have a custom Erlang build. The reason
I've done this to begin the process of building a fully-self contained
Erlang program. One of the biggest benefits of the infrastructure in
Go is that they're fully self-contained statically linked binaries. In
Erlang, although you can statically link beam itself, unfortunately,
the releases themselves are messy, in that you have a lot of files and
getting the execution environment right isn't always trivial.

Eventually, the purpose is to build an init system -- something that
will be primarily used by sysadmin-types. These need to be statically
linked, and often times, the file system isn't fully-functioning when
they're starting.

 I've done two things:

1) I've modified the set of preloaded files in the VM. It would be
really nice if there was a way to do this without forking OTP. I
didn't add any existing beam files, but I just changed around some of
the file, and code loading code to my need.

2) I've modified the final, linked product to include custom objects
in the final ELF binary.

This is kind of terrible, as it requires I maintain a fork of OTP.
Does anyone have any recommendations for solving this problem another
way?

My proposal would be to output a beam.a file with the requisite
objects to build a BEAM installation. It would be valuable if I could
statically link against this binary, and own main, if it had a
separate entrypoint that I could then call into. I've toyed around
with this, and for the unix build, it seems possible. Is there any
reason why Erlang isn't distributed this way already?

If there was a public API for extending the loader, or adding
preloaded modules via sections, I imagine that solving (1) would be
easy as well.

Is anyone open to accepting this upstream if the work is done?
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Joe Armstrong-2
This is great (I think) - if you've done what I think you might have done

Have you seen following this thread?

http://erlang.org/pipermail/erlang-questions/2017-September/093468.html

I posted a comment here:

http://erlang.org/pipermail/erlang-questions/2017-September/093496.html

If we can link all the DLL's to a single together with a small number
of precompiled
erlang modules then I think making stand-alone applications is doable.

I experimented with self-modifying code - you basically take the executable
and append some beam code at the end of the executable. The precompiled
part checks to see if any code has been added to the end of the executable
and if so loads and runs it.

The nice part about this is you can write the self-modifying part in Erlang.

What is needed is a script that can make a stand-alone program with a
number of precompiled erlang modules, say something like this:

    > make_stand_alone_executable x.erl y.erl z.erl  -start x -out prog

This would turn x.erl y.erl and z.erl into C (this is easy) then build
a stand alone
executable called prog.

The command "prog args" would start execution of x:main(args)

I thought to myself this must be possible since all the C code necessary
is available - the fact that erlexec is small and does not statically link
the emulator must have to do with the makefiles - Unfortunately I could
not easily figure out how the makefiles were constructed.

I tried to do this a while back - and remember there was one other
problem - the code in erlexec presupposes a bin directory (or
something like that)

There would be another problem with NIFs - since these are dynamically loaded
they would have to be statically linked instead.

If you can make a statically linked binary containing a small number of
pre-compiled beam files the rest should be easy. The only thing
to watch for is to make sure that that static binary does not require
and funny environment variables or directory paths to be set.

To me "stand alone" means "stand alone" (ie the program should not break if
you relocate it because you forgot to set some environment variable.

If you have the basis for a 'make_stand_alone_executable' script it would be
great - how to integrate this with all the autoconf stuff is a mystery to me.

Cheers

/Joe











On Sun, Sep 24, 2017 at 10:25 AM, Sargun Dhillon <[hidden email]> wrote:

> I've customized my OTP build to have a custom Erlang build. The reason
> I've done this to begin the process of building a fully-self contained
> Erlang program. One of the biggest benefits of the infrastructure in
> Go is that they're fully self-contained statically linked binaries. In
> Erlang, although you can statically link beam itself, unfortunately,
> the releases themselves are messy, in that you have a lot of files and
> getting the execution environment right isn't always trivial.
>
> Eventually, the purpose is to build an init system -- something that
> will be primarily used by sysadmin-types. These need to be statically
> linked, and often times, the file system isn't fully-functioning when
> they're starting.
>
>  I've done two things:
>
> 1) I've modified the set of preloaded files in the VM. It would be
> really nice if there was a way to do this without forking OTP. I
> didn't add any existing beam files, but I just changed around some of
> the file, and code loading code to my need.
>
> 2) I've modified the final, linked product to include custom objects
> in the final ELF binary.
>
> This is kind of terrible, as it requires I maintain a fork of OTP.
> Does anyone have any recommendations for solving this problem another
> way?
>
> My proposal would be to output a beam.a file with the requisite
> objects to build a BEAM installation. It would be valuable if I could
> statically link against this binary, and own main, if it had a
> separate entrypoint that I could then call into. I've toyed around
> with this, and for the unix build, it seems possible. Is there any
> reason why Erlang isn't distributed this way already?
>
> If there was a public API for extending the loader, or adding
> preloaded modules via sections, I imagine that solving (1) would be
> easy as well.
>
> Is anyone open to accepting this upstream if the work is done?
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Benoit Chesneau-2
In reply to this post by Sargun Dhillon

> On 24 Sep 2017, at 10:25, Sargun Dhillon <[hidden email]> wrote:
>
> I've customized my OTP build to have a custom Erlang build. The reason
> I've done this to begin the process of building a fully-self contained
> Erlang program. One of the biggest benefits of the infrastructure in
> Go is that they're fully self-contained statically linked binaries. In
> Erlang, although you can statically link beam itself, unfortunately,
> the releases themselves are messy, in that you have a lot of files and
> getting the execution environment right isn't always trivial.
>
> Eventually, the purpose is to build an init system -- something that
> will be primarily used by sysadmin-types. These need to be statically
> linked, and often times, the file system isn't fully-functioning when
> they're starting.
>
> I've done two things:
>
> 1) I've modified the set of preloaded files in the VM. It would be
> really nice if there was a way to do this without forking OTP. I
> didn't add any existing beam files, but I just changed around some of
> the file, and code loading code to my need.

+1
>
> 2) I've modified the final, linked product to include custom objects
> in the final ELF binary.

What files did you changed for it?
>
> This is kind of terrible, as it requires I maintain a fork of OTP.

Is this something public?

> Does anyone have any recommendations for solving this problem another
> way?
>
> My proposal would be to output a beam.a file with the requisite
> objects to build a BEAM installation. It would be valuable if I could
> statically link against this binary, and own main, if it had a
> separate entrypoint that I could then call into. I've toyed around
> with this, and for the unix build, it seems possible. Is there any
> reason why Erlang isn't distributed this way already?
>
> If there was a public API for extending the loader, or adding
> preloaded modules via sections, I imagine that solving (1) would be
> easy as well.
>
> Is anyone open to accepting this upstream if the work is done?
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Tristan Sloughter-4
In reply to this post by Sargun Dhillon
I like the idea of easily built static builds but for dealing with
releases being messy (if I'm understanding what you mean) is something
that may be solved simply with existing tools like makeself:

http://makeself.io/

--
  Tristan Sloughter
  "I am not a crackpot" - Abe Simpson
  [hidden email]

On Sun, Sep 24, 2017, at 01:25 AM, Sargun Dhillon wrote:

> I've customized my OTP build to have a custom Erlang build. The reason
> I've done this to begin the process of building a fully-self contained
> Erlang program. One of the biggest benefits of the infrastructure in
> Go is that they're fully self-contained statically linked binaries. In
> Erlang, although you can statically link beam itself, unfortunately,
> the releases themselves are messy, in that you have a lot of files and
> getting the execution environment right isn't always trivial.
>
> Eventually, the purpose is to build an init system -- something that
> will be primarily used by sysadmin-types. These need to be statically
> linked, and often times, the file system isn't fully-functioning when
> they're starting.
>
>  I've done two things:
>
> 1) I've modified the set of preloaded files in the VM. It would be
> really nice if there was a way to do this without forking OTP. I
> didn't add any existing beam files, but I just changed around some of
> the file, and code loading code to my need.
>
> 2) I've modified the final, linked product to include custom objects
> in the final ELF binary.
>
> This is kind of terrible, as it requires I maintain a fork of OTP.
> Does anyone have any recommendations for solving this problem another
> way?
>
> My proposal would be to output a beam.a file with the requisite
> objects to build a BEAM installation. It would be valuable if I could
> statically link against this binary, and own main, if it had a
> separate entrypoint that I could then call into. I've toyed around
> with this, and for the unix build, it seems possible. Is there any
> reason why Erlang isn't distributed this way already?
>
> If there was a public API for extending the loader, or adding
> preloaded modules via sections, I imagine that solving (1) would be
> easy as well.
>
> Is anyone open to accepting this upstream if the work is done?
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Dmitry Kolesnikov-2

On 25 Sep 2017, at 22.32, Tristan Sloughter <[hidden email]> wrote:

may be solved simply with existing tools like makeself

+1. 

There are not issue to package Erlang release to tarball and then to self-extractable archive. This is what I am doing on daily basic. I am trying to understand why community rejects this approach and targets some complex tooling? Is this because of Windows platform support or escripts? Or something else?

- Dmitry

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Tristan Sloughter-4
There are not issue to package Erlang release to tarball and then to self-extractable archive. This is what I am doing on daily basic. I am trying to understand why community rejects this approach and targets some complex tooling? Is this because of Windows platform support or escripts? Or something else?

escripts are a separate use case. And not working on Windows is certainly limiting but I doubt it is why the pattern isn't used much.

Personally I just use containers these days, so no real need for building self-extractable archives, and this is probably the case of many now. Plus the fact that unpacking a tarball and running `bin/<script>` isn't that much more work :). And of course, none of this works with release upgrades or sharing libs between releases to save space.

That said, I meant to also put an example:

rebar3 as prod release
cd _build/prod/rel/<release name>
makeself --bzip2 ./ <release name> "Description" ./bin/<release name>

And then you can just do:

./<release name> console


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Joe Armstrong-2
In reply to this post by Tristan Sloughter-4
Everything would be a lot simpler if I could build a static version of erlexec

At some level abstraction we have a list of C files that must be compiled
and linked together.

If the C files and the header files were all in one directory this would be
easy - as it is the makefiles are generated automatically and the C
files that are needed to build the system are all over the place.

All I need is a makefile to build a static version of erlexec. It is not obvious
to me how to do this (otherwise I would have done it :-)

Cheers

/Joe



On Mon, Sep 25, 2017 at 9:32 PM, Tristan Sloughter <[hidden email]> wrote:

> I like the idea of easily built static builds but for dealing with
> releases being messy (if I'm understanding what you mean) is something
> that may be solved simply with existing tools like makeself:
>
> http://makeself.io/
>
> --
>   Tristan Sloughter
>   "I am not a crackpot" - Abe Simpson
>   [hidden email]
>
> On Sun, Sep 24, 2017, at 01:25 AM, Sargun Dhillon wrote:
>> I've customized my OTP build to have a custom Erlang build. The reason
>> I've done this to begin the process of building a fully-self contained
>> Erlang program. One of the biggest benefits of the infrastructure in
>> Go is that they're fully self-contained statically linked binaries. In
>> Erlang, although you can statically link beam itself, unfortunately,
>> the releases themselves are messy, in that you have a lot of files and
>> getting the execution environment right isn't always trivial.
>>
>> Eventually, the purpose is to build an init system -- something that
>> will be primarily used by sysadmin-types. These need to be statically
>> linked, and often times, the file system isn't fully-functioning when
>> they're starting.
>>
>>  I've done two things:
>>
>> 1) I've modified the set of preloaded files in the VM. It would be
>> really nice if there was a way to do this without forking OTP. I
>> didn't add any existing beam files, but I just changed around some of
>> the file, and code loading code to my need.
>>
>> 2) I've modified the final, linked product to include custom objects
>> in the final ELF binary.
>>
>> This is kind of terrible, as it requires I maintain a fork of OTP.
>> Does anyone have any recommendations for solving this problem another
>> way?
>>
>> My proposal would be to output a beam.a file with the requisite
>> objects to build a BEAM installation. It would be valuable if I could
>> statically link against this binary, and own main, if it had a
>> separate entrypoint that I could then call into. I've toyed around
>> with this, and for the unix build, it seems possible. Is there any
>> reason why Erlang isn't distributed this way already?
>>
>> If there was a public API for extending the loader, or adding
>> preloaded modules via sections, I imagine that solving (1) would be
>> easy as well.
>>
>> Is anyone open to accepting this upstream if the work is done?
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Peer Stritzinger-3
Hi Joe,

On 26. Sep 2017, at 16:12, Joe Armstrong <[hidden email]> wrote:

My only (I think) problem are the makefiles. I tried reading them
and they were pretty complex - I couldn't see where to change the code to
make a static image.

You shouldn’t need to touch the Makefiles  key is:


You make a copy of one of the files in there and edit that copy.

Together with the description how to cross compile Erlang:


Then for the target system set CC CFLAGS LDFLAGS etc in your copy in a way that makes your C compiler link statically.

I think If I could make a static version of erlexec, it would become clearer
to me.

Unfortunately the argument massaging is done in a separate executable and then the proper emulator i.e. erl_main.c is exec() called.

That makes it also harder for us since we need to duplicate what erlexec does to the environment and the args.  We just leave away erlexec and set the environment variables before calling erl_main() for arguments we keep to the subset that are understood by erl_main() directly without massaging.

I wrote the original ring0 etc stuff but have never been deeply in the C and
makefiles stuff.

Any advice would be welcome.

Cheers

/Joe

On Tue, Sep 26, 2017 at 3:49 PM, Peer Stritzinger <[hidden email]> wrote:
For GRiSP we do have a statically linked version of Erlang, what’s the problem of statically linking it?  It just needs the right build system with static libraries.

As for the forking OTP to get some specialized version, we have a workaround in GRiSPs tooling.   There is a rebar3 plugin that is able to build a specialized version of OTP from source.  One issue we need to solve: adding our own statically linked drivers
and static NIFs to this statically linked beam.  The plugin collects all C drivers etc from all the dependencies and patches them into the OTP source.  Some just get copied over
but e.g. one of the Makefile.in in OTP needs to be patched to add our static drivers to the driver table.

We even cross build with this setup and go from nothing to SD card our NAND flash filesystem contents with cross built statically linked beam plus Erlang release.

BTW cross building could be a way to build a special Erlang for your hostsystem ... nothing prevents from the cross target to be the same as the host architecture with some build flags tweaked.

One caveat remains: there are a few applications in OTP that rely on dynamic linking.  These can be either configured out (currently we still have our patched OTP but we are moving towards getting rid of this requirement) or since there is support for static NIFs in OTP now be made possible static.

Cheers,
— Peer

On 25. Sep 2017, at 22:06, Joe Armstrong <[hidden email]> wrote:

Everything would be a lot simpler if I could build a static version of erlexec

At some level abstraction we have a list of C files that must be compiled
and linked together.

If the C files and the header files were all in one directory this would be
easy - as it is the makefiles are generated automatically and the C
files that are needed to build the system are all over the place.

All I need is a makefile to build a static version of erlexec. It is not obvious
to me how to do this (otherwise I would have done it :-)

Cheers

/Joe



On Mon, Sep 25, 2017 at 9:32 PM, Tristan Sloughter <[hidden email]> wrote:
I like the idea of easily built static builds but for dealing with
releases being messy (if I'm understanding what you mean) is something
that may be solved simply with existing tools like makeself:

http://makeself.io/

--
Tristan Sloughter
"I am not a crackpot" - Abe Simpson
[hidden email]

On Sun, Sep 24, 2017, at 01:25 AM, Sargun Dhillon wrote:
I've customized my OTP build to have a custom Erlang build. The reason
I've done this to begin the process of building a fully-self contained
Erlang program. One of the biggest benefits of the infrastructure in
Go is that they're fully self-contained statically linked binaries. In
Erlang, although you can statically link beam itself, unfortunately,
the releases themselves are messy, in that you have a lot of files and
getting the execution environment right isn't always trivial.

Eventually, the purpose is to build an init system -- something that
will be primarily used by sysadmin-types. These need to be statically
linked, and often times, the file system isn't fully-functioning when
they're starting.

I've done two things:

1) I've modified the set of preloaded files in the VM. It would be
really nice if there was a way to do this without forking OTP. I
didn't add any existing beam files, but I just changed around some of
the file, and code loading code to my need.

2) I've modified the final, linked product to include custom objects
in the final ELF binary.

This is kind of terrible, as it requires I maintain a fork of OTP.
Does anyone have any recommendations for solving this problem another
way?

My proposal would be to output a beam.a file with the requisite
objects to build a BEAM installation. It would be valuable if I could
statically link against this binary, and own main, if it had a
separate entrypoint that I could then call into. I've toyed around
with this, and for the unix build, it seems possible. Is there any
reason why Erlang isn't distributed this way already?

If there was a public API for extending the loader, or adding
preloaded modules via sections, I imagine that solving (1) would be
easy as well.

Is anyone open to accepting this upstream if the work is done?
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Sargun Dhillon
In reply to this post by Benoit Chesneau-2
On Mon, Sep 25, 2017 at 12:13 PM, Benoit Chesneau <[hidden email]> wrote:

>
>> On 24 Sep 2017, at 10:25, Sargun Dhillon <[hidden email]> wrote:
>>
>> I've customized my OTP build to have a custom Erlang build. The reason
>> I've done this to begin the process of building a fully-self contained
>> Erlang program. One of the biggest benefits of the infrastructure in
>> Go is that they're fully self-contained statically linked binaries. In
>> Erlang, although you can statically link beam itself, unfortunately,
>> the releases themselves are messy, in that you have a lot of files and
>> getting the execution environment right isn't always trivial.
>>
>> Eventually, the purpose is to build an init system -- something that
>> will be primarily used by sysadmin-types. These need to be statically
>> linked, and often times, the file system isn't fully-functioning when
>> they're starting.
>>
>> I've done two things:
>>
>> 1) I've modified the set of preloaded files in the VM. It would be
>> really nice if there was a way to do this without forking OTP. I
>> didn't add any existing beam files, but I just changed around some of
>> the file, and code loading code to my need.
>
> +1
>>
>> 2) I've modified the final, linked product to include custom objects
>> in the final ELF binary.
>
> What files did you changed for it?
I redid a significant portion of ./erts/preloaded -- erl_prim_loader,
and prim_file. I also changed this mechanism to be able to add my own
files (modules) in as well. The second part of the compilation was
just done by modifying Makefiles "by hand".

>>
>> This is kind of terrible, as it requires I maintain a fork of OTP.
>
> Is this something public?
Not yet. It's very much "builds for me" -- and nobody else.

>> Does anyone have any recommendations for solving this problem another
>> way?
>>
>> My proposal would be to output a beam.a file with the requisite
>> objects to build a BEAM installation. It would be valuable if I could
>> statically link against this binary, and own main, if it had a
>> separate entrypoint that I could then call into. I've toyed around
>> with this, and for the unix build, it seems possible. Is there any
>> reason why Erlang isn't distributed this way already?
>>
>> If there was a public API for extending the loader, or adding
>> preloaded modules via sections, I imagine that solving (1) would be
>> easy as well.
>>
>> Is anyone open to accepting this upstream if the work is done?
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Customizing Beam Build

Sargun Dhillon
In reply to this post by Dmitry Kolesnikov-2
On Mon, Sep 25, 2017 at 12:38 PM, Dmitry Kolesnikov
<[hidden email]> wrote:

>
> On 25 Sep 2017, at 22.32, Tristan Sloughter <[hidden email]> wrote:
>
> may be solved simply with existing tools like makeself
>
>
> +1.
>
> There are not issue to package Erlang release to tarball and then to
> self-extractable archive. This is what I am doing on daily basic. I am
> trying to understand why community rejects this approach and targets some
> complex tooling? Is this because of Windows platform support or escripts? Or
> something else?
>
> - Dmitry
What I build is a container runtime. This means, I want to be able to
setns into a namespace, and do an execat in an arbitrary, and even
perhaps hostile environment. This also means that either the
filesystem is readonly, or I don't want to pollute the user's
filesystem. Often times, we give people user namespaces, so they can
setup all sorts of interesting mounts that break our software. I also
have to be able to be re-executed in place -- which requires cleanup.

In order to do something like a self-extracting installer, I can
create my own mount namespace using CLONE_NEWNS and mark it as a slave
namespace. I can then setup tmpfs, but then all of my code has to be
in-memory. Unfortunately, some users are running older kernels, and
others are running without these privileges.

I do not care about Windows, only Linux and OS X.


>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions