tiny erlang

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

tiny erlang

Joe Armstrong-2
Hello

I want to make a small erlang distribution for an embedded device - the process
is rather simple:

    1) strip the beam executable
    2) for each library in {stdlib, kernerl, compiler, sasl} do
       Compress all the beam code and remove debug symbols
       squash everthing into a single packed file
    3) Hack the code loader to read from the single packed beam file
    4) Tweak the shell scripts to start Erlang

At a guess we're down to c. 6 MB

I have done this on *several* occassions - and am getting a bit fed up
doing it over and over again.

Has anybody done this recently - and should not something like this be
in the standard distribution?

Cheers

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: tiny erlang

João Henrique Freitas
If it does not fit in your need, I could implement it

Of course meta-erlang should be used with Yocto/Openembedded.

Thanks

On Thu, Feb 18, 2016 at 10:21 AM, Joe Armstrong <[hidden email]> wrote:
Hello

I want to make a small erlang distribution for an embedded device - the process
is rather simple:

    1) strip the beam executable
    2) for each library in {stdlib, kernerl, compiler, sasl} do
       Compress all the beam code and remove debug symbols
       squash everthing into a single packed file
    3) Hack the code loader to read from the single packed beam file
    4) Tweak the shell scripts to start Erlang

At a guess we're down to c. 6 MB

I have done this on *several* occassions - and am getting a bit fed up
doing it over and over again.

Has anybody done this recently - and should not something like this be
in the standard distribution?

Cheers

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



--
João Henrique Ferreira de Freitas - joaohf_at_gmail.com
Campinas-SP-Brasil

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: tiny erlang

Håkan Mattsson
In reply to this post by Joe Armstrong-2

How much do you gain doing it your way compared to use Reltool to build the release?

Assuming you instruct Reltool to only include your four applications, build compressed .ez files, strip the beams, skip unneccessary system files etc.

/Håkan

On Thu, Feb 18, 2016 at 1:21 PM, Joe Armstrong <[hidden email]> wrote:
Hello

I want to make a small erlang distribution for an embedded device - the process
is rather simple:

    1) strip the beam executable
    2) for each library in {stdlib, kernerl, compiler, sasl} do
       Compress all the beam code and remove debug symbols
       squash everthing into a single packed file
    3) Hack the code loader to read from the single packed beam file
    4) Tweak the shell scripts to start Erlang

At a guess we're down to c. 6 MB

I have done this on *several* occassions - and am getting a bit fed up
doing it over and over again.

Has anybody done this recently - and should not something like this be
in the standard distribution?

Cheers

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: tiny erlang

Matthias Lang
In reply to this post by Joe Armstrong-2
Hi,

On 18. February 2016, Joe Armstrong wrote:

> I want to make a small erlang distribution for an embedded device -
> the process is rather simple:

>     1) strip the beam executable
>     2) for each library in {stdlib, kernerl, compiler, sasl} do
>        Compress all the beam code and remove debug symbols
>        squash everthing into a single packed file
>     3) Hack the code loader to read from the single packed beam file
>     4) Tweak the shell scripts to start Erlang
>
> At a guess we're down to c. 6 MB
>
> I have done this on *several* occassions - and am getting a bit fed up
> doing it over and over again.
>
> Has anybody done this recently - and should not something like this be
> in the standard distribution?

I do this every few years, using a Makefile. I've handed that file out
a few times. It's a get-stuff-done solution, not a masterpiece. Also,
I'm not on the bleeding edge of Erlang; on some hardware I use R14B03,
on other hardware 17.4.

Some things I do slightly differently to your steps:

   2. I don't think you need to compress the beam code; Erlang's build
      system does it for you by default when cross compiling, since at
      least R14B03.

   2. I don't ship the compiler library.

   2. I strip some .beams within libraries,
      e.g. 'stdlib/win32reg.beam' is useless on my hardware.

   3. I don't think you need to hack the code loader; I don't. .ez files
      files work as-is since at least R14B03

For R14B03, that gets me down to 2.5M. For 17.4, it's 3M.

---

Over the years, I've tried a number of different solutions to "which
library .beam files do I ship?".

  Whitelist: the script only grabs the libraries I list. That's
             what you suggest above.

  Blacklist: the script deletes a list of libraries. That's what I do
             now. This 'fails' more gracefully when OTP gets a new
             library; I have to actively decide I don't need it.

  test suite driven: I make a release with all the .beams in it, run
             our (comprehensive!) test suite and delete all .beams
             which didn't get loaded, using information from
             code:all_loaded/0. This makes the smallest releases.

I've settled on the 'blacklist' approach. It's robust and
release size doesn't matter as much as it did in 2001 when I
started doing this, mainly because flash memory has gotten far cheaper.

Matt

----------------------------------------------------------------------
Here's the 'Makefile' I use:

# Build and slim Erlang

# Change the line below if you want to use a different version of Erlang
# (Keep in mind that patches may not apply cleanly)
erlang_version=otp_src_R14B03

erlang_source_file:=$(PACKAGE_SOURCES)/$(erlang_version)/$(erlang_version).tar.gz
erlang_root=$(GTH_ERLANG_ROOT)/$(erlang_version)

target_prefix=/opt/erlang

ifneq ($(COR_TARGET), GTH3)
ifneq ($(COR_TARGET), STH3)
ifneq ($(COR_TARGET), GTH2D)
  $(error no COR_TARGET set, or set to an unknown target, giving up)
endif
endif
endif
xconf_file=erl-xcomp-armle-linux-gnu.conf

install_prefix=$($(COR_TARGET)_INSTALL_BASE)

prefix=$(install_prefix)$(target_prefix)

ifeq (,$(findstring $(erlang_version),$(erlang_root)))
default:
        @echo "Aborting because Erlang versions do not match"
        @echo "erlang_root=$(erlang_root)"
        @echo "erlang_version=$(erlang_version)"
        @/bin/false
endif

# Build the actual code
# The 'unset' commands are needed because otp_build misbehaves if it realises
# it's being run by a makefile. See mail to erlang-bugs 2010-06-22/23
build: $(erlang_version)
        @if test -d $(prefix); then \
                echo "$(prefix) already contains a (partial?) Erlang dist. Aborting, because building over the top of a dist is known to break. Use 'make clean_target' to wipe $(prefix)" ; \
                exit 1 ; \
        fi
        sh -c "unset MAKELEVEL; unset MAKEFLAGS; cd $(erlang_version); ./otp_build boot -a"
        sh -c "cd $(erlang_version); make release RELEASE_ROOT=$(prefix)"

# untar, patch, run configure
$(erlang_version): $(erlang_source_file)
        tar -xzf $(erlang_source_file)
        sh -c "cd $(erlang_version); patch -p 1 < ../patches/better_to_erl_messages"
        sh -c "cd $(erlang_version); patch -p 1 < ../patches/extern_inline_c99"
        sh -c "cd $(erlang_version); patch -p 1 < ../patches/perl_5_22_defined_fix"
        sh -c "cd $(erlang_version); ./otp_build configure --xcomp-conf=../$(xconf_file)"

install: run_otp_install slim strip
        make zip_libraries  # run in sub-make; dependencies appeared just now

.PHONY run_otp_install:
        sh -c "cd $(prefix); umask 022; ./Install -cross -minimal /opt/erlang"

slim_binaries =$(addprefix $(prefix)/bin/, beam.smp dialyzer epmd erlc heart typer escript ct_run)
slim_binaries+=$(addprefix $(prefix)/erts*/bin/, \
        beam.smp dialyzer dyn_erl epmd erlc heart start typer escript ct_run *.src)

slim_sources +=$(prefix)/erts*/src/
slim_sources +=$(prefix)/erts*/include/
slim_sources +=$(prefix)/erts*/doc/
slim_sources +=$(prefix)/erts*/man/
slim_sources +=$(prefix)/erts*/lib/internal/README
slim_sources +=$(prefix)/lib/*/src/
slim_sources +=$(prefix)/lib/*/examples/
slim_sources +=$(prefix)/lib/*/include/
slim_sources +=$(prefix)/lib/*/man/
slim_sources +=$(prefix)/usr/
slim_sources +=$(prefix)/misc/

slim_applications +=$(prefix)/lib/appmon*/
slim_applications +=$(prefix)/lib/asn1*/
slim_applications +=$(prefix)/lib/common_test*/
slim_applications +=$(prefix)/lib/compiler*/
slim_applications +=$(prefix)/lib/cos*/
slim_applications +=$(prefix)/lib/debugger*/
slim_applications +=$(prefix)/lib/dialyzer*/
slim_applications +=$(prefix)/lib/diameter*/
slim_applications +=$(prefix)/lib/docbuilder*/
slim_applications +=$(prefix)/lib/edoc*/
slim_applications +=$(prefix)/lib/erl_docgen*/
slim_applications +=$(prefix)/lib/erl_interface*/
slim_applications +=$(prefix)/lib/et*/
slim_applications +=$(prefix)/lib/eunit*/
slim_applications +=$(prefix)/lib/gs*/
slim_applications +=$(prefix)/lib/hipe*/
slim_applications +=$(prefix)/lib/ic*/
slim_applications +=$(prefix)/lib/inets*/
slim_applications +=$(prefix)/lib/inviso*/
slim_applications +=$(prefix)/lib/jinterface*/
slim_applications +=$(prefix)/lib/megaco*/
slim_applications +=$(prefix)/lib/mnesia*/
slim_applications +=$(prefix)/lib/observer*/
slim_applications +=$(prefix)/lib/orber*/
slim_applications +=$(prefix)/lib/os_mon*/
slim_applications +=$(prefix)/lib/otp_mibs*/
slim_applications +=$(prefix)/lib/parsetools*/
slim_applications +=$(prefix)/lib/percept*/
slim_applications +=$(prefix)/lib/pman*/
slim_applications +=$(prefix)/lib/public_key*/
slim_applications +=$(prefix)/lib/reltool*/
slim_applications +=$(prefix)/lib/runtime_tools*/
slim_applications +=$(prefix)/lib/snmp*/
slim_applications +=$(prefix)/lib/syntax_tools*/
slim_applications +=$(prefix)/lib/test_server*/
slim_applications +=$(prefix)/lib/toolbar*/
slim_applications +=$(prefix)/lib/tools*/
slim_applications +=$(prefix)/lib/typer*/
slim_applications +=$(prefix)/lib/tv*/
slim_applications +=$(prefix)/lib/webtool*/
slim_applications +=$(prefix)/lib/wx*/
slim_applications +=$(prefix)/lib/xmerl*/

slim_applications +=$(prefix)/lib/stdlib*/ebin/win32reg.beam
slim_applications +=$(prefix)/lib/sasl*/ebin/systools*.beam
slim_applications +=$(prefix)/lib/sasl*/ebin/rb.beam
slim_applications +=$(prefix)/lib/sasl*/ebin/rb_format_supp.beam
slim_applications +=$(prefix)/lib/sasl*/ebin/erlsrv.beam
slim_applications +=$(prefix)/lib/kernel*/ebin/inet6_sctp.beam
slim_applications +=$(prefix)/lib/kernel*/ebin/array.beam

slim_other_junk +=$(prefix)/erts*/lib/
slim_other_junk +=$(prefix)/Install
slim_other_junk +=$(prefix)/releases/
slim_other_junk +=$(prefix)/bin/start_clean.boot

slim:
        rm -f $(slim_binaries)
        rm -rf $(slim_sources)
        rm -rf $(slim_applications)
        rm -rf $(slim_other_junk)
        $(erlang_version)/bootstrap/bin/erl -noshell -noinput -eval "beam_lib:strip_release(\"$(prefix)\"),init:stop()"
        rm -rf $(prefix)/bin/run_erl
        rm -rf $(prefix)/bin/to_erl
        sh -c "cd $(prefix)/bin; ln -s ../erts*/bin/run_erl ."
        sh -c "cd $(prefix)/bin; ln -s ../erts*/bin/to_erl ."

zip_libraries: $(addsuffix .ez, $(wildcard $(prefix)/lib/*))

%.ez: %
        sh -c "cd `dirname $@`; zip -n .beam -mr $@ `basename $^`"

STRIP=$(CROSS_COMPILE)strip
strip:
        $(STRIP) $(prefix)/erts*/bin/run_erl
        $(STRIP) $(prefix)/erts*/bin/to_erl
        $(STRIP) $(prefix)/erts*/bin/beam
        $(STRIP) $(prefix)/erts*/bin/inet_gethost
        $(STRIP) $(prefix)/erts*/bin/erlexec
        $(STRIP) $(prefix)/erts*/bin/child_setup
        $(STRIP) $(prefix)/bin/run_erl
        $(STRIP) $(prefix)/bin/to_erl

clean:
        rm -rf $(erlang_version)

# removes Erlang from the GTH/STH/GMH target directory
clean_target:
        rm -rf $(prefix)

# even deletes downloaded source code
dist_clean: clean
        rm $(erlang_source_file)
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: tiny erlang

Max Lapshin-2
3 MB is very cool!

We have reduced to 10 MB, but it includes developer and debugging tools.

Smaller package — faster loading.
Also it is very important on slow flash to change configuration of code server so that it doesn't scan all ebin directories in ERL_LIBS for each file, but loads them in a more efficient way.

We do it more dumb (we do not use releases): just scan all .beam files and load them. It makes startup faster.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: tiny erlang

Tony Rogvall-2
I recently put together som instructions on how to cross compile erlang
and user applications.
It includes how to cross compile zlib/openssl/ncurses to get a decent
crypto environment and a working terminal up and running.

You can find it here:


This is not a HOWTO make a tiny erlang but can be used as a template for
how to build the important parts.

/Tony

On 24 feb 2016, at 13:14, Max Lapshin <[hidden email]> wrote:

3 MB is very cool!

We have reduced to 10 MB, but it includes developer and debugging tools.

Smaller package — faster loading.
Also it is very important on slow flash to change configuration of code server so that it doesn't scan all ebin directories in ERL_LIBS for each file, but loads them in a more efficient way.

We do it more dumb (we do not use releases): just scan all .beam files and load them. It makes startup faster.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

signature.asc (817 bytes) Download Attachment