trying to make official docker images for erlang

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

trying to make official docker images for erlang

books
Hi to Erlang users,

this is effort trying to make official docker images for erlang otp
community, please comment if you like to run it with docker:

https://github.com/docker-library/official-images/pull/1075

Nowadays, docker is the popular way to run many applications, have a
look on docker hub, many popular programming languages and
applications have an official image there, beginners can easily pull a
docker image to start playing without hassling their host Linux,

https://hub.docker.com/_/python/
https://hub.docker.com/_/golang/
https://hub.docker.com/_/ruby/
...

But there is no official one for erlang yet, let's try to make it happen,
https://hub.docker.com/_/erlang/

I have researched some existing effort like these, looks like many
ones are already on this way just haven't communicated thru
@erlang.org yet

- https://hub.docker.com/r/correl/erlang/
   this might be the earliest currently have the most stars by `docker
search erlang`, provided erlang-otp-17.5 and rebar and relx in a
single image, and compiling each one from source code, I haven't tried
it, but presumably would be close to 1GB image;
- https://hub.docker.com/r/unbalancedparentheses/erlang
   this one support all versions of erlang R6B-0, R7, R8, ... R16, up to 17.4
   from https://github.com/unbalancedparentheses/docker-erlang/blob/master/17.4/install.sh
it also provided erlang & rebar & relx all compiled from source code
- https://hub.docker.com/r/msaraiva/erlang/
   this one is providing erlang-17.5 and 18, on top of Alpine Linux
docker image, it's very slim, as small as 16.78 MB, while erlang lib
is broken into very small packages, like most OS distro does, broken
into erlang-compiler, erlang-dialyzer, erlang-otp, erlang-snmp, and
etc. its base erlang:18 image has 5 packages under
/usr/lib/erlang/lib/... (compared a full erlang-otp has 52 lib
packages)
  http://dl-4.alpinelinux.org/alpine/edge/main/x86_64/ (search erlang)
   this one also provided elixir images in different Dockerfile,
presumably also slim
- https://hub.docker.com/r/voidlock/erlang
    is very similar to my way, support erlang R16 thru 18.1; I would
not start my project if I know this earlier, but it installed
update-locale for UTF-8 I'm not sure for what, is that required by
erlang runtime ?
- https://github.com/synctree/docker-erlang/blob/master/R17/Dockerfile
  this one installs erlang solutions precompiled deb on top of
debian:8 (or debian:jessie) image.

$ docker search erlang
NAME                           DESCRIPTION
        STARS     OFFICIAL   AUTOMATED
correl/erlang                  Erlang/OTP for Docker
        14                   [OK]
...

While I just started from scratch, before above PR to official images
got merged, you can try it with cloning this repo, and build images
locally,

https://github.com/c0b/docker-erlang-otp

So there are two ways to make images:
1) build from source code, from standard debian:8 image, start with
apt-get install build-essential and gcc and some lib...-dev and
download erlang-otp source code and build, this usually ends up with a
fat image, close to 1GB;
2) install from some binary erlang packages, like the one from
erlang-solutions, could end up with smaller image; packages from most
OS distributions also provided erlang but relatively not up to date
like debian https://packages.debian.org/source/sid/erlang fedora
centos similar

Here from my repo I mainted one each for latest 4 erlang releases,
(R15, R16, 17, 18), each with different variant, following the best
practices from docker official images guideline, should end up with
full feature while relatively slim images:
https://github.com/docker-library/official-images

1. the standard variant erlang:18, erlang:17, erlang:R16, erlang:R15
builds from source code, on top of
https://hub.docker.com/_/buildpack-deps/ :jessie, it covered gcc
compiler and some popular -dev packages, for port driver written in C;
while it doesn't have java compiler so out of the standard erlang-otp
provided 52 packages under /usr/lib/erlang/lib/... from this one odbc
/ jinterface / wxwidgets won't work, I assume to run GUI programs in
docker is not popular, so here we can save space; jinterface is
similar, the java dependencies are too fat, I assume demand is low;
2. the -onbuild variant for each erlang version, to utilize ONBUILD
instruction from Dockerfile, those are for starters
3. -esl variant is to pull erlang-solutions deb package to install on
top of debian:jessie, results in relatively slim image, but I am
trying to avoid wxwidgets / jinterface dependencies, reasons same as
above.

All these images are almost full featured Erlang-OTP images (except
wxwidgets & jinterface), you can run it like this once build locally
(or pull over docker hub if above PR can be merged),

$ docker run -it --rm erlang:18.1
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:8:8]
[async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.1  (abort with ^G)
1> uptime().                                          # the new
uptime() shell command since OTP 18
3 seconds
ok
2> application:which_applications().
[{stdlib,"ERTS  CXC 138 10","2.6"},
 {kernel,"ERTS  CXC 138 10","4.1"}]
3>
User switch command
 --> q
root@6d2c1ad52ae8:/# ls /usr/local/lib/erlang/lib/
asn1-4.0     cosProperty-1.2  edoc-0.7.17     gs-1.6    observer-2.1
 public_key-1.0.1 stdlib-2.6   xmerl-1.3.8 common_test-1.11
cosTime-1.2  eldap-1.2     hipe-3.13    orber-3.8    reltool-0.7
syntax_tools-1.7 compiler-6.0.1     cosTransactions-1.3
erl_docgen-0.4     ic-4.4    os_mon-2.4    runtime_tools-1.9.1
test_server-3.9 cosEvent-2.2     crypto-3.6.1  erl_interface-3.8
inets-6.0.1    ose-1.1    sasl-2.6 tools-2.8.1 cosEventDomain-1.2
debugger-4.1.1  erts-7.1     kernel-4.1     otp_mibs-1.1    snmp-5.2
typer-0.9.9 cosFileTransfer-1.2  dialyzer-2.8.1  et-1.5.1
megaco-3.18    parsetools-2.1  ssh-4.1 webtool-0.9 cosNotification-1.2
 diameter-1.11  eunit-2.2.11     mnesia-4.13.1  percept-0.8.11
ssl-7.1 wx-1.5
root@4cfd3172e8cc:/# ls /usr/local/lib/erlang/lib/ | wc -l
50

Size:

$ docker images |grep ^erlang
erlang     18.1-esl            138c797adec7        5 days ago          286.9 MB
erlang     18.1                27ad0fc44644        5 days ago          741.5 MB
erlang     R16B03-1            e0deec5e1e72        6 days ago          740.2 MB
erlang     18.0.3              52d4a7a4a281        6 days ago          743.7 MB

Comments are welcome.


Thanks,
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: trying to make official docker images for erlang

books
Please click Reply to all to make comments available thru @erlang.org ML,
http://erlang.org/pipermail/erlang-questions/2015-September/thread.html#86239

And this PR is requesting comments from language upstream, so please
also comment there if you could; thanks
https://github.com/docker-library/official-images/pull/1075

On Wed, Sep 30, 2015 at 4:28 AM, Jesper Louis Andersen
<[hidden email]> wrote:

>
> On Wed, Sep 30, 2015 at 5:42 AM, derek <[hidden email]> wrote:
>>
>> this is effort trying to make official docker images for erlang otp
>> community, please comment if you like to run it with docker:
>
>
> Nice idea, and really needed. I'm not too familiar with docker, so I can't
> help you out in the details of things, but it seems like a worthwhile thing
> to have!
>
>
> --
> J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: trying to make official docker images for erlang

Marlus Saraiva
In reply to this post by books
Hi derek,

I'm the maintainer of both, the erlang packages at http://dl-4.alpinelinux.org/alpine/edge/main/x86_64/
and the docker images you've mentioned at https://hub.docker.com/u/msaraiva/

Having minimal containers is something that I take seriously and I think it makes the whole docker experience much more pleasant. The only reason I wouldn't recommend using Alpine Linux for the official image is that there's no "official" support from the OTP team. As far as I know, they don't build/test Erlang/OTP on any Linux distribution based on musl libc and I have no idea if they have any interest in doing that. So, since the official image is intended to reach a broader audience, I guess you'd better stick with the larger images.

In case you need more info about Erlang for Alpine Linux, take a look at:
It also contains information about building Erlang/OTP against musl libc. So, if you want to give it a try,
you can easily create an image containing a full Erlang/OTP of any version and with as many erlang libs as you want.


Cheers,

--
Marlus Saraiva


2015-09-30 0:42 GMT-03:00 derek <[hidden email]>:
Hi to Erlang users,

this is effort trying to make official docker images for erlang otp
community, please comment if you like to run it with docker:

https://github.com/docker-library/official-images/pull/1075

Nowadays, docker is the popular way to run many applications, have a
look on docker hub, many popular programming languages and
applications have an official image there, beginners can easily pull a
docker image to start playing without hassling their host Linux,

https://hub.docker.com/_/python/
https://hub.docker.com/_/golang/
https://hub.docker.com/_/ruby/
...

But there is no official one for erlang yet, let's try to make it happen,
https://hub.docker.com/_/erlang/

I have researched some existing effort like these, looks like many
ones are already on this way just haven't communicated thru
@erlang.org yet

- https://hub.docker.com/r/correl/erlang/
   this might be the earliest currently have the most stars by `docker
search erlang`, provided erlang-otp-17.5 and rebar and relx in a
single image, and compiling each one from source code, I haven't tried
it, but presumably would be close to 1GB image;
- https://hub.docker.com/r/unbalancedparentheses/erlang
   this one support all versions of erlang R6B-0, R7, R8, ... R16, up to 17.4
   from https://github.com/unbalancedparentheses/docker-erlang/blob/master/17.4/install.sh
it also provided erlang & rebar & relx all compiled from source code
- https://hub.docker.com/r/msaraiva/erlang/
   this one is providing erlang-17.5 and 18, on top of Alpine Linux
docker image, it's very slim, as small as 16.78 MB, while erlang lib
is broken into very small packages, like most OS distro does, broken
into erlang-compiler, erlang-dialyzer, erlang-otp, erlang-snmp, and
etc. its base erlang:18 image has 5 packages under
/usr/lib/erlang/lib/... (compared a full erlang-otp has 52 lib
packages)
  http://dl-4.alpinelinux.org/alpine/edge/main/x86_64/ (search erlang)
   this one also provided elixir images in different Dockerfile,
presumably also slim
- https://hub.docker.com/r/voidlock/erlang
    is very similar to my way, support erlang R16 thru 18.1; I would
not start my project if I know this earlier, but it installed
update-locale for UTF-8 I'm not sure for what, is that required by
erlang runtime ?
- https://github.com/synctree/docker-erlang/blob/master/R17/Dockerfile
  this one installs erlang solutions precompiled deb on top of
debian:8 (or debian:jessie) image.

$ docker search erlang
NAME                           DESCRIPTION
        STARS     OFFICIAL   AUTOMATED
correl/erlang                  Erlang/OTP for Docker
        14                   [OK]
...

While I just started from scratch, before above PR to official images
got merged, you can try it with cloning this repo, and build images
locally,

https://github.com/c0b/docker-erlang-otp

So there are two ways to make images:
1) build from source code, from standard debian:8 image, start with
apt-get install build-essential and gcc and some lib...-dev and
download erlang-otp source code and build, this usually ends up with a
fat image, close to 1GB;
2) install from some binary erlang packages, like the one from
erlang-solutions, could end up with smaller image; packages from most
OS distributions also provided erlang but relatively not up to date
like debian https://packages.debian.org/source/sid/erlang fedora
centos similar

Here from my repo I mainted one each for latest 4 erlang releases,
(R15, R16, 17, 18), each with different variant, following the best
practices from docker official images guideline, should end up with
full feature while relatively slim images:
https://github.com/docker-library/official-images

1. the standard variant erlang:18, erlang:17, erlang:R16, erlang:R15
builds from source code, on top of
https://hub.docker.com/_/buildpack-deps/ :jessie, it covered gcc
compiler and some popular -dev packages, for port driver written in C;
while it doesn't have java compiler so out of the standard erlang-otp
provided 52 packages under /usr/lib/erlang/lib/... from this one odbc
/ jinterface / wxwidgets won't work, I assume to run GUI programs in
docker is not popular, so here we can save space; jinterface is
similar, the java dependencies are too fat, I assume demand is low;
2. the -onbuild variant for each erlang version, to utilize ONBUILD
instruction from Dockerfile, those are for starters
3. -esl variant is to pull erlang-solutions deb package to install on
top of debian:jessie, results in relatively slim image, but I am
trying to avoid wxwidgets / jinterface dependencies, reasons same as
above.

All these images are almost full featured Erlang-OTP images (except
wxwidgets & jinterface), you can run it like this once build locally
(or pull over docker hub if above PR can be merged),

$ docker run -it --rm erlang:18.1
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:8:8]
[async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.1  (abort with ^G)
1> uptime().                                          # the new
uptime() shell command since OTP 18
3 seconds
ok
2> application:which_applications().
[{stdlib,"ERTS  CXC 138 10","2.6"},
 {kernel,"ERTS  CXC 138 10","4.1"}]
3>
User switch command
 --> q
root@6d2c1ad52ae8:/# ls /usr/local/lib/erlang/lib/
asn1-4.0     cosProperty-1.2  edoc-0.7.17     gs-1.6    observer-2.1
 public_key-1.0.1 stdlib-2.6   xmerl-1.3.8 common_test-1.11
cosTime-1.2  eldap-1.2     hipe-3.13    orber-3.8    reltool-0.7
syntax_tools-1.7 compiler-6.0.1     cosTransactions-1.3
erl_docgen-0.4     ic-4.4    os_mon-2.4    runtime_tools-1.9.1
test_server-3.9 cosEvent-2.2     crypto-3.6.1  erl_interface-3.8
inets-6.0.1    ose-1.1    sasl-2.6 tools-2.8.1 cosEventDomain-1.2
debugger-4.1.1  erts-7.1     kernel-4.1     otp_mibs-1.1    snmp-5.2
typer-0.9.9 cosFileTransfer-1.2  dialyzer-2.8.1  et-1.5.1
megaco-3.18    parsetools-2.1  ssh-4.1 webtool-0.9 cosNotification-1.2
 diameter-1.11  eunit-2.2.11     mnesia-4.13.1  percept-0.8.11
ssl-7.1 wx-1.5
root@4cfd3172e8cc:/# ls /usr/local/lib/erlang/lib/ | wc -l
50

Size:

$ docker images |grep ^erlang
erlang     18.1-esl            138c797adec7        5 days ago          286.9 MB
erlang     18.1                27ad0fc44644        5 days ago          741.5 MB
erlang     R16B03-1            e0deec5e1e72        6 days ago          740.2 MB
erlang     18.0.3              52d4a7a4a281        6 days ago          743.7 MB

Comments are welcome.


Thanks,
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: trying to make official docker images for erlang

books
On Thu, Oct 1, 2015 at 6:45 AM, Marlus Saraiva <[hidden email]> wrote:
> Hi derek,
>
> I'm the maintainer of both, the erlang packages at
> http://dl-4.alpinelinux.org/alpine/edge/main/x86_64/

yesterday I've seen that there are 53 erlang-*.apk packages, similar
like debian maintaining packages into smaller pieces, but more up to
date;
while how are those pieces got compiled, does Alpine also have similar
structure like debian packaging? have files like debian/rules in deb
packaging? for a deb pkg, just look at the debian
https://packages.debian.org/source/sid/erlang   (find
erlang_18.0-dfsg-2.debian.tar.xz has all instructions how to build and
split into smaller pieces)

https://pkgs.alpinelinux.org/package/main/x86_64/erlang

erlang-18.1-r0.apk2015-Oct-01 14:43:111.9Mapplication/octet-stream
erlang-asn1-18.1-r0.apk2015-Oct-01 14:43:12773.4Kapplication/octet-stream
erlang-common-test-18.1-r0.apk2015-Oct-01 14:43:12816.5Kapplication/octet-stream
erlang-compiler-18.1-r0.apk2015-Oct-01 14:43:121.1Mapplication/octet-stream
[...]

and it looks like your alpine base image is smaller than the official
one, do you think your alpine image is better than the official alpine
image? if so, could you try to push it to become new official ?
https://hub.docker.com/r/msaraiva/alpine/
https://hub.docker.com/_/alpine/

> and the docker images you've mentioned at https://hub.docker.com/u/msaraiva/

>
> Having minimal containers is something that I take seriously and I think it
> makes the whole docker experience much more pleasant. The only reason I
> wouldn't recommend using Alpine Linux for the official image is that there's
> no "official" support from the OTP team. As far as I know, they don't
> build/test Erlang/OTP on any Linux distribution based on musl libc and I
> have no idea if they have any interest in doing that. So, since the official
> image is intended to reach a broader audience, I guess you'd better stick
> with the larger images.

I guess so, to make an official image is mainly targeting for new
audiences, for newbies to easily start programming in erlang, and
provides a standard Linux with glibc, and bash (and all tools like
sed, awk for bash shell scripting), might be easier to troubleshoot
than the alpine busybox musl if problems rise up;

Comparing to python official image, it's also building from source
code, with a similar standard ~700MB size should be ok,
https://hub.docker.com/_/python/

>
> In case you need more info about Erlang for Alpine Linux, take a look at:
> https://github.com/msaraiva/alpine-erlang
> It also contains information about building Erlang/OTP against musl libc.
> So, if you want to give it a try,
> you can easily create an image containing a full Erlang/OTP of any version
> and with as many erlang libs as you want.

I see there are 3 patches required to be built with musl, and it seems
targeting to be a replacement of glibc, I am not familiar about that
and not sure what mature level is it;    so for general audience,
might be easier to start an official image FROM debian:jessie and
default Linux glibc bash shell utils
http://www.musl-libc.org/faq.html

As advocated in official-images page:
https://github.com/docker-library/official-images#library-definition-files

there is request for language upstream's opinion, so please also
comment there if you could, would be much appreciated
https://github.com/docker-library/official-images/pull/1075#issuecomment-143883926

I don't see what open source license do you put your code under? would
you mind I copy some code (from your msaraiva/alpine-erlang) and
provide another alpine-slim varaint?
https://github.com/c0b/docker-erlang-otp/tree/master/18


Thanks a lot!

>
>
> Cheers,
>
> --
> Marlus Saraiva
> https://github.com/msaraiva
> https://twitter.com/MarlusSaraiva
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions