Erlang and Docker

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Erlang and Docker

Stefan Hellkvist-2
Hi,

Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on.

Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).

Stefan


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Rad Gruchalski
I was playing with deploying erlang with docker, just standard trusty64 image. But out of curiosity, what's wrong with releases? Docker sounds plausible for building releases.

Sent from Outlook




On Wed, Jun 17, 2015 at 11:10 PM -0700, "Stefan Hellkvist" <[hidden email]> wrote:

Hi,

Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on. 

Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).

Stefan


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

dmkolesnikov
In reply to this post by Stefan Hellkvist-2
Hello,

We are using Docker + Erlang for multiple purposes:

a) cross platform assembly of Erlang application release (e.g. using Mac to build production Linux images)

b) in production to achieve blue-green deployment, isolate complex external dependencies or production deployment of Erlang application performed on hosts running vanilla Linux distribution. The major assumption -- Erlang/OTP runtime is not installed on target host. The application needs to package and deliver Erlang runtime along with its code.

Erlang VM has very good isolation and cross-platform support by itself. The docker might look as an overhead. The best what is can bring is the optimisation of management practice.

I’ve played with Docker public / private repositories and container image assembly from Docker file on target host. The image assembly is more appealing to me and it is successfully used by us in production.

We are using Amazon, and I’ve tried to use Elastic Beanstalk and EC2 Container service. They are fine if you are building Erlang-based front-end where nodes do not talk each other. The challenge comes if you are building complicated Erlang cluster or tries to deploy existed Erlang software (e.g. Riak). Thus Chef / OpsWorks is more optimal to make life-cycle management and deployment practices.


Best Regards,
Dmitry


> On 18 Jun 2015, at 09:10, Stefan Hellkvist <[hidden email]> wrote:
>
> Hi,
>
> Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on.
>
> Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).
>
> Stefan
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Christopher Vance-8
We build and deploy an Erlang service using two Debian Jessie images.

The first image contains an Erlang compiler and is used to build a release into a host-based volume.

The second image is built using that release. The only Erlang system on the second image is the release. This is the one we deploy.

On Thu, Jun 18, 2015 at 8:19 PM, Dmitry Kolesnikov <[hidden email]> wrote:
Hello,

We are using Docker + Erlang for multiple purposes:

a) cross platform assembly of Erlang application release (e.g. using Mac to build production Linux images)

b) in production to achieve blue-green deployment, isolate complex external dependencies or production deployment of Erlang application performed on hosts running vanilla Linux distribution. The major assumption -- Erlang/OTP runtime is not installed on target host. The application needs to package and deliver Erlang runtime along with its code.

Erlang VM has very good isolation and cross-platform support by itself. The docker might look as an overhead. The best what is can bring is the optimisation of management practice.

I’ve played with Docker public / private repositories and container image assembly from Docker file on target host. The image assembly is more appealing to me and it is successfully used by us in production.

We are using Amazon, and I’ve tried to use Elastic Beanstalk and EC2 Container service. They are fine if you are building Erlang-based front-end where nodes do not talk each other. The challenge comes if you are building complicated Erlang cluster or tries to deploy existed Erlang software (e.g. Riak). Thus Chef / OpsWorks is more optimal to make life-cycle management and deployment practices.


Best Regards,
Dmitry


> On 18 Jun 2015, at 09:10, Stefan Hellkvist <[hidden email]> wrote:
>
> Hi,
>
> Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on.
>
> Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).
>
> Stefan
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



--
Christopher Vance

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Marlus Saraiva
In reply to this post by Stefan Hellkvist-2
Hi Stefan,

In case you're looking for images with small footprints, give Alpine Linux a try. You can have a minimal Erlang image starting from 16.22MB. More info and examples can be found here:

Cheers,
-marlus 

2015-06-18 3:10 GMT-03:00 Stefan Hellkvist <[hidden email]>:
Hi,

Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on.

Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).

Stefan


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Rick Pettit
We are in the process of trying out Erlang-in-Docker (but haven’t actually gotten that working just yet):

  https://github.com/taotaotheripper/Erlang-In-Docker

-Rick

> On Jun 18, 2015, at 6:18 AM, Marlus Saraiva <[hidden email]> wrote:
>
> Hi Stefan,
>
> In case you're looking for images with small footprints, give Alpine Linux a try. You can have a minimal Erlang image starting from 16.22MB. More info and examples can be found here:
> https://github.com/msaraiva/docker-alpine
>
> Cheers,
> -marlus
>
> 2015-06-18 3:10 GMT-03:00 Stefan Hellkvist <[hidden email]>:
> Hi,
>
> Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on.
>
> Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).
>
> Stefan
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Stefan Hellkvist-2
Thank you for the information. I will look into both the Alpine image and the Erlang-In-Docker project. For my current use cases there is currently no need for erlang communication outside the container (I use docker primarily to have a unified deployment format in a multi-language service and for the namespaces separation) but it sounds like an interesting feature for a distributed erlang scenario. Thanks! 

On Thu, Jun 18, 2015 at 4:31 PM, Rick Pettit <[hidden email]> wrote:
We are in the process of trying out Erlang-in-Docker (but haven’t actually gotten that working just yet):

  https://github.com/taotaotheripper/Erlang-In-Docker

-Rick

> On Jun 18, 2015, at 6:18 AM, Marlus Saraiva <[hidden email]> wrote:
>
> Hi Stefan,
>
> In case you're looking for images with small footprints, give Alpine Linux a try. You can have a minimal Erlang image starting from 16.22MB. More info and examples can be found here:
> https://github.com/msaraiva/docker-alpine
>
> Cheers,
> -marlus
>
> 2015-06-18 3:10 GMT-03:00 Stefan Hellkvist <[hidden email]>:
> Hi,
>
> Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on.
>
> Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).
>
> Stefan
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

David Goehrig
At work, we've been deploying a wide range of Erlang apps on Docker.  I'm working on a blog post ( and presentation ) on how we manage our containers across multiple cloud providers.  The biggest gotchas tend to be around:

* epmd port mapping and exposing a range of ports for clustering
* volume mounting and mnesia data stores
* short names and docker's dns implementation

We deploy Erlang application on both Ubunutu and Centos, and deploy in Rackspace, OpenStack, MS Azure, EC2, and VirtualBox.

When the post is ready I'll ping the list, we're redoing our developer blog this week.

Dave

On Mon, Jun 22, 2015 at 1:59 AM, Stefan Hellkvist <[hidden email]> wrote:
Thank you for the information. I will look into both the Alpine image and the Erlang-In-Docker project. For my current use cases there is currently no need for erlang communication outside the container (I use docker primarily to have a unified deployment format in a multi-language service and for the namespaces separation) but it sounds like an interesting feature for a distributed erlang scenario. Thanks! 

On Thu, Jun 18, 2015 at 4:31 PM, Rick Pettit <[hidden email]> wrote:
We are in the process of trying out Erlang-in-Docker (but haven’t actually gotten that working just yet):

  https://github.com/taotaotheripper/Erlang-In-Docker

-Rick

> On Jun 18, 2015, at 6:18 AM, Marlus Saraiva <[hidden email]> wrote:
>
> Hi Stefan,
>
> In case you're looking for images with small footprints, give Alpine Linux a try. You can have a minimal Erlang image starting from 16.22MB. More info and examples can be found here:
> https://github.com/msaraiva/docker-alpine
>
> Cheers,
> -marlus
>
> 2015-06-18 3:10 GMT-03:00 Stefan Hellkvist <[hidden email]>:
> Hi,
>
> Just out of curiosity, is anyone out there using Docker to deploy your Erlang releases? Any experiences captured in any blog out there? I am in particular interested in what docker images people are basing their deployments on.
>
> Personally I have used various Ubuntu images of varying sizes and also the phusion/baseimage (https://github.com/phusion/baseimage-docker) with good results but I would like to hear particularly if anyone has found any images with small footprints out there, capable of running Erlang releases (which is not a well defined requirement I guess, but still).
>
> Stefan
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




--
-=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

kevin montuori
>>>>> "dg" == David Goehrig <[hidden email]> writes:

    dg> At work, we've been deploying a wide range of Erlang apps on
    dg> Docker.  I'm working on a blog post ( and presentation ) on how
    dg> we manage our containers across multiple cloud providers.
   
Hi David --

I realize this is digging up a post from nearly six months ago but I'm
wondering if this blog post materialized?  Or if anyone out there had
new thoughts on distributed Erlang and Docker?

Thanks!
k.


--
  Kevin Montuori
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Felix Gallo-2
Erlang is already like a miniaturized operating system implementing well-scheduled lightweight microservices, and releases (http://www.erlang.org/doc/design_principles/release_structure.html) are already a redistributable, self-contained, minimizable unit of deployment that can be monitored, inspected and logged using standard unix workflow.

In my opinion, the cost-benefit of using docker is highly questionable even in its optimal use case, where the dev team is composed entirely of inexperienced front end javascript developers trying to deploy monolithic microservices and pretending they're being lightweight.  Maybe there's a case to be made for tooling which takes languages with terrible manageability and hides them behind something, but it's not at all clear that docker is the answer to that or any other question.  It makes a negative amount of sense to layer the highly ceremonial, unperformant, problem-multiplying docker abstraction on top of an actually light, already existing microservices architecture, and I wouldn't wish ownership of the resulting mutant chimera on my worst enemy.  But again -- my personal opinion.

F.

On Sat, Nov 21, 2015 at 11:04 AM, Kevin Montuori <[hidden email]> wrote:
>>>>> "dg" == David Goehrig <[hidden email]> writes:

    dg> At work, we've been deploying a wide range of Erlang apps on
    dg> Docker.  I'm working on a blog post ( and presentation ) on how
    dg> we manage our containers across multiple cloud providers.

Hi David --

I realize this is digging up a post from nearly six months ago but I'm
wondering if this blog post materialized?  Or if anyone out there had
new thoughts on distributed Erlang and Docker?

Thanks!
k.


--
  Kevin Montuori
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Rad Gruchalski
Nevertheless, it would be nice to read the article. Having been thinking about it, Erlang is the interesting unicorn case. Still, I’d like to read what the reasoning was.

Kind regards,

Radek Gruchalski
[hidden email][hidden email]
de.linkedin.com/in/radgruchalski/

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.

On Saturday, 21 November 2015 at 21:52, Felix Gallo wrote:

Erlang is already like a miniaturized operating system implementing well-scheduled lightweight microservices, and releases (http://www.erlang.org/doc/design_principles/release_structure.html) are already a redistributable, self-contained, minimizable unit of deployment that can be monitored, inspected and logged using standard unix workflow.

In my opinion, the cost-benefit of using docker is highly questionable even in its optimal use case, where the dev team is composed entirely of inexperienced front end javascript developers trying to deploy monolithic microservices and pretending they're being lightweight.  Maybe there's a case to be made for tooling which takes languages with terrible manageability and hides them behind something, but it's not at all clear that docker is the answer to that or any other question.  It makes a negative amount of sense to layer the highly ceremonial, unperformant, problem-multiplying docker abstraction on top of an actually light, already existing microservices architecture, and I wouldn't wish ownership of the resulting mutant chimera on my worst enemy.  But again -- my personal opinion.

F.

On Sat, Nov 21, 2015 at 11:04 AM, Kevin Montuori <[hidden email]> wrote:
>>>>> "dg" == David Goehrig <[hidden email]> writes:

    dg> At work, we've been deploying a wide range of Erlang apps on
    dg> Docker.  I'm working on a blog post ( and presentation ) on how
    dg> we manage our containers across multiple cloud providers.

Hi David --

I realize this is digging up a post from nearly six months ago but I'm
wondering if this blog post materialized?  Or if anyone out there had
new thoughts on distributed Erlang and Docker?

Thanks!
k.


--
  Kevin Montuori
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Stefan Hellkvist-2
In reply to this post by Felix Gallo-2


21 nov. 2015 kl. 21:52 skrev Felix Gallo <[hidden email]>:

Erlang is already like a miniaturized operating system implementing well-scheduled lightweight microservices, and releases (http://www.erlang.org/doc/design_principles/release_structure.html) are already a redistributable, self-contained, minimizable unit of deployment that can be monitored, inspected and logged using standard unix workflow.

In my opinion, the cost-benefit of using docker is highly questionable even in its optimal use case, where the dev team is composed entirely of inexperienced front end javascript developers trying to deploy monolithic microservices and pretending they're being lightweight.  Maybe there's a case to be made for tooling which takes languages with terrible manageability and hides them behind something, but it's not at all clear that docker is the answer to that or any other question.  It makes a negative amount of sense to layer the highly ceremonial, unperformant, problem-multiplying docker abstraction on top of an actually light, already existing microservices architecture, and I wouldn't wish ownership of the resulting mutant chimera on my worst enemy.  But again -- my personal opinion.

F.

Perhaps this is true for homogeneous systems such as systems only built with Erlang components. Many systems are however hybrid systems with different component types and a container architecture (such as docker) could help in making things more homogeneous.

Stefan





On Sat, Nov 21, 2015 at 11:04 AM, Kevin Montuori <[hidden email]> wrote:
>>>>> "dg" == David Goehrig <[hidden email]> writes:

    dg> At work, we've been deploying a wide range of Erlang apps on
    dg> Docker.  I'm working on a blog post ( and presentation ) on how
    dg> we manage our containers across multiple cloud providers.

Hi David --

I realize this is digging up a post from nearly six months ago but I'm
wondering if this blog post materialized?  Or if anyone out there had
new thoughts on distributed Erlang and Docker?

Thanks!
k.


--
  Kevin Montuori
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

zxq9-2
On 2015年11月22日 日曜日 11:19:45 Stefan Hellkvist wrote:
> > 21 nov. 2015 kl. 21:52 skrev Felix Gallo <[hidden email]>:
> > Erlang is already like a miniaturized operating system implementing well-scheduled lightweight microservices, and releases (http://www.erlang.org/doc/design_principles/release_structure.html) are already a redistributable, self-contained, minimizable unit of deployment that can be monitored, inspected and logged using standard unix workflow.
> >
> > In my opinion, the cost-benefit of using docker is highly questionable even in its optimal use case, where the dev team is composed entirely of inexperienced front end javascript developers trying to deploy monolithic microservices and pretending they're being lightweight.  Maybe there's a case to be made for tooling which takes languages with terrible manageability and hides them behind something, but it's not at all clear that docker is the answer to that or any other question.  It makes a negative amount of sense to layer the highly ceremonial, unperformant, problem-multiplying docker abstraction on top of an actually light, already existing microservices architecture, and I wouldn't wish ownership of the resulting mutant chimera on my worst enemy.  But again -- my personal opinion.
> >
> > F.
>
> Perhaps this is true for homogeneous systems such as systems only built with Erlang components. Many systems are however hybrid systems with different component types and a container architecture (such as docker) could help in making things more homogeneous.

[BEGIN RANT: Docker is not a best practice]

Then those systems have not either been appropriately bundled (lots of releases drop other language bundles in priv/) or abstracted across the network as separate services (this is SO FREAKING NOT HARD TO DO AND MAKES LIFE EASIER FOREVER WHY ISN'T THIS THE DEFAULT MODE OF THOUGHT WTF AHHHHHHH!). The reason I've grown skeptical of both docker and "fire and forget" virtualization is that I generally see containerization as a way for devops to barely cobble an environment together in a rush before some deadline, slap a label like "WorkingBaseSystem X" on it -- and consider their job with that particular platform as complete, forever.

Which means...

- It is never maintained (until something like heartbleed... dhoh!).
- How it was made to work in the first place is lightly/never documented.
- The interactions among components are not documented.
- Interfaces are not defined.
- Sloppy hacks are very hard to discover.
- Updating the system is a *terrifying* prospect.
- Reviewing the old code to document structure/flow becomes archaeology.

They become security landmines until the end of time, and most management teams seem to think "its not broke; don't fix it", but are confused as to why their devs becomes paralyzed the moment some new feature or bugfix is required in the old system -- so instead begin replicating functionality buried in the old system.

In the case where the containerized system represents the core business, sure, that is maintained and updated -- and in those cases containerization isn't really much of an asset because its not competing with other configurations for primacy over settings. In the case where the containerized system is not the core business containerization encourages the horrible trends I wrote about above. In the bad case this means that there will probably never *be* another major version release of whatever system lives in the "hybrid system with different component types and a container architecture" because nobody will ever be able to figure either what that system does inside, or never figure out how to resolve the christmas-lights-tangle of settings and version choices that has been containerized.

If the system is well maintained, these things are obvious and docker isn't buying you much; if the system is not well maintained docker is just covering up the accumulating stench by dousing it in fabreeze and wrapping it in a blanket really tight.

With regard specifically to Erlang, a release basically *is* a container, though placing that again within a VM/docker/container/whatever doesn't really hurt anything -- its just not as big a win, and it certainly does make things more complex for no reason (now you have to make two releases, one nested within the other).

"But nobody has time to [do X]"
  when X :: document the mysterious system
          : define interfaces
          : figure out what versions of what work with what else
          : update component subsystems
          : whatever other time-consuming task
Then you
 - Make time
 - Appreciate that you are going to swim in mud until you make enough
   money to break into open water
 - Start looking for another job before your current company implodes

Theoretically maybe docker has some really super great benefits for Erlang... but in practice I've generally just seen it (and not just docker, again, all forms of containerization) become an excuse for (and later evolve into a justification of) profoundly sloppy operational practices. For some reason I see this a bit less with VM-based solutions than docker (but seen it there, too).

"Gee, Craig, why all the vinegar?" you may ask.

Because the difference between a heterogeneous/homogeneous environment should not be a major design issue, which implies that methods to deal with different components as totally separate, totally abstracted entities should be a fundamental part of any large system design (this method is usually called "sockets", and they don't all have to be HTTP... >gasp!<). You shouldn't be running gleefully along and just suddenly realize you've built not just a monolithic code base, but a Jenga game of the supporting system that it runs on as well. This is a solution to a problem that shouldn't exist, and the way to do better things is not to just sweep it under the rug and consider this "a best practice", but rather regard the need for this form of deployment as a sign that you've screwed up and painted yourself into a corner. If you find yourself having to use docker, especially for an Erlang-based system, you should think "thank goodness we can bundle this awful mess up for now until we get our fundamental problems sorted out".
[END RANT: DINABP]

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Tristan Sloughter-4
I have to say that I partially agree with those here pointing out the
use of a target system providing you with many similar benefits to the
use of a docker image. And I'm no docker fan, but the argument that a
release is basically a container is simply false and scrubs away all the
useful purposes of a container, jail, zone, etc.

A release is not a replacement for the cases you need a cgroup, jail or
zone. But if the only reason you think you need such a thing is to start
an Erlang node you may be engaging in overkill that will be technical
debt.

--
  Tristan Sloughter
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

zxq9-2
On 2015年11月22日 日曜日 07:45:42 Tristan Sloughter wrote:

> I have to say that I partially agree with those here pointing out the
> use of a target system providing you with many similar benefits to the
> use of a docker image. And I'm no docker fan, but the argument that a
> release is basically a container is simply false and scrubs away all the
> useful purposes of a container, jail, zone, etc.
>
> A release is not a replacement for the cases you need a cgroup, jail or
> zone. But if the only reason you think you need such a thing is to start
> an Erlang node you may be engaging in overkill that will be technical
> debt.

Absolutely.

To expand the discussion a bit further though...

I've seen more folks blindly employ segregation facilities like cgoups
and especially jails because they thought they were primary security
features than realizing they are environmental segregations. "Scrubs
away all the useful purposes of a container, jail, zone, etc." I don't
find these particularly useful compared to simple virtualization -- and
virtualization itself is also not a primary security feature. (The term
"jail" was poorly chosen, in retrospect.)

With this in mind, I tend to view things rather more black and white: VMs
or local execution -- pretty much anything else is indeed technical debt.
But its the cool-feeling kind of technical debt that comes complete with
a bodyguard of totally kickass websites, buzzwords and ready-made
consultant army. (otoh, if you have to pay a fee per OS image or committed
core, then VMs can easily move you from "technical" to "finanical" debt)

I'm not saying these things don't work -- they do. But they have never
actually made deployment or maintenance any easier over the long term --
instead merely move the complexity involved in those tasks elsewhere for
a while. I wind up having to learn and *maintain* several new skills on
top of >gasp< already having to deploy in VMs anyway. Splitting system
components up as network services, though -- that actually *has* been a
big win, and I don't really care how those are deployed, what language
they are written in, or how they are hosted.

Containerization, of any form, encourages coupling at the grand
architecture level because it hides the long-term pain of coupling
tradeoffs very well. That is really what I'm most concerned about.

A well written service should be deployable to any target platform in a
simple way. We just skip this step when we aren't writing client-side
software -- but its not so hard to get right if we start out with this
in mind instead of just barely getting a development environment working
and then letting devops make that the blueprint for production. Here I
am certainly not talking just about Erlang-only services -- it is a broad
problem.

I'm not particularly fond of Erlang releases for *everything* -- I think
it is very often overkill and understanding what a release even is seems
to repel a lot of folks who like the language and runtime but don't grok
that aspect of the environment (after all, most Erlang code written today
is not targetting embedded phone switches with constrained resources and
limited underlying system facilities), but it *does* simplify deployment
dramatically (once you come to terms with the technical debt involved in
getting Erlang releases to work right...), and ease-of-deployment seems
to be the focus of most chatter about containerization.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

kevin montuori
>>>>> "z" == zxq9  <[hidden email]> writes:

    z> I'm not saying these things don't work -- they do. But they have
    z> never actually made deployment or maintenance any easier over the
    z> long term -- instead merely move the complexity involved in those
    z> tasks elsewhere for a while.

Bringing up Docker certainly roused up some passions.  I'm not sold on
containers as the solution to the world's ills but I'm also not willing
to eschew their use just because they're not always ideal.  Right tool
for the job and all that.

Anyhow, I apologize for not being specific with the question I asked.

I have a use case where containers are indicated and need a very small
distributed cache.  Mnesia's my first choice but only if I don't have to
figure out the Erlang-in-Docker clustering myself; I was hoping someone
might have insight into (or a recipe for!) making that work.  If not
there are other easily implementable solutions that I'll investigate.

Thanks all ... the discussion has made interesting reading!

k.

--
  Kevin Montuori
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

zxq9-2
On 2015年11月22日 日曜日 11:27:47 Kevin Montuori wrote:

> >>>>> "z" == zxq9  <[hidden email]> writes:
>
>     z> I'm not saying these things don't work -- they do. But they have
>     z> never actually made deployment or maintenance any easier over the
>     z> long term -- instead merely move the complexity involved in those
>     z> tasks elsewhere for a while.
>
> Bringing up Docker certainly roused up some passions.  I'm not sold on
> containers as the solution to the world's ills but I'm also not willing
> to eschew their use just because they're not always ideal.  Right tool
> for the job and all that.

Well put.

> Anyhow, I apologize for not being specific with the question I asked.

I apologize for becoming passionate -- it drove me out on a tangent.
Cathartic, but still a tangent.

> I have a use case where containers are indicated and need a very small
> distributed cache.  Mnesia's my first choice but only if I don't have to
> figure out the Erlang-in-Docker clustering myself; I was hoping someone
> might have insight into (or a recipe for!) making that work.  If not
> there are other easily implementable solutions that I'll investigate.

This is a curious phrase. You want "containers to have a cache"... ?

Mnesia is *ideal* for creating caches. I actually don't really like DETS
so much because it distracts from what Mnesia is really good at, and that
is core cache with genuine db features.

But... containers and in-memory caches with interesting features are
usually viewed as orthogonal features. In the case of Erlang, they are
really per-node concepts (unless you count DETS into that, then the
discussion changes a bit, but is still manageable).

Can you elaborate? I almost guarantee that someone will have a lot of
interesting things to say about whatever it is you are dealing with. My
suspicion is that the point at which the word "docker" entered the
discussion is the point at which it became an X-Y problem.

(And I went off on my tangent -- (>.<) dhoh. I appreciate your grace
on this point.)

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

kevin montuori
>>>>> "z" == zxq9  <[hidden email]> writes:

    >> I have a use case where containers are indicated and need a very small
    >> distributed cache.  Mnesia's my first choice but only if I don't have to
    >> figure out the Erlang-in-Docker clustering myself; I was hoping someone
    >> might have insight into (or a recipe for!) making that work.  If not
    >> there are other easily implementable solutions that I'll investigate.

    z> This is a curious phrase. You want "containers to have a cache"... ?

    z> Mnesia is *ideal* for creating caches. I actually don't really
    z> like DETS so much because it distracts from what Mnesia is really
    z> good at, and that is core cache with genuine db features.


I should probably have phrased that as:

  I have an Erlang application where containers are indicated; that
  application requires a small cache.

In a strictly OTP application deployment I would without doubt use
Mnesia.  But because the application has the Docker requirement things
are a little more difficult.

I agree 100% with Mnesia as distributed cache (and, honestly, for
durable storage as well): it's treated me well over the years.

k.


--
  Kevin Montuori
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

books

try the official image https://hub.docker.com/_/erlang/ for a good start for beginners, and there are other efforts like https://hub.docker.com/r/msaraiva/erlang/ making really minimal image, I think msaraiva is already running that in production

➸ docker run -it --rm erlang
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V7.1 (abort with ^G)
1> uptime().
3 seconds
ok

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang and Docker

Nathaniel Waisbrot
In reply to this post by Stefan Hellkvist-2
I'm using Docker with Erlang and have found it extremely useful.

I think that there's some hostility towards Docker because much of the buzz around Docker seems to treat it as the perfect technology for solving all your problems. This is untrue, of course, but I think "stupid people like it" is a bad reason to reject something. So now I'll out myself as a stupid person...

Here's an example of how I'm using Docker:
https://gist.github.com/waisbrot/3415d53f7c0b66a36013

I use Kerl to install Erlang into a base image and then use that base image to build my applications. The applications are simply releases made with Rebar. There's a little Docker work to keep it from needing to download all dependencies every time, which is optional but nice. Inside the container, everything is normal Erlang world and there's no surprises.

You could use the official image, but we wanted to customize our images and having a shared base image is valuable for saving downloading time and disk space.

So that's the "how". As for the "why":
- We started out in Node.js and Python. Containerization was helpful here, mostly to segregate Python2 and Python3 headaches (a bunch of tools that incompatibly expected "python" to be the right Python for them).
- We were doing deployments via `git`, which was not great. Using Docker as a packaging system was nice because it's simpler to learn than RPMs and is uniform across all languages (don't have to learn wheels/eggs/gems/etc).
- We didn't have any Ops staff to handle deployment and manage the machines
- After the container infrastructure is set up, it's easy to introduce new languages and we were able to ease Erlang into the mix
- Our development environment runs the same containers as our production environment, just using container-linking and running all on one machine
- Or CI environment does the same thing. We regularly run our functional test suite against multiple copies of the production system simultaneously on a single machine (one run per pull-request), and it doesn't take any more effort than a single production deploy (which is also easy)

My main complaint with Docker so far is that if I want my containers to launch with the system I need to set their restart policy to "always", but this causes it to restart crashed containers which interferes with Erlang's own restart system.

Some day in the far future, we may reach a point where Erlang is the only language we're running (or at least the only one on a particular server) and then it might be attractive to un-containerize it. On the other hand, maybe not. Adding another layer of virtualization in the form of Docker has cost us very little so far.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
12