Deployment rebar3

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Deployment rebar3

Code Wiget
Hey everyone,

I’ve recently built a few applications that handle high volume message passing and all communicate with each other using Erlang’s ssl distribution. I’m working on deploying these to multiple servers - right now I am just using rebar3 release and creating a zipped tar of the release directory, moving it to a new box, and changing the some vm.args and sys.config options on the new machines. This doesn’t seem totally intuitive though… it take far longer than I feel like it should. 

I’ve read about Docker and Kubernetes, and those seem promising, but our IT guys aren’t ready to move to containers and are using static nodes right now. 

So, what is the best way to quickly distribute your erlang binaries and configure the enviornment? Have a quick script to do it automatically? Is the answer just containerize with Docker and wait a few months for the IT team to catch up with it? Or is there some other option out there?

Thanks for your help!

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Deployment rebar3

Stefan Hellkvist-2
What's "best" is very much depending on who you ask, but, for starters you may consider wrapping your configuration with something that allows you to override config variables with environment variables (see example below). This would make it easier to run things without changing files and it would also prepare you for whenever you enter docker-/kube-land later also...should you decide to take that step. 

/Stefan




...an example 

-module(config).

-export([get/1]).

%% reads a configuration value Key. Values are placed in sys.config as atoms
%% but can be overridden by setting environment variables in the os. The environment
%% variable name is given by the atom name converted to a string/list and then uppercased
%% example: my_atom_key would translate to the environment variable MY_ATOM_KEY
get(Key) when is_atom(Key) ->
    {ok, Default} = application:get_env(axon, Key),
    EnvName = string:to_upper(atom_to_list(Key)),
    case os:getenv(EnvName) of
        false ->
            Default; %% assume it has the right type in config

        OSVal ->
            convert(OSVal, Default)
    end.

convert(Val, Default) when is_list(Val), is_integer(Default) ->
    list_to_integer(Val);
convert(Val, Default) when is_list(Val), is_binary(Default) ->
    list_to_binary(Val);
convert(Val, Default) when is_list(Val), is_boolean(Default) ->
    list_to_atom(Val);
convert(Val, Default) when is_list(Val), is_list(Default) ->
    Val.

On Mon, May 7, 2018 at 3:45 PM, asdf asdf <[hidden email]> wrote:
Hey everyone,

I’ve recently built a few applications that handle high volume message passing and all communicate with each other using Erlang’s ssl distribution. I’m working on deploying these to multiple servers - right now I am just using rebar3 release and creating a zipped tar of the release directory, moving it to a new box, and changing the some vm.args and sys.config options on the new machines. This doesn’t seem totally intuitive though… it take far longer than I feel like it should. 

I’ve read about Docker and Kubernetes, and those seem promising, but our IT guys aren’t ready to move to containers and are using static nodes right now. 

So, what is the best way to quickly distribute your erlang binaries and configure the enviornment? Have a quick script to do it automatically? Is the answer just containerize with Docker and wait a few months for the IT team to catch up with it? Or is there some other option out there?

Thanks for your help!

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Deployment rebar3

Guilherme Andrade


On 7 May 2018 at 15:03, Stefan Hellkvist <[hidden email]> wrote:
What's "best" is very much depending on who you ask, but, for starters you may consider wrapping your configuration with something that allows you to override config variables with environment variables (see example below).

rebar3 releases provide this out of the box:

https://www.rebar3.org/docs/releases#dynamic-configuration
 

--
Guilherme

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Deployment rebar3

Stefan Hellkvist-2


7 maj 2018 kl. 16:44 skrev Guilherme Andrade <[hidden email]>:



On 7 May 2018 at 15:03, Stefan Hellkvist <[hidden email]> wrote:
What's "best" is very much depending on who you ask, but, for starters you may consider wrapping your configuration with something that allows you to override config variables with environment variables (see example below).

rebar3 releases provide this out of the box:

https://www.rebar3.org/docs/releases#dynamic-configuration

It is a relx feature, yes, which works fine also in erlang.mk, but I find it sometimes inconvenient that you then HAVE to set the environment variable every time. But, agreed, it is also a good option if you start your release with a script.

Stefan


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Deployment rebar3

Code Wiget
Using OS env variables would definitely speed up the process. I could have a “master” config file that is exported to the ENV and then each node would get all of the important info. Really, the changing information that just got annoying was entering the node name and IP address for the local network and the public IP addresses to listen on for certain information. This could certainly hold us over until we move into containerized systems.

Thanks!

On May 7, 2018, 10:51 AM -0400, Stefan Hellkvist <[hidden email]>, wrote:


7 maj 2018 kl. 16:44 skrev Guilherme Andrade <[hidden email]>:



On 7 May 2018 at 15:03, Stefan Hellkvist <[hidden email]> wrote:
What's "best" is very much depending on who you ask, but, for starters you may consider wrapping your configuration with something that allows you to override config variables with environment variables (see example below).

rebar3 releases provide this out of the box:

https://www.rebar3.org/docs/releases#dynamic-configuration

It is a relx feature, yes, which works fine also in erlang.mk, but I find it sometimes inconvenient that you then HAVE to set the environment variable every time. But, agreed, it is also a good option if you start your release with a script.

Stefan


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Deployment rebar3

Tristan Sloughter-4
In reply to this post by Code Wiget
Note that you, assuming you aren't already, should use `rebar3 tar` and not tar/zip the release directory manually.

How to deploy a release is  more of a general problem. You should be able to use whatever your IT guys are already using for copying a release to a node and setting up configuration -- ansible, chef, etc? You can use either OS vars as someone pointed out or simply have one of those configuration tools create a whole new sys.config/vm.args and overwrite the existing files after unpacking the release.

Docker doesn't really bring you much if not also using something like kubernetes. You still have to get the image to a node and setup per-node configuration, not much different from doing it with the tarred release.

But kubernetes is really great for this :). I use kubernetes deployments and stateful sets to deploy Erlang releases, with env vars for configuring sys.config/vm.args per pod and kubernetes dns for node discovery/clustering.

Tristan

On Mon, May 7, 2018, at 6:45 AM, asdf asdf wrote:
Hey everyone,

I’ve recently built a few applications that handle high volume message passing and all communicate with each other using Erlang’s ssl distribution. I’m working on deploying these to multiple servers - right now I am just using rebar3 release and creating a zipped tar of the release directory, moving it to a new box, and changing the some vm.args and sys.config options on the new machines. This doesn’t seem totally intuitive though… it take far longer than I feel like it should. 

I’ve read about Docker and Kubernetes, and those seem promising, but our IT guys aren’t ready to move to containers and are using static nodes right now. 

So, what is the best way to quickly distribute your erlang binaries and configure the enviornment? Have a quick script to do it automatically? Is the answer just containerize with Docker and wait a few months for the IT team to catch up with it? Or is there some other option out there?

Thanks for your help!
_______________________________________________
erlang-questions mailing list


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Deployment rebar3

Jesper Louis Andersen-2
In reply to this post by Guilherme Andrade
On Mon, May 7, 2018 at 4:44 PM Guilherme Andrade <[hidden email]> wrote:
On 7 May 2018 at 15:03, Stefan Hellkvist <[hidden email]> wrote:
What's "best" is very much depending on who you ask, but, for starters you may consider wrapping your configuration with something that allows you to override config variables with environment variables (see example below).

rebar3 releases provide this out of the box:

https://www.rebar3.org/docs/releases#dynamic-configuration
 


When it works :)

If you use it, copy `sys.config` to `sys.config.orig` as well and deliver this to the machine. That way, the script won't try to save `sys.config.orig` if it is not present. This makes your dynamic configuration more static and can avoid some situations where the expanded `sys.config` is being copied into `sys.config.orig`. This then removes the dynamic configuration altogether.




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions