Can't connect to Erlang node in docker

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Can't connect to Erlang node in docker

Roger Lipscombe-2
I've got an Erlang node running in docker[1]. How do I connect to it
from the host?

I've tried every combination of -name, -sname, etc., that I can think
of, and all I'm getting is 'pang'.

I can telnet to the distribution port on the container; if I use
erl_epmd:names/1, passing the DNS name of the docker container, it
returns the expected Erlang nodes.

But I can't get Erlang distribution to connect from the host to the
node in the docker container.

Interestingly, if I connect from the node inside docker to a node on
the host, it all starts working. But that's not useful for what I'm
trying to do.

[1] actually several Erlang nodes in several docker containers.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Can't connect to Erlang node in docker

Juan Jose Comellas-3
This is most probably not a problem with Erlang but missing configuration settings in Docker. Have you published or exposed the ports outside of the Docker container [1]? Are you starting the containers individually or are you using docker-compose or something like that?

I would recommend reading up on the different options that Docker provides for networking. If you send some sample Dockerfile or docker-compose.yml file with the arguments you're using to start the nodes, we could provide better feedback.



On Wed, Jun 5, 2019 at 9:11 AM Roger Lipscombe <[hidden email]> wrote:
I've got an Erlang node running in docker[1]. How do I connect to it
from the host?

I've tried every combination of -name, -sname, etc., that I can think
of, and all I'm getting is 'pang'.

I can telnet to the distribution port on the container; if I use
erl_epmd:names/1, passing the DNS name of the docker container, it
returns the expected Erlang nodes.

But I can't get Erlang distribution to connect from the host to the
node in the docker container.

Interestingly, if I connect from the node inside docker to a node on
the host, it all starts working. But that's not useful for what I'm
trying to do.

[1] actually several Erlang nodes in several docker containers.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Can't connect to Erlang node in docker

Roger Lipscombe-2
On Wed, 5 Jun 2019 at 13:28, Juan Jose Comellas <[hidden email]> wrote:
> This is most probably not a problem with Erlang but missing configuration settings in Docker. Have you published or exposed the ports outside of the Docker container [1]? Are you starting the containers individually or are you using docker-compose or something like that?

Using docker-compose. The ports are not forwarded to localhost,
because I've got a whole bunch of Erlang nodes in docker containers,
and that approach won't scale.

> I would recommend reading up on the different options that Docker provides for networking.

Docker networking is fine; using the IP address of the container (or
DNS name; I've got dnsmasq forwarding a domain suffix), I can telnet
to the epmd port and to the distribution port (and to various HTTP
ports, etc.). It's Erlang's node discovery/connection process that's
not working. erl_epmd:names works fine if I run it on the host, and
specify one of the containers.

So I'm guessing that Erlang doesn't like the lashed-up DNS resolution,
or something, but it won't connect using the IP address in the node
name either.

As I pointed out, if I connect from a node in the container to a node
on the host, it works fine, which also implies that it's not a
networking problem, per se. But rather it's a discovery/resolution
problem. But I don't understand how that actually works in OTP, so...

> If you send some sample Dockerfile or docker-compose.yml file with the arguments you're using to start the nodes, we could provide better feedback.

The nodes are using relx-generated startup scripts, identical to the
ones we're using in production (where they're not running in
containers). We start them using s6-overlay.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Can't connect to Erlang node in docker

Roger Lipscombe-2
OK. I think I discovered the relevant caveat in the documentation:

"A node with a long node name cannot communicate with a node with a
short node name."

The node I'd like to connect to is using short names, but I'm having
to use a long name on the host (because of the differing domain
suffix).

*But* when I set the remote node to use long name mode, I *still* can't connect.

*However*, I found various places (e.g. [1]) which state that you
can't mix short and long names at all. This is obviously incorrect,
'cos I successfully connected from the short-name node to the
long-name node. Also, in that documentation, "communicate with"
implies (to me, at least) both directions. Did something accidentally
get "fixed", allowing short-to-long, but not long-to-short?

[1] https://stackoverflow.com/questions/26474591/connecting-erlang-nodes-when-an-internal-and-external-ip-address-are-at-play

On Wed, 5 Jun 2019 at 14:22, Roger Lipscombe <[hidden email]> wrote:

>
> On Wed, 5 Jun 2019 at 13:28, Juan Jose Comellas <[hidden email]> wrote:
> > This is most probably not a problem with Erlang but missing configuration settings in Docker. Have you published or exposed the ports outside of the Docker container [1]? Are you starting the containers individually or are you using docker-compose or something like that?
>
> Using docker-compose. The ports are not forwarded to localhost,
> because I've got a whole bunch of Erlang nodes in docker containers,
> and that approach won't scale.
>
> > I would recommend reading up on the different options that Docker provides for networking.
>
> Docker networking is fine; using the IP address of the container (or
> DNS name; I've got dnsmasq forwarding a domain suffix), I can telnet
> to the epmd port and to the distribution port (and to various HTTP
> ports, etc.). It's Erlang's node discovery/connection process that's
> not working. erl_epmd:names works fine if I run it on the host, and
> specify one of the containers.
>
> So I'm guessing that Erlang doesn't like the lashed-up DNS resolution,
> or something, but it won't connect using the IP address in the node
> name either.
>
> As I pointed out, if I connect from a node in the container to a node
> on the host, it works fine, which also implies that it's not a
> networking problem, per se. But rather it's a discovery/resolution
> problem. But I don't understand how that actually works in OTP, so...
>
> > If you send some sample Dockerfile or docker-compose.yml file with the arguments you're using to start the nodes, we could provide better feedback.
>
> The nodes are using relx-generated startup scripts, identical to the
> ones we're using in production (where they're not running in
> containers). We start them using s6-overlay.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Can't connect to Erlang node in docker

Magnus Henoch-7
In reply to this post by Roger Lipscombe-2

On 05/06/2019 13:10, Roger Lipscombe wrote:

I've got an Erlang node running in docker[1]. How do I connect to it
from the host?

I've tried every combination of -name, -sname, etc., that I can think
of, and all I'm getting is 'pang'.

You need a couple of things to succeed:

  • Both nodes must have the same cookie
  • The nodes must be using either both -name or both -sname; no mixing long and short names
  • The local node must use the exact same node name (including host name) that the remote node thinks it has
  • The local node must be able to resolve the host name part of the node name of the remote node

So if the remote node is running with node name foo@mycontainer, you should be able to connect to it with erl -sname whatever -remsh foo@mycontainer. You can check the node name of the remote node by looking at the prompt in the Erlang shell. Note that epmd only displays the "local" part of the node name, not the host name, so you may be able to open a connection to the remote node and still have it rejected and closed if the host name part doesn't match.

If mycontainer doesn't resolve to the correct IP address, you can use an inet configuration file: http://erlang.org/doc/apps/erts/inet_cfg.html
For example, to hard-code an IP address for a host name and force the Erlang node to use this file instead of native name resolution:

{host, {10,1,2,3}, ["mycontainer"]}.
{lookup, [file]}.

Then pass the name of this file like this:

erl -sname whatever -remsh foo@mycontainer -kernel inetrc '"./myinetrc"'

Hope this helps,
Magnus



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Can't connect to Erlang node in docker

Roger Lipscombe-2
On Wed, 5 Jun 2019 at 15:06, Magnus Henoch <[hidden email]> wrote:
> Both nodes must have the same cookie

They are.

> The nodes must be using either both -name or both -sname; no mixing long and short names

Except that you can connect from a short name to a long name; see my later post.

> The local node must use the exact same node name (including host name) that the remote node thinks it has

Does "host name" include the domain name suffix for this purpose?

> The local node must be able to resolve the host name part of the node name of the remote node

It can.

> So if the remote node is running with node name foo@mycontainer, you should be able to connect to it with erl -sname whatever -remsh foo@mycontainer.

This works fine provided I put 'mycontainer' in /etc/hosts; what
doesn't seem to work is 'mycontainer.docker_network', where I've got
local dnsmasq forwarding '.docker_network' to the docker resolver.

> If mycontainer doesn't resolve to the correct IP address, you can use an inet configuration file: http://erlang.org/doc/apps/erts/inet_cfg.html
> For example, to hard-code an IP address for a host name and force the Erlang node to use this file instead of native name resolution:

That's not a particularly scalable solution, given that docker
hostnames and IP addresses change all the time. I guess I can script
it, though.

> Hope this helps,
> Magnus

Thanks,
Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions