unix domain sockets with abstract namespace: can't use all 108 bytes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G

Hi All,

 

I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html

 

Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

 

> Sun_path_length=108.

108

 

> UDS_path = <<"whatever">>.

<<"whatever">>

 

> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).

792

 

> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.

{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,...>>}

 

> Port = 0.

0


> gen_tcp:connect(Address,Port,[local],infinity).

(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)

(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)

(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})

(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}

** exception exit: badarg

     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)

(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}

(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}

 

If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:

connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)

When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.

 

Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.

Seems to me that this is a bug in the Erlang implementation. What do you think?

 

Thank you very much,

 

Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Alex S.
I have a strong suspicion the binary might be automatically zero-terminated.
27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]> написал(а):


Hi All,
 
I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html
 
Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:
 
> Sun_path_length=108.
108
 
> UDS_path = <<"whatever">>.
<<"whatever">>
 
> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).
792
 
> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.
{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,...>>}
 
> Port = 0.
0

> gen_tcp:connect(Address,Port,[local],infinity).
(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)
(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)
(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})
(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}
** exception exit: badarg
     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)
(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}
(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}
 
If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:
connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)
When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.
 
Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.
Seems to me that this is a bug in the Erlang implementation. What do you think?
 
Thank you very much,
 
Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   




NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.





_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2
In reply to this post by Bekes, Andras G

> On 28/03/2017, at 5:03 AM, Bekes, Andras G <[hidden email]> wrote:
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.

Be careful there.  OSX 10.11.3 manual page:

    UNIX-domain addresses are variable-length filesystem pathnames
    of at most 104 characters.

The Single Unix Specification, version 4, section un.h, says

    The size of sun_path has intentionally been left undefined.
    This is because different implementations use different sizes.
    For example, 4.3 BSD uses a size of 108, and 4.4 BSD uses a size of 104.
    Since most implementations originate from BSD versions, the size is
    typically in the range 92 to 108.

    Applications should not assume a particular length for sun_path
    or assume that it can hold {_POSIX_PATH_MAX} bytes (256).

Expect the last byte to be used for the NUL terminator of a C string,
so the length of the actual name could be limited to 91..107.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G
Thank you very much for the heads up. I am happy with a non-portable implementation. I only need to target Linux.

I suspect the below holds for unix domain specific paths in general.
The abstract namespace feature seems to be Linux-specific.

-----Original Message-----
From: Richard A. O'Keefe [mailto:[hidden email]]
Sent: Monday, March 27, 2017 11:12 PM
To: Bekes, Andras G (IST)
Cc: [hidden email]
Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes


> On 28/03/2017, at 5:03 AM, Bekes, Andras G <[hidden email]> wrote:
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.

Be careful there.  OSX 10.11.3 manual page:

    UNIX-domain addresses are variable-length filesystem pathnames
    of at most 104 characters.

The Single Unix Specification, version 4, section un.h, says

    The size of sun_path has intentionally been left undefined.
    This is because different implementations use different sizes.
    For example, 4.3 BSD uses a size of 108, and 4.4 BSD uses a size of 104.
    Since most implementations originate from BSD versions, the size is
    typically in the range 92 to 108.

    Applications should not assume a particular length for sun_path
    or assume that it can hold {_POSIX_PATH_MAX} bytes (256).

Expect the last byte to be used for the NUL terminator of a C string, so the length of the actual name could be limited to 91..107.




--------------------------------------------------------------------------------

NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G
In reply to this post by Alex S.

I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux.

Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109.

 

From: Alex S. [mailto:[hidden email]]
Sent: Monday, March 27, 2017 8:00 PM
To: Bekes, Andras G (IST)
Cc: [hidden email]
Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes

 

I have a strong suspicion the binary might be automatically zero-terminated.

27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]> написал(а):

 

 

Hi All,

 

I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html

 

Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

 

> Sun_path_length=108.

108

 

> UDS_path = <<"whatever">>.

<<"whatever">>

 

> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).

792

 

> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.

{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,...>>}

 

> Port = 0.

0


> gen_tcp:connect(Address,Port,[local],infinity).

(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)

(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)

(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})

(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}

** exception exit: badarg

     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)

(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}

(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}

 

If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:

connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)

When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.

 

Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.

Seems to me that this is a bug in the Erlang implementation. What do you think?

 

Thank you very much,

 

Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.

 

 

 

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

 





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
In reply to this post by Bekes, Andras G
On Mon, Mar 27, 2017 at 04:03:40PM +0000, Bekes, Andras G wrote:
> Hi All,
>
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
> If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
> http://man7.org/linux/man-pages/man7/unix.7.html

That is correct.  The third argument shall always be the size of the
address structure.

>
> Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

Yes.  Since it is so unspecified whether the sun_path should be zero
terminated or not, inet_drv tries to do something sensible.  It first
zeros the whole sun_path, which on linux is 108 bytes, and then copies the
whole given address binary into it.  But if the binary is longer than 107
bytes it is rejected since that might overwrite the last zero, which might
be dangerous if the kernel was expecting a zero terminated address.  And
the length is set to the length of the binary (without zero termination)
since that seems to be expected for file names.

I see no good way to know for all platforms if the address is supposed to
be zero terminated or not.  Linux utilizes the special case that if the
first byte is zero then it is not zero terminated.  I do not know if that
is a safe indicator on all platforms.

An fix that sounds safe and would work in this case is to check if the
last byte is zero then allow it to fill sun_path and use its actual length.

A riskier fix would be to allow it to fill sun_path and use its acual
length if the first byte is zero since that is what Linux says.
Riskier since *BSD says it should be zero terminated and the calculated
length to use should not count the zero termination.

/ Raimo Niskanen, Erlang/OTP


>
> > Sun_path_length=108.
> 108
>
> > UDS_path = <<"whatever">>.
> <<"whatever">>
>
> > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).
> 792
>
> > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.
> {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,...>>}
>
> > Port = 0.
> 0
>
> > gen_tcp:connect(Address,Port,[local],infinity).
> (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)
> (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)
> (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})
> (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}
> ** exception exit: badarg
>      in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)
> (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}
> (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}
>
> If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:
> connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)
> When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.
>
> Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.
> Seems to me that this is a bug in the Erlang implementation. What do you think?
>
> Thank you very much,
>
> Andras G. Bekes, Vice President
> Morgan Stanley | Institutional Securities Tech
> Lechner Odon Fasor 6 | Floor 06
> Budapest, 1095
> Phone: +36 1 882-0791
> [hidden email]<mailto:[hidden email]>
>
>
>
> ________________________________
>
> NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.

> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions


--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2

> On 28/03/2017, at 11:05 PM, Raimo Niskanen <[hidden email]> wrote:
> I see no good way to know for all platforms if the address is supposed to
> be zero terminated or not.  Linux utilizes the special case that if the
> first byte is zero then it is not zero terminated.  I do not know if that
> is a safe indicator on all platforms.

The Linux manual page is quite specific:
  "a Unix domain socket can be bound to a >>>null-terminated<<<
   file system pathname"

  "an abstract socket address" begins with a NUL byte. "All of the
  remaining bytes define the 'name' of the socket."  "The abstract
  socket namespace is a non-portable Linux extension."

Sounds like any "fix" in Erlang's guts would need to be Linux-specific
and leading NUL bytes should probably be reported as errors on other
systems.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Wed, Mar 29, 2017 at 02:08:42PM +1300, Richard A. O'Keefe wrote:

>
> > On 28/03/2017, at 11:05 PM, Raimo Niskanen <[hidden email]> wrote:
> > I see no good way to know for all platforms if the address is supposed to
> > be zero terminated or not.  Linux utilizes the special case that if the
> > first byte is zero then it is not zero terminated.  I do not know if that
> > is a safe indicator on all platforms.
>
> The Linux manual page is quite specific:
>   "a Unix domain socket can be bound to a >>>null-terminated<<<
>    file system pathname"
>
>   "an abstract socket address" begins with a NUL byte. "All of the
>   remaining bytes define the 'name' of the socket."  "The abstract
>   socket namespace is a non-portable Linux extension."
>
> Sounds like any "fix" in Erlang's guts would need to be Linux-specific
> and leading NUL bytes should probably be reported as errors on other
> systems.

That is uncomfortable since the spirit of configure tests is to test for
features and to not rely on a particular system name implying a feature.

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2
I wrote that any support in Erlang for the 'abstract socket namespace'
of Linux would have to depend on the system *being* Linux.

> On 30/03/2017, at 2:29 AM, Raimo Niskanen <[hidden email]> wrote:
>
> That is uncomfortable since the spirit of configure tests is to test for
> features and to not rely on a particular system name implying a feature.

Where did I say anything about testing for a system name?
What I meant is simply that
 *IF* Erlang ever supports this Linux feature (and I do not say that it
 should; the last time I wanted to write system-specific code was the
 last day I programmed a DEC-10)
 it should take care to check for a leading NUL byte and give an
 informative error on systems that DON'T support this "feature".
I have no idea what the best way to detect such support might be.
What I cared about is that Erlang shouldn't just go ahead and
accept leading NUL bytes on all systems because someone might be
expecting the Linux "feature", so an informative error message is
needed.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Thu, Mar 30, 2017 at 12:12:15PM +1300, Richard A. O'Keefe wrote:

> I wrote that any support in Erlang for the 'abstract socket namespace'
> of Linux would have to depend on the system *being* Linux.
>
> > On 30/03/2017, at 2:29 AM, Raimo Niskanen <[hidden email]> wrote:
> >
> > That is uncomfortable since the spirit of configure tests is to test for
> > features and to not rely on a particular system name implying a feature.
>
> Where did I say anything about testing for a system name?
> What I meant is simply that
>  *IF* Erlang ever supports this Linux feature (and I do not say that it
>  should; the last time I wanted to write system-specific code was the
>  last day I programmed a DEC-10)
>  it should take care to check for a leading NUL byte and give an
>  informative error on systems that DON'T support this "feature".
> I have no idea what the best way to detect such support might be.
> What I cared about is that Erlang shouldn't just go ahead and
> accept leading NUL bytes on all systems because someone might be
> expecting the Linux "feature", so an informative error message is
> needed.

Ah.  Sorry, I misread you.  I also have no idea how to detect if this
"feature" is supported, so I thought you actually suggested to check if
the system was "Linux" since you had no other suggestion...

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G
In reply to this post by Alex S.

Did anyone have a chance to take a closer look?

 

I still think that this is a bug in Erlang.

 

From: Bekes, Andras G (IST)
Sent: Tuesday, March 28, 2017 9:55 AM
To: 'Alex S.'
Cc: [hidden email]
Subject: RE: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes

 

I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux.

Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109.

 

From: Alex S. [mailto:[hidden email]]
Sent: Monday, March 27, 2017 8:00 PM
To: Bekes, Andras G (IST)
Cc: [hidden email]
Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes

 

I have a strong suspicion the binary might be automatically zero-terminated.

27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]> написал(а):

 

 

Hi All,

 

I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html

 

Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

 

> Sun_path_length=108.

108

 

> UDS_path = <<"whatever">>.

<<"whatever">>

 

> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).

792

 

> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.

{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,...>>}

 

> Port = 0.

0


> gen_tcp:connect(Address,Port,[local],infinity).

(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)

(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)

(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})

(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}

** exception exit: badarg

     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)

(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}

(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}

 

If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:

connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)

When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.

 

Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.

Seems to me that this is a bug in the Erlang implementation. What do you think?

 

Thank you very much,

 

Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   

 



NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.

 

 

 

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

 





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Wed, Apr 19, 2017 at 12:13:26PM +0000, Bekes, Andras G wrote:
> Did anyone have a chance to take a closer look?
>
> I still think that this is a bug in Erlang.

So far no-one have had any good suggestion on how to handle this
in a platform independent way, nor any suggestion on how to detect
this misfeature with configure.  And digging into this is not
on our prioritized list.

I am leaning towards special treatment for this Linux abnormality by
observing the leading 0 and in this special case use all bytes of
the given binary.

Then we might pass a length to the kernel that does not match
the string length of the file path.  But if such a platform uses
zero termination there is no problem, and if it uses
the size information there is no problem, it is only if it uses both
that might be a problem, and that should be very unlikely...

Something like this should do it:
------------------- erts/emulator/drivers/common/inet_drv.c -------------------
index 0fe5183..abcc289 100644
@@ -602,14 +602,6 @@ static size_t my_strnlen(const char *s, size_t maxlen)
     return i;
 }
 
-/* Check that some character in the buffer != '\0' */
-static int is_nonzero(const char *s, size_t n)
-{
-    size_t i;
-    for (i = 0;  i < n;  i++) if (s[i] != '\0') return !0;
-    return 0;
-}
-
 #endif
 
 #ifdef VALGRIND
@@ -4018,7 +4010,15 @@ static char* inet_set_address(int family,
inet_address* dst,
         int n;
         if (*len == 0) return str_einval;
  n = *((unsigned char*)(*src)); /* Length field */
- if ((*len < 1+n) || (sizeof(dst->sal.sun_path) < n+1)) {
+ if (*len < 1+n) return str_einval;
+ /* Portability fix: Make sure the address gets zero terminated
+         * except when the first byte is \0 because then it is sort
+         * of zero terminated although the zero termination comes
+         * before the address...  This fix handles Linux's
+         * abstract socket address nonportable extension.
+ */
+ if (n + ((*len) > 1 && (*src)[1] == '\0' ? 0 : 1)
+            > sizeof(dst->sal.sun_path)) {
     return str_einval;
  }
  sys_memzero((char*)dst, sizeof(struct sockaddr_un));
@@ -4176,8 +4176,7 @@ static int inet_get_address(char* dst, inet_address*
src, unsigned int* len)
  * nonportable extension.
  */
         m = my_strnlen(src->sal.sun_path, n);
- if ((m == 0) && is_nonzero(src->sal.sun_path, n))
-    m = n;
+ if (m == 0)  m = n;
         dst[0] = INET_AF_LOCAL;
         dst[1] = (char) ((unsigned char) m);
         sys_memcpy(dst+2, src->sal.sun_path, m);
@@ -4241,8 +4240,7 @@ inet_address_to_erlang(char *dst, inet_address **src,
SOCKLEN_T sz) {
  * nonportable extension.
  */
         m = my_strnlen((*src)->sal.sun_path, n);
- if ((m == 0) && is_nonzero((*src)->sal.sun_path, n))
-    m = n;
+ if (m == 0)  m = n;
  if (dst) {
     dst[0] = INET_AF_LOCAL;
     dst[1] = (char) ((unsigned char) m);



>
> From: Bekes, Andras G (IST)
> Sent: Tuesday, March 28, 2017 9:55 AM
> To: 'Alex S.'
> Cc: [hidden email]
> Subject: RE: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes
>
> I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux.
> Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109.
>
> From: Alex S. [mailto:[hidden email]]
> Sent: Monday, March 27, 2017 8:00 PM
> To: Bekes, Andras G (IST)
> Cc: [hidden email]
> Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes
>
> I have a strong suspicion the binary might be automatically zero-terminated.
> 27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]<mailto:[hidden email]>> написал(а):
>
>
> Hi All,
>
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
> If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
> http://man7.org/linux/man-pages/man7/unix.7.html
>
> Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:
>
> > Sun_path_length=108.
> 108
>
> > UDS_path = <<"whatever">>.
> <<"whatever">>
>
> > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).
> 792
>
> > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.
> {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,...>>}
>
> > Port = 0.
> 0
>
> > gen_tcp:connect(Address,Port,[local],infinity).
> (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)
> (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)
> (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})
> (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}
> ** exception exit: badarg
>      in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)
> (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}
> (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}
>
> If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:
> connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)
> When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.
>
> Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.
> Seems to me that this is a bug in the Erlang implementation. What do you think?
>
> Thank you very much,
>
> Andras G. Bekes, Vice President
> Morgan Stanley | Institutional Securities Tech
> Lechner Odon Fasor 6 | Floor 06
> Budapest, 1095
> Phone: +36 1 882-0791
> [hidden email]<mailto:[hidden email]>
>
>


--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Kenneth Lakin
On 04/24/2017 03:01 AM, Raimo Niskanen wrote:
> So far no-one have had any good suggestion on how to handle this
> in a platform independent way, nor any suggestion on how to detect
> this misfeature with configure.

This _appears_ to be a useful feature that seems to make domain sockets
work more like IP sockets. [0] Why do you consider it a misfeature?

[0] https://utcc.utoronto.ca/~cks/space/blog/linux/SocketAbstractNamespace



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Mon, Apr 24, 2017 at 11:09:34AM -0700, Kenneth Lakin wrote:
> On 04/24/2017 03:01 AM, Raimo Niskanen wrote:
> > So far no-one have had any good suggestion on how to handle this
> > in a platform independent way, nor any suggestion on how to detect
> > this misfeature with configure.
>
> This _appears_ to be a useful feature that seems to make domain sockets
> work more like IP sockets. [0] Why do you consider it a misfeature?

Well, it is just because on all other Unix-like OS:es the address is
(loosly/de-facto) defined to be zero terminated, and there are macros to
calculate the address structure size that use strlen on the string.

Therefore the Linux way to use the address type contradicts the BSD:s and I
guess Solaris.  Nobody has so far found a way to figure out how to use
configure to detect if a size larger than the string length is OK or might
crash the program some day.

Other than that I am sure it is a useful feature, so maybe not a
misfeature, more an API abuse.

>
> [0] https://utcc.utoronto.ca/~cks/space/blog/linux/SocketAbstractNamespace
>
>



--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2

> On 26/04/2017, at 2:14 AM, Raimo Niskanen <[hidden email]> wrote:
>
> Well, it is just because on all other Unix-like OS:es the address is
> (loosly/de-facto) defined to be zero terminated, and there are macros to
> calculate the address structure size that use strlen on the string.

It's not *that* loose.  POSIX explicitly defines sun_path to hold a "pathname"
and a path name is defined as a character string and those are defined to be
NUL terminated.  (See <sys/un.h> and then check definitions for "path name"
and "string".)
>
> Therefore the Linux way to use the address type contradicts the BSD:s and I
> guess Solaris.

And POSIX.  And HP-UX and AIX, as far as I can tell, and to the degree that
anyone cares.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Wed, Apr 26, 2017 at 03:52:52PM +1200, Richard A. O'Keefe wrote:

>
> > On 26/04/2017, at 2:14 AM, Raimo Niskanen <[hidden email]> wrote:
> >
> > Well, it is just because on all other Unix-like OS:es the address is
> > (loosly/de-facto) defined to be zero terminated, and there are macros to
> > calculate the address structure size that use strlen on the string.
>
> It's not *that* loose.  POSIX explicitly defines sun_path to hold a "pathname"
> and a path name is defined as a character string and those are defined to be
> NUL terminated.  (See <sys/un.h> and then check definitions for "path name"
> and "string".)

Ok.  That is not as loose as I thought.

I can not find (in Posix) any clue to which address_len you shall use with
a sockaddr_un structure.  Is it the size of the whole structure, or is it
the size of the structure ending with the terminating 0 in the path?
There is an example:
    http://pubs.opengroup.org/onlinepubs/9699919799/ ->
       bind - bind a name to a socket
that uses the size of the structure, ignoring the path length in .sun_path.
That would speak for that the Linux use is creative but not violating Posix
since the path is 0 terminated and the address size is large enough.
They just encode an address after the empty path...  But Posix also states
that bind() shall fail with [ENOENT] if the pathname is an empty string. :-)

And e.g FreeBSD unix(4) tells you to use the macro SUN_LEN() that uses
strlen(.sun_path) to calculate the address size.  But it seems Posix does
not define this macro.

So it seems it is not a violation of Posix to use a su->sun_path that starts
with a 0 combined with an address_len of sizeof(*su).  Or maybe even between
SUN_LEN(su) and sizeof(*su)?  Or?

> >
> > Therefore the Linux way to use the address type contradicts the BSD:s and I
> > guess Solaris.
>
> And POSIX.  And HP-UX and AIX, as far as I can tell, and to the degree that
> anyone cares.

I am sorry, since I am not a native english speaker; what does "and to the
degree that anyone cares" mean in this context?

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Kenneth Lakin
Disclaimer: I've only read the man pages, I've not actually worked with
this stuff.

On 04/26/2017 01:55 AM, Raimo Niskanen wrote:
> Well, it is just because on all other Unix-like OS:es
> the address is (loosly/de-facto) defined to be zero
> terminated...

unix(7) on my Linux system [0] agrees that sockaddr_un.sun_path is
typically null-terminated. (Though, it sounds like abstract socket
addresses are always 108 bytes long, as embedded NULLs other than the
very first one are not treated specially.)

The BUGS section claims to describe two different portable methods to
retrieve the specified socket address when there might be exactly 108
bytes of address. The man page makes it really sound like Linux will Do
The Right Thing, as long as sun_path is no longer than 108 bytes.

So, just make sure that the Erlang string or binary containing the
socket address isn't longer than 108 bytes, fill sun_path (or the entire
sockaddr_un) with 0 before you copy the address in, and you should be fine?

I'm _fairly_ certain the second paragraph in the BUGS section is talking
about the behavior of non-Linux systems, and can be ignored when writing
Linux-specific code. Someone _please_ correct me if I'm wrong about that.

> Nobody has so far found a way to figure out how to use
> configure to detect if a size larger than the string length
> is OK or might crash the program some day.

As mentioned above, it _looks_ like sun_paths up to 108 bytes are A-OK.

> I can not find (in Posix) any clue to which address_len you shall use with
> a sockaddr_un structure. Is it the size of the whole structure[?]

Yeah, it _looks_ like it is the size of the whole structure.

The "Pathname sockets" section of the unix(7) man page suggests:

"or, more simply, addrlen can be specified as
 sizeof(struct sockaddr_un)."

The EXAMPLE section in bind(2) [1] as well as the EXAMPLE section in
unix(7) indicates that passing sizeof(struct sockaddr_un) as bind's
third argument is the correct way to specify addrlen to bind a Unix
domain socket.

_Additionally_, in the POSIX reference that you linked to in your
message, the man page for bind(2) [2] agrees with the Linux man page
that sizeof(struct sockaddr_un) is the correct thing to pass to bind's
third argument.

Or did I misunderstand, and you're talking about a different address
length parameter?

[0] http://man7.org/linux/man-pages/man7/unix.7.html
[1] http://man7.org/linux/man-pages/man2/bind.2.html
[2] http://pubs.opengroup.org/onlinepubs/9699919799/functions/bind.html



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2
In reply to this post by Raimo Niskanen-2

> On 26/04/2017, at 8:55 PM, Raimo Niskanen <[hidden email]> wrote:
>
> Ok.  That is not as loose as I thought.
>
> I can not find (in Posix) any clue to which address_len you shall use with
> a sockaddr_un structure.  Is it the size of the whole structure, or is it
> the size of the structure ending with the terminating 0 in the path?

The bind() description says that
"address_len Specifies the length of the sockaddr structure
 pointed to by the address argument."
It does not say the *size* of the structure but the *length* of the structure.
Unfortunately, I can find no definition of "length of a structure".  Sigh.

> There is an example:
>    http://pubs.opengroup.org/onlinepubs/9699919799/ ->
>       bind - bind a name to a socket
> that uses the size of the structure, ignoring the path length in .sun_path.
> That would speak for that the Linux use is creative but not violating Posix
> since the path is 0 terminated and the address size is large enough.

The Linux extension puts a NUL byte at the front of the path, thus
unambiguously providing the path "".  Bytes after that are not part
of the required path name.  It is *deeply* regrettable that the Linux
extenders did not do the clean thing and add a new address family.

I feel that I have to say this: if you look at the TOP500 list, 498 out
of the top 500 supercomputers in the world run "Linux".  (The other 2
run "Unix".)  While I enjoy exegesis of POSIX, at some point reality
must be acknowledged...

Solaris documentation is clear that you should use
bind(s, (struct sockaddr *) &addr,
                strlen(addr.sun_path) + sizeof (addr.sun_family))

Poking around in the OpenSolaris kernel source code, the function
copyin_name() suggests to me that any value between that and the
full size of the structure should work.

> So it seems it is not a violation of Posix to use a su->sun_path that starts
> with a 0 combined with an address_len of sizeof(*su).

It may not be, but the path is still "" and thus a run-time error.

Turning errors into extensions is common enough, so this leaves the situation
where we found it, really.  POSIX systems *shouldn't* do this, but Linux *does*.
>
>>> Therefore the Linux way to use the address type contradicts the BSD:s and I
>>> guess Solaris.
>>
>> And POSIX.  And HP-UX and AIX, as far as I can tell, and to the degree that
>> anyone cares.
>
> I am sorry, since I am not a native english speaker; what does "and to the
> degree that anyone cares" mean in this context?

What I meant in that paragraph was "I have checked the AIX and HP-UX manuals
and they certainly do not mention the Linux extension and can be read as
consistent with POSIX and the BSDs, so Erlang code using this Linux feature
would not port to AIX or HP-UX, but does that have any practical significance?
Does anyone *care* about Erlang on AIX or HP-UX?"

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Kenneth Lakin
NB: The observations of a Linux weenie (that is, me) follow:

On 04/26/2017 05:46 PM, Richard A. O'Keefe wrote:
> Solaris documentation is clear that you should use
> bind(s, (struct sockaddr *) &addr,
> strlen(addr.sun_path) + sizeof (addr.sun_family))

Which is _really_ confusing to me.

Even on Solaris, sockaddr_un.sun_path is an array of fixed size, right?
And for INET and INET6 sockets, it seem that the Solaris documentation
says that argument three to bind is the length of the _entire struct_.

Why does it make sense to force client code to special-case UNIX domain
sockets and get strlen involved in calculating bind's third argument?
Why not have the kernel switch on sa_family_t and have any required
special-casing neatly hidden from the client? [0]

>> So it seems it is not a violation of Posix to use a su->sun_path that starts
>> with a 0 combined with an address_len of sizeof(*su).
>
> It may not be, but the path is still "" and thus a run-time error.
> ...
> POSIX systems *shouldn't* do this, but Linux *does*.

My search skills might be weak, but I'm unable to find the section in
the POSIX documentation that fully addresses UNIX domain sockets. I
think the most it says on the topic is in the documentation for
sys/un.h, which pretty much says "sockaddr_un must have sun_family and
sun_path members. sun_path might be any size at all. Cast this struct to
struct_sockaddr when you use it.". [1] What did I miss?

If we move from thinking of sockaddr_un.sun_path as a null-terminated
string, and move to thinking of it as a fixed-length sequence of bytes
that might happen to be a null-terminated string, but might also be
interpreted in any other way by the underlying OS, does Linux's behavior
seem more sensible? I mean, if we're no longer constrained by the
restriction that sun_path _must_ be something that's a valid filename,
[2] we can get very creative with the interpretation of sun_path, no?

From a practical perspective, it seems to me that as long as you zero
out the entirety of the sockaddr_un before you start filling it and
ensure that whatever you copy into sun_path is not longer than sun_path,
then your code _should_ just work.

And, mostly unrelated to all that:

> It does not say the *size* of the structure but the
> *length* of the structure.

To be fair, the documentation seems to consistently uses "length of
[the] structure" where one might expect to see "size". See the
documentation for connect, for example. ;)

[0] And while we're talking about inconsistent interfaces, why do the
BSDs have a one-byte length as the first member of all(?) of its socket
structs? This wart appears to be absent in both Solaris and Linux.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_un.h.html
[2] Because Linux's abstract namespace UNIX sockets are (just like IP
sockets) AFAICT _not_ represented as files in the filesystem.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Wed, Apr 26, 2017 at 10:51:38PM -0700, Kenneth Lakin wrote:

> NB: The observations of a Linux weenie (that is, me) follow:
>
> On 04/26/2017 05:46 PM, Richard A. O'Keefe wrote:
> > Solaris documentation is clear that you should use
> > bind(s, (struct sockaddr *) &addr,
> > strlen(addr.sun_path) + sizeof (addr.sun_family))
>
> Which is _really_ confusing to me.
>
> Even on Solaris, sockaddr_un.sun_path is an array of fixed size, right?
> And for INET and INET6 sockets, it seem that the Solaris documentation
> says that argument three to bind is the length of the _entire struct_.

Well, maybe.
But AF_INET and AF_INET6 socket addresses are not variable length.

That it is an array of fixed size only means that you have a pre-allocated
area whithin which the address shall be stored.  The alternative would be to
have a pointer but then we get the problem of having to allocate and free
the memory for the address correctly.

This is an example of an variable length address.  In fact Linux does not
always use all 108 bytes.  You can create a socket with an empty address
and then you get an address with a leading '\0' and some 6 characters not
corresponding to a filesystem name.

>
> Why does it make sense to force client code to special-case UNIX domain
> sockets and get strlen involved in calculating bind's third argument?
> Why not have the kernel switch on sa_family_t and have any required
> special-casing neatly hidden from the client? [0]

Since you for AF_UNIX have to know that the address corresponds to a
filesystem path name and that you have to explicitly use unlink(2)
to remove the file that bind(3) creates you already are forced to
specialize client code.

>
> >> So it seems it is not a violation of Posix to use a su->sun_path that starts
> >> with a 0 combined with an address_len of sizeof(*su).
> >
> > It may not be, but the path is still "" and thus a run-time error.
> > ...
> > POSIX systems *shouldn't* do this, but Linux *does*.
>
> My search skills might be weak, but I'm unable to find the section in
> the POSIX documentation that fully addresses UNIX domain sockets. I
> think the most it says on the topic is in the documentation for
> sys/un.h, which pretty much says "sockaddr_un must have sun_family and
> sun_path members. sun_path might be any size at all. Cast this struct to
> struct_sockaddr when you use it.". [1] What did I miss?

There is an example in the Posix documentation of bind() that says sun_path
is a path name and that the empty string should result in ENOENT.

>
> If we move from thinking of sockaddr_un.sun_path as a null-terminated
> string, and move to thinking of it as a fixed-length sequence of bytes
> that might happen to be a null-terminated string, but might also be
> interpreted in any other way by the underlying OS, does Linux's behavior
> seem more sensible? I mean, if we're no longer constrained by the
> restriction that sun_path _must_ be something that's a valid filename,
> [2] we can get very creative with the interpretation of sun_path, no?

Yes.  Creative.  But it would have been more kosher if Linux had defined a
new address family.  They have violated old standards in a creative way.

>
> From a practical perspective, it seems to me that as long as you zero
> out the entirety of the sockaddr_un before you start filling it and
> ensure that whatever you copy into sun_path is not longer than sun_path,
> then your code _should_ just work.

Yes.  But what happens if you do that on FreeBSD?  Probably nothing
since sun_path is zero terminated within the size of the structure.
But unix(4) tells you to use SUN_LEN(su).  So there is no way to fulfill
both Linux's and FreeBSD's API definitions.  And so far nobody has
provided a way to configure test for it.  Thank you Linus!

I am inclined to just ignore the FreeBSD man page advice since it _may_
allow SUN_LEN(su) =< su->sun_len =< sizeof(*su)...

>
> And, mostly unrelated to all that:
>
> > It does not say the *size* of the structure but the
> > *length* of the structure.
>
> To be fair, the documentation seems to consistently uses "length of
> [the] structure" where one might expect to see "size". See the
> documentation for connect, for example. ;)
>
> [0] And while we're talking about inconsistent interfaces, why do the
> BSDs have a one-byte length as the first member of all(?) of its socket
> structs? This wart appears to be absent in both Solaris and Linux.

This simplifies handling variable length addresses for example for routing
sockets. [3]  It was added with 4.3BSD-Reno (1990) and probably existed in
SunOS 4 but did not propagate to Solaris.

> [1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_un.h.html
> [2] Because Linux's abstract namespace UNIX sockets are (just like IP
> sockets) AFAICT _not_ represented as files in the filesystem.

In blatant contradiction to Posix.

>
>

[3]: W. Richard Stevens: Unix Network Programming Volume 1 3:rd Edition
     Chapter 3.2 Socket Address Structures.


--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
12