Quantcast

unix domain sockets with abstract namespace: can't use all 108 bytes

Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G

Hi All,

 

I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html

 

Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

 

> Sun_path_length=108.

108

 

> UDS_path = <<"whatever">>.

<<"whatever">>

 

> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).

792

 

> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.

{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,...>>}

 

> Port = 0.

0


> gen_tcp:connect(Address,Port,[local],infinity).

(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)

(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)

(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})

(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}

** exception exit: badarg

     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)

(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}

(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}

 

If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:

connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)

When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.

 

Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.

Seems to me that this is a bug in the Erlang implementation. What do you think?

 

Thank you very much,

 

Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Alex S.
I have a strong suspicion the binary might be automatically zero-terminated.
27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]> написал(а):


Hi All,
 
I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html
 
Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:
 
> Sun_path_length=108.
108
 
> UDS_path = <<"whatever">>.
<<"whatever">>
 
> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).
792
 
> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.
{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,...>>}
 
> Port = 0.
0

> gen_tcp:connect(Address,Port,[local],infinity).
(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)
(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)
(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})
(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}
** exception exit: badarg
     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)
(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}
(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}
 
If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:
connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)
When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.
 
Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.
Seems to me that this is a bug in the Erlang implementation. What do you think?
 
Thank you very much,
 
Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   




NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.





_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2
In reply to this post by Bekes, Andras G

> On 28/03/2017, at 5:03 AM, Bekes, Andras G <[hidden email]> wrote:
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.

Be careful there.  OSX 10.11.3 manual page:

    UNIX-domain addresses are variable-length filesystem pathnames
    of at most 104 characters.

The Single Unix Specification, version 4, section un.h, says

    The size of sun_path has intentionally been left undefined.
    This is because different implementations use different sizes.
    For example, 4.3 BSD uses a size of 108, and 4.4 BSD uses a size of 104.
    Since most implementations originate from BSD versions, the size is
    typically in the range 92 to 108.

    Applications should not assume a particular length for sun_path
    or assume that it can hold {_POSIX_PATH_MAX} bytes (256).

Expect the last byte to be used for the NUL terminator of a C string,
so the length of the actual name could be limited to 91..107.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G
Thank you very much for the heads up. I am happy with a non-portable implementation. I only need to target Linux.

I suspect the below holds for unix domain specific paths in general.
The abstract namespace feature seems to be Linux-specific.

-----Original Message-----
From: Richard A. O'Keefe [mailto:[hidden email]]
Sent: Monday, March 27, 2017 11:12 PM
To: Bekes, Andras G (IST)
Cc: [hidden email]
Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes


> On 28/03/2017, at 5:03 AM, Bekes, Andras G <[hidden email]> wrote:
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.

Be careful there.  OSX 10.11.3 manual page:

    UNIX-domain addresses are variable-length filesystem pathnames
    of at most 104 characters.

The Single Unix Specification, version 4, section un.h, says

    The size of sun_path has intentionally been left undefined.
    This is because different implementations use different sizes.
    For example, 4.3 BSD uses a size of 108, and 4.4 BSD uses a size of 104.
    Since most implementations originate from BSD versions, the size is
    typically in the range 92 to 108.

    Applications should not assume a particular length for sun_path
    or assume that it can hold {_POSIX_PATH_MAX} bytes (256).

Expect the last byte to be used for the NUL terminator of a C string, so the length of the actual name could be limited to 91..107.




--------------------------------------------------------------------------------

NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G
In reply to this post by Alex S.

I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux.

Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109.

 

From: Alex S. [mailto:[hidden email]]
Sent: Monday, March 27, 2017 8:00 PM
To: Bekes, Andras G (IST)
Cc: [hidden email]
Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes

 

I have a strong suspicion the binary might be automatically zero-terminated.

27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]> написал(а):

 

 

Hi All,

 

I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html

 

Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

 

> Sun_path_length=108.

108

 

> UDS_path = <<"whatever">>.

<<"whatever">>

 

> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).

792

 

> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.

{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,...>>}

 

> Port = 0.

0


> gen_tcp:connect(Address,Port,[local],infinity).

(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)

(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)

(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})

(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}

** exception exit: badarg

     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)

(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}

(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}

 

If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:

connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)

When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.

 

Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.

Seems to me that this is a bug in the Erlang implementation. What do you think?

 

Thank you very much,

 

Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.

 

 

 

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

 





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
In reply to this post by Bekes, Andras G
On Mon, Mar 27, 2017 at 04:03:40PM +0000, Bekes, Andras G wrote:
> Hi All,
>
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
> If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
> http://man7.org/linux/man-pages/man7/unix.7.html

That is correct.  The third argument shall always be the size of the
address structure.

>
> Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

Yes.  Since it is so unspecified whether the sun_path should be zero
terminated or not, inet_drv tries to do something sensible.  It first
zeros the whole sun_path, which on linux is 108 bytes, and then copies the
whole given address binary into it.  But if the binary is longer than 107
bytes it is rejected since that might overwrite the last zero, which might
be dangerous if the kernel was expecting a zero terminated address.  And
the length is set to the length of the binary (without zero termination)
since that seems to be expected for file names.

I see no good way to know for all platforms if the address is supposed to
be zero terminated or not.  Linux utilizes the special case that if the
first byte is zero then it is not zero terminated.  I do not know if that
is a safe indicator on all platforms.

An fix that sounds safe and would work in this case is to check if the
last byte is zero then allow it to fill sun_path and use its actual length.

A riskier fix would be to allow it to fill sun_path and use its acual
length if the first byte is zero since that is what Linux says.
Riskier since *BSD says it should be zero terminated and the calculated
length to use should not count the zero termination.

/ Raimo Niskanen, Erlang/OTP


>
> > Sun_path_length=108.
> 108
>
> > UDS_path = <<"whatever">>.
> <<"whatever">>
>
> > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).
> 792
>
> > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.
> {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,...>>}
>
> > Port = 0.
> 0
>
> > gen_tcp:connect(Address,Port,[local],infinity).
> (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)
> (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)
> (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})
> (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}
> ** exception exit: badarg
>      in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)
> (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}
> (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}
>
> If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:
> connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)
> When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.
>
> Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.
> Seems to me that this is a bug in the Erlang implementation. What do you think?
>
> Thank you very much,
>
> Andras G. Bekes, Vice President
> Morgan Stanley | Institutional Securities Tech
> Lechner Odon Fasor 6 | Floor 06
> Budapest, 1095
> Phone: +36 1 882-0791
> [hidden email]<mailto:[hidden email]>
>
>
>
> ________________________________
>
> NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.

> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions


--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2

> On 28/03/2017, at 11:05 PM, Raimo Niskanen <[hidden email]> wrote:
> I see no good way to know for all platforms if the address is supposed to
> be zero terminated or not.  Linux utilizes the special case that if the
> first byte is zero then it is not zero terminated.  I do not know if that
> is a safe indicator on all platforms.

The Linux manual page is quite specific:
  "a Unix domain socket can be bound to a >>>null-terminated<<<
   file system pathname"

  "an abstract socket address" begins with a NUL byte. "All of the
  remaining bytes define the 'name' of the socket."  "The abstract
  socket namespace is a non-portable Linux extension."

Sounds like any "fix" in Erlang's guts would need to be Linux-specific
and leading NUL bytes should probably be reported as errors on other
systems.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Wed, Mar 29, 2017 at 02:08:42PM +1300, Richard A. O'Keefe wrote:

>
> > On 28/03/2017, at 11:05 PM, Raimo Niskanen <[hidden email]> wrote:
> > I see no good way to know for all platforms if the address is supposed to
> > be zero terminated or not.  Linux utilizes the special case that if the
> > first byte is zero then it is not zero terminated.  I do not know if that
> > is a safe indicator on all platforms.
>
> The Linux manual page is quite specific:
>   "a Unix domain socket can be bound to a >>>null-terminated<<<
>    file system pathname"
>
>   "an abstract socket address" begins with a NUL byte. "All of the
>   remaining bytes define the 'name' of the socket."  "The abstract
>   socket namespace is a non-portable Linux extension."
>
> Sounds like any "fix" in Erlang's guts would need to be Linux-specific
> and leading NUL bytes should probably be reported as errors on other
> systems.

That is uncomfortable since the spirit of configure tests is to test for
features and to not rely on a particular system name implying a feature.

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Richard A. O'Keefe-2
I wrote that any support in Erlang for the 'abstract socket namespace'
of Linux would have to depend on the system *being* Linux.

> On 30/03/2017, at 2:29 AM, Raimo Niskanen <[hidden email]> wrote:
>
> That is uncomfortable since the spirit of configure tests is to test for
> features and to not rely on a particular system name implying a feature.

Where did I say anything about testing for a system name?
What I meant is simply that
 *IF* Erlang ever supports this Linux feature (and I do not say that it
 should; the last time I wanted to write system-specific code was the
 last day I programmed a DEC-10)
 it should take care to check for a leading NUL byte and give an
 informative error on systems that DON'T support this "feature".
I have no idea what the best way to detect such support might be.
What I cared about is that Erlang shouldn't just go ahead and
accept leading NUL bytes on all systems because someone might be
expecting the Linux "feature", so an informative error message is
needed.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Thu, Mar 30, 2017 at 12:12:15PM +1300, Richard A. O'Keefe wrote:

> I wrote that any support in Erlang for the 'abstract socket namespace'
> of Linux would have to depend on the system *being* Linux.
>
> > On 30/03/2017, at 2:29 AM, Raimo Niskanen <[hidden email]> wrote:
> >
> > That is uncomfortable since the spirit of configure tests is to test for
> > features and to not rely on a particular system name implying a feature.
>
> Where did I say anything about testing for a system name?
> What I meant is simply that
>  *IF* Erlang ever supports this Linux feature (and I do not say that it
>  should; the last time I wanted to write system-specific code was the
>  last day I programmed a DEC-10)
>  it should take care to check for a leading NUL byte and give an
>  informative error on systems that DON'T support this "feature".
> I have no idea what the best way to detect such support might be.
> What I cared about is that Erlang shouldn't just go ahead and
> accept leading NUL bytes on all systems because someone might be
> expecting the Linux "feature", so an informative error message is
> needed.

Ah.  Sorry, I misread you.  I also have no idea how to detect if this
"feature" is supported, so I thought you actually suggested to check if
the system was "Linux" since you had no other suggestion...

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Bekes, Andras G
In reply to this post by Alex S.

Did anyone have a chance to take a closer look?

 

I still think that this is a bug in Erlang.

 

From: Bekes, Andras G (IST)
Sent: Tuesday, March 28, 2017 9:55 AM
To: 'Alex S.'
Cc: [hidden email]
Subject: RE: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes

 

I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux.

Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109.

 

From: Alex S. [mailto:[hidden email]]
Sent: Monday, March 27, 2017 8:00 PM
To: Bekes, Andras G (IST)
Cc: [hidden email]
Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes

 

I have a strong suspicion the binary might be automatically zero-terminated.

27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]> написал(а):

 

 

Hi All,

 

I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
http://man7.org/linux/man-pages/man7/unix.7.html

 

Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:

 

> Sun_path_length=108.

108

 

> UDS_path = <<"whatever">>.

<<"whatever">>

 

> Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).

792

 

> Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.

{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,...>>}

 

> Port = 0.

0


> gen_tcp:connect(Address,Port,[local],infinity).

(<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)

(<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

         0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)

(<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,

          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})

(<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}

** exception exit: badarg

     in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)

(<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}

(<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}

 

If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:

connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)

When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.

 

Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.

Seems to me that this is a bug in the Erlang implementation. What do you think?

 

Thank you very much,

 

Andras G. Bekes, Vice President   
Morgan Stanley | Institutional Securities Tech   
Lechner Odon Fasor 6 | Floor 06   
Budapest, 1095   
Phone: +36 1 882-0791   
[hidden email]   
   

 



NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.

 

 

 

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

 





NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: unix domain sockets with abstract namespace: can't use all 108 bytes

Raimo Niskanen-2
On Wed, Apr 19, 2017 at 12:13:26PM +0000, Bekes, Andras G wrote:
> Did anyone have a chance to take a closer look?
>
> I still think that this is a bug in Erlang.

So far no-one have had any good suggestion on how to handle this
in a platform independent way, nor any suggestion on how to detect
this misfeature with configure.  And digging into this is not
on our prioritized list.

I am leaning towards special treatment for this Linux abnormality by
observing the leading 0 and in this special case use all bytes of
the given binary.

Then we might pass a length to the kernel that does not match
the string length of the file path.  But if such a platform uses
zero termination there is no problem, and if it uses
the size information there is no problem, it is only if it uses both
that might be a problem, and that should be very unlikely...

Something like this should do it:
------------------- erts/emulator/drivers/common/inet_drv.c -------------------
index 0fe5183..abcc289 100644
@@ -602,14 +602,6 @@ static size_t my_strnlen(const char *s, size_t maxlen)
     return i;
 }
 
-/* Check that some character in the buffer != '\0' */
-static int is_nonzero(const char *s, size_t n)
-{
-    size_t i;
-    for (i = 0;  i < n;  i++) if (s[i] != '\0') return !0;
-    return 0;
-}
-
 #endif
 
 #ifdef VALGRIND
@@ -4018,7 +4010,15 @@ static char* inet_set_address(int family,
inet_address* dst,
         int n;
         if (*len == 0) return str_einval;
  n = *((unsigned char*)(*src)); /* Length field */
- if ((*len < 1+n) || (sizeof(dst->sal.sun_path) < n+1)) {
+ if (*len < 1+n) return str_einval;
+ /* Portability fix: Make sure the address gets zero terminated
+         * except when the first byte is \0 because then it is sort
+         * of zero terminated although the zero termination comes
+         * before the address...  This fix handles Linux's
+         * abstract socket address nonportable extension.
+ */
+ if (n + ((*len) > 1 && (*src)[1] == '\0' ? 0 : 1)
+            > sizeof(dst->sal.sun_path)) {
     return str_einval;
  }
  sys_memzero((char*)dst, sizeof(struct sockaddr_un));
@@ -4176,8 +4176,7 @@ static int inet_get_address(char* dst, inet_address*
src, unsigned int* len)
  * nonportable extension.
  */
         m = my_strnlen(src->sal.sun_path, n);
- if ((m == 0) && is_nonzero(src->sal.sun_path, n))
-    m = n;
+ if (m == 0)  m = n;
         dst[0] = INET_AF_LOCAL;
         dst[1] = (char) ((unsigned char) m);
         sys_memcpy(dst+2, src->sal.sun_path, m);
@@ -4241,8 +4240,7 @@ inet_address_to_erlang(char *dst, inet_address **src,
SOCKLEN_T sz) {
  * nonportable extension.
  */
         m = my_strnlen((*src)->sal.sun_path, n);
- if ((m == 0) && is_nonzero((*src)->sal.sun_path, n))
-    m = n;
+ if (m == 0)  m = n;
  if (dst) {
     dst[0] = INET_AF_LOCAL;
     dst[1] = (char) ((unsigned char) m);



>
> From: Bekes, Andras G (IST)
> Sent: Tuesday, March 28, 2017 9:55 AM
> To: 'Alex S.'
> Cc: [hidden email]
> Subject: RE: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes
>
> I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux.
> Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109.
>
> From: Alex S. [mailto:[hidden email]]
> Sent: Monday, March 27, 2017 8:00 PM
> To: Bekes, Andras G (IST)
> Cc: [hidden email]
> Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes
>
> I have a strong suspicion the binary might be automatically zero-terminated.
> 27 марта 2017 г., в 19:03, Bekes, Andras G <[hidden email]<mailto:[hidden email]>> написал(а):
>
>
> Hi All,
>
> I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used.
> If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does).
> http://man7.org/linux/man-pages/man7/unix.7.html
>
> Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception:
>
> > Sun_path_length=108.
> 108
>
> > UDS_path = <<"whatever">>.
> <<"whatever">>
>
> > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1).
> 792
>
> > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}.
> {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,...>>}
>
> > Port = 0.
> 0
>
> > gen_tcp:connect(Address,Port,[local],infinity).
> (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity)
> (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>          0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false)
> (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>           0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval})
> (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval}
> ** exception exit: badarg
>      in function  gen_tcp:connect/4 (gen_tcp.erl, line 149)
> (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval}
> (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg}
>
> If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall:
> connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused)
> When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path.
>
> Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108.
> Seems to me that this is a bug in the Erlang implementation. What do you think?
>
> Thank you very much,
>
> Andras G. Bekes, Vice President
> Morgan Stanley | Institutional Securities Tech
> Lechner Odon Fasor 6 | Floor 06
> Budapest, 1095
> Phone: +36 1 882-0791
> [hidden email]<mailto:[hidden email]>
>
>


--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Loading...