UDP questions

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

UDP questions

Joe Armstrong-2
Hello,

I have a simple UDP client/server

The server is on a fixed port 4567 - I open it like this

    {ok, Socket} = gen_udp:open(4567, [binary]),

then wait for registrations
    receive
{udp, Socket, _Ip, _Port, <<"register">>) ->


Clients do this:

     {ok, Socket} = gen_udp:open(0, [binary]),
     gen_udp:send(Socket, "localhost", 44567, <<"register">>),
     ...

I'm testing on "localhost"

After registration the processes send binaries to each other

This works fine for small binaries but gen_udp:send fails  with
{error,emsgsize} for large binaries

So I have a few questions:

    1) Can I find out the max binary size that can be sent/received?
    2) Can I increase the max size?
    3) Is there a guaranteed minimum packet length that will not be fragemented?
    3) Can received packets be fragmented - TCP has a  {packet,N}
       option and some built-in defragmentation, what's the story for
       UDP?

I think I know the answers to these but I'm not really sure - could somebody
who really knows enlighten me?

Thanks

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Paul Peregud-2
This boils down to UDP being datagram protocol. It can't fragment
packets by design, since it does not assume that it is operating on
stream of data. Also, every packet is treated as something that can be
dropped.

So it may get dropped by anyone: network router, network adapter,
kernel, or BEAM. On both sending and receiving side. As for reasons
for dropping - packet is too big to fit into single ethernet frame,
buffer (of any device or program) being full, connection being
congested. In general UDP user needs to take care of all those things
by himself.

On bright side - UDP is cheaper and is better in some circumstances
than TCP. I'm not aware though of any libraries that simplify UDP -
but I'm not an expert at all.

On Wed, Jan 27, 2016 at 4:27 PM, Joe Armstrong <[hidden email]> wrote:

> Hello,
>
> I have a simple UDP client/server
>
> The server is on a fixed port 4567 - I open it like this
>
>     {ok, Socket} = gen_udp:open(4567, [binary]),
>
> then wait for registrations
>     receive
> {udp, Socket, _Ip, _Port, <<"register">>) ->
>
>
> Clients do this:
>
>      {ok, Socket} = gen_udp:open(0, [binary]),
>      gen_udp:send(Socket, "localhost", 44567, <<"register">>),
>      ...
>
> I'm testing on "localhost"
>
> After registration the processes send binaries to each other
>
> This works fine for small binaries but gen_udp:send fails  with
> {error,emsgsize} for large binaries
>
> So I have a few questions:
>
>     1) Can I find out the max binary size that can be sent/received?
>     2) Can I increase the max size?
>     3) Is there a guaranteed minimum packet length that will not be fragemented?
>     3) Can received packets be fragmented - TCP has a  {packet,N}
>        option and some built-in defragmentation, what's the story for
>        UDP?
>
> I think I know the answers to these but I'm not really sure - could somebody
> who really knows enlighten me?
>
> Thanks
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions



--
Best regards,
Paul Peregud
+48602112091
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Antonio SJ Musumeci
I think in practice, given what Paul mentioned, it can't really be explicitly known or queried. Perhaps a simple handshake between the client and server could be done which tries to establish a practical msg size?

On Wed, Jan 27, 2016 at 10:43 AM, Paul Peregud <[hidden email]> wrote:
This boils down to UDP being datagram protocol. It can't fragment
packets by design, since it does not assume that it is operating on
stream of data. Also, every packet is treated as something that can be
dropped.

So it may get dropped by anyone: network router, network adapter,
kernel, or BEAM. On both sending and receiving side. As for reasons
for dropping - packet is too big to fit into single ethernet frame,
buffer (of any device or program) being full, connection being
congested. In general UDP user needs to take care of all those things
by himself.

On bright side - UDP is cheaper and is better in some circumstances
than TCP. I'm not aware though of any libraries that simplify UDP -
but I'm not an expert at all.

On Wed, Jan 27, 2016 at 4:27 PM, Joe Armstrong <[hidden email]> wrote:
> Hello,
>
> I have a simple UDP client/server
>
> The server is on a fixed port 4567 - I open it like this
>
>     {ok, Socket} = gen_udp:open(4567, [binary]),
>
> then wait for registrations
>     receive
> {udp, Socket, _Ip, _Port, <<"register">>) ->
>
>
> Clients do this:
>
>      {ok, Socket} = gen_udp:open(0, [binary]),
>      gen_udp:send(Socket, "localhost", 44567, <<"register">>),
>      ...
>
> I'm testing on "localhost"
>
> After registration the processes send binaries to each other
>
> This works fine for small binaries but gen_udp:send fails  with
> {error,emsgsize} for large binaries
>
> So I have a few questions:
>
>     1) Can I find out the max binary size that can be sent/received?
>     2) Can I increase the max size?
>     3) Is there a guaranteed minimum packet length that will not be fragemented?
>     3) Can received packets be fragmented - TCP has a  {packet,N}
>        option and some built-in defragmentation, what's the story for
>        UDP?
>
> I think I know the answers to these but I'm not really sure - could somebody
> who really knows enlighten me?
>
> Thanks
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions



--
Best regards,
Paul Peregud
<a href="tel:%2B48602112091" value="+48602112091">+48602112091
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

zxq9-2
In reply to this post by Joe Armstrong-2
Hi Joe.

tl;dr: Problems with UDP boil down to limitations of the protocol.

On 2016年1月27日 水曜日 16:27:13 Joe Armstrong wrote:

> Hello,
>
> I have a simple UDP client/server
>
> The server is on a fixed port 4567 - I open it like this
>
>     {ok, Socket} = gen_udp:open(4567, [binary]),
>
> then wait for registrations
>     receive
> {udp, Socket, _Ip, _Port, <<"register">>) ->
>
>
> Clients do this:
>
>      {ok, Socket} = gen_udp:open(0, [binary]),
>      gen_udp:send(Socket, "localhost", 44567, <<"register">>),
>      ...
>
> I'm testing on "localhost"
>
> After registration the processes send binaries to each other
>
> This works fine for small binaries but gen_udp:send fails  with
> {error,emsgsize} for large binaries
>
> So I have a few questions:
>
>     1) Can I find out the max binary size that can be sent/received?
>     2) Can I increase the max size?
>     3) Is there a guaranteed minimum packet length that will not be fragemented?
>     3) Can received packets be fragmented - TCP has a  {packet,N}
>        option and some built-in defragmentation, what's the story for
>        UDP?
>
> I think I know the answers to these but I'm not really sure - could somebody
> who really knows enlighten me?

The max datagram size is supposed to be discoverable with GET_MAXSIZES() as
per RFC 1122 4.1.4 ( https://tools.ietf.org/html/rfc1122#page-79 ) but in
practice I've never seen anyone actually query this. Discovery, at least in
the game server world, tends to happen using ICMP because the question is
almost never "what is the max datagram size only to localhost".

Generally speaking you've got 512 - 8 bytes of safe space to use in IPv4
UDP -- though over the localhost... I'm not really sure what the limit would
be; but you should be able to configure the local interface to be some
almost arbitrarily large jumboframe size.

512 - 8 is a guideline based on MTU based fragmentation. I'm assuming you
want the code to not care whether its all localhost or if the system is
extended across the network. ?

As far as breaking larger things up into several pieces, you either wind
up reinventing TCP streams in chunks, or reinventing SCTP sequential packet
mode. UDP will never guarantee the ordering or receipt of any given message.
I've never lost a UDP packet to localhost, but I'm pretty sure you could
create a system load that would cause this to happen.

I think really the ideal protocol is SCTP. Each association in sequential
packet mode is like a UDP datagram, but ordered and guaranteed to be
received (deals with the fragmentation and ordering problem for you),
and any association can be "peeled off" into a stream socket connection
when you need a stream of bytes (like for transferring a large binary
blob). Also, a single connection can be one-to-many, and each connection
can have several channels -- thus also solving the problem of needing
several bands.

But only geeks even know about SCTP and its not supported everywhere, so
that might put a dent in the plan; I assume this is in the context of the
blog posts you just put up about stitching together program components
over UDP sockets.

You're either going to need multiple channels (UDP for your control messages
and TCP connections for large transfer), or a multi-channel connection
protocol like SCTP to fully accomplish what you're trying to do.

This approach, incidentally, is my preferred way of thinking about system
design -- and it makes me so sad that people think HTTP is the only protocol
that exists! (And JSON and XML are the only serialization formats!)

People just don't remember how to program sockets today -- and we don't make
it better by ignoring the problem since it is so simple in Erlang, but can
suck so hard and confusing in many other languages. Its pretty obvious why
more programs don't support things like IPv6... nobody wants to touch the
networking code if they can avoid it. (Much easier to just make an "API"
over HTTP and pray that one day the web will be magically secure, bandwidth
will be infinite and FavoriteWebServer will support feature X that you
really wish you could use -- in other words, easier to stick one's head in
the sand.)

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Pagayon, Ruel
In reply to this post by Joe Armstrong-2
>> 1) Can I find out the max binary size that can be sent/received?

I believe you're getting that error because you exceeded the default value of the receive buffer, which is 8192 bytes. It's on the gen_udp doc.

>> 2) Can I increase the max size?

You can set the option {recbuf, ...} to change the max size.

>> 3) Is there a guaranteed minimum packet length that will not be fragemented?
>> 3) Can received packets be fragmented - TCP has a  {packet,N}
>>        option and some built-in defragmentation, what's the story for
>>        UDP?

In the network layer in general, both TCP and UDP can be sent through an interface, disassembled, and re-assembled at its destination. This can happen if the packet size exceeds the interface's MTU (e.g. 1500).

In the application layer, with the {packet, N} option is viable in TCP because it's reliable and it has flow control. UDP can drop packets along the way and can be received out of order.

Cheers,
Ruel


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Serge Aleynikov-3
In reply to this post by Joe Armstrong-2
In addition to what others have said, the max UDP binary size that can be sent/received is:
64k - (sizeof(IPHeader) + sizeof(UDPHeader)) = 65535- (20 + 8) = 65507.  This is the UDP protocol's limitation.  I would say if you use UDP, the {packet, N} option makes little sense, since each udp message is delivered as a whole (UDP is message oriented vs. TCP that is byte-oriented).

I see between your 3rd and 4th questions there's some confusion. UDP fragmentation may happen at the network layer (IP) when sending a large datagram (> MTU size), but it's defragmented by the network stack, and delivered to the transport layer (UDP) and consequently the user space as a whole message (preserving message boundaries).

Regards,

Serge

On Wed, Jan 27, 2016 at 10:27 AM, Joe Armstrong <[hidden email]> wrote:
Hello,

I have a simple UDP client/server

The server is on a fixed port 4567 - I open it like this

    {ok, Socket} = gen_udp:open(4567, [binary]),

then wait for registrations
    receive
{udp, Socket, _Ip, _Port, <<"register">>) ->


Clients do this:

     {ok, Socket} = gen_udp:open(0, [binary]),
     gen_udp:send(Socket, "localhost", 44567, <<"register">>),
     ...

I'm testing on "localhost"

After registration the processes send binaries to each other

This works fine for small binaries but gen_udp:send fails  with
{error,emsgsize} for large binaries

So I have a few questions:

    1) Can I find out the max binary size that can be sent/received?
    2) Can I increase the max size?
    3) Is there a guaranteed minimum packet length that will not be fragemented?
    3) Can received packets be fragmented - TCP has a  {packet,N}
       option and some built-in defragmentation, what's the story for
       UDP?

I think I know the answers to these but I'm not really sure - could somebody
who really knows enlighten me?

Thanks

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Artie Gold
If, of course, it arrives at all. Larger messages, spanning several MTUs are more likely to be dropped.

Sent from my iPhone

On Jan 27, 2016, at 7:24 PM, Serge Aleynikov <[hidden email]> wrote:

In addition to what others have said, the max UDP binary size that can be sent/received is:
64k - (sizeof(IPHeader) + sizeof(UDPHeader)) = 65535- (20 + 8) = 65507.  This is the UDP protocol's limitation.  I would say if you use UDP, the {packet, N} option makes little sense, since each udp message is delivered as a whole (UDP is message oriented vs. TCP that is byte-oriented).

I see between your 3rd and 4th questions there's some confusion. UDP fragmentation may happen at the network layer (IP) when sending a large datagram (> MTU size), but it's defragmented by the network stack, and delivered to the transport layer (UDP) and consequently the user space as a whole message (preserving message boundaries).

Regards,

Serge

On Wed, Jan 27, 2016 at 10:27 AM, Joe Armstrong <[hidden email]> wrote:
Hello,

I have a simple UDP client/server

The server is on a fixed port 4567 - I open it like this

    {ok, Socket} = gen_udp:open(4567, [binary]),

then wait for registrations
    receive
{udp, Socket, _Ip, _Port, <<"register">>) ->


Clients do this:

     {ok, Socket} = gen_udp:open(0, [binary]),
     gen_udp:send(Socket, "localhost", 44567, <<"register">>),
     ...

I'm testing on "localhost"

After registration the processes send binaries to each other

This works fine for small binaries but gen_udp:send fails  with
{error,emsgsize} for large binaries

So I have a few questions:

    1) Can I find out the max binary size that can be sent/received?
    2) Can I increase the max size?
    3) Is there a guaranteed minimum packet length that will not be fragemented?
    3) Can received packets be fragmented - TCP has a  {packet,N}
       option and some built-in defragmentation, what's the story for
       UDP?

I think I know the answers to these but I'm not really sure - could somebody
who really knows enlighten me?

Thanks

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Felix Gallo-2
In reply to this post by Serge Aleynikov-3
I've used OSC pretty heavily with Erlang recently, and:

1) use wired connections where possible; wireless connections experience significantly higher packet loss, and at least two different models of wireless router displayed unusual behavior in which a steady train of UDP packets would get increasingly geometrically delayed for no apparent reason.  

2) consult musician forums about wireless router brand / model / firmware combinations that seem to work with the specific tools you want to use, if you are going to use something like an iPad with TouchOSC or Lemur etc. as a manual controller.  

3) use OSC bundles where possible; not all languages have libraries which are good at that particular section of the spec, but properly implemented bundles can significantly improve sync and a feeling of snappiness in non-musical applications, such as higher level control.

4) Limit your packet sizes to 1500 bytes or lower on a LAN.  For single messages this is usually pretty easy unless you're trying to send data instead of commands, which I don't recommend.  You can breach the limit if you pack a bunch of commands into a bundle, but if your application layer is smart, it can fill up an outgoing packet to 500 bytes or whatever, and then save off any further messages for the next packet.  With good switches, good PHYs (e.g. Intel), good OS with well-tuned buffers or good defaults, you should be able to run at 100Mb line rate.  In the real world, sometimes you don't get to pick all of those things, so having your OSC code be tunable (or self-tunable) down in packet size and packet rate is a good practice.

5) OSC is super awesome because there's a bunch of controllers, libraries, real time graphical monitors, etc., for it, and it's pretty familiar in the entertainment space, but it's also pretty verbose and hefty on the wire.  Cap'n Proto, protobufs and thrift are nearby cousins that deserve a look or two if you're really pushing OSC out of its comfort envelope.

F.


On Wed, Jan 27, 2016 at 5:24 PM, Serge Aleynikov <[hidden email]> wrote:
In addition to what others have said, the max UDP binary size that can be sent/received is:
64k - (sizeof(IPHeader) + sizeof(UDPHeader)) = 65535- (20 + 8) = 65507.  This is the UDP protocol's limitation.  I would say if you use UDP, the {packet, N} option makes little sense, since each udp message is delivered as a whole (UDP is message oriented vs. TCP that is byte-oriented).

I see between your 3rd and 4th questions there's some confusion. UDP fragmentation may happen at the network layer (IP) when sending a large datagram (> MTU size), but it's defragmented by the network stack, and delivered to the transport layer (UDP) and consequently the user space as a whole message (preserving message boundaries).

Regards,

Serge

On Wed, Jan 27, 2016 at 10:27 AM, Joe Armstrong <[hidden email]> wrote:
Hello,

I have a simple UDP client/server

The server is on a fixed port 4567 - I open it like this

    {ok, Socket} = gen_udp:open(4567, [binary]),

then wait for registrations
    receive
{udp, Socket, _Ip, _Port, <<"register">>) ->


Clients do this:

     {ok, Socket} = gen_udp:open(0, [binary]),
     gen_udp:send(Socket, "localhost", 44567, <<"register">>),
     ...

I'm testing on "localhost"

After registration the processes send binaries to each other

This works fine for small binaries but gen_udp:send fails  with
{error,emsgsize} for large binaries

So I have a few questions:

    1) Can I find out the max binary size that can be sent/received?
    2) Can I increase the max size?
    3) Is there a guaranteed minimum packet length that will not be fragemented?
    3) Can received packets be fragmented - TCP has a  {packet,N}
       option and some built-in defragmentation, what's the story for
       UDP?

I think I know the answers to these but I'm not really sure - could somebody
who really knows enlighten me?

Thanks

/Joe
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

zxq9-2
In reply to this post by Artie Gold
On 2016年1月27日 水曜日 19:34:02 Artie Gold wrote:
> If, of course, it arrives at all. Larger messages, spanning several MTUs
> are more likely to be dropped.

Exactly.

That's why we tend to test for MTU and then stay below it, with the
generally safe (and blind) guideline being 512 bytes total. This is
pretty small, but control and status update messages tend not to be
anywhere near this large in practice. Reliable transfer of data has
to happen on a separate TCP connection, perhaps created in response
to a UDP control message.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Andreas Schultz
In reply to this post by Serge Aleynikov-3
Hi,

On 01/28/2016 02:24 AM, Serge Aleynikov wrote:
> In addition to what others have said, the max UDP binary size that can be sent/received is:
> 64k - (sizeof(IPHeader) + sizeof(UDPHeader)) = 65535- (20 + 8) = 65507.  This is the UDP protocol's limitation.  I would say if you use UDP,
> the {packet, N} option makes little sense, since each udp message is delivered as a whole (UDP is message oriented vs. TCP that is
> byte-oriented).
>
> I see between your 3rd and 4th questions there's some confusion. UDP fragmentation may happen at the network layer (IP) when sending a large
> datagram (> MTU size), but it's defragmented by the network stack, and delivered to the transport layer (UDP) and consequently the user
> space as a whole message (preserving message boundaries).

What you describe here is happens when the DF (don't fragment) bit in the IP header is not set.

When the DF bit set, the first hop (which might be your local host) with a MTU smaller that your datagram size, will send an ICMP error with
the reason set to "fragmentation needed". This ICMP will also include the largest MTU that this host can pass on. This procedure is also
known as Path MTU discovery (PMTU disc).

Whether your local host actually gets the ICMP error, depends on the (mis-)configuration of firewalls between the reporting hop and your
host. This is what makes PMTU discovery somewhat unreliable.

If your sending host does indeed get the ICMP report, it will usually store the new PMTU for the original destination IP. On Linux you
should be able to inquire that value through the IP_MTU socket option (see 'man 7 ip' on Linux).

A UDP datagram that was dropped due to a fragmentation needed error, even when your host did get the ICMP error, is not resent and you will
not get a error from the socket. Only when you attempt to send another packet that is above the PMTU will you get a error from the socket.

Note: The UDP socket error behaviour description above is what Erlang supports. At least on Linux you could configure the socket to report
errors as OOB data and poll/select for that data. This error data would then need to be decode and forwarded to the socket owner as
udp_error report. I have no idea if/how this work on other OSes.

On Linux, UDP sockets are opened with systems defaults for the DF bit. In most cases this means the DF will be set on outgoing UDP
datagrams. So any non localhost communication this will mean, that in the best case, your maximum usable UDP payload size will be 1472 (1500
- 20 - 8), the guaranteed minimum for IPv4 is 548 (576 - 20 - 8) and for IPv6 around 1200 (minimum MTU is 1280, but IPv6 header size can vary).

It would be absolutely fantastic if Erlang would support proper PMTU discovery and error reporting on all supported OSes. I could contribute
an Linux only version if there is a chance of having this as an officially supported feature.

Regards,
Andreas

>
> Regards,
>
> Serge
>
> On Wed, Jan 27, 2016 at 10:27 AM, Joe Armstrong <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hello,
>
>     I have a simple UDP client/server
>
>     The server is on a fixed port 4567 - I open it like this
>
>          {ok, Socket} = gen_udp:open(4567, [binary]),
>
>     then wait for registrations
>          receive
>     {udp, Socket, _Ip, _Port, <<"register">>) ->
>
>
>     Clients do this:
>
>           {ok, Socket} = gen_udp:open(0, [binary]),
>           gen_udp:send(Socket, "localhost", 44567, <<"register">>),
>           ...
>
>     I'm testing on "localhost"
>
>     After registration the processes send binaries to each other
>
>     This works fine for small binaries but gen_udp:send fails  with
>     {error,emsgsize} for large binaries
>
>     So I have a few questions:
>
>          1) Can I find out the max binary size that can be sent/received?
>          2) Can I increase the max size?
>          3) Is there a guaranteed minimum packet length that will not be fragemented?
>          3) Can received packets be fragmented - TCP has a  {packet,N}
>             option and some built-in defragmentation, what's the story for
>             UDP?
>
>     I think I know the answers to these but I'm not really sure - could somebody
>     who really knows enlighten me?
>
>     Thanks
>
>     /Joe
>     _______________________________________________
>     erlang-questions mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Joe Armstrong-2
In reply to this post by Felix Gallo-2
On Thu, Jan 28, 2016 at 2:59 AM, Felix Gallo <[hidden email]> wrote:

> I've used OSC pretty heavily with Erlang recently, and:
>
> 1) use wired connections where possible; wireless connections experience
> significantly higher packet loss, and at least two different models of
> wireless router displayed unusual behavior in which a steady train of UDP
> packets would get increasingly geometrically delayed for no apparent reason.
>
> 2) consult musician forums about wireless router brand / model / firmware
> combinations that seem to work with the specific tools you want to use, if
> you are going to use something like an iPad with TouchOSC or Lemur etc. as a
> manual controller.

I want to connect iPad/Mac/various synths together. I hadn't heard of Lemur

I just got a midi-to-osc bridge working (midi half in swift) - osc-in erlang :-)

Just need bonjour and get http://nexusosc.com/ working then I can have fun ...


>
> 3) use OSC bundles where possible; not all languages have libraries which
> are good at that particular section of the spec, but properly implemented
> bundles can significantly improve sync and a feeling of snappiness in
> non-musical applications, such as higher level control.
>
> 4) Limit your packet sizes to 1500 bytes or lower on a LAN.  For single
> messages this is usually pretty easy unless you're trying to send data
> instead of commands, which I don't recommend.  You can breach the limit if
> you pack a bunch of commands into a bundle, but if your application layer is
> smart, it can fill up an outgoing packet to 500 bytes or whatever, and then
> save off any further messages for the next packet.  With good switches, good
> PHYs (e.g. Intel), good OS with well-tuned buffers or good defaults, you
> should be able to run at 100Mb line rate.  In the real world, sometimes you
> don't get to pick all of those things, so having your OSC code be tunable
> (or self-tunable) down in packet size and packet rate is a good practice.
>
> 5) OSC is super awesome because there's a bunch of controllers, libraries,
> real time graphical monitors, etc., for it, and it's pretty familiar in the
> entertainment space, but it's also pretty verbose and hefty on the wire.
> Cap'n Proto, protobufs and thrift are nearby cousins that deserve a look or
> two if you're really pushing OSC out of its comfort envelope.
>

I agree - the music world has a load of very cool devices - I've been
playing with Pure Data and my head hurts

/Joe


> F.
>
>
> On Wed, Jan 27, 2016 at 5:24 PM, Serge Aleynikov <[hidden email]>
> wrote:
>>
>> In addition to what others have said, the max UDP binary size that can be
>> sent/received is:
>> 64k - (sizeof(IPHeader) + sizeof(UDPHeader)) = 65535- (20 + 8) = 65507.
>> This is the UDP protocol's limitation.  I would say if you use UDP, the
>> {packet, N} option makes little sense, since each udp message is delivered
>> as a whole (UDP is message oriented vs. TCP that is byte-oriented).
>>
>> I see between your 3rd and 4th questions there's some confusion. UDP
>> fragmentation may happen at the network layer (IP) when sending a large
>> datagram (> MTU size), but it's defragmented by the network stack, and
>> delivered to the transport layer (UDP) and consequently the user space as a
>> whole message (preserving message boundaries).
>>
>> Regards,
>>
>> Serge
>>
>> On Wed, Jan 27, 2016 at 10:27 AM, Joe Armstrong <[hidden email]> wrote:
>>>
>>> Hello,
>>>
>>> I have a simple UDP client/server
>>>
>>> The server is on a fixed port 4567 - I open it like this
>>>
>>>     {ok, Socket} = gen_udp:open(4567, [binary]),
>>>
>>> then wait for registrations
>>>     receive
>>> {udp, Socket, _Ip, _Port, <<"register">>) ->
>>>
>>>
>>> Clients do this:
>>>
>>>      {ok, Socket} = gen_udp:open(0, [binary]),
>>>      gen_udp:send(Socket, "localhost", 44567, <<"register">>),
>>>      ...
>>>
>>> I'm testing on "localhost"
>>>
>>> After registration the processes send binaries to each other
>>>
>>> This works fine for small binaries but gen_udp:send fails  with
>>> {error,emsgsize} for large binaries
>>>
>>> So I have a few questions:
>>>
>>>     1) Can I find out the max binary size that can be sent/received?
>>>     2) Can I increase the max size?
>>>     3) Is there a guaranteed minimum packet length that will not be
>>> fragemented?
>>>     3) Can received packets be fragmented - TCP has a  {packet,N}
>>>        option and some built-in defragmentation, what's the story for
>>>        UDP?
>>>
>>> I think I know the answers to these but I'm not really sure - could
>>> somebody
>>> who really knows enlighten me?
>>>
>>> Thanks
>>>
>>> /Joe
>>> _______________________________________________
>>> erlang-questions mailing list
>>> [hidden email]
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: UDP questions

Joe Armstrong-2
In reply to this post by Serge Aleynikov-3
On Thu, Jan 28, 2016 at 2:24 AM, Serge Aleynikov <[hidden email]> wrote:

> In addition to what others have said, the max UDP binary size that can be
> sent/received is:
> 64k - (sizeof(IPHeader) + sizeof(UDPHeader)) = 65535- (20 + 8) = 65507.
> This is the UDP protocol's limitation.  I would say if you use UDP, the
> {packet, N} option makes little sense, since each udp message is delivered
> as a whole (UDP is message oriented vs. TCP that is byte-oriented).
>
> I see between your 3rd and 4th questions there's some confusion. UDP
> fragmentation may happen at the network layer (IP) when sending a large
> datagram (> MTU size), but it's defragmented by the network stack, and
> delivered to the transport layer (UDP) and consequently the user space as a
> whole message (preserving message boundaries).

Yes - it was my understanding that the UDP packet if it is received
by the application is not fragmented - but I wasn't sure.

/Joe

>
> Regards,
>
> Serge
>
> On Wed, Jan 27, 2016 at 10:27 AM, Joe Armstrong <[hidden email]> wrote:
>>
>> Hello,
>>
>> I have a simple UDP client/server
>>
>> The server is on a fixed port 4567 - I open it like this
>>
>>     {ok, Socket} = gen_udp:open(4567, [binary]),
>>
>> then wait for registrations
>>     receive
>> {udp, Socket, _Ip, _Port, <<"register">>) ->
>>
>>
>> Clients do this:
>>
>>      {ok, Socket} = gen_udp:open(0, [binary]),
>>      gen_udp:send(Socket, "localhost", 44567, <<"register">>),
>>      ...
>>
>> I'm testing on "localhost"
>>
>> After registration the processes send binaries to each other
>>
>> This works fine for small binaries but gen_udp:send fails  with
>> {error,emsgsize} for large binaries
>>
>> So I have a few questions:
>>
>>     1) Can I find out the max binary size that can be sent/received?
>>     2) Can I increase the max size?
>>     3) Is there a guaranteed minimum packet length that will not be
>> fragemented?
>>     3) Can received packets be fragmented - TCP has a  {packet,N}
>>        option and some built-in defragmentation, what's the story for
>>        UDP?
>>
>> I think I know the answers to these but I'm not really sure - could
>> somebody
>> who really knows enlighten me?
>>
>> Thanks
>>
>> /Joe
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions