Quantcast

Erlang get_tcp:recv data length

classic Classic list List threaded Threaded
25 messages Options
12
shk
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang get_tcp:recv data length

shk
Hello,

I user gen_tcp:recv(Socket, 0). for data receiveng, but i can receive only 1418 bytes for 1 time. How can i receive how much data was sent?

Thank you.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang get_tcp:recv data length

Edward Wang
What's your question again? And by the way, number 1418 is very likely your
MTU value.

-Ed
On Feb 23, 2011 1:14 AM, "shk" <[hidden email]> wrote:
>
> Hello,
>
> I user gen_tcp:recv(Socket, 0). for data receiveng, but i can receive only
> 1418 bytes for 1 time. How can i receive how much data was sent?
>
> Thank you.
> --
> View this message in context:
http://erlang.2086793.n4.nabble.com/Erlang-get-tcp-recv-data-length-tp3319635p3319635.html
> Sent from the Erlang Questions mailing list archive at Nabble.com.
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:[hidden email]
>
shk
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang get_tcp:recv data length

shk
Thank you for reply.

>And by the way, number 1418 is very likely your MTU value.

Then how can i recieve more than 1 packet?

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang get_tcp:recv data length

Roberto Ostinelli
2011/2/22 Alexander Kuleshov <[hidden email]>

> Thank you for reply.
>
> >And by the way, number 1418 is very likely your MTU value.
>
> Then how can i recieve more than 1 packet?
>

..looping?

http://www.erlang.org/doc/man/gen_tcp.html

server() ->
    {ok, LSock} = gen_tcp:listen(5678, [binary, {packet, 0},
                                        {active, false}]),
    {ok, Sock} = gen_tcp:accept(LSock),
    {ok, Bin} = do_recv(Sock, []),
    ok = gen_tcp:close(Sock),
    Bin.
do_recv(Sock, Bs) ->
    case gen_tcp:recv(Sock, 0) of
        {ok, B} ->
            do_recv(Sock, [Bs, B]);
        {error, closed} ->
            {ok, list_to_binary(Bs)}
    end.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang get_tcp:recv data length

Edward Wang
In reply to this post by shk
You recv again. Document http://www.erlang.org/doc/man/gen_tcp.html

Edward.
On Feb 23, 2011 1:43 AM, "Alexander Kuleshov" <[hidden email]>
wrote:

> Thank you for reply.
>
>>And by the way, number 1418 is very likely your MTU value.
>
> Then how can i recieve more than 1 packet?
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:[hidden email]
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: [erlang-questions] Erlang get_tcp:recv data length

info-1679
In reply to this post by shk
What is the criteria of end ? the "tcp close" or a terminator ?
Alexander didn't say how many packets he could receive, the separator(s) between packets, the terminator after the last packet !
And he didn't say if his client closes the connection after the last packet !

Therefore, is your example always valid ?

J-Ph. Constantin
ITS3 Genève
www.its3.ch
shk
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: [erlang-questions] Erlang get_tcp:recv data length

shk
>Alexander didn't say how many packets he could receive, the separator


I don't know how many packets i must receive. I receive smtp must data message
Data
bla bla bla
.
Separator  dot - .\r\n

>And he didn't say if his client closes the connection after the last packet !
The client don't close connection after it.

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

info-1679
Then the solution given by Roberto doesn't solve the problem ! correct or not ?

J-Ph. Constantin
ITS3 Genève
www.its3.ch
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: [erlang-questions] Erlang get_tcp:recv data length

Andrew Thompson-2
In reply to this post by shk
On Wed, Feb 23, 2011 at 03:46:38PM +0000, Alexander Kuleshov wrote:
> >Alexander didn't say how many packets he could receive, the separator
>
>
> I don't know how many packets i must receive. I receive smtp must data message
> Data
> bla bla bla
> .
> Separator  dot - .\r\n
>
Actually, you need to check for \r\n.\r\n, just .\r\n by itself isn't
guranteed to be the end of data.

I receive the message body in gen_smtp by spawning a new process and
looping doing a recv until the buffer contains "\r\n.\r\n", then I
return the resulting binary and any trailing data back to the original
process. I do it in a subprocess to help the garbage collector be able
to clean all the mess up when the subprocess exits.

https://github.com/Vagabond/gen_smtp/blob/master/src/gen_smtp_server_session.erl#L701

The recv size adjustment is kinda ugly, but it seems to work pretty
well.

Also, you can use the SMTP SIZE extension to get a vague idea of how
much data you're going to get. Just don't trust it, its only a
guideline.

Andrew

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

shk
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: [erlang-questions] Erlang get_tcp:recv data length

shk
On Wed, Feb 23, 2011 at 7:29 PM, Andrew Thompson <[hidden email]> wrote:

> On Wed, Feb 23, 2011 at 03:46:38PM +0000, Alexander Kuleshov wrote:
>> >Alexander didn't say how many packets he could receive, the separator
>>
>>
>> I don't know how many packets i must receive. I receive smtp must data message
>> Data
>> bla bla bla
>> .
>> Separator  dot - .\r\n
>>
> Actually, you need to check for \r\n.\r\n, just .\r\n by itself isn't
> guranteed to be the end of data.
>
> I receive the message body in gen_smtp by spawning a new process and
> looping doing a recv until the buffer contains "\r\n.\r\n", then I
> return the resulting binary and any trailing data back to the original
> process. I do it in a subprocess to help the garbage collector be able
> to clean all the mess up when the subprocess exits.
>
> https://github.com/Vagabond/gen_smtp/blob/master/src/gen_smtp_server_session.erl#L701
>
> The recv size adjustment is kinda ugly, but it seems to work pretty
> well.
>
> Also, you can use the SMTP SIZE extension to get a vague idea of how
> much data you're going to get. Just don't trust it, its only a
> guideline.
>
> Andrew
>

First of all thank you for your advices.

>Read in packets one at a time and use the binary module to match/split
>on the end-marker in each new package. If you have accumulated up data
>this way, it should be easy to get a boundary point in the right
>packet and then use the part and all preceeding packets as the
>message.

Yes, i just do it.

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: [erlang-questions] Erlang get_tcp:recv data length

Max Lapshin-2
In fact, using gen_tcp:recv is a bad idea for a "rock solid" server,
because you will not be able to stop process or reload its code.

Proper way is to inet:setops(Socket, [{active,once}]), use buffer and
use erlang:decode_packet to decode lines

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

info-1679
In my case, the client can send packets of messages separated by <cr><lf>.
I don't know how many messages contain each packet (the number can change according to the time).

The solution given as example doesn't solve my problem. How can I do ?
I tried to increase recbuf but I observe (with Network Monitor) that the TCP frames are truncated to 1024 bytes.

J-Ph. Constantin
ITS3 Genève
www.its3.ch
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

info-1679
Be explicit ! How to detect the last message of a packet ?

J-Ph. Constantin
ITS3 Genève
www.its3.ch
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

Johan Wärlander
That's totally application dependent. You must know how to separate
messages, and what marks the end of a number of messages or a user session.
You can never rely on gen_tcp:recv to get the full contents of an
application's message in one go.

It basically comes down to, what does the protocol look like that you're
using?

On Thu, Feb 24, 2011 at 2:25 PM, info <[hidden email]> wrote:

> Be explicit ! How to detect the last message of a packet ?
>
> J-Ph. Constantin
> ITS3 Genève
> www.its3.ch
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

info-1679
I know the "separator" but there is no "end of packet" (according to my sniff).
I make reverse engineering on a device. There is no standard protocol.

J-Ph. Constantin
ITS3 Genève
www.its3.ch
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang get_tcp:recv data length

Kevin A. Smith-2
In reply to this post by info-1679
What about using {packet, line} to receive the data? This would allow you to receive and process each message separately.

--Kevin
On Feb 24, 2011, at 4:11 AM, info wrote:

> In my case, the client can send packets of messages separated by <cr><lf>.
> I don't know how many messages contain each packet (the number can change according to the time).
>
> The solution given as example doesn't solve my problem. How can I do ?
> I tried to increase recbuf but I observe (with Network Monitor) that the TCP frames are truncated to 1024 bytes.
>
> J-Ph. Constantin
> ITS3 Genève
> www.its3.ch


________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

mazenharake
In reply to this post by info-1679
When you create your socket use {active, false} and {packet, line} as
options. This will first of all stop all the receiving messages until
you are ready to accept them and second it will "split" all incoming
data based on lines (I.e. on \r\n or \n etc.).

When you are ready to receive data you do inet:setopts(Socket,
[{active, once}]) which will send you 1 message with the first line of
the message.
If the socket has closed you will get a {tcp_closed, Socket} message.
If the line is too long it will be truncated to max of the packet
buffer (and the ending chars will be absent from the message)

In your case it seems that the lines are too long. If they are, then
the alternative is to recv the data and continously scan what you got
until you reach the delimiter, when you do you keep your own buffer of
received data so that the "extra" data you get after the delimiter
becomes the start of the next. Also as Max pointed out, recv() blocks
so consider using a timeout and recursion to give time inbetween for
updates etc.

Hope this is more clear.

/M



On 24 February 2011 14:25, info <[hidden email]> wrote:
> Be explicit ! How to detect the last message of a packet ?
>
> J-Ph. Constantin
> ITS3 Genève
> www.its3.ch

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

info-1679
In reply to this post by info-1679
Yes, it's more clear ! Thank you. However two questions:
- Is it possible to define another terminator string ? {packet,line} accept only <LF>. If I want to define "<CR><LF>" or "<NULL>" or "#" ?
- I suppose that your explanation is also valid for "receive {tcp,Socket,Data} ->  ... end" ?
Rgds,

J-Ph. Constantin
ITS3 Genève
www.its3.ch
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [erlang-questions] Erlang get_tcp:recv data length

mazenharake
No I think that in that case you have to keep track of it yourself and
so you accept the data packages and accumulate the data and as the
same time search for the delimiter. If you find the delimiter you take
what you have accumulated and send it somewhere and save the rest as
an accumulator for the next data coming in. This should be fairly
straight forward. For this you should not set the packet to line but
rather to binary (or list if you prefer).

My explanation assumes "receive {tcp,Socket,Data} -> ... end", because
you do inet:setopts(Socket, [{active,once}]) right before like this:

foo(Socket, BinAcc) ->
  inet:setopts(Socket, [{active, once}]),
  receive
    {tcp,Socket,Data} ->
      %% search for delimiter here and add to BinAcc... then recurse.
    {tcp_closed,Socket} ->
      %% something else here...
  end.

The only delimiters you can accept are those that are specified in the
packet option in setopts, you can find it here:
http://www.erlang.org/doc/man/inet.html#setopts-2


On 25 February 2011 16:21, info <[hidden email]> wrote:

> Yes, it's more clear ! Thank you. However two questions:
> - Is it possible to define another terminator string ? {packet,line} accept
> only <LF>. If I want to define "<CR><LF>" or "<NULL>" or "#" ?
> - I suppose that your explanation is also valid for "receive
> {tcp,Socket,Data} ->  ... end" ?
> Rgds,
>
> J-Ph. Constantin
> ITS3 Genève
> www.its3.ch

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang get_tcp:recv data length

Alain O'Dea
Mazen, your {active, once} solution supports any delimiter if used in conjuction with {packet, raw}.

J-Ph, it really doesn't matter how big or fragmented the packets are so long as you accumulate fragments until you get a message.  The message boundary could easily be in the middle of a packet, but Mazen's solution makes that easy to handle :)

Here is an example of accumulating binary data:
OldAccBinary = <<"Hello ">>,
Data = <<"World!">>.
BinAcc = <<OldBinAcc/binary, Data/binary>>.



On 2011-02-25, at 12:05, Mazen Harake <[hidden email]> wrote:

> No I think that in that case you have to keep track of it yourself and
> so you accept the data packages and accumulate the data and as the
> same time search for the delimiter. If you find the delimiter you take
> what you have accumulated and send it somewhere and save the rest as
> an accumulator for the next data coming in. This should be fairly
> straight forward. For this you should not set the packet to line but
> rather to binary (or list if you prefer).
>
> My explanation assumes "receive {tcp,Socket,Data} -> ... end", because
> you do inet:setopts(Socket, [{active,once}]) right before like this:
>
> foo(Socket, BinAcc) ->
> inet:setopts(Socket, [{active, once}]),
> receive
>   {tcp,Socket,Data} ->
>     %% search for delimiter here and add to BinAcc... then recurse.
>   {tcp_closed,Socket} ->
>     %% something else here...
> end.
>
> The only delimiters you can accept are those that are specified in the
> packet option in setopts, you can find it here:
> http://www.erlang.org/doc/man/inet.html#setopts-2
>
>
> On 25 February 2011 16:21, info <[hidden email]> wrote:
>> Yes, it's more clear ! Thank you. However two questions:
>> - Is it possible to define another terminator string ? {packet,line} accept
>> only <LF>. If I want to define "<CR><LF>" or "<NULL>" or "#" ?
>> - I suppose that your explanation is also valid for "receive
>> {tcp,Socket,Data} ->  ... end" ?
>> Rgds,
>>
>> J-Ph. Constantin
>> ITS3 Genève
>> www.its3.ch
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:[hidden email]
>


________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

12
Loading...