Are you using {packet, http} ?

[Sverker Eriksson]

A message from the OTP team:

There is an undocumented socket packet mode that provides HTTP parsing.
We are planning to make this packet mode official and possibly also
change the format of the tuples returned in this mode. One current big
user is the web server Yaws. We would like to know of other applications
using this HTTP mode and what kind of impact such an incompatible change
would have.

A socket using {packet, http} returns tuples like this in *passive* mode:

{ok, {http_request, ...}}
{ok, {http_response, ...}}
{ok, {http_header, ...}}


and tuples like this in *active* mode:

{http_request, Socket, ...}
{http_response, Socket, ...}
{http_header, Socket, ...}


The proposed change would only affect the active mode like this:

{http, Socket, {http_request, ...}}
{http, Socket, {http_response, ...}}
{http, Socket, {http_header, ...}}
 

The purpose is to make the inner tuples look the same regardless of how
they were received. This would simplify both the implementation and the
documentation as well as any applications using both receive modes.
Applications only using http in passive mode (like Yaws) will not be
affected.

Am I stirring up any worried http-users out there?

/Sverker, Erlang/OTP Ericsson

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Hans Hübner-3]

On Mon, Jul 14, 2008 at 16:20, Sverker Eriksson
<[hidden email]> wrote:
> A message from the OTP team:
>
> There is an undocumented socket packet mode that provides HTTP parsing.
[...]
> Am I stirring up any worried http-users out there?

I am currently looking into Erlang because I need a web server front
end that multiplexes many external connections to one HTTP backend
server on a single persistent HTTP/1.1 connection.  I was about to ask
in this list if there are any http client and server libraries that I
could use, and if I understand your message correctly, the HTTP packet
modes are just what I need.  So:  I can't be stirred up, but I'd
really like to play around with this, so if you can point me to source
code or anything else that might help me understand of to put sockets
into HTTP packet mode, I'd greatly appreciate it.

Thanks,
Hans
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Justin Sheehy]

Sverker,

Mochiweb also uses the undocumented http mode for sockets.

I think (offhand, without checking) that it uses it in passive mode and
wouldn't care about the change, but you may want to get one of the mochiweb
developers would need to chime in as an authoritative voice.  But as a heavy
user of mochiweb I wanted to make sure that you were aware of it as another
implementation relying on this.

I'm very glad you're making it official.  It's a great feature.

Thanks!

-Justin



On 7/14/08 10:20 AM, "Sverker Eriksson" <[hidden email]> wrote:

> A message from the OTP team:
>
> There is an undocumented socket packet mode that provides HTTP parsing.
> We are planning to make this packet mode official and possibly also
> change the format of the tuples returned in this mode. One current big
> user is the web server Yaws. We would like to know of other applications
> using this HTTP mode and what kind of impact such an incompatible change
> would have.
>
> A socket using {packet, http} returns tuples like this in *passive* mode:
>
> {ok, {http_request, ...}}
> {ok, {http_response, ...}}
> {ok, {http_header, ...}}
>
>
> and tuples like this in *active* mode:
>
> {http_request, Socket, ...}
> {http_response, Socket, ...}
> {http_header, Socket, ...}
>
>
> The proposed change would only affect the active mode like this:
>
> {http, Socket, {http_request, ...}}
> {http, Socket, {http_response, ...}}
> {http, Socket, {http_header, ...}}
>
>
> The purpose is to make the inner tuples look the same regardless of how
> they were received. This would simplify both the implementation and the
> documentation as well as any applications using both receive modes.
> Applications only using http in passive mode (like Yaws) will not be
> affected.
>
> Am I stirring up any worried http-users out there?
>
> /Sverker, Erlang/OTP Ericsson
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Joseph Wayne Norton]

Just in case you haven't seen this article on trapexit.  Please see here:

http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_features

On Mon, 14 Jul 2008 23:41:29 +0900, Hans Huebner <[hidden email]>  
wrote:

> On Mon, Jul 14, 2008 at 16:20, Sverker Eriksson
> <[hidden email]> wrote:
>> A message from the OTP team:
>>
>> There is an undocumented socket packet mode that provides HTTP parsing.
> [...]
>> Am I stirring up any worried http-users out there?
>
> I am currently looking into Erlang because I need a web server front
> end that multiplexes many external connections to one HTTP backend
> server on a single persistent HTTP/1.1 connection.  I was about to ask
> in this list if there are any http client and server libraries that I
> could use, and if I understand your message correctly, the HTTP packet
> modes are just what I need.  So:  I can't be stirred up, but I'd
> really like to play around with this, so if you can point me to source
> code or anything else that might help me understand of to put sockets
> into HTTP packet mode, I'd greatly appreciate it.
>
> Thanks,
> Hans
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-questions

--
[hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Christian S-2]

> Am I stirring up any worried http-users out there?

I checked "iserve", it does passive http sockets.

If it does active once somewhere that is a very simple change anyway.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Richard Kelsall]

Sverker Eriksson wrote:
> There is an undocumented socket packet mode that provides HTTP parsing.
> We are planning to make this packet mode official and possibly also
> change the format of the tuples returned in this mode.
...
> Am I stirring up any worried http-users out there?

This is great. Thank you. I am not using it yet but have played around
with the simple program here :

http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_features

that I think uses it :

"The notable thing about this code is the use of undocumented socket
options to set up the initial state of connections made to the web
server port.
{backlog, 30} specifies the length of the OS accept queue.
{packet, http} puts the socket into http mode. ..."

I hope to get round to using this program at some point. I vote in
favour of any improvements you want to make.


Richard.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Bob Ippolito]

mochiweb does indeed use {packet, http} only in passive mode.

On Mon, Jul 14, 2008 at 7:52 AM, Justin Sheehy <[hidden email]> wrote:

> Sverker,
>
> Mochiweb also uses the undocumented http mode for sockets.
>
> I think (offhand, without checking) that it uses it in passive mode and
> wouldn't care about the change, but you may want to get one of the mochiweb
> developers would need to chime in as an authoritative voice.  But as a heavy
> user of mochiweb I wanted to make sure that you were aware of it as another
> implementation relying on this.
>
> I'm very glad you're making it official.  It's a great feature.
>
> Thanks!
>
> -Justin
>
>
>
> On 7/14/08 10:20 AM, "Sverker Eriksson" <[hidden email]> wrote:
>
>> A message from the OTP team:
>>
>> There is an undocumented socket packet mode that provides HTTP parsing.
>> We are planning to make this packet mode official and possibly also
>> change the format of the tuples returned in this mode. One current big
>> user is the web server Yaws. We would like to know of other applications
>> using this HTTP mode and what kind of impact such an incompatible change
>> would have.
>>
>> A socket using {packet, http} returns tuples like this in *passive* mode:
>>
>> {ok, {http_request, ...}}
>> {ok, {http_response, ...}}
>> {ok, {http_header, ...}}
>>
>>
>> and tuples like this in *active* mode:
>>
>> {http_request, Socket, ...}
>> {http_response, Socket, ...}
>> {http_header, Socket, ...}
>>
>>
>> The proposed change would only affect the active mode like this:
>>
>> {http, Socket, {http_request, ...}}
>> {http, Socket, {http_response, ...}}
>> {http, Socket, {http_header, ...}}
>>
>>
>> The purpose is to make the inner tuples look the same regardless of how
>> they were received. This would simplify both the implementation and the
>> documentation as well as any applications using both receive modes.
>> Applications only using http in passive mode (like Yaws) will not be
>> affected.
>>
>> Am I stirring up any worried http-users out there?
>>
>> /Sverker, Erlang/OTP Ericsson
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Valentin Micic]

I'm so glad taht I haven't been aware of this before. So glad that I am now
;-).

V.

PS
Hope that {active, once} will be supported as per usual.


----- Original Message -----
From: "Sverker Eriksson" <[hidden email]>
To: "erlang-questions" <[hidden email]>
Sent: Monday, July 14, 2008 4:20 PM
Subject: [erlang-questions] Are you using {packet, http} ?

>A message from the OTP team:
>
> There is an undocumented socket packet mode that provides HTTP parsing.
> We are planning to make this packet mode official and possibly also
> change the format of the tuples returned in this mode. One current big
> user is the web server Yaws. We would like to know of other applications
> using this HTTP mode and what kind of impact such an incompatible change
> would have.
>
> A socket using {packet, http} returns tuples like this in *passive* mode:
>
> {ok, {http_request, ...}}
> {ok, {http_response, ...}}
> {ok, {http_header, ...}}
>
>
> and tuples like this in *active* mode:
>
> {http_request, Socket, ...}
> {http_response, Socket, ...}
> {http_header, Socket, ...}
>
>
> The proposed change would only affect the active mode like this:
>
> {http, Socket, {http_request, ...}}
> {http, Socket, {http_response, ...}}
> {http, Socket, {http_header, ...}}
>
>
> The purpose is to make the inner tuples look the same regardless of how
> they were received. This would simplify both the implementation and the
> documentation as well as any applications using both receive modes.
> Applications only using http in passive mode (like Yaws) will not be
> affected.
>
> Am I stirring up any worried http-users out there?
>
> /Sverker, Erlang/OTP Ericsson
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-questions 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Sean Hinde]

The tutorial can be updated easily enough if needs be.

Sean

On 14 Jul 2008, at 16:07, Joseph Wayne Norton wrote:

> Just in case you haven't seen this article on trapexit.  Please see  
> here:
>
> http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_features
>
> On Mon, 14 Jul 2008 23:41:29 +0900, Hans Huebner <[hidden email]
> >
> wrote:
>
>> On Mon, Jul 14, 2008 at 16:20, Sverker Eriksson
>> <[hidden email]> wrote:
>>> A message from the OTP team:
>>>
>>> There is an undocumented socket packet mode that provides HTTP  
>>> parsing.
>> [...]
>>> Am I stirring up any worried http-users out there?
>>
>> I am currently looking into Erlang because I need a web server front
>> end that multiplexes many external connections to one HTTP backend
>> server on a single persistent HTTP/1.1 connection.  I was about to  
>> ask
>> in this list if there are any http client and server libraries that I
>> could use, and if I understand your message correctly, the HTTP  
>> packet
>> modes are just what I need.  So:  I can't be stirred up, but I'd
>> really like to play around with this, so if you can point me to  
>> source
>> code or anything else that might help me understand of to put sockets
>> into HTTP packet mode, I'd greatly appreciate it.
>>
>> Thanks,
>> Hans
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>
>
> --
> [hidden email]
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Paulo Sérgio Almeida]

Sverker Eriksson wrote:
> A message from the OTP team:
>
> There is an undocumented socket packet mode that provides HTTP parsing.
> We are planning to make this packet mode official and possibly also
> change the format of the tuples returned in this mode. One current big

Nice!

I use {active, once}, but will have no problem changing the code to
adapt to the new message format.

As you are making it official, here is something you may consider as
well, if not now, maybe for a future version: supporting the "binary"
option. As it is now, the binary option is ignored (at least when
returning the path and headers, which is almost all that matters in the
most common case, "GET"). This means that several lists of chars will be
created which (specially in 64 bit architectures) may use several KBytes
per request, just for the headers. If binaries were returned, memory
consumption would be better and it would make less pressure on GC (I
guess). Considering that pattern matching on binaries in nice and
efficient nowadays, it would be an interesting feature.

A possibility would be the implementation to return sub-binaries
pointing to the single block in the memory in the common case when the
request arrives in a single TCP packet. It would be very efficient. The
programmer would have to be aware of the implications, and not get hold
of sub-binaries. But I see this as a minor issue. Headers are typically
either ignored or acted upon to decide something and discarded soon
afterwards, not stored for the long term (is this true?).

In any case chosing "binary" is optional anyway, we can always choose
getting lists.

Regards,
Paulo

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[chamila piyasena]

I also have a similar problem as Paulo.

By using this can we get the original binary of the http message or should we have to create the original message using the details in the reciving tuples?




cheers,
chamila


http://chamilar.blogspot.com/

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Sverker Eriksson]

Paulo Sérgio Almeida wrote:
> As you are making it official, here is something you may consider as
> well, if not now, maybe for a future version: supporting the "binary"
> option. As it is now, the binary option is ignored [...]
 
I suppose you mean to return the same http-tuples but with binaries
instead of lists for the string values.
One problem here is backward compatibility (again). At least Yaws is
already using the combination [binary,{packet,http}]. We could of course
argue our right to change an undocumented feature, but is it worth it?
Maybe a new {packet, http_bin} is a future solution even though using
the binary option could be more elegant.


/Sverker, Erlang/OTP Ericsson

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Paulo Sérgio Almeida]

Hi,

Sverker Eriksson wrote:

> I suppose you mean to return the same http-tuples but with binaries
> instead of lists for the string values.

Yes.

> One problem here is backward compatibility (again). At least Yaws is
> already using the combination [binary,{packet,http}]. We could of course

Looks like it is in just a couple of places. In the others it uses plain
{packet, http} without binary, or it uses binary in others after
switching to raw or line mode. It should be simple to change. But
perhaps Claes can answer that.

> argue our right to change an undocumented feature, but is it worth it?

The point is that I think having the possibility is useful. And if we
are to fix it, it is now (that it is becoming official) or never.

Regards,
Paulo

> Maybe a new {packet, http_bin} is a future solution even though using
> the binary option could be more elegant.
>
>
> /Sverker, Erlang/OTP Ericsson
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[chamila piyasena]

Hi,
When we receive a http response what is the  format of  the body of the responce(i.e : the html content)?
Is it like ,
 {ok,  http_response, { body, ...........}}            


Cheers,
Chamila

On Mon, Jul 14, 2008 at 7:50 PM, Sverker Eriksson <[hidden email]> wrote:

A message from the OTP team:

There is an undocumented socket packet mode that provides HTTP parsing.
We are planning to make this packet mode official and possibly also
change the format of the tuples returned in this mode. One current big
user is the web server Yaws. We would like to know of other applications
using this HTTP mode and what kind of impact such an incompatible change
would have.

A socket using {packet, http} returns tuples like this in *passive* mode:

{ok, {http_request, ...}}
{ok, {http_response, ...}}
{ok, {http_header, ...}}


and tuples like this in *active* mode:

{http_request, Socket, ...}
{http_response, Socket, ...}
{http_header, Socket, ...}


The proposed change would only affect the active mode like this:

{http, Socket, {http_request, ...}}
{http, Socket, {http_response, ...}}
{http, Socket, {http_header, ...}}


The purpose is to make the inner tuples look the same regardless of how
they were received. This would simplify both the implementation and the
documentation as well as any applications using both receive modes.
Applications only using http in passive mode (like Yaws) will not be
affected.

Am I stirring up any worried http-users out there?

/Sverker, Erlang/OTP Ericsson

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

--


http://chamilar.blogspot.com/

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Christian S-2]

2008/7/23 chamila piyasena <[hidden email]>:
> Hi,
> When we receive a http response what is the  format of  the body of the
> responce(i.e : the html content)?
> Is it like ,
>  {ok,  http_response, { body, ...........}}

You change into plain mode after the end of headers arrive.

It is a little bit annoying that no chunked-mode body exists.
Implementable in erlang though.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[chamila piyasena]

On Wed, Jul 23, 2008 at 5:05 PM, Christian S <[hidden email]> wrote:

2008/7/23 chamila piyasena <[hidden email]>:

> Hi,
> When we receive a http response what is the  format of  the body of the
> responce(i.e : the html content)?
> Is it like ,
>  {ok,  http_response, { body, ...........}}

You change into plain mode after the end of headers arrive.

It is a little bit annoying that no chunked-mode body exists.
Implementable in erlang though.

Thank you for your reply Christian,
You mean It automatically swiches in? If we originally set [binary, {packet, http}] then after end of header to what form it will be swiches to.
      for an example {packet, 0}
      and are we getting the rest as {tcp, Socket, Bin}





Cheers,
Chamila


http://chamilar.blogspot.com/

_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions

[Christian S-2]

>>
>> You change into plain mode after the end of headers arrive.
>>
>> It is a little bit annoying that no chunked-mode body exists.
>> Implementable in erlang though.
>
>
>
> Thank you for your reply Christian,
> You mean It automatically swiches in? If we originally set [binary, {packet,
> http}] then after end of header to what form it will be swiches to.
>       for an example {packet, 0}
>       and are we getting the rest as {tcp, Socket, Bin}

No, you do
  inet:setopts(Sock, [{packet, raw}]),
when you get http_eoh

Find some code that uses this and read it. Suggested example:

http://www.tornkvist.org/gitweb?p=iserve.git;a=blob;f=src/iserve_socket.erl;h=6b0c0f5c37f75b7a8236029f23d8cac78f1e2b42;hb=HEAD

Notice how it only sports an unencoded body (a size specified in a
content length header) and chunked encoding POSTing would make it cry.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-questions