httpc broke URL parsing in OTP-21.0?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

httpc broke URL parsing in OTP-21.0?

Roger Lipscombe-2
20.3...

1> inets:start().
ok
2> httpc:request(get,
{"http://localhost:31316/EluoueLg-ueL/?x=32&y=54&z[0]=1&z[1]=2&q[]=5&q[]=4&p=9&p=8",
[]}, [], []).
{error,{failed_connect,[{to_address,{"localhost",31316}},
                        {inet,[inet],econnrefused}]}}

...fair enough (no server is running).

21.0...

1> inets:start().
ok
2> httpc:request(get,
{"http://localhost:31316/EluoueLg-ueL/?x=32&y=54&z[0]=1&z[1]=2&q[]=5&q[]=4&p=9&p=8",
[]}, [], []).
** exception error: no function clause matching
uri_string:parse({error,invalid_uri,":"}) (uri_string.erl, line 337)
     in function  httpc:request/5 (httpc.erl, line 179)

Not good :(

Cheers,
Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: httpc broke URL parsing in OTP-21.0?

Roger Lipscombe-2
Pared it down:

17> httpc:request(get, {"http://localhost/?x[0]=1", []}, [], []).
** exception error: no function clause matching
uri_string:parse({error,invalid_uri,":"}) (uri_string.erl, line 337)
     in function  httpc:request/5 (httpc.erl, line 179)
18> httpc:request(get, {"http://localhost/?x=1", []}, [], []).
{ok,{{"HTTP/1.1",200,"OK"}, ...}

It doesn't seem to like the square brackets.

On 25 June 2018 at 15:28, Roger Lipscombe <[hidden email]> wrote:

> 20.3...
>
> 1> inets:start().
> ok
> 2> httpc:request(get,
> {"http://localhost:31316/EluoueLg-ueL/?x=32&y=54&z[0]=1&z[1]=2&q[]=5&q[]=4&p=9&p=8",
> []}, [], []).
> {error,{failed_connect,[{to_address,{"localhost",31316}},
>                         {inet,[inet],econnrefused}]}}
>
> ...fair enough (no server is running).
>
> 21.0...
>
> 1> inets:start().
> ok
> 2> httpc:request(get,
> {"http://localhost:31316/EluoueLg-ueL/?x=32&y=54&z[0]=1&z[1]=2&q[]=5&q[]=4&p=9&p=8",
> []}, [], []).
> ** exception error: no function clause matching
> uri_string:parse({error,invalid_uri,":"}) (uri_string.erl, line 337)
>      in function  httpc:request/5 (httpc.erl, line 179)
>
> Not good :(
>
> Cheers,
> Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: httpc broke URL parsing in OTP-21.0?

Andreas Schultz-3
Square brackets are not permitted in the query part of an URL. You need to percent encode them (see https://tools.ietf.org/html/rfc3986#section-3.4).

6> httpc:request(get, {"http://localhost:8080/?x%5B0%5D=1", []}, [], []).
{ok,{{"HTTP/1.1",200,"OK"},...}

However, many browsers and web servers will accept square brackets in the URI. And some (probably many) broken clients will not encode square brackets in requests.

e.g.:

$  wget --debug -O /dev/null -S 'http://localhost:8080/?x [0]=1'
...
---request begin---
GET /?x%20[0]=1 HTTP/1.1

Space is encoded, brackets are not...

Erlang will also (rightly) complain about spaces:

7> httpc:request(get, {"http://localhost:8080/?x =1", []}, [], []).      
** exception error: no function clause matching uri_string:parse({error,invalid_uri,":"}) (uri_string.erl, line 337)
     in function  httpc:request/5 (httpc.erl, line 179)

Andreas

Roger Lipscombe <[hidden email]> schrieb am Mo., 25. Juni 2018 um 16:31 Uhr:
Pared it down:

17> httpc:request(get, {"http://localhost/?x[0]=1", []}, [], []).
** exception error: no function clause matching
uri_string:parse({error,invalid_uri,":"}) (uri_string.erl, line 337)
     in function  httpc:request/5 (httpc.erl, line 179)
18> httpc:request(get, {"http://localhost/?x=1", []}, [], []).
{ok,{{"HTTP/1.1",200,"OK"}, ...}

It doesn't seem to like the square brackets.

On 25 June 2018 at 15:28, Roger Lipscombe <[hidden email]> wrote:
> 20.3...
>
> 1> inets:start().
> ok
> 2> httpc:request(get,
> {"http://localhost:31316/EluoueLg-ueL/?x=32&y=54&z[0]=1&z[1]=2&q[]=5&q[]=4&p=9&p=8",
> []}, [], []).
> {error,{failed_connect,[{to_address,{"localhost",31316}},
>                         {inet,[inet],econnrefused}]}}
>
> ...fair enough (no server is running).
>
> 21.0...
>
> 1> inets:start().
> ok
> 2> httpc:request(get,
> {"http://localhost:31316/EluoueLg-ueL/?x=32&y=54&z[0]=1&z[1]=2&q[]=5&q[]=4&p=9&p=8",
> []}, [], []).
> ** exception error: no function clause matching
> uri_string:parse({error,invalid_uri,":"}) (uri_string.erl, line 337)
>      in function  httpc:request/5 (httpc.erl, line 179)
>
> Not good :(
>
> Cheers,
> Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
--
--
Dipl.-Inform. Andreas Schultz

----------------------- enabling your networks ----------------------
Travelping GmbH                     Phone:  +49-391-81 90 99 0
Roentgenstr. 13                     Fax:    +49-391-81 90 99 299
39108 Magdeburg                     Email:  [hidden email]
GERMANY                             Web:    http://www.travelping.com

Company Registration: Amtsgericht Stendal        Reg No.:   HRB 10578
Geschaeftsfuehrer: Holger Winkelmann          VAT ID No.: DE236673780
---------------------------------------------------------------------

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: httpc broke URL parsing in OTP-21.0?

Roger Lipscombe-2
On 25 June 2018 at 17:05, Andreas Schultz
<[hidden email]> wrote:
> Square brackets are not permitted in the query part of an URL. You need to
> percent encode them (see https://tools.ietf.org/html/rfc3986#section-3.4).

Sure, but this is a breaking change. Source that did work now doesn't.

> However, many browsers and web servers will accept square brackets in the
> URI. And some (probably many) broken clients will not encode square brackets
> in requests.

Chrome passes the square brackets straight through if you type them
into the address bar; haven't checked any other browsers (yet).

> $  wget --debug -O /dev/null -S 'http://localhost:8080/?x [0]=1'

curl (provided you specify --globoff; and, tbf, it *does* have a
warning in the manpage) also passes them straight through.

I've fixed my code -- it was just one line -- but ... breaking change.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions