http:request memory leak in R12B04

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

http:request memory leak in R12B04

Lev Walkin

Hi,

The R12B04 release brought a reliable memory leak to http:request that  
was never in there before.

Here's how it manifests itself:

========================
Erlang R13B04 (erts-5.7.5) [source] [rq:1] [async-threads:0] [hipe]  
[kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> inets:start().
ok
2> http:request("http://google.com/nonexistent"), [S || S = {size, _}  
<- ets:info(httpc_manager__handler_db)].
[{size,1}]
3> http:request("http://google.com/nonexistent"), [S || S = {size, _}  
<- ets:info(httpc_manager__handler_db)].
[{size,2}]
4> http:request("http://google.com/nonexistent"), [S || S = {size, _}  
<- ets:info(httpc_manager__handler_db)].
[{size,3}]
5> a=b.
** exception error: no match of right hand side value b
6> http:request("http://google.com/nonexistent"), [S || S = {size, _}  
<- ets:info(httpc_manager__handler_db)].
[{size,4}]
7>
========================

After a few (million) requests this stuff eats all available memory. I  
suppose this is not the intended behavior for the http module.

Here's the cure:

===
--- ./httpc_handler.erl.orig     2010-03-22 12:34:07.000000000 +0300
+++ ./httpc_handler.erl 2010-03-22 16:50:34.000000000 +0300
@@ -1407,7 +1407,7 @@
            State#state{status = close}
     end.

-answer_request(Request, Msg, #state{timers = Timers} = State) ->
+answer_request(Request, Msg, #state{timers = Timers, profile_name =  
ProfileName} = State) ->
     ?hcrt("answer request", [{request, Request}]),
     httpc_response:send(Request#request.from, Msg),
     RequestTimers = Timers#timers.request_timers,
@@ -1415,6 +1415,7 @@
        proplists:get_value(Request#request.id, RequestTimers,  
undefined),
     Timer = {Request#request.id, TimerRef},
     cancel_timer(TimerRef, {timeout, Request#request.id}),
+    httpc_manager:request_done(Request#request.id, ProfileName),
     State#state{request = Request#request{from = answer_sent},
                timers =
                Timers#timers{request_timers =

--- ./httpc_manager.erl.orig     2010-03-22 12:36:58.000000000 +0300
+++ ./httpc_manager.erl 2010-03-22 16:50:46.000000000 +0300
@@ -30,6 +30,7 @@
         request/2,
         cancel_request/2,
         request_canceled/2,
+        request_done/2,
         retry_request/2,
         redirect_request/2,
         insert_session/2,
@@ -169,6 +170,9 @@
request_canceled(RequestId, ProfileName) ->
     cast(ProfileName, {request_canceled, RequestId}).

+request_done(RequestId, ProfileName) ->
+    cast(ProfileName, {request_done, RequestId}).
+

%%--------------------------------------------------------------------
%% Function: insert_session(Session, ProfileName) -> _
@@ -486,6 +490,10 @@
           {noreply, State}
     end;

+handle_cast({request_done, RequestId}, State) ->
+    ets:delete(State#state.handler_db, RequestId),
+    {noreply, State};
+
handle_cast({set_options, Options}, State = #state{options =  
OldOptions}) ->
     ?hcrv("set options", [{options, Options}, {old_options,  
OldOptions}]),
     NewOptions =
===


--
vlm


________________________________________________________________
erlang-bugs (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: http:request memory leak in R12B04

Lev Walkin

No idea. I haven't received any follow-up, and the available code base  
on git never changed in this respect, as far as I can tell.

We are running with a custom version on our servers and thinking of  
dropping Erlang in favor of PHP.


On May 28, 2010, at 1:50 PM, mats cronqvist wrote:

> Lev Walkin <[hidden email]> writes:
>
>  what's up with this? renders httpc unusable, as far as I can tell.
>
>  mats
>
>> The R12B04 release brought a reliable memory leak to http:request  
>> that
>> was never in there before.

--
vlm


________________________________________________________________
erlang-bugs (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: http:request memory leak in R12B04

Lev Walkin

I must correct myself here. Just checked the OTP source code and it  
turned out the bug was finally fixed in inets-5.3.2, roughly in accord  
with the patch we've submitted on March 22.

Here's the proof:

http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49


On May 28, 2010, at 5:49 PM, Lev Walkin wrote:

>
> No idea. I haven't received any follow-up, and the available code  
> base on git never changed in this respect, as far as I can tell.
>
> We are running with a custom version on our servers and thinking of  
> dropping Erlang in favor of PHP.
>
>
> On May 28, 2010, at 1:50 PM, mats cronqvist wrote:
>
>> Lev Walkin <[hidden email]> writes:
>>
>> what's up with this? renders httpc unusable, as far as I can tell.
>>
>> mats
>>
>>> The R12B04 release brought a reliable memory leak to http:request  
>>> that
>>> was never in there before.
>
> --
> vlm
>
>
> ________________________________________________________________
> erlang-bugs (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:[hidden email]
>

--
vlm


________________________________________________________________
erlang-bugs (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]