Possible leak in inets httpc.
Locked
Possible leak in inets httpc.
 
|
Hello,
I have application that makes huge number of HTTP requests with httpc and I have spotted that `httpc_manager__handler_db` ETS table growing in size and doesn't clean up. That is the example records from this table (ets:i(httpc_manager__handler_db)): <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} and processes with handler pids are no longer alive. Also, note that I have no timeout setting in my http:request HTTPOptions, but I think inets should not leak even in that case. Thanks. -- Andrey Popp phone: +7 911 740 24 91 e-mail: [hidden email] ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
Re: Possible leak in inets httpc.
|
|
It seems it has been fixed with [1] in inets-5.3.2. Sorry.
[1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <[hidden email]> wrote: > Hello, > > I have application that makes huge number of HTTP requests with httpc > and I have spotted that `httpc_manager__handler_db` ETS table growing > in size and doesn't clean up. That is the example records from this > table (ets:i(httpc_manager__handler_db)): > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} > > and processes with handler pids are no longer alive. Also, note that I > have no timeout setting in my http:request HTTPOptions, but I think > inets should not leak even in that case. > > Thanks. > > -- > Andrey Popp > > phone: +7 911 740 24 91 > e-mail: [hidden email] > -- Andrey Popp phone: +7 911 740 24 91 e-mail: [hidden email] ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
Re: Possible leak in inets httpc.
|
Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process
of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of httpc this will bite me. Ideally, I can just grab the appropriate files from git and plop them into my R13B04 when I build an rpm (of course I need to remember enough git to do that :) ), so wondering if it's possible or if there are deps outside of the inets directory which matter? Thanks, -Anthony On Thu, Jul 08, 2010 at 02:12:07PM +0400, Andrey Popp wrote: > It seems it has been fixed with [1] in inets-5.3.2. Sorry. > > [1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 > > On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <[hidden email]> wrote: > > Hello, > > > > I have application that makes huge number of HTTP requests with httpc > > and I have spotted that `httpc_manager__handler_db` ETS table growing > > in size and doesn't clean up. That is the example records from this > > table (ets:i(httpc_manager__handler_db)): > > > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} > > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} > > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} > > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} > > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} > > > > and processes with handler pids are no longer alive. Also, note that I > > have no timeout setting in my http:request HTTPOptions, but I think > > inets should not leak even in that case. > > > > Thanks. > > > > -- > > Andrey Popp > > > > phone: +7 911 740 24 91 > > e-mail: [hidden email] > > > > > > -- > Andrey Popp > > phone: +7 911 740 24 91 > e-mail: [hidden email] > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:[hidden email] > -- ------------------------------------------------------------------------ Anthony Molinaro <[hidden email]> ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
Re: Possible leak in inets httpc.
|
|
Well, looks like the answer is yes and no. I was able to get a patch
which applied cleanly, however, the problem does not appear to be fixed. The ets table still fills up with every request that times out (I unfortunately don't have a standalone test, but I've tested repeatedly and it always seem to get an extra entry for every request that times out). This is sort of a show stopper for me upgrading, so I'll be trying to figure out a fix, but if anyone more familiar with the code wants to help me out, it would be appreciated. Thanks, -Anthony On Thu, Jul 08, 2010 at 01:13:41PM -0700, Anthony Molinaro wrote: > Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process > of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of > httpc this will bite me. Ideally, I can just grab the appropriate files from > git and plop them into my R13B04 when I build an rpm (of course I need to > remember enough git to do that :) ), so wondering if it's possible or if > there are deps outside of the inets directory which matter? > > Thanks, > > -Anthony > > On Thu, Jul 08, 2010 at 02:12:07PM +0400, Andrey Popp wrote: > > It seems it has been fixed with [1] in inets-5.3.2. Sorry. > > > > [1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 > > > > On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <[hidden email]> wrote: > > > Hello, > > > > > > I have application that makes huge number of HTTP requests with httpc > > > and I have spotted that `httpc_manager__handler_db` ETS table growing > > > in size and doesn't clean up. That is the example records from this > > > table (ets:i(httpc_manager__handler_db)): > > > > > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} > > > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} > > > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} > > > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} > > > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} > > > > > > and processes with handler pids are no longer alive. Also, note that I > > > have no timeout setting in my http:request HTTPOptions, but I think > > > inets should not leak even in that case. > > > > > > Thanks. > > > > > > -- > > > Andrey Popp > > > > > > phone: +7 911 740 24 91 > > > e-mail: [hidden email] > > > > > > > > > > > -- > > Andrey Popp > > > > phone: +7 911 740 24 91 > > e-mail: [hidden email] > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:[hidden email] > > > > -- > ------------------------------------------------------------------------ > Anthony Molinaro <[hidden email]> -- ------------------------------------------------------------------------ Anthony Molinaro <[hidden email]> ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
Re: Possible leak in inets httpc.
|
|
We have updated to R14 (thanks to deb packages in debian squeeze
repository) and all seems to be ok for now. On Fri, Jul 9, 2010 at 1:34 AM, Anthony Molinaro <[hidden email]> wrote: > Well, looks like the answer is yes and no. I was able to get a patch > which applied cleanly, however, the problem does not appear to be fixed. > The ets table still fills up with every request that times out > (I unfortunately don't have a standalone test, but I've tested > repeatedly and it always seem to get an extra entry for every request > that times out). > > This is sort of a show stopper for me upgrading, so I'll be trying to > figure out a fix, but if anyone more familiar with the code wants to > help me out, it would be appreciated. > > Thanks, > > -Anthony > > On Thu, Jul 08, 2010 at 01:13:41PM -0700, Anthony Molinaro wrote: >> Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process >> of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of >> httpc this will bite me. Ideally, I can just grab the appropriate files from >> git and plop them into my R13B04 when I build an rpm (of course I need to >> remember enough git to do that :) ), so wondering if it's possible or if >> there are deps outside of the inets directory which matter? >> >> Thanks, >> >> -Anthony >> >> On Thu, Jul 08, 2010 at 02:12:07PM +0400, Andrey Popp wrote: >> > It seems it has been fixed with [1] in inets-5.3.2. Sorry. >> > >> > [1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 >> > >> > On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <[hidden email]> wrote: >> > > Hello, >> > > >> > > I have application that makes huge number of HTTP requests with httpc >> > > and I have spotted that `httpc_manager__handler_db` ETS table growing >> > > in size and doesn't clean up. That is the example records from this >> > > table (ets:i(httpc_manager__handler_db)): >> > > >> > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} >> > > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} >> > > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} >> > > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} >> > > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} >> > > >> > > and processes with handler pids are no longer alive. Also, note that I >> > > have no timeout setting in my http:request HTTPOptions, but I think >> > > inets should not leak even in that case. >> > > >> > > Thanks. >> > > >> > > -- >> > > Andrey Popp >> > > >> > > phone: +7 911 740 24 91 >> > > e-mail: [hidden email] >> > > >> > >> > >> > >> > -- >> > Andrey Popp >> > >> > phone: +7 911 740 24 91 >> > e-mail: [hidden email] >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:[hidden email] >> > >> >> -- >> ------------------------------------------------------------------------ >> Anthony Molinaro <[hidden email]> > > -- > ------------------------------------------------------------------------ > Anthony Molinaro <[hidden email]> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:[hidden email] > > -- Andrey Popp phone: +7 911 740 24 91 e-mail: [hidden email] ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
Re: Possible leak in inets httpc.
|
This problem was fixed in inets-5.3.2. The fix was reported by Lev Walkin:
http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1766 Unfortunately, github was not updated in one commit, but in several, here are some of them: http://github.com/erlang/otp/commit/4b87b22ce97abf2759eb551222a862e17c5f4dcb http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 So you have to do a bit of cut-and-paste. The last version of inets in R13B was 5.3.3, but I don't know if that made it into github. Regards, /BMK On Fri, Jul 9, 2010 at 8:12 AM, Andrey Popp <[hidden email]> wrote: > We have updated to R14 (thanks to deb packages in debian squeeze > repository) and all seems to be ok for now. > > On Fri, Jul 9, 2010 at 1:34 AM, Anthony Molinaro > <[hidden email]> wrote: >> Well, looks like the answer is yes and no. I was able to get a patch >> which applied cleanly, however, the problem does not appear to be fixed. >> The ets table still fills up with every request that times out >> (I unfortunately don't have a standalone test, but I've tested >> repeatedly and it always seem to get an extra entry for every request >> that times out). >> >> This is sort of a show stopper for me upgrading, so I'll be trying to >> figure out a fix, but if anyone more familiar with the code wants to >> help me out, it would be appreciated. >> >> Thanks, >> >> -Anthony >> >> On Thu, Jul 08, 2010 at 01:13:41PM -0700, Anthony Molinaro wrote: >>> Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process >>> of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of >>> httpc this will bite me. Ideally, I can just grab the appropriate files from >>> git and plop them into my R13B04 when I build an rpm (of course I need to >>> remember enough git to do that :) ), so wondering if it's possible or if >>> there are deps outside of the inets directory which matter? >>> >>> Thanks, >>> >>> -Anthony >>> >>> On Thu, Jul 08, 2010 at 02:12:07PM +0400, Andrey Popp wrote: >>> > It seems it has been fixed with [1] in inets-5.3.2. Sorry. >>> > >>> > [1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 >>> > >>> > On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <[hidden email]> wrote: >>> > > Hello, >>> > > >>> > > I have application that makes huge number of HTTP requests with httpc >>> > > and I have spotted that `httpc_manager__handler_db` ETS table growing >>> > > in size and doesn't clean up. That is the example records from this >>> > > table (ets:i(httpc_manager__handler_db)): >>> > > >>> > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} >>> > > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} >>> > > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} >>> > > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} >>> > > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} >>> > > >>> > > and processes with handler pids are no longer alive. Also, note that I >>> > > have no timeout setting in my http:request HTTPOptions, but I think >>> > > inets should not leak even in that case. >>> > > >>> > > Thanks. >>> > > >>> > > -- >>> > > Andrey Popp >>> > > >>> > > phone: +7 911 740 24 91 >>> > > e-mail: [hidden email] >>> > > >>> > >>> > >>> > >>> > -- >>> > Andrey Popp >>> > >>> > phone: +7 911 740 24 91 >>> > e-mail: [hidden email] >>> > >>> > ________________________________________________________________ >>> > erlang-questions (at) erlang.org mailing list. >>> > See http://www.erlang.org/faq.html >>> > To unsubscribe; mailto:[hidden email] >>> > >>> >>> -- >>> ------------------------------------------------------------------------ >>> Anthony Molinaro <[hidden email]> >> >> -- >> ------------------------------------------------------------------------ >> Anthony Molinaro <[hidden email]> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:[hidden email] >> >> > > > > -- > Andrey Popp > > phone: +7 911 740 24 91 > e-mail: [hidden email] > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:[hidden email] > > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
Re: Possible leak in inets httpc.
|
|
Yeah, I managed to get the entire set of patches, basically used
git diff 201ccdd08bd2a591fe1e348a9a1df3d94a3606e4 141ff3c087ca856d21d0 lib/inets Which seemed to get them all. However, I still see memory leaked Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> inets:start(). ok 2> ets:i(). ... 16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager httpc_manager__handler_db httpc_manager__handler_db set 0 316 httpc_manager httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager ... 3> http:request (post, { "http://www.example.com", [], "application/octet-stream", <<"hello">>}, [{connect_timeout, 100},{timeout, 10}], [{body_format, binary} ]). {error,timeout} 4> ets:i(). ... 16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager httpc_manager__handler_db httpc_manager__handler_db set 1 333 httpc_manager httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager ... 5> ets:i(httpc_manager__handler_db). <1 > {handler_info,#Ref<0.0.0.142>,undefined,<0.60.0>,<0.33.0>,operational} 6> m(inets). Module inets compiled: Date: July 8 2010, Time: 06.45 Compiler options: [{d,'SERVER_SOFTWARE',"inets/5.3.3"}, {cwd,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app"}, {outdir,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app/../../ebin"}, {attribute,insert,app_vsn,"inets-5.3.3"}, {parse_transform,sys_pre_attributes}, debug_info] Object file: /usr/lib64/erlang/lib/inets-5.3.3/ebin/inets.beam ... So it seems if you connect but then timeout you leak some memory even with 5.3.3. I've not tried 5.4 which is in dev but the only changes there seem to be related to ssl... Can anyone else confirm this is still broken? -Anthony On Fri, Jul 09, 2010 at 12:12:13PM +0200, Micael Karlberg wrote: > This problem was fixed in inets-5.3.2. The fix was reported by Lev Walkin: > > http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1766 > > Unfortunately, github was not updated in one commit, but in several, here > are some of them: > > http://github.com/erlang/otp/commit/4b87b22ce97abf2759eb551222a862e17c5f4dcb > http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 > > So you have to do a bit of cut-and-paste. The last version of inets in > R13B was 5.3.3, > but I don't know if that made it into github. > > Regards, > /BMK > > On Fri, Jul 9, 2010 at 8:12 AM, Andrey Popp <[hidden email]> wrote: > > We have updated to R14 (thanks to deb packages in debian squeeze > > repository) and all seems to be ok for now. > > > > On Fri, Jul 9, 2010 at 1:34 AM, Anthony Molinaro > > <[hidden email]> wrote: > >> Well, looks like the answer is yes and no. I was able to get a patch > >> which applied cleanly, however, the problem does not appear to be fixed. > >> The ets table still fills up with every request that times out > >> (I unfortunately don't have a standalone test, but I've tested > >> repeatedly and it always seem to get an extra entry for every request > >> that times out). > >> > >> This is sort of a show stopper for me upgrading, so I'll be trying to > >> figure out a fix, but if anyone more familiar with the code wants to > >> help me out, it would be appreciated. > >> > >> Thanks, > >> > >> -Anthony > >> > >> On Thu, Jul 08, 2010 at 01:13:41PM -0700, Anthony Molinaro wrote: > >>> Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process > >>> of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of > >>> httpc this will bite me. Ideally, I can just grab the appropriate files from > >>> git and plop them into my R13B04 when I build an rpm (of course I need to > >>> remember enough git to do that :) ), so wondering if it's possible or if > >>> there are deps outside of the inets directory which matter? > >>> > >>> Thanks, > >>> > >>> -Anthony > >>> > >>> On Thu, Jul 08, 2010 at 02:12:07PM +0400, Andrey Popp wrote: > >>> > It seems it has been fixed with [1] in inets-5.3.2. Sorry. > >>> > > >>> > [1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 > >>> > > >>> > On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <[hidden email]> wrote: > >>> > > Hello, > >>> > > > >>> > > I have application that makes huge number of HTTP requests with httpc > >>> > > and I have spotted that `httpc_manager__handler_db` ETS table growing > >>> > > in size and doesn't clean up. That is the example records from this > >>> > > table (ets:i(httpc_manager__handler_db)): > >>> > > > >>> > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} > >>> > > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} > >>> > > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} > >>> > > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} > >>> > > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} > >>> > > > >>> > > and processes with handler pids are no longer alive. Also, note that I > >>> > > have no timeout setting in my http:request HTTPOptions, but I think > >>> > > inets should not leak even in that case. > >>> > > > >>> > > Thanks. > >>> > > > >>> > > -- > >>> > > Andrey Popp > >>> > > > >>> > > phone: +7 911 740 24 91 > >>> > > e-mail: [hidden email] > >>> > > > >>> > > >>> > > >>> > > >>> > -- > >>> > Andrey Popp > >>> > > >>> > phone: +7 911 740 24 91 > >>> > e-mail: [hidden email] > >>> > > >>> > ________________________________________________________________ > >>> > erlang-questions (at) erlang.org mailing list. > >>> > See http://www.erlang.org/faq.html > >>> > To unsubscribe; mailto:[hidden email] > >>> > > >>> > >>> -- > >>> ------------------------------------------------------------------------ > >>> Anthony Molinaro <[hidden email]> > >> > >> -- > >> ------------------------------------------------------------------------ > >> Anthony Molinaro <[hidden email]> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:[hidden email] > >> > >> > > > > > > > > -- > > Andrey Popp > > > > phone: +7 911 740 24 91 > > e-mail: [hidden email] > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:[hidden email] > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:[hidden email] > -- ------------------------------------------------------------------------ Anthony Molinaro <[hidden email]> ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
Re: Possible leak in inets httpc.
|
|
Well, the leak was there in a big way under production traffic patterns, so
I switched from httpc to ibrowse. All the leaks went away. Just an FYI for anyone out there making a decision between httpc and ibrowse. -Anthony On Fri, Jul 09, 2010 at 09:26:48AM -0700, Anthony Molinaro wrote: > Yeah, I managed to get the entire set of patches, basically used > > git diff 201ccdd08bd2a591fe1e348a9a1df3d94a3606e4 141ff3c087ca856d21d0 lib/inets > > Which seemed to get them all. However, I still see memory leaked > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] > [kernel-poll:false] > > Eshell V5.7.5 (abort with ^G) > 1> inets:start(). > ok > 2> ets:i(). > ... > 16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager > httpc_manager__handler_db httpc_manager__handler_db set 0 316 httpc_manager > httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager > ... > 3> http:request (post, { "http://www.example.com", [], "application/octet-stream", <<"hello">>}, [{connect_timeout, 100},{timeout, 10}], [{body_format, binary} ]). > {error,timeout} > 4> ets:i(). > ... > 16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager > httpc_manager__handler_db httpc_manager__handler_db set 1 333 httpc_manager > httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager > ... > 5> ets:i(httpc_manager__handler_db). > <1 > {handler_info,#Ref<0.0.0.142>,undefined,<0.60.0>,<0.33.0>,operational} > > 6> m(inets). > Module inets compiled: Date: July 8 2010, Time: 06.45 > Compiler options: [{d,'SERVER_SOFTWARE',"inets/5.3.3"}, > {cwd,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app"}, > {outdir,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app/../../ebin"}, > {attribute,insert,app_vsn,"inets-5.3.3"}, > {parse_transform,sys_pre_attributes}, > debug_info] > Object file: /usr/lib64/erlang/lib/inets-5.3.3/ebin/inets.beam > ... > > So it seems if you connect but then timeout you leak some memory even with > 5.3.3. I've not tried 5.4 which is in dev but the only changes there seem > to be related to ssl... > > Can anyone else confirm this is still broken? > > -Anthony > > On Fri, Jul 09, 2010 at 12:12:13PM +0200, Micael Karlberg wrote: > > This problem was fixed in inets-5.3.2. The fix was reported by Lev Walkin: > > > > http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1766 > > > > Unfortunately, github was not updated in one commit, but in several, here > > are some of them: > > > > http://github.com/erlang/otp/commit/4b87b22ce97abf2759eb551222a862e17c5f4dcb > > http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 > > > > So you have to do a bit of cut-and-paste. The last version of inets in > > R13B was 5.3.3, > > but I don't know if that made it into github. > > > > Regards, > > /BMK > > > > On Fri, Jul 9, 2010 at 8:12 AM, Andrey Popp <[hidden email]> wrote: > > > We have updated to R14 (thanks to deb packages in debian squeeze > > > repository) and all seems to be ok for now. > > > > > > On Fri, Jul 9, 2010 at 1:34 AM, Anthony Molinaro > > > <[hidden email]> wrote: > > >> Well, looks like the answer is yes and no. I was able to get a patch > > >> which applied cleanly, however, the problem does not appear to be fixed. > > >> The ets table still fills up with every request that times out > > >> (I unfortunately don't have a standalone test, but I've tested > > >> repeatedly and it always seem to get an extra entry for every request > > >> that times out). > > >> > > >> This is sort of a show stopper for me upgrading, so I'll be trying to > > >> figure out a fix, but if anyone more familiar with the code wants to > > >> help me out, it would be appreciated. > > >> > > >> Thanks, > > >> > > >> -Anthony > > >> > > >> On Thu, Jul 08, 2010 at 01:13:41PM -0700, Anthony Molinaro wrote: > > >>> Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process > > >>> of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of > > >>> httpc this will bite me. Ideally, I can just grab the appropriate files from > > >>> git and plop them into my R13B04 when I build an rpm (of course I need to > > >>> remember enough git to do that :) ), so wondering if it's possible or if > > >>> there are deps outside of the inets directory which matter? > > >>> > > >>> Thanks, > > >>> > > >>> -Anthony > > >>> > > >>> On Thu, Jul 08, 2010 at 02:12:07PM +0400, Andrey Popp wrote: > > >>> > It seems it has been fixed with [1] in inets-5.3.2. Sorry. > > >>> > > > >>> > [1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49 > > >>> > > > >>> > On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <[hidden email]> wrote: > > >>> > > Hello, > > >>> > > > > >>> > > I have application that makes huge number of HTTP requests with httpc > > >>> > > and I have spotted that `httpc_manager__handler_db` ETS table growing > > >>> > > in size and doesn't clean up. That is the example records from this > > >>> > > table (ets:i(httpc_manager__handler_db)): > > >>> > > > > >>> > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational} > > >>> > > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational} > > >>> > > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational} > > >>> > > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational} > > >>> > > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational} > > >>> > > > > >>> > > and processes with handler pids are no longer alive. Also, note that I > > >>> > > have no timeout setting in my http:request HTTPOptions, but I think > > >>> > > inets should not leak even in that case. > > >>> > > > > >>> > > Thanks. > > >>> > > > > >>> > > -- > > >>> > > Andrey Popp > > >>> > > > > >>> > > phone: +7 911 740 24 91 > > >>> > > e-mail: [hidden email] > > >>> > > > > >>> > > > >>> > > > >>> > > > >>> > -- > > >>> > Andrey Popp > > >>> > > > >>> > phone: +7 911 740 24 91 > > >>> > e-mail: [hidden email] > > >>> > > > >>> > ________________________________________________________________ > > >>> > erlang-questions (at) erlang.org mailing list. > > >>> > See http://www.erlang.org/faq.html > > >>> > To unsubscribe; mailto:[hidden email] > > >>> > > > >>> > > >>> -- > > >>> ------------------------------------------------------------------------ > > >>> Anthony Molinaro <[hidden email]> > > >> > > >> -- > > >> ------------------------------------------------------------------------ > > >> Anthony Molinaro <[hidden email]> > > >> > > >> ________________________________________________________________ > > >> erlang-questions (at) erlang.org mailing list. > > >> See http://www.erlang.org/faq.html > > >> To unsubscribe; mailto:[hidden email] > > >> > > >> > > > > > > > > > > > > -- > > > Andrey Popp > > > > > > phone: +7 911 740 24 91 > > > e-mail: [hidden email] > > > > > > ________________________________________________________________ > > > erlang-questions (at) erlang.org mailing list. > > > See http://www.erlang.org/faq.html > > > To unsubscribe; mailto:[hidden email] > > > > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:[hidden email] > > > > -- > ------------------------------------------------------------------------ > Anthony Molinaro <[hidden email]> -- ------------------------------------------------------------------------ Anthony Molinaro <[hidden email]> ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
«
Return to Erlang Questions
|
1 view|%1 views
Loading...
| Powered by Nabble | Edit this page |