|
Hi, all,
I tried to create 1K TCP/TLS client connections on a Windows box, and got {error, emfile} error, any idea to solve it? I have no problem on Linux after raising the ulimit. Thanks, /Kaiduan ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
|
On Mon, Nov 29, 2010 at 11:18:21PM -0500, Kaiduan Xie wrote:
> I tried to create 1K TCP/TLS client connections on a Windows box, and > got {error, emfile} error, any idea to solve it? I have no problem on > Linux after raising the ulimit. Whilst in theory windows has a limit of around 16M fds per process, the Erlang windows port seems to use bits of the MS C runtime. This seems to limit severely wrt number of available sockets. Whilst erlang:system_info(check_io) reports 2048 under Windows, that's clearly wrong. But hey, a few years ago, I found that fsync under the Windows port was implemented as "return 1;". Such is the quality of the Windows port. Matthew ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
|
On Windows there is no limit to set in the shell like in UNIX/Linux
that the Erlang VM can read and adjust according to. Therefore you can use the environment variable ERL_MAX_PORTS to set a higher limit thant the default (1024). But bevare that the number you set will imply an allocation of memory to hold all these port data structures. So don't set a higher value than you really need. You can actually set this on the command line when you start erlang, like this example: erl -env ERL_MAX_PORTS 4096 See the documentation at: http://www.erlang.org/doc/efficiency_guide/advanced.html#id64904 (I admit that it is not easy to find this, but the Efficiency Guide is important to know about and to read through for all system builders) We are thinking of changing this to a totally dynamic number of ports only limited by the OS setup, but that work is not scheduled for a specific release yet. On Tue, Nov 30, 2010 at 12:20 PM, Matthew Sackman <[hidden email]> wrote: > On Mon, Nov 29, 2010 at 11:18:21PM -0500, Kaiduan Xie wrote: >> I tried to create 1K TCP/TLS client connections on a Windows box, and >> got {error, emfile} error, any idea to solve it? I have no problem on >> Linux after raising the ulimit. > > Whilst in theory windows has a limit of around 16M fds per process, the > Erlang windows port seems to use bits of the MS C runtime. This seems to > limit severely wrt number of available sockets. Well this is not a correct statement as already described. > > Whilst erlang:system_info(check_io) reports 2048 under Windows, that's > clearly wrong. But hey, a few years ago, I found that fsync under the > Windows port was implemented as "return 1;". Such is the quality of the > Windows port. This is not the case now, fsync is definitely implemented. Windows has not been our main target platform for Erlang but I still think the quality is good even on Windows. With an increasing number of developers doing serious applications with Erlang on Windows the functionality and quality will become even better. /Kenneth Erlang/OTP, Ericsson > > Matthew > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:[hidden email] > > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
|
Kenneth,
Setting ERL_MAX_PORT still does not resolve the problem, it can only open 31 TLS connections. C:\erl5.7.1\usr>..\bin\erl.exe -env ERL_MAX_PORTS 4096 Eshell V5.7.1 (abort with ^G) 1> application:start(ssl). ok 2> Socks = test_tls_sip:connect("10.9.22.85", 5061, 512, 1). ** exception error: no match of right hand side value {error,emfile} in function test_tls_sip:'-connect/4-fun-0-'/5 in call from lists:foldl/3 connect(Addr, Port, N, TimerValue) -> lists:foldl(fun(_I, Acc) -> {ok, Socket} = ssl:connect(Addr, Port, [{active, true}]), {ok, Pid} = gen_server:start(erlang_tls_connection, {Socket, TimerValue}, []), ssl:controlling_process(Socket, Pid), [Pid|Acc] end, [], lists:seq(1, N)). Thanks, /Kaiduan On Tue, Nov 30, 2010 at 9:08 AM, Kenneth Lundin <[hidden email]> wrote: > On Windows there is no limit to set in the shell like in UNIX/Linux > that the Erlang VM > can read and adjust according to. > Therefore you can use the environment variable ERL_MAX_PORTS to set a > higher limit > thant the default (1024). But bevare that the number you set will > imply an allocation > of memory to hold all these port data structures. So don't set a > higher value than you really need. > You can actually set this on the command line when you start erlang, > like this example: > erl -env ERL_MAX_PORTS 4096 > > See the documentation at: > http://www.erlang.org/doc/efficiency_guide/advanced.html#id64904 > (I admit that it is not easy to find this, but the Efficiency Guide is > important to know > about and to read through for all system builders) > > We are thinking of changing this to a totally dynamic number of ports > only limited by the OS setup, but that work is not scheduled for a > specific release yet. > > > On Tue, Nov 30, 2010 at 12:20 PM, Matthew Sackman <[hidden email]> wrote: >> On Mon, Nov 29, 2010 at 11:18:21PM -0500, Kaiduan Xie wrote: >>> I tried to create 1K TCP/TLS client connections on a Windows box, and >>> got {error, emfile} error, any idea to solve it? I have no problem on >>> Linux after raising the ulimit. >> >> Whilst in theory windows has a limit of around 16M fds per process, the >> Erlang windows port seems to use bits of the MS C runtime. This seems to >> limit severely wrt number of available sockets. > > Well this is not a correct statement as already described. >> >> Whilst erlang:system_info(check_io) reports 2048 under Windows, that's >> clearly wrong. But hey, a few years ago, I found that fsync under the >> Windows port was implemented as "return 1;". Such is the quality of the >> Windows port. > This is not the case now, fsync is definitely implemented. > > Windows has not been our main target platform for Erlang but I still > think the quality is good even on Windows. > > With an increasing number of developers doing serious applications > with Erlang on > Windows the functionality and quality will become even better. > > > /Kenneth Erlang/OTP, Ericsson > >> >> Matthew >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:[hidden email] >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:[hidden email] > > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
|
In reply to this post by Kenneth Lundin
On Tue, Nov 30, 2010 at 03:08:33PM +0100, Kenneth Lundin wrote:
> See the documentation at: > http://www.erlang.org/doc/efficiency_guide/advanced.html#id64904 > (I admit that it is not easy to find this, but the Efficiency Guide is > important to know about and to read through for all system builders) Interesting - I've read that several times before ... except clearly not the bits about ports and sockets. Thanks for the pointer. Could you clarify the documentation a bit though - for example, on my Linux machine, I have a ulimit -n of 100000 and I don't seem to need to configure anything further (certainly I've never touched ERL_MAX_PORTS) for Erlang to be able to use many sockets and fds. Thus for platforms which have an Erlang port and ulimit, which of the ulimit -n and ERL_MAX_PORTS is used if both are set? > We are thinking of changing this to a totally dynamic number of ports > only limited by the OS setup, but that work is not scheduled for a > specific release yet. So under Windows, various sources eg http://blogs.technet.com/markrussinovich/archive/2009/09/29/3283844.aspx suggest that the limit is 16M. You've suggested that unnecessarily high numbers are likely to waste memory, so would the limit be detected and set dynamically at startup or would memory only be allocated as needed? Also, under unices, file descriptors and sockets come from the same pool, but under Windows, my understanding is that this is not the case. Could you confirm that in the Windows port, file descriptors are win32 file handles, and sockets are winsock socket handles? Is there going to be a supported+documented API to be able to inspect the limits of each and the number of each allocated that works on both Windows and unices? > > Whilst in theory windows has a limit of around 16M fds per process, the > > Erlang windows port seems to use bits of the MS C runtime. This seems to > > limit severely wrt number of available sockets. > > Well this is not a correct statement as already described. Indeed not. Thanks for the illumination. Matthew ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:[hidden email] |
| Powered by Nabble | Edit this page |
