Quantcast

{error, emfile} on windows

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

{error, emfile} on windows

Kaiduan Xie-2
Hi, all,

I tried to create 1K TCP/TLS client connections on a Windows box, and
got {error, emfile} error, any idea to solve it? I have no problem on
Linux after raising the ulimit.

Thanks,

/Kaiduan

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: {error, emfile} on windows

Matthew Sackman-2
On Mon, Nov 29, 2010 at 11:18:21PM -0500, Kaiduan Xie wrote:
> I tried to create 1K TCP/TLS client connections on a Windows box, and
> got {error, emfile} error, any idea to solve it? I have no problem on
> Linux after raising the ulimit.

Whilst in theory windows has a limit of around 16M fds per process, the
Erlang windows port seems to use bits of the MS C runtime. This seems to
limit severely wrt number of available sockets.

Whilst erlang:system_info(check_io) reports 2048 under Windows, that's
clearly wrong. But hey, a few years ago, I found that fsync under the
Windows port was implemented as "return 1;". Such is the quality of the
Windows port.

Matthew

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: {error, emfile} on windows

Kenneth Lundin
On Windows there is no limit to set in the shell like in UNIX/Linux
that the Erlang VM
can read and adjust according to.
Therefore you can use the environment variable ERL_MAX_PORTS to set a
higher limit
thant the default (1024). But bevare that the number you set will
imply an allocation
of  memory to hold all these port data structures. So don't set a
higher value than you really need.
You can actually set this on the command line when you start erlang,
like this example:
erl -env ERL_MAX_PORTS 4096

See the documentation at:
http://www.erlang.org/doc/efficiency_guide/advanced.html#id64904
(I admit that it is not easy to find this, but the Efficiency Guide is
important to know
about and to read through for all system builders)

We are thinking of changing this to a totally dynamic number of ports
only limited by the OS setup, but that work is not scheduled for a
specific release yet.


On Tue, Nov 30, 2010 at 12:20 PM, Matthew Sackman <[hidden email]> wrote:
> On Mon, Nov 29, 2010 at 11:18:21PM -0500, Kaiduan Xie wrote:
>> I tried to create 1K TCP/TLS client connections on a Windows box, and
>> got {error, emfile} error, any idea to solve it? I have no problem on
>> Linux after raising the ulimit.
>
> Whilst in theory windows has a limit of around 16M fds per process, the
> Erlang windows port seems to use bits of the MS C runtime. This seems to
> limit severely wrt number of available sockets.

Well this is not a correct statement as already described.
>
> Whilst erlang:system_info(check_io) reports 2048 under Windows, that's
> clearly wrong. But hey, a few years ago, I found that fsync under the
> Windows port was implemented as "return 1;". Such is the quality of the
> Windows port.
This is not the case now, fsync is definitely implemented.

Windows has not been our main target platform for Erlang but I still
think the quality is good even on Windows.

With an increasing number of developers doing serious applications
with Erlang on
Windows the functionality and quality will become even better.


/Kenneth Erlang/OTP, Ericsson

>
> Matthew
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:[hidden email]
>
>

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: {error, emfile} on windows

Kaiduan Xie-2
Kenneth,

Setting ERL_MAX_PORT still does not resolve the problem, it can only
open 31 TLS connections.

C:\erl5.7.1\usr>..\bin\erl.exe -env ERL_MAX_PORTS 4096
Eshell V5.7.1  (abort with ^G)
1> application:start(ssl).
ok
2> Socks = test_tls_sip:connect("10.9.22.85", 5061, 512, 1).
** exception error: no match of right hand side value {error,emfile}
     in function  test_tls_sip:'-connect/4-fun-0-'/5
     in call from lists:foldl/3

connect(Addr, Port, N, TimerValue) ->
    lists:foldl(fun(_I, Acc) ->
        {ok, Socket} = ssl:connect(Addr, Port, [{active, true}]),

        {ok, Pid} = gen_server:start(erlang_tls_connection,
                                     {Socket, TimerValue}, []),
        ssl:controlling_process(Socket, Pid),
        [Pid|Acc] end, [], lists:seq(1, N)).

Thanks,

/Kaiduan

On Tue, Nov 30, 2010 at 9:08 AM, Kenneth Lundin
<[hidden email]> wrote:

> On Windows there is no limit to set in the shell like in UNIX/Linux
> that the Erlang VM
> can read and adjust according to.
> Therefore you can use the environment variable ERL_MAX_PORTS to set a
> higher limit
> thant the default (1024). But bevare that the number you set will
> imply an allocation
> of  memory to hold all these port data structures. So don't set a
> higher value than you really need.
> You can actually set this on the command line when you start erlang,
> like this example:
> erl -env ERL_MAX_PORTS 4096
>
> See the documentation at:
> http://www.erlang.org/doc/efficiency_guide/advanced.html#id64904
> (I admit that it is not easy to find this, but the Efficiency Guide is
> important to know
> about and to read through for all system builders)
>
> We are thinking of changing this to a totally dynamic number of ports
> only limited by the OS setup, but that work is not scheduled for a
> specific release yet.
>
>
> On Tue, Nov 30, 2010 at 12:20 PM, Matthew Sackman <[hidden email]> wrote:
>> On Mon, Nov 29, 2010 at 11:18:21PM -0500, Kaiduan Xie wrote:
>>> I tried to create 1K TCP/TLS client connections on a Windows box, and
>>> got {error, emfile} error, any idea to solve it? I have no problem on
>>> Linux after raising the ulimit.
>>
>> Whilst in theory windows has a limit of around 16M fds per process, the
>> Erlang windows port seems to use bits of the MS C runtime. This seems to
>> limit severely wrt number of available sockets.
>
> Well this is not a correct statement as already described.
>>
>> Whilst erlang:system_info(check_io) reports 2048 under Windows, that's
>> clearly wrong. But hey, a few years ago, I found that fsync under the
>> Windows port was implemented as "return 1;". Such is the quality of the
>> Windows port.
> This is not the case now, fsync is definitely implemented.
>
> Windows has not been our main target platform for Erlang but I still
> think the quality is good even on Windows.
>
> With an increasing number of developers doing serious applications
> with Erlang on
> Windows the functionality and quality will become even better.
>
>
> /Kenneth Erlang/OTP, Ericsson
>
>>
>> Matthew
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:[hidden email]
>>
>>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:[hidden email]
>
>

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: {error, emfile} on windows

Matthew Sackman-2
In reply to this post by Kenneth Lundin
On Tue, Nov 30, 2010 at 03:08:33PM +0100, Kenneth Lundin wrote:
> See the documentation at:
> http://www.erlang.org/doc/efficiency_guide/advanced.html#id64904
> (I admit that it is not easy to find this, but the Efficiency Guide is
> important to know about and to read through for all system builders)

Interesting - I've read that several times before ... except clearly not
the bits about ports and sockets. Thanks for the pointer.

Could you clarify the documentation a bit though - for example, on my
Linux machine, I have a ulimit -n of 100000 and I don't seem to need to
configure anything further (certainly I've never touched ERL_MAX_PORTS)
for Erlang to be able to use many sockets and fds.

Thus for platforms which have an Erlang port and ulimit, which of the
ulimit -n and ERL_MAX_PORTS is used if both are set?

> We are thinking of changing this to a totally dynamic number of ports
> only limited by the OS setup, but that work is not scheduled for a
> specific release yet.

So under Windows, various sources eg
http://blogs.technet.com/markrussinovich/archive/2009/09/29/3283844.aspx
suggest that the limit is 16M. You've suggested that unnecessarily high
numbers are likely to waste memory, so would the limit be detected and
set dynamically at startup or would memory only be allocated as needed?

Also, under unices, file descriptors and sockets come from the same
pool, but under Windows, my understanding is that this is not the case.
Could you confirm that in the Windows port, file descriptors are win32
file handles, and sockets are winsock socket handles? Is there going to
be a supported+documented API to be able to inspect the limits of each
and the number of each allocated that works on both Windows and unices?

> > Whilst in theory windows has a limit of around 16M fds per process, the
> > Erlang windows port seems to use bits of the MS C runtime. This seems to
> > limit severely wrt number of available sockets.
>
> Well this is not a correct statement as already described.

Indeed not. Thanks for the illumination.

Matthew

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:[hidden email]

Loading...