Yaws's "sendfile" linked-in driver : "stealing control of fd" error

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Zabrane Mickael
Hi OTP team & Steve,

I'm using the "sendfile" (thanks to Steve Vinoski) linked-in driver
borrowed from "Yaws"
in one of my pet project.

Today, I noticed this strange error:
=ERROR REPORT==== 17-Apr-2011::20:49:18 ===
driver_select(0x0000000000000c31, 24, ERL_DRV_WRITE, 1) by
sendfile_drv driver #Port<0.3121> stealing control of fd=24 from
output driver tcp_inet #Port<0.10029>

Searching "stealing control of fd=" in the "otp_src_R14B02" source
code returns this:

otp_src_R14B02> find . -type "f" | xargs grep -n "stealing control of" | head -1
./erts/emulator/sys/common/erl_check_io.c:883:

The C function printing out this error line is:
static void
steal(erts_dsprintf_buf_t *dsbufp, ErtsDrvEventState *state, int mode)
{
    erts_dsprintf(dsbufp, "stealing control of fd=%d from ", (int) state->fd);
    switch (state->type) {
    case ERTS_EV_TYPE_DRV_SEL: {
        int deselect_mode = 0;
        Eterm iid = state->driver.select->inport;
        Eterm oid = state->driver.select->outport;
        if ((mode & ERL_DRV_READ) && (is_not_nil(iid))) {
            erts_dsprintf(dsbufp, "input driver ");
            print_driver_name(dsbufp, iid);
            erts_dsprintf(dsbufp, "%T ", iid);
            deselect_mode |= ERL_DRV_READ;
        }


Could someone please tell me what's the aim of this C function?
Should I consider this as an error or can I forget about it?

Regards,
Zab
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Steve Vinoski-2
On Sun, Apr 17, 2011 at 5:23 PM, zabrane Mikael <[hidden email]> wrote:

> Hi OTP team & Steve,
>
> I'm using the "sendfile" (thanks to Steve Vinoski) linked-in driver
> borrowed from "Yaws"
> in one of my pet project.
>
> Today, I noticed this strange error:
> =ERROR REPORT==== 17-Apr-2011::20:49:18 ===
> driver_select(0x0000000000000c31, 24, ERL_DRV_WRITE, 1) by
> sendfile_drv driver #Port<0.3121> stealing control of fd=24 from
> output driver tcp_inet #Port<0.10029>

I can't speak authoritatively about exactly what causes this, but I
can say that most of the times I've seen this were when serious
problems were occurring -- most often prior to a fatal VM crash, but I
think it's also occasionally occurred when the VM was shutting down
normally. It's been awhile since I've seen it at all, though.

> Could someone please tell me what's the aim of this C function?
> Should I consider this as an error or can I forget about it?

Someone familiar with the "steal" code should correct me if I'm wrong
(please!), but I would recommend treating this as an error. Can you
provide details about what your app is doing when it causes this error
report?

--steve
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Sverker Eriksson
On Sun, Apr 17, 2011 at 5:23 PM, zabrane Mikael <[hidden email]> wrote:

>> Hi OTP team & Steve,
>>
>> I'm using the "sendfile" (thanks to Steve Vinoski) linked-in driver
>> borrowed from "Yaws"
>> in one of my pet project.
>>
>> Today, I noticed this strange error:
>> =ERROR REPORT==== 17-Apr-2011::20:49:18 ===
>> driver_select(0x0000000000000c31, 24, ERL_DRV_WRITE, 1) by
>> sendfile_drv driver #Port<0.3121> stealing control of fd=24 from
>> output driver tcp_inet #Port<0.10029>
>>    
>
>  
The error message kind of says it all:
The driver "sendfile_drv" is calling driver_select() to subscribe for
output events on file descriptor 24. BUT, driver "tcp_inet" is already
subscribing to output events on that very same file descriptor 24.
Driver "sendfile_drv" thus steals the file descriptor from "tcp_inet" as
only one port at a time can subscribe to a particular file descriptor event.

I think you should consider this as an error. Drivers are not supposed
to act like this. The normal cause of this is a driver that forgets to
deselect a file descriptor before closing it. Despite that I would
regard sendfile_drv as the primary suspect here as the new kid on the block.

/Sverker, Erlang/OTP

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Zabrane Mickael
Hi Sverker,

Thanks a lot for these explanations.
Hope that Steve (and Tuncer with his "sendfile" version) can fix that very soon.

Regards
Zab

Le 18 avr. 2011 à 12:02, Sverker Eriksson a écrit :

> On Sun, Apr 17, 2011 at 5:23 PM, zabrane Mikael <[hidden email]> wrote:
>>> Hi OTP team & Steve,
>>>
>>> I'm using the "sendfile" (thanks to Steve Vinoski) linked-in driver
>>> borrowed from "Yaws"
>>> in one of my pet project.
>>>
>>> Today, I noticed this strange error:
>>> =ERROR REPORT==== 17-Apr-2011::20:49:18 ===
>>> driver_select(0x0000000000000c31, 24, ERL_DRV_WRITE, 1) by
>>> sendfile_drv driver #Port<0.3121> stealing control of fd=24 from
>>> output driver tcp_inet #Port<0.10029>
>>>    
>>
>>  
> The error message kind of says it all:
> The driver "sendfile_drv" is calling driver_select() to subscribe for output events on file descriptor 24. BUT, driver "tcp_inet" is already subscribing to output events on that very same file descriptor 24. Driver "sendfile_drv" thus steals the file descriptor from "tcp_inet" as only one port at a time can subscribe to a particular file descriptor event.
>
> I think you should consider this as an error. Drivers are not supposed to act like this. The normal cause of this is a driver that forgets to deselect a file descriptor before closing it. Despite that I would regard sendfile_drv as the primary suspect here as the new kid on the block.
>
> /Sverker, Erlang/OTP
>


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Zabrane Mickael
In reply to this post by Sverker Eriksson
Hi guys,

I can confirm that using "sendfile:compat_send/4" worked perfectly in my case
without any error message.

Culprit found ;-)

Regards,
Zabrane

Le 18 avr. 2011 à 12:02, Sverker Eriksson a écrit :

> On Sun, Apr 17, 2011 at 5:23 PM, zabrane Mikael <[hidden email]> wrote:
>>> Hi OTP team & Steve,
>>>
>>> I'm using the "sendfile" (thanks to Steve Vinoski) linked-in driver
>>> borrowed from "Yaws"
>>> in one of my pet project.
>>>
>>> Today, I noticed this strange error:
>>> =ERROR REPORT==== 17-Apr-2011::20:49:18 ===
>>> driver_select(0x0000000000000c31, 24, ERL_DRV_WRITE, 1) by
>>> sendfile_drv driver #Port<0.3121> stealing control of fd=24 from
>>> output driver tcp_inet #Port<0.10029>
>>>    
>>
>>  
> The error message kind of says it all:
> The driver "sendfile_drv" is calling driver_select() to subscribe for output events on file descriptor 24. BUT, driver "tcp_inet" is already subscribing to output events on that very same file descriptor 24. Driver "sendfile_drv" thus steals the file descriptor from "tcp_inet" as only one port at a time can subscribe to a particular file descriptor event.
>
> I think you should consider this as an error. Drivers are not supposed to act like this. The normal cause of this is a driver that forgets to deselect a file descriptor before closing it. Despite that I would regard sendfile_drv as the primary suspect here as the new kid on the block.
>
> /Sverker, Erlang/OTP
>


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Steve Vinoski-2
In reply to this post by Sverker Eriksson
On Mon, Apr 18, 2011 at 6:02 AM, Sverker Eriksson
<[hidden email]> wrote:

> On Sun, Apr 17, 2011 at 5:23 PM, zabrane Mikael <[hidden email]> wrote:
>>>
>>> Hi OTP team & Steve,
>>>
>>> I'm using the "sendfile" (thanks to Steve Vinoski) linked-in driver
>>> borrowed from "Yaws"
>>> in one of my pet project.
>>>
>>> Today, I noticed this strange error:
>>> =ERROR REPORT==== 17-Apr-2011::20:49:18 ===
>>> driver_select(0x0000000000000c31, 24, ERL_DRV_WRITE, 1) by
>>> sendfile_drv driver #Port<0.3121> stealing control of fd=24 from
>>> output driver tcp_inet #Port<0.10029>
>>>
>>
>>
>
> The error message kind of says it all:
> The driver "sendfile_drv" is calling driver_select() to subscribe for output
> events on file descriptor 24. BUT, driver "tcp_inet" is already subscribing
> to output events on that very same file descriptor 24. Driver "sendfile_drv"
> thus steals the file descriptor from "tcp_inet" as only one port at a time
> can subscribe to a particular file descriptor event.
>
> I think you should consider this as an error. Drivers are not supposed to
> act like this. The normal cause of this is a driver that forgets to deselect
> a file descriptor before closing it. Despite that I would regard
> sendfile_drv as the primary suspect here as the new kid on the block.

Hi Sverker, I agree it's an error in the sendfile driver, but so far
we have no evidence that it results in erroneous transfers or
incorrect data. As I explained earlier, I've only ever seen it under
catastrophic circumstances, but the fatal errors I saw weren't caused
by this issue; rather, this issue was a side effect of the fatal
errors. What Mikael is seeing, though, has nothing to do with fatal
errors, so I think for cases like his it's just a rare and innocuous
error report. I suspect it occurs due to the tcp driver closing the
socket before the sendfile driver has dropped interest in it but
that's only a guess.

I'll see if I can't come up with a way to reliably reproduce this, but
either way it might not be fixable in the current code since the
sendfile driver can't know if or when the tcp driver is also
registered for the socket being writeable. Maybe this can be handled
correctly as part of Tuncer's effort to add the sendfile functionality
directly into Erlang/OTP.

thanks,
--steve
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Zabrane Mickael-2
Hi Steve,

> Hi Sverker, I agree it's an error in the sendfile driver, but so far
> we have no evidence that it results in erroneous transfers or
> incorrect data. As I explained earlier, I've only ever seen it under
> catastrophic circumstances, but the fatal errors I saw weren't caused
> by this issue; rather, this issue was a side effect of the fatal
> errors. What Mikael is seeing, though, has nothing to do with fatal
> errors, so I think for cases like his it's just a rare and innocuous
> error report. I suspect it occurs due to the tcp driver closing the
> socket before the sendfile driver has dropped interest in it but
> that's only a guess.

That's correct. I confirm that in my case, the error occurs only when the Socket is
closed.

> I'll see if I can't come up with a way to reliably reproduce this, but
> either way it might not be fixable in the current code since the
> sendfile driver can't know if or when the tcp driver is also
> registered for the socket being writeable. Maybe this can be handled
> correctly as part of Tuncer's effort to add the sendfile functionality
> directly into Erlang/OTP.


If you need help, don't hesitate.

Regards,
Zab



Reply | Threaded
Open this post in threaded view
|

Re: Yaws's "sendfile" linked-in driver : "stealing control of fd" error

Tuncer Ayaz
In reply to this post by Zabrane Mickael
To test yaws with sendfile_drv instead of yaws_sendfile_drv get
the temporary 'sendfile_drv' branch from my yaws fork and run:
$ rebar get-deps compile
or
$ rebar g-d co

$ bin/yaws --conf etc/yaws.conf --pa deps/sendfile/ebin
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions