docs on group leaders?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

docs on group leaders?

Xavier Noria
I have read descriptions about what is a group leader and understand the concept, but don't know which role plays exactly in the grand schema of things.

I have checked books and online resources but don't get pass their functional description. For example, what is the point of having each application master become the group leader of its application? When do you change group leaders in an Erlang/Elixir application?

Any pointers appreciated!


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: docs on group leaders?

Fred Hebert-2
On 04/04, Xavier Noria wrote:

>I have read descriptions about what is a group leader and understand the
>concept, but don't know which role plays exactly in the grand schema of
>things.
>
>I have checked books and online resources but don't get pass their
>functional description. For example, what is the point of having each
>application master become the group leader of its application? When do you
>change group leaders in an Erlang/Elixir application?
>
>Any pointers appreciated!

Group leaders let you redirect I/O to the right endpoint. It is one of
the few properties that is inherited by one process to the other,
creating a chain.

By default, an Erlang node has one group leader called 'user'. This
process owns communication with the stdio channels, and all input
requests and output messages transit through that one.

Then, each shell you start becomes its own group leader. This means that
any function you run from the shell will send all its IO data to that
shell process. When you enter the job mode with ^G and pick a shell (c
<number>), what this does is let a special shell-handling process
(usr_drv if memory serves) to pick which input and output to show. If
you have 50 shells from 50 different users, each one has to send their
traffic to the right endpoint.

As such, a slave node, or a shell from a remote node will set its group
leader to a foreign pid and automatically get all its descendent
processes' IO data rerouted to the right place automatically.

So that handles it for the very interactive side of things.

Each OTP application also has its own process--the application
master--becoming a group leader. By default this has no apparent use,
except for two things.

The first one is allowing each process to know which app it belongs to,
cheating with `application:get_env(Var)', as a shorter form than
`application:get_env(App, Var)`. If the process is started within the
app, it has access to the config; if not, it doesn't have it. This is
the only reason documented in a code comment for the usage of group
leaders.

The second usage is for when the application stops. This starts from the
otp-level, shutting down the supervision tree as normal. Then there's
one final step.  Once the whole tree is shut down, the application
master calles 'processes()' to get all the pids on the node, and then
scans their group leaders to find out which pids belong to this specific
application, and then it kills them.

This is a final "garbage collecting" step for application processes.

Both usages are hijacking (or so it seems) the inheritance of group
leaders to track unsupervised children that are running wild out there,
and grouping them within the application family.

I don't know or remember if there's any use for them there aside from
that. It would be easy to imagine redirecting the output of some
applications to remote nodes specifically, on a per-app basis, but
currently the structure of things does not allow that as far as I can
tell.

Regards,
Fred.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: docs on group leaders?

Xavier Noria
Awesome Fred!!!

I was precisely right now thinking about that grouping usage, since application_controller.erl has code like

    ets:match(ac_tab, {{application_master, '$1'}, Master})

used by application.erl as in

    application_controller:get_pid_all_env(group_leader())

which is like grepping based on the group leader. I think I read about this trick somewhere, but has checked several of the resources I normally consult and have failed to find it. Perhaps it was that code comment you mentioned.

The strange thing for someone like me studying this is that this use case seems unrelated to I/O, it is like hijacking the idea of group leader for something entirely different, so your confirmation helps.

Indeed, application masters seem to be just delegating to their previous group leader since it seems all I/O messages get forwarded up as in

     receive
     IoReq when element(1, IoReq) =:= io_request ->
        State#state.gleader ! IoReq,
        ...

Also, if some subtree of the application changed the group leader, that grouping trick wouldn't work. Since the trick is in place in such a core module, I guess the conclusion is that you are generally not supposed to do that unless you know very well what you are doing.

José Valim also explained to me the use case of shells a while back.

Thanks a lot Fred!!!


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: docs on group leaders?

Xavier Noria
There it is, in application_master.erl

    kill_children(Children) ->
        lists:foreach(fun(Pid) -> exit(Pid, kill) end, Children),
        kill_all_procs().

    kill_all_procs() ->
        kill_all_procs_1(processes(), self(), 0).

    kill_all_procs_1([Self|Ps], Self, N) ->
        kill_all_procs_1(Ps, Self, N);
    kill_all_procs_1([P|Ps], Self, N) ->
        case process_info(P, group_leader) of
    {group_leader,Self} ->
        exit(P, kill),
        kill_all_procs_1(Ps, Self, N+1);
    _ ->
        kill_all_procs_1(Ps, Self, N)
        end;
    kill_all_procs_1([], _, 0) -> ok;
    kill_all_procs_1([], _, _) -> kill_all_procs().

I read this file from top to bottom in the last months to do some groundwork, perhaps it was there that I saw the grouping trick instead of reading it in prose.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: docs on group leaders?

Xavier Noria
In reply to this post by Xavier Noria
On Wed, Apr 4, 2018 at 2:32 PM, Xavier Noria <[hidden email]> wrote:

what is the point of having each application master become the group leader of its application?

For the archives, this is now documented: https://github.com/erlang/otp/pull/1771

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions