How to configure processes, (and applications)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Anders Nygren-5
Sorry,
but it got sent before I was finished.

Hi
I am pondering the question of how to best get configuration data
to processes and/or applications in an OTP based system.

So far I have as a precondition that
1 - Useful default values are in the application resource file
2 - Installation specific values are in the sys.config file

But my question is, "How do I best get the data into the processes
that need it?"

In my first OTP based system I had the processes do application:env/1
whenever they needed the data, but I am not very happy with this.

Now I am considering having the main supervisor of the application
read the configuration data and pass it in the start_link calls to the
different processes.

I believe that this will be better, easier to reuse, test before the entire
system is developed and so on.

So is this a good idea?
How does the the big guys do it, (Ulf??)

/Anders Nygren


Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Vance Shipley-2
Anders,

Argument passing is the Erlang way of doing things.  So looking
up application variables in the supervisor and passing them as
arguments to the start functions of workers is better.  What I
tend to do though is look them up in the application callback
module and pass them to the top level supervisor.  When you have
multiple instances of the same tree this saves more work.

The thing to consider of course is when they will change.  The
application callback module can export config_change/3 which will
be called after a code replacement if there are any changes to
the configuration parameters.

You also might want to change configurations parameters in runtime
yourself with application:set_env/3.  If you do so you will then
need to propagate the change to the running processes somehow.
If you had left the workers to look them up each time you wouldn't
have to.

        -Vance


On Mon, Feb 21, 2005 at 10:11:29AM -0600, Anders Nygren wrote:
}
}  I am pondering the question of how to best get configuration data
}  to processes and/or applications in an OTP based system.


Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Vladimir Sekissov
In reply to this post by Anders Nygren-5
Good day,

anders.nygren> In my first OTP based system I had the processes do application:env/1
anders.nygren> whenever they needed the data, but I am not very happy with this.
anders.nygren>
anders.nygren> Now I am considering having the main supervisor of the application
anders.nygren> read the configuration data and pass it in the start_link calls to the
anders.nygren> different processes.
anders.nygren>
anders.nygren> I believe that this will be better, easier to reuse, test before the entire
anders.nygren> system is developed and so on.

I also use the similar approach.  My application usually looks like

                         
                    +------------------------+
                    |  Application supervisor|
                    +------------------------+
                       |                   |
                       |   |
                       V                   V  
          +--------------+               +----------------------+
          |Config manager|               | Subprocess supervisor|
          +--------------+               +----------------------+
                 ^                                  ...
                 +----------------------------->Subprocesses

Configuration loading is similar to INETs http-server:

0. Application supervisor starts 'Config' manager and 'Subprocess
   supervisor'. They are restarted and killed together(one_for_one policy).

1. Config manager loads configuration data from application environment
or text file.

2. Bootstraps extracting information about subsystem configuration modules.

3. If config was in text file - parse it calling subsystems callbacks
 if exist, callbacks can add their own data. Config is processed in
 the usual top-down-left-right manner. Example:

parse_config(Path=[login, authen], [User], UserContent, _) ->
  {{authen, login, User}, UserContent};
parse_config(Path=[authen], [], Content, _) ->
  Content;
....

4. Prepare config to store in the same manner:

store_config({Key={authen, login, User}, UserInfo}, AllEntries) ->
  {Key, UserInfo ++ find_user_identity(User, AllEntries)};
.....

and store result configuration in desired backend (application env,
ets/mnesia table).

5. Start subprocesses providing
'Config manager' and 'Subprocess supervisor' pids to callbacks:

postload_config(Config, Sup) ->
  EventServer = ?server_spec
  {ok, _} = supervisor:start_child(Sup, EventServer),
  case config:gv_all(Config, event_handler) of
    [] ->
      ok;
    HlrTypes ->
      ok = lists:foreach(fun (T) -> ok = add_relay_handler(T) end, HlrTypes)
  end;

IMHO this way is more verbose but more flexible than direct OTP
one(but compatible).

I can send you code if you want one.

Best Regards,
Vladimir Sekissov



Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Anders Nygren-5
On Mon, 21 Feb 2005 22:35:59 +0500 (YEKT), Vladimir Sekissov
<svg> wrote:
> Good day,

SNIP

>
> IMHO this way is more verbose but more flexible than direct OTP
> one(but compatible).
>
> I can send you code if you want one.
>
Hi
I am not sure I completely understand what You are doing
but please send some code as this looks interesting.

Does Your method also support configuration changes
while the system is running?

/Anders


Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Vladimir Sekissov

anders.nygren> I am not sure I completely understand what You are doing
anders.nygren> but please send some code as this looks interesting.
anders.nygren>
anders.nygren> Does Your method also support configuration changes
anders.nygren> while the system is running?

I don't know how this process could be generalized but in any way it
is ordinary OTP application, follow OTP rules and you can customize
its behaviour accordong to your needs (for my current needs restarting
the whole subsystem is enough).

The code is in attachement. As example of usage you can look at

tacacs/src/tac_app.erl
tacacs/src/tac_conf.erl

Example config file: tacacs/templates/tacacs.conf.stl

You must make at top level to compile everything. You need GNU make >=
3.80 for this.

If you want to use application environment instead of text file you
can use {list, Config} as parameter instead of {file, Name}.

To change backend to application environment from ets, modify the
following property_server callbacks:

bootstrap/2
new_config/1
delete_config/1
store_entry/1
handle_op/2
config_to_list/1

Best Regargs,
Vladimir Sekissov
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ws.tar.gz
Type: application/octet-stream
Size: 288491 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20050221/82c8b066/attachment.obj>

Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Vladimir Sekissov
Oops,

I'm awfully sorry for all subscribers.
I've sent ~1Mb attachment to the mail-list.

Best Regards,
Vladimir Sekissov


Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Sean Hinde-2
In reply to this post by Anders Nygren-5

On 21 Feb 2005, at 16:11, Anders Nygren wrote:

> Sorry,
> but it got sent before I was finished.
>
> Hi
> I am pondering the question of how to best get configuration data
> to processes and/or applications in an OTP based system.
>
> So far I have as a precondition that
> 1 - Useful default values are in the application resource file
> 2 - Installation specific values are in the sys.config file
>
> But my question is, "How do I best get the data into the processes
> that need it?"

Most of our apps use a simple mechanism where the supervisor reads the
config and passes it into the start_link function. It works well.

We have moved away from using sys.config for our own apps (i.e. stuff
where we can influence this - not inets, kernel and friends) and now
have a well known config directory location which holds a file called
app_name.conf for any application which needs config data. This is
normally something which can be read with file:consult/1.

This allows us to have a single standard sys.config for all new systems
which makes multiple configs easier to manage, and crucially a restart
is not needed to re-read the config.

Changing config on the fly is done by exporting config change functions
for each gen_server and putting code to handle this in the call
callbacks. We also have a persistent storage for config data modelled
along the same api as application:get_env etc. Applications are free to
use that instead of or as well as config files.

I'm a big fan of K.I.S.S!

Sean




Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Hal Snyder-2
In reply to this post by Anders Nygren-5
Anders Nygren <anders.nygren> writes:

> On Mon, 21 Feb 2005 22:35:59 +0500 (YEKT), Vladimir Sekissov
> <svg> wrote:
>> Good day,
>
> SNIP
>
>>
>> IMHO this way is more verbose but more flexible than direct OTP
>> one(but compatible).
>>
>> I can send you code if you want one.
>>
> Hi
> I am not sure I completely understand what You are doing
> but please send some code as this looks interesting.
>
> Does Your method also support configuration changes
> while the system is running?

On a telephony platform we regularly have the following requirements:

- change config without restarting servers/nodes/processes

- change persists when things are restarted

- change is picked up when systems are replaced or added

Specifying configuration in config files or command line commits you
to real-time edits in the above cases.

So, is there a way to store things in a distributed, non-volatile,
scalable database (which is read-mostly, if you want to take advantage
of that property) in Erlang/OTP? Answer is yes, there is: mnesia.

There are bootstrap issues making sure OTP nodes all find each other
and sync up properly. See extra_db_nodes posting to this list, e.g.:

http://www.erlang.org/ml-archive/erlang-questions/200502/msg00176.html

Also with any replicated database, you will go nuts with
partition-and-reconnect unless you consider the requirements of the
situation and system topology during design.


Reply | Threaded
Open this post in threaded view
|

How to configure processes, (and applications)

Mikael Karlsson
In reply to this post by Vladimir Sekissov
Actually your attachement looks pretty cool,
could you elaborate on what it does, the
documentation seems to be much in the code :-)

Best Regards
Mikael
 
m?ndag 21 februari 2005 21:56 skrev Vladimir Sekissov:
> Oops,
>
> I'm awfully sorry for all subscribers.
> I've sent ~1Mb attachment to the mail-list.
>
> Best Regards,
> Vladimir Sekissov