Erlang filesystem

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Erlang filesystem

Chris Pressey
To expand on what I was saying last night while I was falling asleep:

On Tue, 6 May 2003 Ulf Wiger <etxuwig> wrote:
> - Do not use distributed erlang for your user interface.

That's essentially my rationale for disallowing the evaluation of
arbitrary Erlang functions from tellerl.  Of course one could go and
write a tellerl handler that allows it to be done, but that'd be silly.
'Eval Scheme'-type silly.

(Looking at it in the light of day, the Erlang side of tellerl needs to
be seriously rewritten, using a behaviour and maybe proplists -
hopefully next week.  The Perl side looks just dandy though.)

On Thu, 17 Apr 2003 Ulf Wiger <etxuwig> wrote:
> A common setup is to keep the same code on all nodes -- i.e.
> identical application trees. One very good reason for doing
> this is to avoid having different versions of the same
> module in the system.

This is one of the things that got me thinking about a replicated file
system.

It would be interesting to know how people handle keeping their code in
synch across nodes - i.e., what's most popular - rpc:multicall(code,
load_binary, [Mod, Bin, File])?  rsync?  Manually copying the code over
when it changes?  Using the release handler somehow?

Also - this is embarrassing - but I am starting to suspect that for the
past two years I've had the wrong impression about the file server (i.e.
IoDevices that you get when you call file:open/2 without specifying
'raw'.)  I was operating under the assumption that it was trivial to
open a file on another node (because I swear I did it once, but my
memory might be playing tricks on me, because I've been banging my head
against the wall trying to do it again.)

It IS trivial to read an entire file from another node:

  > rpc:call(catbus, file, read_file, [".cshrc"]).
  {ok,<<35,32,36,70,114,101,101,66,83,68,58,32,115,114,99,47,115,
  104,97,114,101,47,115,107,101,108,47,...>>}

But when I try just opening it:

  > {ok,H} = rpc:call(catbus, file, open, [".cshrc",
  [read]]).
  {ok,<3931.59.0>}
  > rpc:call(catbus, file, read, [H, 100]).
  {error,terminated}
  > file:format_error({error,terminated}).
  "unknown POSIX error"

Same result if I try merely file:read(H, 100).

I'm sure I could get around this by writing a file server process and
running it on catbus, but I thought that's why file_server_2 existed in
the first place!  (I like reinventing the wheel, but not THAT much :)

-Chris


Reply | Threaded
Open this post in threaded view
|

Erlang filesystem

Ulf Wiger-4
On Fri, 9 May 2003, Chris Pressey wrote:

>On Thu, 17 Apr 2003 Ulf Wiger <etxuwig> wrote:
>> A common setup is to keep the same code on all nodes -- i.e.
>> identical application trees. One very good reason for doing
>> this is to avoid having different versions of the same
>> module in the system.
>
>This is one of the things that got me thinking about a
>replicated file system.
>
>It would be interesting to know how people handle keeping
>their code in synch across nodes - i.e., what's most
>popular - rpc:multicall(code, load_binary, [Mod, Bin,
>File])?  rsync?  Manually copying the code over when it
>changes?  Using the release handler somehow?

The release handler works well for this. It's designed to
support synchronized upgrade on multiple nodes.


>It IS trivial to read an entire file from another node:
>
>  > rpc:call(catbus, file, read_file, [".cshrc"]).
>  {ok,<<35,32,36,70,114,101,101,66,83,68,58,32,115,114,99,47,115,
>  104,97,114,101,47,115,107,101,108,47,...>>}
>
>But when I try just opening it:
>
>  > {ok,H} = rpc:call(catbus, file, open, [".cshrc",
>  [read]]).
>  {ok,<3931.59.0>}
>  > rpc:call(catbus, file, read, [H, 100]).
>  {error,terminated}
>  > file:format_error({error,terminated}).
>  "unknown POSIX error"

Not sure why you get 'terminated', but a problem with doing
it this way is that the rpc:call/3 will be executed in a
temporary process on catbus; when the call completes, the
process is terminated, and since it is the owner of the file
descriptor, the file will be closed automatically.

I think what you want to do is to run your other nodes as
loader slaves to catbus. It's not incredibly clear from the
documentation (or I haven't found the place in the
documentation where it's clearly described) how to do this.
You may try with erl -loader inet -hosts ...

or reading the code in file_server.erl, one might perhaps
try erl -master catbus, but that doesn't seem
to work for me, and I don't find any documentation on it.

(I'm sorry I can't give you a working example. From a
cursory glance, it's not obvious to me how one tests an
erlang node using another erlang node on the same host as a
boot loader (it seems consistent with the erlang philosophy
that one should be able to do this), and I'm not sure if you
need to use a boot server in order to have a file server.
I'm sure this is explained in the docs somewhere, but it's
late, and I intend to go home and get some sleep.)

/Uffe
--
Ulf Wiger, Senior Specialist,
   / / /   Architecture & Design of Carrier-Class Software
  / / /    Strategic Product & System Management
 / / /     Ericsson AB, Connectivity and Control Nodes



Reply | Threaded
Open this post in threaded view
|

Erlang filesystem

Chris Pressey
On Fri, 9 May 2003 23:54:05 +0200 (MEST)
Ulf Wiger <etxuwig> wrote:

> On Fri, 9 May 2003, Chris Pressey wrote:
> >  > {ok,H} = rpc:call(catbus, file, open, [".cshrc",
> >  [read]]).
> >  {ok,<3931.59.0>}
> >  > rpc:call(catbus, file, read, [H, 100]).
> >  {error,terminated}
> >  > file:format_error({error,terminated}).
> >  "unknown POSIX error"
>
> Not sure why you get 'terminated', but a problem with doing
> it this way is that the rpc:call/3 will be executed in a
> temporary process on catbus; when the call completes, the
> process is terminated, and since it is the owner of the file
> descriptor, the file will be closed automatically.

Ah, that makes sense.

I tried various ways of working around this, but as they got
increasingly complex with still no luck, I gave up on this approach.

> I think what you want to do is to run your other nodes as
> loader slaves to catbus. It's not incredibly clear from the
> documentation (or I haven't found the place in the
> documentation where it's clearly described) how to do this.
> You may try with erl -loader inet -hosts ...

That's pretty neat :)  After I figured out that I had to start erl with
-mode embedded as well, I got catbus to load all its code from the other
computer (whose node started erl_boot_server, so actually catbus was the
loader slave) - and proved it to myself by removing $ROOT/lib ... neat!

Only weirdness was the following messages on catbus on startup:

  exec: ose_inet: not found

And on exiting:

  Error in process <0.2.0> with exit value: {badarg,[{erlang,unlink,
  [noport]},{erl_prim_loader,stop_port,1},{erl_prim_loader,loop,3}]}

But it otherwise worked fine.

> or reading the code in file_server.erl, one might perhaps
> try erl -master catbus, but that doesn't seem
> to work for me, and I don't find any documentation on it.

Me neither, it just hangs when I try it.

> (I'm sorry I can't give you a working example. From a
> cursory glance, it's not obvious to me how one tests an
> erlang node using another erlang node on the same host as a
> boot loader (it seems consistent with the erlang philosophy
> that one should be able to do this), and I'm not sure if you
> need to use a boot server in order to have a file server.
> I'm sure this is explained in the docs somewhere, but it's
> late, and I intend to go home and get some sleep.)
>
> /Uffe

No problem, thanks for pointing me in what seems to be the right
direction.  I've gained a lot of insight just wrestling with this.

If the raison d'etre of the distributed file server is to load code,
then it might not be what I'm looking for (not that I know exactly what
I'm looking for...)

Feels like it'd be limited to erl_prim_loader:get_file/1.

Looking at file.erl, it mostly seems to look for processes named
file_server or file_server_2 on the *local* node.  I'm not sure how one
would tell it to look for that process on some external node, besides
hacking the source.

Although, an intuitive way from a usage perspective would be

  file:open(".cshrc", [read, {on_node, catbus}]).

Or, using "Joe logic", perhaps

  file:open({catbus, ".cshrc"}, [read]).

to make it orthogonal with file:delete({Node, FileName}) et al.

Other than that, I don't know where I'm going with this, and I'll have
to re-examine my assumptions.

-Chris