Starting Erlang

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Mikael Pettersson
On Wed, 24 Apr 2002 14:50:01 +0200 (CEST), Joe Armstrong wrote:

> 2) An application is (usually) three files.
>
>   foo.bat  (windows launcher)
>   foo.sh   (unix launcher)
>   foo.ear  (Erlang archive)
>
>           foo.bar and foo.sh would be one liners
>           containing just
>
>   erlRun foo.ear -s Mod Func Arg1 Arg2 ... ArgN
>
>   foo.ear is assumed to be a Mod x Code archive -
>   The code in this is loaded (lazily)

Please please please make .ear be a standard (e.g. POSIX) archive
format, usable by standard tools (e.g. tar/cpio/pax).
The world doesn't need Yet Another Pointless Archive Format(TM).

/Mikael


Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Joe Williams-2
On Wed, 24 Apr 2002, Mikael Pettersson wrote:

> On Wed, 24 Apr 2002 14:50:01 +0200 (CEST), Joe Armstrong wrote:
> > 2) An application is (usually) three files.
> >
> >   foo.bat  (windows launcher)
> >   foo.sh   (unix launcher)
> >   foo.ear  (Erlang archive)
> >
> >           foo.bar and foo.sh would be one liners
> >           containing just
> >
> >   erlRun foo.ear -s Mod Func Arg1 Arg2 ... ArgN
> >
> >   foo.ear is assumed to be a Mod x Code archive -
> >   The code in this is loaded (lazily)
>
> Please please please make .ear be a standard (e.g. POSIX) archive
> format, usable by standard tools (e.g. tar/cpio/pax).
> The world doesn't need Yet Another Pointless Archive Format(TM).
>
> /Mikael
>

  Ummm - It's not clear to me the advantages of following a standard
format outweigh the advantages of using a non-standard format....

  Since I expect an Erlang program to produce the YAPAF file and
another Erlang program to consume the YAPAF I don't see why it matters
what the format is. A symmetric pair of term_to_binary and
binary_to_term calls is all that is needed - not exactly complex code.

  Using a YAPAF file means the standard tools like tar/cpio/... cannot
work on these formats - and this you can view as either an advantage
or a disadvantage depending upon your perspective. I think it's an
advantage since other people cannot (easily) mess around with your
files :-)

  In my experience doing things in pure Erlang is usually quicker than
mixing tools (apart from if the non Erlang thing is itself terribly
complex).

  Standard tools seem in my experience not to be so standard (or maybe
I've just been unlucky) - and don't work identically on all platforms
- I know they should - but they often don't.

  Even if they *do* work they are usually not available on all
platforms - I would not, for example, want to have to install the
Cygnus gnu utilities on windows in order to list the contents of a
archive file.

  I would (possibly) agree if unpacking the archive file occurred late
in the boot sequence - the problem is that the *first* file read in
the boot sequence *is* the archive file. So basically all I want to do
is execute a minimal bit of code that says, something like:

        {ok, Bin} = prim_file_read(BootFile),
        {Mods, {M,F,A}} = binary_to_term(Bin),
        map(fun({Mod,Bin}) -> load_code(Mod, Bin) end, Mods),
        apply(M,F,A)

  I don't (at this stage) want to execute complex code to unpack a tar
file format before I can get started - remember this happens very
*early* in the boot sequence so I want this to be as fast as possible
(so I can do single line shell scripts for scripting applications)
hopefully much faster than (say) perl.

  I don't mind unpacking tar files etc. *late* in the boot sequence -
since by then all my code for error handling and recovery is loaded -
but not early in the boot sequence.

  /Joe




Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Vance Shipley-2


On Thu, Apr 25, 2002 at 09:42:50AM +0200, Joe Armstrong wrote:
}  
}    The vast majority of systems seem to fetch their code from a file system
}  either local, NFS mounted or flashed.
}  
}    init.erl has the possibility to fetch code from the net.
}  
}    *does anybody use the -loader inet -hosts ...* stuff ?????


On Thu, Apr 25, 2002 at 09:57:27AM +0200, Ulf Wiger wrote:
}
}  We have at least discussed this option as we move towards running
}  Erlang on our device boards. A simple NFS mount might do the
}  trick, though. .....


We have been thinking along the same lines.  Erlang/OTP is well
suited for a more distributed control architecture and we had been
planning to run some nodes embedded on cPCI peripheral cards.  In
this scenario I had thought that using the cPSB ethernet bus to load
code and do distribution was a perfect fit.

        -Vance

--
Vance Shipley
Motivity Telecom Inc.
+1 519 579 5816
vances


Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Thomas Lindgren-3
In reply to this post by Joe Williams-2

>> Please please please make .ear be a standard (e.g. POSIX) archive
>> format, usable by standard tools (e.g. tar/cpio/pax).
>> The world doesn't need Yet Another Pointless Archive Format(TM).
>>
>> /Mikael
>
>  Ummm - It's not clear to me the advantages of following a standard
>format outweigh the advantages of using a non-standard format....
>
>  Since I expect an Erlang program to produce the YAPAF file and
>another Erlang program to consume the YAPAF I don't see why it matters
>what the format is. A symmetric pair of term_to_binary and
>binary_to_term calls is all that is needed - not exactly complex code.

If there is any interest, I have not one but two .ear formats with
encoder/decoders :-) Yes, they are YAPAFs, but straightforward ones.(On the
bright side, I have archived and installed the entire OTP .beam collection
with both. On Solaris and Linux.)

The main advantage of doing it in Erlang is portability, hence availability.
(This doesn't argue for a YAPAF, of course, since we could implement some
well-known format in Erlang.) We can use the same code on Windows and
VxWorks as on various Unices. (Using a YAPAF, we also get the double-edged
sword of flexibility :-)

-- Thomas



Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Miguel Barreiro Paz-2
In reply to this post by Vance Shipley-2


>  We have at least discussed this option as we move towards running
>  Erlang on our device boards. A simple NFS mount might do the
>  trick, though. .....

        The problem with NFS is that upon network failures tends to behave
in rather unpredictable ways, from clean unmounting to mounts hanging or
not recovering for an extremely long time (and thus processes with their
cwd or open descriptors on that filesystem either hanging or dying). One
certainly wants a process reading boot code to work or fail, not to wait
for a long time in an uncertain state. Also events like rebooting a
server and not the clients may range from harmless to a recipe for
disaster depending on client/server combinations.

        (I had to say that, just after fighting a mountain of Linux, AIX,
IRIX and Solaris boxes of various ages using NFS 2 and 3, over TCP and
UDP, with automount and autofs or nothing at all, with soft and hard
mounts, with... agh)

Regards,

Miguel




Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Hal Snyder-2
Miguel Barreiro Paz <enano> writes:

> The problem with NFS is that upon network failures tends to behave
> in rather unpredictable ways, from clean unmounting to mounts
> hanging or not recovering for an extremely long time (and thus
> processes with their cwd or open descriptors on that filesystem
> either hanging or dying)...

Agreed.

All too often, NFS turns small failures into large ones. And it
thereby introduces a single point of failure into a nicely distributed
OTP system, no?


Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Per Hedeland-4
In reply to this post by Thomas Lindgren-3
>                                                                    
> >> Please please please make .ear be a standard (e.g. POSIX) archive
> >> format, usable by standard tools (e.g. tar/cpio/pax).            
> >> The world doesn't need Yet Another Pointless Archive Format(TM).
> >>                                                                  
> >> /Mikael                                                          
> >                                                                  
> >  Ummm - It's not clear to me the advantages of following a        
standard                                                              
> >format outweigh the advantages of using a non-standard format....  
> >                                                                  
> >  Since I expect an Erlang program to produce the YAPAF file and  
> >another Erlang program to consume the YAPAF I don't see why it    
matters                                                              
> >what the format is. A symmetric pair of term_to_binary and        
> >binary_to_term calls is all that is needed - not exactly complex  
code.                                                                
>                                                                    
> If there is any interest, I have not one but two .ear formats with  
> encoder/decoders :-) Yes, they are YAPAFs, but straightforward      
ones.(On the                                                          
> bright side, I have archived and installed the entire OTP .beam    
collection                                                            
> with both. On Solaris and Linux.)                                  
>                                                                    
> The main advantage of doing it in Erlang is portability, hence      
availability.                                                        
> (This doesn't argue for a YAPAF, of course, since we could implement
some                                                                  
> well-known format in Erlang.) We can use the same code on Windows  
and                                                                  
> VxWorks as on various Unices. (Using a YAPAF, we also get the      
double-edged                                                          
> sword of flexibility :-)                                            
>                                                                    
> -- Thomas                                                          
>                                                                    
As we discussed before Thomas, I think that support for other        
resources than .beam files should be supported (i.e. loadable drivers
but make it generic). Support for different architectures in parallell
should be included.                                                  
Next had this with its fat binaries.                                  
(It also had utilities to slim the binaries ;-)                      
                                                                     
BTW, Lack of support for other files is than .beam files is what makes
the network bootloader useless.                                      
                                                                     
/PER                                                                  
                                                                     
=========================================================            
Per Bergqvist                                                        
Synapse Systems AB                                                    
Phone: +46 709 686 685                                                
Email: per                                                  


Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Martin Bjorklund-2
Per Bergqvist <per> wrote:

> As we discussed before Thomas, I think that support for other        
> resources than .beam files should be supported (i.e. loadable drivers
> but make it generic).

Very good point!  Also external port programs and data files. I fixed
this for the R7 SAE that Joe did (I sent that to you Joe).  We're
using this modified SAE to distribute RPMs (one with sae itself, one
for the app).  The main thing that needs to be done if you don't want
to modify existing code is to support code:priv_dir/1.  In my
modification I added that function to Joe's simple code.erl and let
each RPM package install it's priv dir into
/usr/local/lib/erlsae/AppName/priv/.  The reason for this was that
existing code usually calls priv_dir/1 and then use the file.erl
functions to read the files.  Of course it would be even better if you
could provide a new API for application specific files which is not
dependent on the local filesystem (or maybe add a new filedescriptor
so the existing file.erl functions could be used)..


/martin





Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Scott Lystig Fritchie-3
In reply to this post by Hal Snyder-2
>>>>> "hs" == Hal Snyder <hal> writes:

hs> Miguel Barreiro Paz <enano> writes:
>> The problem with NFS is that upon network failures tends to behave
>> in rather unpredictable ways, [...]

hs> All too often, NFS turns small failures into large ones. And it
hs> thereby introduces a single point of failure into a nicely
hs> distributed OTP system, no?

No.  At least, not the OTP system that I worked on at my last gig.
But then again, we implemented the NFS client in Erlang and chose
quite intentionally not to implement the braindamaged behavior that
most UNIX kernels have evolved and that no one (myself included) has
bothered To Do a Better Way.  :-)

To be fair, that work also meant making the applications aware that
their "disk" operations might fail.  Many developers treat the file
system like the ancients regarded fish in the sea: it's just there.
Developers that do check file op errors almost never do it with the
mindset of, "Oh, that one might be only temporary."  So, most UNIX
flavors try to hide from user space apps the funky & weird things can
(and do) happen regularly to networks.  The mishmash result, without a
doubt, sucks.

Having said all that malicious and quite true stuff about kernel NFS
client implementations ... If you have decent & sane ways of dealing
with network errors, there are several NFS servers that have had
dozens of person years spent on them to be able to move huge amounts
of data across a network very quickly....

-Scott


Reply | Threaded
Open this post in threaded view
|

Starting Erlang

Thomas Lindgren-3
In reply to this post by Per Hedeland-4

>As we discussed before Thomas, I think that support for other
>resources than .beam files should be supported (i.e. loadable drivers
>but make it generic). Support for different architectures in parallell
>should be included.
>Next had this with its fat binaries.
>(It also had utilities to slim the binaries ;-)
>
>BTW, Lack of support for other files is than .beam files is what makes
>the network bootloader useless.

Yes, good point.

After you mentioned it last time, I wrote some code to take care of that
problem. Nominally, the archivers (at least one of them) support archiving
and installing non-beam files as well, in particular drivers. I haven't had
the opportunity to check that it really works, though. Nor that it works the
Right Way.

(The development of those .ear programs was driven by listening to Per, in
case anyone wonders :-)

-- Thomas