I/O from binaries

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

I/O from binaries

Bengt Kleberg-3


> Subject: I/o frmo binaries
> MIME-Version: 1.0
> Content-ID: <1360.1004902273.0>
> Date: Sun, 04 Nov 2001 20:32:08 +0100
> From: Robert Virding <rv>
>
> Many have requested a module which allows doing i/o to/from binaries.
> Here is a small module I have fixed which does this.  It only
> preliminary but seems to work ok.
>
> The API is:
>
> bin_io:open_read(Binary) -> {ok,IoStream}
> bin_io:open_create() -> {ok,IoStream}
> bin_io:open_append(Binary) -> {ok,IoStream}

Any reasons to avoid:

bin_io:open( Binary, [read] )

etc?

> bin_io:close(IoStream) -> ok | {ok,Binary}
>
> If the binary was opened for write or append then bin_io:close will
> return the resultant binary, otherwise just ok.

Are there any possibilities of errors? (Not only for close(), but all of them)

> In this version you can only read *or* write to a binary IoStream
> depending on how it was opened.  Would it be interesting to be able to
> do the file operations like positioning, reading and writing, etc?

Yes.

> As an extra bonus I include a modified epp which can read from an
> already opened IoStream, like a bin_io.  To use is you do:
...deleted
> N.B. Epp does not close the IoStream as it did not open it.

Good design.


bengt



Reply | Threaded
Open this post in threaded view
|

I/O from binaries

Chris Pressey
On Mon, 5 Nov 2001 08:17:24 +0100 (MET)
Bengt Kleberg <eleberg> wrote:

> > From: Robert Virding <rv>
> >
> > Many have requested a module which allows doing i/o to/from binaries.
> > Here is a small module I have fixed which does this.  It only
> > preliminary but seems to work ok.
> >
> > The API is:
> >
> > bin_io:open_read(Binary) -> {ok,IoStream}
> > bin_io:open_create() -> {ok,IoStream}
> > bin_io:open_append(Binary) -> {ok,IoStream}
>
> Any reasons to avoid:
>
> bin_io:open( Binary, [read] )
>
> etc?

I'll go one step further, and ask: is there any reason both the io module
and the bin_io module shouldn't both adhere to an 'io' behaviour?

I realize that modules that implement behaviours are intended as callback
modules, but I don't see why they couldn't be used just as well to
establish an interface for calls of all kinds, not just callbacks.

For example, a function in some other module could take an atom as an
argument, which would be a module name like io or bin_io (or socket_io,
serial_io, etc ad infinitum), and use that module for I/O by calling
Module:open(Object, [read]) and so forth.  This would allow that function
to seamlessly use whatever I/O its caller specifies.

I think the real benefit here comes from the new behaviour_info/1
function, so that conformity with an interface can be checked, even at
compile time.  There are probably other places it could be beneficial, for
example in the epop package, where a custom database driver can be
specified; it could be required to live up to an epop_db behaviour, or
some such.

Any thoughts on this would be very welcome :)

Chris


Reply | Threaded
Open this post in threaded view
|

I/O from binaries

Fredrik Linder-2
Just want to add som thoughts to this.

Maybe it is a bit "forbidden" to talk about OO here (I don't think it should
be), but is not Chris' proposal a bit like inheritance in OO, the good parts
of it? One super-class defines an interface which many other sub-classes
inherits, so that they all can be accessed in the same way (they all share
an interface :-). Does not activation in mnesia implement something like
this?

Yes, there are both strengths and weaknesess with inheritance. An example of
its strength does Chris' give an sample of in his discussion, and an example
of its weaknesses is huge class-tree structures (that are hard to maintain).

Generally, is Chris' proposal not a good solution?

/Fredrik

> Bengt Kleberg <eleberg> wrote:
>
> > > From: Robert Virding <rv>
> > >
> > > Many have requested a module which allows doing i/o to/from binaries.
> > > Here is a small module I have fixed which does this.  It only
> > > preliminary but seems to work ok.
> > >
> > > The API is:
> > >
> > > bin_io:open_read(Binary) -> {ok,IoStream}
> > > bin_io:open_create() -> {ok,IoStream}
> > > bin_io:open_append(Binary) -> {ok,IoStream}
> >
> > Any reasons to avoid:
> >
> > bin_io:open( Binary, [read] )
> >
> > etc?
>
> I'll go one step further, and ask: is there any reason both the io module
> and the bin_io module shouldn't both adhere to an 'io' behaviour?
>
> I realize that modules that implement behaviours are intended as callback
> modules, but I don't see why they couldn't be used just as well to
> establish an interface for calls of all kinds, not just callbacks.
>
> For example, a function in some other module could take an atom as an
> argument, which would be a module name like io or bin_io (or socket_io,
> serial_io, etc ad infinitum), and use that module for I/O by calling
> Module:open(Object, [read]) and so forth.  This would allow that function
> to seamlessly use whatever I/O its caller specifies.
>
> I think the real benefit here comes from the new behaviour_info/1
> function, so that conformity with an interface can be checked, even at
> compile time.  There are probably other places it could be beneficial, for
> example in the epop package, where a custom database driver can be
> specified; it could be required to live up to an epop_db behaviour, or
> some such.
>
> Any thoughts on this would be very welcome :)
>
> Chris
>



Reply | Threaded
Open this post in threaded view
|

I/O from binaries

Chris Pressey
On Tue, 6 Nov 2001 08:23:14 +0100
"Fredrik Linder" <fredrik.linder> wrote:

> Just want to add som thoughts to this.
>
> Maybe it is a bit "forbidden" to talk about OO here (I don't think it
> should
> be), but is not Chris' proposal a bit like inheritance in OO, the good
> parts
> of it?

I would say that inheritance and behaviours are two ways to express a
generalization/specialization pattern.  A subclass specializes its
superclass, and an implementation module specializes a behaviour module.

Whether there's a further similarity is questionable.  Behaviours are (to
me) a bit more like templates for Java-like interfaces, than they are like
object classes.  Erlang doesn't have any built-in "dispatching" strategy,
so any "inheriting" of methods, properties, etc has to be explicitly
coded.

> Yes, there are both strengths and weaknesess with inheritance. An
> example of
> its strength does Chris' give an sample of in his discussion, and an
> example
> of its weaknesses is huge class-tree structures (that are hard to
> maintain).

Actually, I *don't* think I would propose behaviours which themselves
implement behaviours, at this point.  In other words, in my mind there is
generally a simple, non-hierarchical, two-level relationship, between
behaviour and implementation.  I would strongly discourage (over)use of
hierarchical behaviours of behaviours.  If I must impose a hierarchy on
it, I would see all behaviours as implementing a single universal
meta-behaviour that might be described best as an imaginary Erlang module
like so:

  -module(meta_behaviour).
  -export([behaviour_info/1]).
  behaviour_info(callbacks) -> [{behaviour_info, 1}].

Thanks for the thoughts, and the chance to clarify, maybe :)

Chris


Reply | Threaded
Open this post in threaded view
|

I/O from binaries and an I/O behaviour

Robert Virding-4
In reply to this post by Chris Pressey
Chris Pressey <cpressey> writes:

>On Mon, 5 Nov 2001 08:17:24 +0100 (MET)
>
>I'll go one step further, and ask: is there any reason both the io module
>and the bin_io module shouldn't both adhere to an 'io' behaviour?
>
>I realize that modules that implement behaviours are intended as callback
>modules, but I don't see why they couldn't be used just as well to
>establish an interface for calls of all kinds, not just callbacks.
>
>For example, a function in some other module could take an atom as an
>argument, which would be a module name like io or bin_io (or socket_io,
>serial_io, etc ad infinitum), and use that module for I/O by calling
>Module:open(Object, [read]) and so forth.  This would allow that function
>to seamlessly use whatever I/O its caller specifies.
>
>I think the real benefit here comes from the new behaviour_info/1
>function, so that conformity with an interface can be checked, even at
>compile time.  There are probably other places it could be beneficial, for
>example in the epop package, where a custom database driver can be
>specified; it could be required to live up to an epop_db behaviour, or
>some such.
>
>Any thoughts on this would be very welcome :)
>
>Chris

No, real reason.  I did look into to it many years ago, long before
behaviours in fact, and found that it wasn't trivial.  The problem is
that such a generic server would need to define a generic way of
getting bytes from and putting bytes to some device.  Some devices,
like files and binaries, work best with an synchronous call method
while others like terminals work best with a completely asynchronous
method.

Another complication is that for things like terminals you really need
to be able to interlace input and output requests in a sensible way,
like the terminal does now.

The best way I then found was to define a simple generic synchronous
io server which uses defined get and put handlers.  Other more
complicated devices like terminals would then need a separate process
to do their tricky bits.  This went against the current trend,
however, towards fewer processes as the BEAM is not really efficient
in sending lists (unified heap rules).

Actually the basic io protocol is pretty simple so I have not really
felt the need to be great.  I will look into it, however, when I get
the time.

Robert


Reply | Threaded
Open this post in threaded view
|

I/O from binaries and an I/O behaviour

Chris Pressey
On Thu, 08 Nov 2001 10:56:14 +0100
Robert Virding <rv> wrote:

> Chris Pressey <cpressey> writes:
> >On Mon, 5 Nov 2001 08:17:24 +0100 (MET)
> >
> >I'll go one step further, and ask: is there any reason both the io
> >module
> >and the bin_io module shouldn't both adhere to an 'io' behaviour?
> >[...]
>
> No, real reason.  I did look into to it many years ago, long before
> behaviours in fact, and found that it wasn't trivial.  The problem is
> that such a generic server would need to define a generic way of
> getting bytes from and putting bytes to some device.  Some devices,
> like files and binaries, work best with an synchronous call method
> while others like terminals work best with a completely asynchronous
> method.
>
> Another complication is that for things like terminals you really need
> to be able to interlace input and output requests in a sensible way,
> like the terminal does now.
>
> The best way I then found was to define a simple generic synchronous
> io server which uses defined get and put handlers.  Other more
> complicated devices like terminals would then need a separate process
> to do their tricky bits.  This went against the current trend,
> however, towards fewer processes as the BEAM is not really efficient
> in sending lists (unified heap rules).
>
> Actually the basic io protocol is pretty simple so I have not really
> felt the need to be great.  I will look into it, however, when I get
> the time.

Perhaps I/O isn't the best example, and the word 'behaviour' is perhaps
misleading.  There are several other examples in the standard libraries of
modules which can and do expose the same 'interface' - dict and orddict,
for example.

What I'm getting at is probably closer to this suggestion from the
programming guidelines:

"3.9 [...] a consistent system where different modules do things in a
similar manner will be much easier to understand than a system where each
module does things in a different manner."

One way for modules to do things in a similar manner is for these modules
to share an interface.  I see interfaces as a subset of behaviours.  A
behaviour specifies common functionality, plus callback points.  An
interface just specifies callback points *without* providing any common
functionality.

behaviour_info/1 gives Erlang an automated way to check that a module
adheres to an interface.  Without it, we resort to manually documenting
the interface, for example from the docs for Tobbe's epop:

"CHANGING DATABASE
If another database or mail storage facility is going to be used together
with the epop server, then it has to export the following functions:
start/0, get_passwd/1, stat/1, scan_list/1, get_mail/2, lock_maildrop/1,
unlock_maildrop/1, delete_mail/3], add_user/2, rm_user/1, change_passwd/2,
msg_limit/2, store_mail/3."

This "contract between modules" could be expressed in Erlang like so:

-module(gen_epop_db_interface).
-export([behaviour_info/1]).
behaviour_info(callbacks) ->
[
  {start,0}, {get_passwd,1}, {stat,1}, {scan_list,1}, {get_mail,2},
  {lock_maildrop,1}, {unlock_maildrop,1}, {delete_mail,3}, {add_user,2},
  {rm_user,1}, {change_passwd,2}, {msg_limit,2}, {store_mail,3}
].

I like the Erlang version better because it's tighter.  The compiler and
other applications know about the interface, not just the programmer, so
it's less error-prone.

Chris