Why do we need modules at all?

[Joe Armstrong-2]

Why do we need modules at all?

This is a brain-dump-stream-of-consciousness-thing. I've been
thinking about this for a while.

I'm proposing a slightly different way of programming here
The basic idea is

    - do away with modules
    - all functions have unique distinct names
    - all functions have (lots of) meta data
    - all functions go into a global (searchable) Key-value database
    - we need letrec
    - contribution to open source can be as simple as
      contributing a single function
    - there are no "open source projects" - only "the open source
      Key-Value database of all functions"
    - Content is peer reviewed

These are discussed in no particular order below:

Why does Erlang have modules?

There's a good an bad side to modules:

Good: Provides a unit of compilation, a unit of code
distribution. unit of code replacement

Bad: It's very difficult to decide which module to put an individual
function in. Break encapsulation (see later)

Aside: lib_misc.erl

When I'm programming I often get to the point were I say there should
a function foo/2 in lists.erl but their isn't. There should be but
there isn't - foo/2 is a small self contained thing. Why should it be
in lists.erl because it "feels right".

Strings are lists, so why do we have two modules lists.erl and
string.erl how should I decide in which module my new string/list
processing function should go.

To avoid all mental anguish when I need a small function that
should be somewhere else and isn't I stick it in
a module elib1_misc.erl.

My elib1_misc exports the following:

added_files/2                 make_challenge/0
as_bits/1                     make_response/2
as_bits_test/0                make_response_test/0
bdump/2                       make_test_strings/1
bin2hex/1                     make_test_strings_test/0
bin2hex_test/0                make_tmp_filename/2
check_io_list/1               merge_kv/1
collect_atom/1                merge_kv_test/0
collect_atom_test/0           mini_shell/0
collect_int/1                 module_info/0
collect_int_test/0            module_info/1
collect_string/1              ndots/1
collect_string_test/0         nibble_to_hex_char/1
collect_word/1                nibble_to_hex_char_test/0
complete/2                    odd/1
complete_test/0               on_exit/2
dos2unix/1                    out_of_date/2
downcase_char/1               outfile/2
dump/2                        padd/2
dump_tmp/2                    perms/1
duplicates/1                  perms_test/0
ensure_started/2              pmap/2
eval_file/1                   pmap1/2
eval_file_test/0              pmap1_test/0
eval_string/1                 pmap_test/0
eval_string_test/0            priority_receive/0
every/3                       random_seed/0
expand_env_vars/1             random_string/1
expand_file_template/3        random_string/2
expand_string_template/2      read_at_most_n_lines/2
expand_tabs/1                 read_at_most_n_lines_test/0
expand_tabs_test/0            remove_duplicates/1
expand_template/2             remove_duplicates_test/0
extract_attribute/2           remove_leading_and_trailing_whitespace/1
extract_attribute_test/0      remove_leading_and_trailing_whitespace_test/0
extract_prefix/2              remove_leading_whitespace/1
fetch/2                       remove_prefix/2
fetch_test/0                  remove_prefix_test/0
file2lines/1                  remove_trailing_whitespace/1
file2lines_test/0             replace/3
file2md5/1                    root_dir/0
file2numberedlines/1          rpc/2
file2numberedlines_test/0     safe/1
file2paras/1                  show_loaded/1
file2stream/1                 signed_byte_to_hex_string/1
file2string/1                 signed_byte_to_hex_string_test/0
file2template/1               skip_blanks/1
file2term/1                   skip_blanks_test/0
file_size_and_type/1          skip_to_nl/1
find_src/1                    skip_to_nl_test/0
first/1                       sleep/1
flatten_io_list/1             spawn_monitor/3
flush_buffer/0                split_at_char/2
for/3                         split_at_char_test/0
force/1                       split_list/2
foreach_chunk_in_file/3       split_list_test/0
foreach_word_in_file/2        string2exprs/1
foreach_word_in_string/2      string2exprs_test/0
forever/0                     string2html/1
get_erl_section/2             string2latex/1
get_line/1                    string2lines/1
get_line/2                    string2lines_test/0
have_common_prefix/1          string2stream/1
have_common_prefix_test/0     string2stream_test/0
hex2bin/1                     string2template/1
hex2bin_test/0                string2template_test/0
hex2list/1                    string2term/1
hex2list_test/0               string2term_test/0
hex_nibble2int/1              string2toks/1
hex_nibble2int_test/0         string2toks_test/0
id/1                          sub_binary/3
include_dir/0                 template2file/3
include_file/1                term2file/2
interleave/2                  term2string/1
is_alphanum/1                 test/0
is_blank_line/1               test1_test/0
is_prefix/2                   test_function_over_substrings/2
is_prefix_test/0              tex2pdf/1
is_response_correct/3         time_fun/2
keep_alive/2                  time_stamp/0
lines2para/1                  to_lower/1
list2frequency_distribution/1 to_lower_test/0
list2frequency_distribution_tetrim/1
longest_common_prefix/1       trim_test/0
longest_common_prefix_test/0  unconsult/2
lookup/2                      unsigned_byte_to_hex_string/1
lorem/1                       unsigned_byte_to_hex_string_test/0
ls/1                          which/1
                              which_added/1

Now I find this very convenient when I write a new small utility function
I stick in in elib1_misc.erl - no mental anguish in choosing a module
name is involved.

The observation that I find this very-convenient is telling me something
about modules - I like my elib1_misc it feels right.

(aside - It seems many development projects have their own private
lib_miscs ...)

Which brings me to the point of my question.

Do we need module's at all? Erlang programs are composed of lots of small
functions, the only place where modules seem useful is to hide a letrec.

The classic example is fibonacci. We want to expose fib/1 but hide the
helper function fib/3. Using modules we say

-module(math).
-export([fib/1]).

fib(N) ->
    fib(N, 1, 0).

fib(N, A, B) when N < 2 -> A;
fib(N, A, B) -> fib(N-1, A+B, A).

The downside is we have had to *invent* one module name math - whose *only*
purpose is to hide the definition of fib/3 which we don't want to be made
callable.

If we put a second function into the module math, then this second function
could call fib/3 which breaks the encapsulation of fib/3.

We could say:

let fib = fun(N) -> fib(N, 1, 0) end
in
   fib(N, A, B) when N < 2 -> A;
   fib(N, A, B) -> fib(N-1, A+B, A).
end.

I hardly dare suggest a syntax for this since I've been following
another thread in this forum where syntax discussion seem to encourage
much comment.

** Please do suggest alternative syntax's here - but do not comment on
other peoples suggestions ...

I would like to just talk about why we have modules.

Another question:

Does the idea of a module come from the idea that functions have to be
stored somewhere, so we store them in a file, and we slurp the
file (as a unit) into the system, so the file becomes a module?

If all the files were store by themselves in a database would this
change things.

I am thinking more and more that if would be nice to have *all* functions in
a key_value database with unique names.

lookup(foo,2) would get the definition foo foo/2 from a database.

The unique names bit is interesting - is this a good idea. Qualified
names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good
idea but but when I'm programming I have to invent the xxx or the
a.b.c which is very difficult. It also involves the "decision problem"
if the namespaces xxx and a.b.c already exist I have to *choose* which
to put my new function in.

I think there might be a case for alises here joe:foo/2 could be used
while developing "joe" would expand to a horrible random local string the
real name being ab123aZwerasch123123_foo/2  but I would not be able to
publish my code or make it available to a third_part before I had
chosen a sensible name.

(( managing namespaces seems really tricky, a lot of peoople seem
to thing that the problem goes away by adding "." 's to the name
but managing a namespace with namees like foo.bar.baz.z is just as complex
as managing a namespace with names like foo_bar_baz_z or names like
0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic
name like www.a.b to a reference like 123.45.23.12 - but how do we discover
the initial name www.a.b? - there are two answers - a) we are given the name
(ie we click on a link) - we do not know the name but we search fo it ))


When programs are small we can live with "just the code" in "a few
modules" the ratio of code to meta data is high.

When programs are large we need a lot of meta-data to understand them.

I would like to see all functions with all meta-data in a data base.

I'd like to say:

   lookup(foo,2,Attribute) when Attribute =

      code|source|documentation|type signatures|revision history|authors|...

The more I think about it the more I think program development should
viewed as changing the state of a Key-Value database.

So I imagine:

    1) all functions have unique names
    2) there are no modules
    3) we discover the name of a function by searching metadata
       describing the function in a database
    4) all public functions (think open source) are in the same
       database

We could make a system to do this.

I think this would make open-source projects easier, since the
granularity of contribution goes down. You could contribute
a single function - not an entire application.

(( A problem with GUT style open source projects is there is
   not one database of functions, I often what one function from
   this project, another function from another project -- the
   granularity of reusable parts should be the individual function.

   functions are really easy to reuse
   modules are more difficult to reuse
   entire applications are very difficult to reuse
     (Unless there are isolated through a communication channel))

Possible extensions.

    1) Voting for promotion
    2) A review process

Given a raw database will *all* functions in it - we could derive an
"approved" functions database.

Popular functions could be moved to the approved database - the
review process would need to be discussed - so kind of peer-review/wiki
stuff.

Comments?

Volunteers?

/Joe




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Ralf P. Gans]

Hi,

the ability to replace code in a running environment depends, to my knowledge, on modules.

So modules prevent downtimes and allow 7/24 operation.

cheers, Ralf

On 24.05.2011, at 10:06, Joe Armstrong wrote:

> Why do we need modules at all?
>
> This is a brain-dump-stream-of-consciousness-thing. I've been
> thinking about this for a while.
>
> I'm proposing a slightly different way of programming here
> The basic idea is
>
>     - do away with modules
>     - all functions have unique distinct names
>     - all functions have (lots of) meta data
>     - all functions go into a global (searchable) Key-value database
>     - we need letrec
>     - contribution to open source can be as simple as
>       contributing a single function
>     - there are no "open source projects" - only "the open source
>       Key-Value database of all functions"
>     - Content is peer reviewed
>
> These are discussed in no particular order below:
>
> Why does Erlang have modules?
>
> There's a good an bad side to modules:
>
> Good: Provides a unit of compilation, a unit of code
> distribution. unit of code replacement
>
> Bad: It's very difficult to decide which module to put an individual
> function in. Break encapsulation (see later)
>
> Aside: lib_misc.erl
>
> When I'm programming I often get to the point were I say there should
> a function foo/2 in lists.erl but their isn't. There should be but
> there isn't - foo/2 is a small self contained thing. Why should it be
> in lists.erl because it "feels right".
>
> Strings are lists, so why do we have two modules lists.erl and
> string.erl how should I decide in which module my new string/list
> processing function should go.
>
> To avoid all mental anguish when I need a small function that
> should be somewhere else and isn't I stick it in
> a module elib1_misc.erl.
>
> My elib1_misc exports the following:
>
> added_files/2                 make_challenge/0
> as_bits/1                     make_response/2
> as_bits_test/0                make_response_test/0
> bdump/2                       make_test_strings/1
> bin2hex/1                     make_test_strings_test/0
> bin2hex_test/0                make_tmp_filename/2
> check_io_list/1               merge_kv/1
> collect_atom/1                merge_kv_test/0
> collect_atom_test/0           mini_shell/0
> collect_int/1                 module_info/0
> collect_int_test/0            module_info/1
> collect_string/1              ndots/1
> collect_string_test/0         nibble_to_hex_char/1
> collect_word/1                nibble_to_hex_char_test/0
> complete/2                    odd/1
> complete_test/0               on_exit/2
> dos2unix/1                    out_of_date/2
> downcase_char/1               outfile/2
> dump/2                        padd/2
> dump_tmp/2                    perms/1
> duplicates/1                  perms_test/0
> ensure_started/2              pmap/2
> eval_file/1                   pmap1/2
> eval_file_test/0              pmap1_test/0
> eval_string/1                 pmap_test/0
> eval_string_test/0            priority_receive/0
> every/3                       random_seed/0
> expand_env_vars/1             random_string/1
> expand_file_template/3        random_string/2
> expand_string_template/2      read_at_most_n_lines/2
> expand_tabs/1                 read_at_most_n_lines_test/0
> expand_tabs_test/0            remove_duplicates/1
> expand_template/2             remove_duplicates_test/0
> extract_attribute/2           remove_leading_and_trailing_whitespace/1
> extract_attribute_test/0      remove_leading_and_trailing_whitespace_test/0
> extract_prefix/2              remove_leading_whitespace/1
> fetch/2                       remove_prefix/2
> fetch_test/0                  remove_prefix_test/0
> file2lines/1                  remove_trailing_whitespace/1
> file2lines_test/0             replace/3
> file2md5/1                    root_dir/0
> file2numberedlines/1          rpc/2
> file2numberedlines_test/0     safe/1
> file2paras/1                  show_loaded/1
> file2stream/1                 signed_byte_to_hex_string/1
> file2string/1                 signed_byte_to_hex_string_test/0
> file2template/1               skip_blanks/1
> file2term/1                   skip_blanks_test/0
> file_size_and_type/1          skip_to_nl/1
> find_src/1                    skip_to_nl_test/0
> first/1                       sleep/1
> flatten_io_list/1             spawn_monitor/3
> flush_buffer/0                split_at_char/2
> for/3                         split_at_char_test/0
> force/1                       split_list/2
> foreach_chunk_in_file/3       split_list_test/0
> foreach_word_in_file/2        string2exprs/1
> foreach_word_in_string/2      string2exprs_test/0
> forever/0                     string2html/1
> get_erl_section/2             string2latex/1
> get_line/1                    string2lines/1
> get_line/2                    string2lines_test/0
> have_common_prefix/1          string2stream/1
> have_common_prefix_test/0     string2stream_test/0
> hex2bin/1                     string2template/1
> hex2bin_test/0                string2template_test/0
> hex2list/1                    string2term/1
> hex2list_test/0               string2term_test/0
> hex_nibble2int/1              string2toks/1
> hex_nibble2int_test/0         string2toks_test/0
> id/1                          sub_binary/3
> include_dir/0                 template2file/3
> include_file/1                term2file/2
> interleave/2                  term2string/1
> is_alphanum/1                 test/0
> is_blank_line/1               test1_test/0
> is_prefix/2                   test_function_over_substrings/2
> is_prefix_test/0              tex2pdf/1
> is_response_correct/3         time_fun/2
> keep_alive/2                  time_stamp/0
> lines2para/1                  to_lower/1
> list2frequency_distribution/1 to_lower_test/0
> list2frequency_distribution_tetrim/1
> longest_common_prefix/1       trim_test/0
> longest_common_prefix_test/0  unconsult/2
> lookup/2                      unsigned_byte_to_hex_string/1
> lorem/1                       unsigned_byte_to_hex_string_test/0
> ls/1                          which/1
>                               which_added/1
>
> Now I find this very convenient when I write a new small utility function
> I stick in in elib1_misc.erl - no mental anguish in choosing a module
> name is involved.
>
> The observation that I find this very-convenient is telling me something
> about modules - I like my elib1_misc it feels right.
>
> (aside - It seems many development projects have their own private
> lib_miscs ...)
>
> Which brings me to the point of my question.
>
> Do we need module's at all? Erlang programs are composed of lots of small
> functions, the only place where modules seem useful is to hide a letrec.
>
> The classic example is fibonacci. We want to expose fib/1 but hide the
> helper function fib/3. Using modules we say
>
> -module(math).
> -export([fib/1]).
>
> fib(N) ->
>     fib(N, 1, 0).
>
> fib(N, A, B) when N < 2 -> A;
> fib(N, A, B) -> fib(N-1, A+B, A).
>
> The downside is we have had to *invent* one module name math - whose *only*
> purpose is to hide the definition of fib/3 which we don't want to be made
> callable.
>
> If we put a second function into the module math, then this second function
> could call fib/3 which breaks the encapsulation of fib/3.
>
> We could say:
>
> let fib = fun(N) -> fib(N, 1, 0) end
> in
>    fib(N, A, B) when N < 2 -> A;
>    fib(N, A, B) -> fib(N-1, A+B, A).
> end.
>
> I hardly dare suggest a syntax for this since I've been following
> another thread in this forum where syntax discussion seem to encourage
> much comment.
>
> ** Please do suggest alternative syntax's here - but do not comment on
> other peoples suggestions ...
>
> I would like to just talk about why we have modules.
>
> Another question:
>
> Does the idea of a module come from the idea that functions have to be
> stored somewhere, so we store them in a file, and we slurp the
> file (as a unit) into the system, so the file becomes a module?
>
> If all the files were store by themselves in a database would this
> change things.
>
> I am thinking more and more that if would be nice to have *all* functions in
> a key_value database with unique names.
>
> lookup(foo,2) would get the definition foo foo/2 from a database.
>
> The unique names bit is interesting - is this a good idea. Qualified
> names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good
> idea but but when I'm programming I have to invent the xxx or the
> a.b.c which is very difficult. It also involves the "decision problem"
> if the namespaces xxx and a.b.c already exist I have to *choose* which
> to put my new function in.
>
> I think there might be a case for alises here joe:foo/2 could be used
> while developing "joe" would expand to a horrible random local string the
> real name being ab123aZwerasch123123_foo/2  but I would not be able to
> publish my code or make it available to a third_part before I had
> chosen a sensible name.
>
> (( managing namespaces seems really tricky, a lot of peoople seem
> to thing that the problem goes away by adding "." 's to the name
> but managing a namespace with namees like foo.bar.baz.z is just as complex
> as managing a namespace with names like foo_bar_baz_z or names like
> 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic
> name like www.a.b to a reference like 123.45.23.12 - but how do we discover
> the initial name www.a.b? - there are two answers - a) we are given the name
> (ie we click on a link) - we do not know the name but we search fo it ))
>
>
> When programs are small we can live with "just the code" in "a few
> modules" the ratio of code to meta data is high.
>
> When programs are large we need a lot of meta-data to understand them.
>
> I would like to see all functions with all meta-data in a data base.
>
> I'd like to say:
>
>    lookup(foo,2,Attribute) when Attribute =
>
>       code|source|documentation|type signatures|revision history|authors|...
>
> The more I think about it the more I think program development should
> viewed as changing the state of a Key-Value database.
>
> So I imagine:
>
>     1) all functions have unique names
>     2) there are no modules
>     3) we discover the name of a function by searching metadata
>        describing the function in a database
>     4) all public functions (think open source) are in the same
>        database
>
> We could make a system to do this.
>
> I think this would make open-source projects easier, since the
> granularity of contribution goes down. You could contribute
> a single function - not an entire application.
>
> (( A problem with GUT style open source projects is there is
>    not one database of functions, I often what one function from
>    this project, another function from another project -- the
>    granularity of reusable parts should be the individual function.
>
>    functions are really easy to reuse
>    modules are more difficult to reuse
>    entire applications are very difficult to reuse
>      (Unless there are isolated through a communication channel))
>
> Possible extensions.
>
>     1) Voting for promotion
>     2) A review process
>
> Given a raw database will *all* functions in it - we could derive an
> "approved" functions database.
>
> Popular functions could be moved to the approved database - the
> review process would need to be discussed - so kind of peer-review/wiki
> stuff.
>
> Comments?
>
> Volunteers?
>
> /Joe
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Amy Lear]

On Tue, May 24, 2011 at 1:11 AM, Ralf P. Gans <[hidden email]> wrote:

Hi,

the ability to replace code in a running environment depends, to my knowledge, on modules.

So modules prevent downtimes and allow 7/24 operation.

cheers, Ralf

This is only true because that's the implementation we have. The VM's ability to maintain a couple of versions of a particular function could still be maintained on that granular a level..

... Provided there's a way to, with atomicity, ensure that nothing calls the new version of a function without all of the related changes in associated functions rolling into the VM at the same time, since you may be moving quite a bit around if you find yourself refactoring a few functions to add new capabilities. And that IS something that is easier with modules as a complete unit of dependency.

Is there a way in a clean semantic manner track cross-function dependencies (potentially through some type of version tags?) in such a code database?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[mazenharake]

Aren't you just redefining the problem? Your next problem is going to be to name more generic type of functions. Imagine you have 3 different search functions which do different things in different modules. Your solution would instead of:

foo:search(), bar:search(), baz:search()

give

foo_search(), bar_search(), baz_search()

now you want to add a function that does replace; but replace what? say you call it

replace()

now I want to make one because it is specific to my problem so I have a decision problem

foo_replace() ? bar_replace() ?

so I will end up defining a prefix so that my versions of functions (where they collide with the already existing ones) always are unique thus:

mazen_replace()

now everyone starts creating functions with prefixes, now they want to start extending other functions so they build a prefix before the those:

mazen_foo_replace()

and before you know it you will have 20 functions ending with "search()" where search means what ever the prefix hints at...

So you are back to square -1 because now it is worse, you don't have modules and you don't have applications beacuse an application would become what modules was before it.

In fact, this is already a problem, all my opensource applications (and non-opensource as well) have a prefix on the modules because I can't call all my server type modules "server" (there is no namespace).



My 2 cents.

/M

On 24 May 2011 10:06, Joe Armstrong <[hidden email]> wrote:

Why do we need modules at all?

This is a brain-dump-stream-of-consciousness-thing. I've been
thinking about this for a while.

I'm proposing a slightly different way of programming here
The basic idea is

    - do away with modules
    - all functions have unique distinct names
    - all functions have (lots of) meta data
    - all functions go into a global (searchable) Key-value database
    - we need letrec
    - contribution to open source can be as simple as
      contributing a single function
    - there are no "open source projects" - only "the open source
      Key-Value database of all functions"
    - Content is peer reviewed

These are discussed in no particular order below:

Why does Erlang have modules?

There's a good an bad side to modules:

Good: Provides a unit of compilation, a unit of code
distribution. unit of code replacement

Bad: It's very difficult to decide which module to put an individual
function in. Break encapsulation (see later)

Aside: lib_misc.erl

When I'm programming I often get to the point were I say there should
a function foo/2 in lists.erl but their isn't. There should be but
there isn't - foo/2 is a small self contained thing. Why should it be
in lists.erl because it "feels right".

Strings are lists, so why do we have two modules lists.erl and
string.erl how should I decide in which module my new string/list
processing function should go.

To avoid all mental anguish when I need a small function that
should be somewhere else and isn't I stick it in
a module elib1_misc.erl.

My elib1_misc exports the following:

added_files/2                 make_challenge/0
as_bits/1                     make_response/2
as_bits_test/0                make_response_test/0
bdump/2                       make_test_strings/1
bin2hex/1                     make_test_strings_test/0
bin2hex_test/0                make_tmp_filename/2
check_io_list/1               merge_kv/1
collect_atom/1                merge_kv_test/0
collect_atom_test/0           mini_shell/0
collect_int/1                 module_info/0
collect_int_test/0            module_info/1
collect_string/1              ndots/1
collect_string_test/0         nibble_to_hex_char/1
collect_word/1                nibble_to_hex_char_test/0
complete/2                    odd/1
complete_test/0               on_exit/2
dos2unix/1                    out_of_date/2
downcase_char/1               outfile/2
dump/2                        padd/2
dump_tmp/2                    perms/1
duplicates/1                  perms_test/0
ensure_started/2              pmap/2
eval_file/1                   pmap1/2
eval_file_test/0              pmap1_test/0
eval_string/1                 pmap_test/0
eval_string_test/0            priority_receive/0
every/3                       random_seed/0
expand_env_vars/1             random_string/1
expand_file_template/3        random_string/2
expand_string_template/2      read_at_most_n_lines/2
expand_tabs/1                 read_at_most_n_lines_test/0
expand_tabs_test/0            remove_duplicates/1
expand_template/2             remove_duplicates_test/0
extract_attribute/2           remove_leading_and_trailing_whitespace/1
extract_attribute_test/0      remove_leading_and_trailing_whitespace_test/0
extract_prefix/2              remove_leading_whitespace/1
fetch/2                       remove_prefix/2
fetch_test/0                  remove_prefix_test/0
file2lines/1                  remove_trailing_whitespace/1
file2lines_test/0             replace/3
file2md5/1                    root_dir/0
file2numberedlines/1          rpc/2
file2numberedlines_test/0     safe/1
file2paras/1                  show_loaded/1
file2stream/1                 signed_byte_to_hex_string/1
file2string/1                 signed_byte_to_hex_string_test/0
file2template/1               skip_blanks/1
file2term/1                   skip_blanks_test/0
file_size_and_type/1          skip_to_nl/1
find_src/1                    skip_to_nl_test/0
first/1                       sleep/1
flatten_io_list/1             spawn_monitor/3
flush_buffer/0                split_at_char/2
for/3                         split_at_char_test/0
force/1                       split_list/2
foreach_chunk_in_file/3       split_list_test/0
foreach_word_in_file/2        string2exprs/1
foreach_word_in_string/2      string2exprs_test/0
forever/0                     string2html/1
get_erl_section/2             string2latex/1
get_line/1                    string2lines/1
get_line/2                    string2lines_test/0
have_common_prefix/1          string2stream/1
have_common_prefix_test/0     string2stream_test/0
hex2bin/1                     string2template/1
hex2bin_test/0                string2template_test/0
hex2list/1                    string2term/1
hex2list_test/0               string2term_test/0
hex_nibble2int/1              string2toks/1
hex_nibble2int_test/0         string2toks_test/0
id/1                          sub_binary/3
include_dir/0                 template2file/3
include_file/1                term2file/2
interleave/2                  term2string/1
is_alphanum/1                 test/0
is_blank_line/1               test1_test/0
is_prefix/2                   test_function_over_substrings/2
is_prefix_test/0              tex2pdf/1
is_response_correct/3         time_fun/2
keep_alive/2                  time_stamp/0
lines2para/1                  to_lower/1
list2frequency_distribution/1 to_lower_test/0
list2frequency_distribution_tetrim/1
longest_common_prefix/1       trim_test/0
longest_common_prefix_test/0  unconsult/2
lookup/2                      unsigned_byte_to_hex_string/1
lorem/1                       unsigned_byte_to_hex_string_test/0
ls/1                          which/1
                              which_added/1

Now I find this very convenient when I write a new small utility function
I stick in in elib1_misc.erl - no mental anguish in choosing a module
name is involved.

The observation that I find this very-convenient is telling me something
about modules - I like my elib1_misc it feels right.

(aside - It seems many development projects have their own private
lib_miscs ...)

Which brings me to the point of my question.

Do we need module's at all? Erlang programs are composed of lots of small
functions, the only place where modules seem useful is to hide a letrec.

The classic example is fibonacci. We want to expose fib/1 but hide the
helper function fib/3. Using modules we say

-module(math).
-export([fib/1]).

fib(N) ->
    fib(N, 1, 0).

fib(N, A, B) when N < 2 -> A;
fib(N, A, B) -> fib(N-1, A+B, A).

The downside is we have had to *invent* one module name math - whose *only*
purpose is to hide the definition of fib/3 which we don't want to be made
callable.

If we put a second function into the module math, then this second function
could call fib/3 which breaks the encapsulation of fib/3.

We could say:

let fib = fun(N) -> fib(N, 1, 0) end
in
   fib(N, A, B) when N < 2 -> A;
   fib(N, A, B) -> fib(N-1, A+B, A).
end.

I hardly dare suggest a syntax for this since I've been following
another thread in this forum where syntax discussion seem to encourage
much comment.

** Please do suggest alternative syntax's here - but do not comment on
other peoples suggestions ...

I would like to just talk about why we have modules.

Another question:

Does the idea of a module come from the idea that functions have to be
stored somewhere, so we store them in a file, and we slurp the
file (as a unit) into the system, so the file becomes a module?

If all the files were store by themselves in a database would this
change things.

I am thinking more and more that if would be nice to have *all* functions in
a key_value database with unique names.

lookup(foo,2) would get the definition foo foo/2 from a database.

The unique names bit is interesting - is this a good idea. Qualified
names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good
idea but but when I'm programming I have to invent the xxx or the
a.b.c which is very difficult. It also involves the "decision problem"
if the namespaces xxx and a.b.c already exist I have to *choose* which
to put my new function in.

I think there might be a case for alises here joe:foo/2 could be used
while developing "joe" would expand to a horrible random local string the
real name being ab123aZwerasch123123_foo/2  but I would not be able to
publish my code or make it available to a third_part before I had
chosen a sensible name.

(( managing namespaces seems really tricky, a lot of peoople seem
to thing that the problem goes away by adding "." 's to the name
but managing a namespace with namees like foo.bar.baz.z is just as complex
as managing a namespace with names like foo_bar_baz_z or names like
0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic
name like www.a.b to a reference like 123.45.23.12 - but how do we discover
the initial name www.a.b? - there are two answers - a) we are given the name
(ie we click on a link) - we do not know the name but we search fo it ))


When programs are small we can live with "just the code" in "a few
modules" the ratio of code to meta data is high.

When programs are large we need a lot of meta-data to understand them.

I would like to see all functions with all meta-data in a data base.

I'd like to say:

   lookup(foo,2,Attribute) when Attribute =

      code|source|documentation|type signatures|revision history|authors|...

The more I think about it the more I think program development should
viewed as changing the state of a Key-Value database.

So I imagine:

    1) all functions have unique names
    2) there are no modules
    3) we discover the name of a function by searching metadata
       describing the function in a database
    4) all public functions (think open source) are in the same
       database

We could make a system to do this.

I think this would make open-source projects easier, since the
granularity of contribution goes down. You could contribute
a single function - not an entire application.

(( A problem with GUT style open source projects is there is
   not one database of functions, I often what one function from
   this project, another function from another project -- the
   granularity of reusable parts should be the individual function.

   functions are really easy to reuse
   modules are more difficult to reuse
   entire applications are very difficult to reuse
     (Unless there are isolated through a communication channel))

Possible extensions.

    1) Voting for promotion
    2) A review process

Given a raw database will *all* functions in it - we could derive an
"approved" functions database.

Popular functions could be moved to the approved database - the
review process would need to be discussed - so kind of peer-review/wiki
stuff.

Comments?

Volunteers?

/Joe




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Dmitrii Dimandt-2]

>
> So I imagine:
>
>     1) all functions have unique names
>     2) there are no modules
>     3) we discover the name of a function by searching metadata
>        describing the function in a database
>     4) all public functions (think open source) are in the same
>        database
>
> We could make a system to do this.

Hm... you'd still need *some* kind of modules.

For instance, in Webmachine each resource is expected to have some, or all, of predefined public/exported functions: to_html, resource_exists, allowed_methods etc.

So, each resource will have functions with the same name. If we move all functions to a global namespace, then how do we differentiate between these functions? Probably by naming them resource_name_function_name?

Furthermore, what do we do with behaviours? A behaviour is encapsulated within a module, also with a predefined set of exported funcions (and any number of custom exported functions).
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Gordon Guthrie-2]

There are two good things about modules:
* they are a higher level of abstraction
* they support good working practices

Lets take a module that I use a little bit - and which a colleague
introduced me to:
http://erldocs.com/R14B02/stdlib/digraph.html?i=2&search=digra#get_digraph/1

I can use the couple of exposed functions of that that I know,
confident that if it comes to it I can come back and master the rest
of the interface later. I have confidence that there is a rubustness
there that makes code using it maintainable. (my idea of maintainable
is to be able to ignore most stuff, most of the time...). I can learn
about the code at this higher level of abstraction.

We too have a module with loads of random functions in it, well we
have three (hn_util, util, util2), and each of them is just library
functions. In the utility modules the ratio of exported to
not-exported fns is maybe 7 to 1.

In functional modules the exported/not ratio goes from 1-1 (in modules
that define apis) to 1-3 against.

If we had the (promised) -export-to() directive the number of pure
exported fns would drop.

Given that we build our testing strategy around -export() this
separation is critical...

Gordon

On 24 May 2011 09:06, Joe Armstrong <[hidden email]> wrote:

> Why do we need modules at all?
>
> This is a brain-dump-stream-of-consciousness-thing. I've been
> thinking about this for a while.
>
> I'm proposing a slightly different way of programming here
> The basic idea is
>
>     - do away with modules
>     - all functions have unique distinct names
>     - all functions have (lots of) meta data
>     - all functions go into a global (searchable) Key-value database
>     - we need letrec
>     - contribution to open source can be as simple as
>       contributing a single function
>     - there are no "open source projects" - only "the open source
>       Key-Value database of all functions"
>     - Content is peer reviewed
>
> These are discussed in no particular order below:
>
> Why does Erlang have modules?
>
> There's a good an bad side to modules:
>
> Good: Provides a unit of compilation, a unit of code
> distribution. unit of code replacement
>
> Bad: It's very difficult to decide which module to put an individual
> function in. Break encapsulation (see later)
>
> Aside: lib_misc.erl
>
> When I'm programming I often get to the point were I say there should
> a function foo/2 in lists.erl but their isn't. There should be but
> there isn't - foo/2 is a small self contained thing. Why should it be
> in lists.erl because it "feels right".
>
> Strings are lists, so why do we have two modules lists.erl and
> string.erl how should I decide in which module my new string/list
> processing function should go.
>
> To avoid all mental anguish when I need a small function that
> should be somewhere else and isn't I stick it in
> a module elib1_misc.erl.
>
> My elib1_misc exports the following:
>
> added_files/2                 make_challenge/0
> as_bits/1                     make_response/2
> as_bits_test/0                make_response_test/0
> bdump/2                       make_test_strings/1
> bin2hex/1                     make_test_strings_test/0
> bin2hex_test/0                make_tmp_filename/2
> check_io_list/1               merge_kv/1
> collect_atom/1                merge_kv_test/0
> collect_atom_test/0           mini_shell/0
> collect_int/1                 module_info/0
> collect_int_test/0            module_info/1
> collect_string/1              ndots/1
> collect_string_test/0         nibble_to_hex_char/1
> collect_word/1                nibble_to_hex_char_test/0
> complete/2                    odd/1
> complete_test/0               on_exit/2
> dos2unix/1                    out_of_date/2
> downcase_char/1               outfile/2
> dump/2                        padd/2
> dump_tmp/2                    perms/1
> duplicates/1                  perms_test/0
> ensure_started/2              pmap/2
> eval_file/1                   pmap1/2
> eval_file_test/0              pmap1_test/0
> eval_string/1                 pmap_test/0
> eval_string_test/0            priority_receive/0
> every/3                       random_seed/0
> expand_env_vars/1             random_string/1
> expand_file_template/3        random_string/2
> expand_string_template/2      read_at_most_n_lines/2
> expand_tabs/1                 read_at_most_n_lines_test/0
> expand_tabs_test/0            remove_duplicates/1
> expand_template/2             remove_duplicates_test/0
> extract_attribute/2           remove_leading_and_trailing_whitespace/1
> extract_attribute_test/0      remove_leading_and_trailing_whitespace_test/0
> extract_prefix/2              remove_leading_whitespace/1
> fetch/2                       remove_prefix/2
> fetch_test/0                  remove_prefix_test/0
> file2lines/1                  remove_trailing_whitespace/1
> file2lines_test/0             replace/3
> file2md5/1                    root_dir/0
> file2numberedlines/1          rpc/2
> file2numberedlines_test/0     safe/1
> file2paras/1                  show_loaded/1
> file2stream/1                 signed_byte_to_hex_string/1
> file2string/1                 signed_byte_to_hex_string_test/0
> file2template/1               skip_blanks/1
> file2term/1                   skip_blanks_test/0
> file_size_and_type/1          skip_to_nl/1
> find_src/1                    skip_to_nl_test/0
> first/1                       sleep/1
> flatten_io_list/1             spawn_monitor/3
> flush_buffer/0                split_at_char/2
> for/3                         split_at_char_test/0
> force/1                       split_list/2
> foreach_chunk_in_file/3       split_list_test/0
> foreach_word_in_file/2        string2exprs/1
> foreach_word_in_string/2      string2exprs_test/0
> forever/0                     string2html/1
> get_erl_section/2             string2latex/1
> get_line/1                    string2lines/1
> get_line/2                    string2lines_test/0
> have_common_prefix/1          string2stream/1
> have_common_prefix_test/0     string2stream_test/0
> hex2bin/1                     string2template/1
> hex2bin_test/0                string2template_test/0
> hex2list/1                    string2term/1
> hex2list_test/0               string2term_test/0
> hex_nibble2int/1              string2toks/1
> hex_nibble2int_test/0         string2toks_test/0
> id/1                          sub_binary/3
> include_dir/0                 template2file/3
> include_file/1                term2file/2
> interleave/2                  term2string/1
> is_alphanum/1                 test/0
> is_blank_line/1               test1_test/0
> is_prefix/2                   test_function_over_substrings/2
> is_prefix_test/0              tex2pdf/1
> is_response_correct/3         time_fun/2
> keep_alive/2                  time_stamp/0
> lines2para/1                  to_lower/1
> list2frequency_distribution/1 to_lower_test/0
> list2frequency_distribution_tetrim/1
> longest_common_prefix/1       trim_test/0
> longest_common_prefix_test/0  unconsult/2
> lookup/2                      unsigned_byte_to_hex_string/1
> lorem/1                       unsigned_byte_to_hex_string_test/0
> ls/1                          which/1
>                               which_added/1
>
> Now I find this very convenient when I write a new small utility function
> I stick in in elib1_misc.erl - no mental anguish in choosing a module
> name is involved.
>
> The observation that I find this very-convenient is telling me something
> about modules - I like my elib1_misc it feels right.
>
> (aside - It seems many development projects have their own private
> lib_miscs ...)
>
> Which brings me to the point of my question.
>
> Do we need module's at all? Erlang programs are composed of lots of small
> functions, the only place where modules seem useful is to hide a letrec.
>
> The classic example is fibonacci. We want to expose fib/1 but hide the
> helper function fib/3. Using modules we say
>
> -module(math).
> -export([fib/1]).
>
> fib(N) ->
>     fib(N, 1, 0).
>
> fib(N, A, B) when N < 2 -> A;
> fib(N, A, B) -> fib(N-1, A+B, A).
>
> The downside is we have had to *invent* one module name math - whose *only*
> purpose is to hide the definition of fib/3 which we don't want to be made
> callable.
>
> If we put a second function into the module math, then this second function
> could call fib/3 which breaks the encapsulation of fib/3.
>
> We could say:
>
> let fib = fun(N) -> fib(N, 1, 0) end
> in
>    fib(N, A, B) when N < 2 -> A;
>    fib(N, A, B) -> fib(N-1, A+B, A).
> end.
>
> I hardly dare suggest a syntax for this since I've been following
> another thread in this forum where syntax discussion seem to encourage
> much comment.
>
> ** Please do suggest alternative syntax's here - but do not comment on
> other peoples suggestions ...
>
> I would like to just talk about why we have modules.
>
> Another question:
>
> Does the idea of a module come from the idea that functions have to be
> stored somewhere, so we store them in a file, and we slurp the
> file (as a unit) into the system, so the file becomes a module?
>
> If all the files were store by themselves in a database would this
> change things.
>
> I am thinking more and more that if would be nice to have *all* functions in
> a key_value database with unique names.
>
> lookup(foo,2) would get the definition foo foo/2 from a database.
>
> The unique names bit is interesting - is this a good idea. Qualified
> names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good
> idea but but when I'm programming I have to invent the xxx or the
> a.b.c which is very difficult. It also involves the "decision problem"
> if the namespaces xxx and a.b.c already exist I have to *choose* which
> to put my new function in.
>
> I think there might be a case for alises here joe:foo/2 could be used
> while developing "joe" would expand to a horrible random local string the
> real name being ab123aZwerasch123123_foo/2  but I would not be able to
> publish my code or make it available to a third_part before I had
> chosen a sensible name.
>
> (( managing namespaces seems really tricky, a lot of peoople seem
> to thing that the problem goes away by adding "." 's to the name
> but managing a namespace with namees like foo.bar.baz.z is just as complex
> as managing a namespace with names like foo_bar_baz_z or names like
> 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic
> name like www.a.b to a reference like 123.45.23.12 - but how do we discover
> the initial name www.a.b? - there are two answers - a) we are given the name
> (ie we click on a link) - we do not know the name but we search fo it ))
>
>
> When programs are small we can live with "just the code" in "a few
> modules" the ratio of code to meta data is high.
>
> When programs are large we need a lot of meta-data to understand them.
>
> I would like to see all functions with all meta-data in a data base.
>
> I'd like to say:
>
>    lookup(foo,2,Attribute) when Attribute =
>
>       code|source|documentation|type signatures|revision history|authors|...
>
> The more I think about it the more I think program development should
> viewed as changing the state of a Key-Value database.
>
> So I imagine:
>
>     1) all functions have unique names
>     2) there are no modules
>     3) we discover the name of a function by searching metadata
>        describing the function in a database
>     4) all public functions (think open source) are in the same
>        database
>
> We could make a system to do this.
>
> I think this would make open-source projects easier, since the
> granularity of contribution goes down. You could contribute
> a single function - not an entire application.
>
> (( A problem with GUT style open source projects is there is
>    not one database of functions, I often what one function from
>    this project, another function from another project -- the
>    granularity of reusable parts should be the individual function.
>
>    functions are really easy to reuse
>    modules are more difficult to reuse
>    entire applications are very difficult to reuse
>      (Unless there are isolated through a communication channel))
>
> Possible extensions.
>
>     1) Voting for promotion
>     2) A review process
>
> Given a raw database will *all* functions in it - we could derive an
> "approved" functions database.
>
> Popular functions could be moved to the approved database - the
> review process would need to be discussed - so kind of peer-review/wiki
> stuff.
>
> Comments?
>
> Volunteers?
>
> /Joe
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
>

--
Gordon Guthrie
CEO hypernumbers

http://hypernumbers.com
t: hypernumbers
[phone]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Josh Johnston]

On 24/05/2011, at 6:39 PM, Dmitrii Dimandt wrote:

So I imagine:

   1) all functions have unique names

   2) there are no modules

   3) we discover the name of a function by searching metadata

      describing the function in a database

   4) all public functions (think open source) are in the same

      database

We could make a system to do this.

Hm... you'd still need *some* kind of modules.

For instance, in Webmachine each resource is expected to have some, or all, of predefined public/exported functions: to_html, resource_exists, allowed_methods etc.

So, each resource will have functions with the same name. If we move all functions to a global namespace, then how do we differentiate between these functions? Probably by naming them resource_name_function_name?

In a normal language this would be a problem, but in erlang we can get the desired behaviour by using pattern matching, right?

So instead of

to_html()

resource_exists()

etc

We have

resource(to_html, ...)

resource(exists, ...)

etc

I'm not disagreeing with you - we do need some good way to do this. But I don't think modules are necessary in this case.

Josh. 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Gordon Guthrie-2]

...and also...

following on from what Dimitri said while I was writing me e-mail, we
hang unit test onto modules and design them at that level of
abstraction

On 24 May 2011 09:48, Gordon Guthrie <[hidden email]> wrote:

> There are two good things about modules:
> * they are a higher level of abstraction
> * they support good working practices
>
> Lets take a module that I use a little bit - and which a colleague
> introduced me to:
> http://erldocs.com/R14B02/stdlib/digraph.html?i=2&search=digra#get_digraph/1
>
> I can use the couple of exposed functions of that that I know,
> confident that if it comes to it I can come back and master the rest
> of the interface later. I have confidence that there is a rubustness
> there that makes code using it maintainable. (my idea of maintainable
> is to be able to ignore most stuff, most of the time...). I can learn
> about the code at this higher level of abstraction.
>
> We too have a module with loads of random functions in it, well we
> have three (hn_util, util, util2), and each of them is just library
> functions. In the utility modules the ratio of exported to
> not-exported fns is maybe 7 to 1.
>
> In functional modules the exported/not ratio goes from 1-1 (in modules
> that define apis) to 1-3 against.
>
> If we had the (promised) -export-to() directive the number of pure
> exported fns would drop.
>
> Given that we build our testing strategy around -export() this
> separation is critical...
>
> Gordon
>
> On 24 May 2011 09:06, Joe Armstrong <[hidden email]> wrote:
>> Why do we need modules at all?
>>
>> This is a brain-dump-stream-of-consciousness-thing. I've been
>> thinking about this for a while.
>>
>> I'm proposing a slightly different way of programming here
>> The basic idea is
>>
>>     - do away with modules
>>     - all functions have unique distinct names
>>     - all functions have (lots of) meta data
>>     - all functions go into a global (searchable) Key-value database
>>     - we need letrec
>>     - contribution to open source can be as simple as
>>       contributing a single function
>>     - there are no "open source projects" - only "the open source
>>       Key-Value database of all functions"
>>     - Content is peer reviewed
>>
>> These are discussed in no particular order below:
>>
>> Why does Erlang have modules?
>>
>> There's a good an bad side to modules:
>>
>> Good: Provides a unit of compilation, a unit of code
>> distribution. unit of code replacement
>>
>> Bad: It's very difficult to decide which module to put an individual
>> function in. Break encapsulation (see later)
>>
>> Aside: lib_misc.erl
>>
>> When I'm programming I often get to the point were I say there should
>> a function foo/2 in lists.erl but their isn't. There should be but
>> there isn't - foo/2 is a small self contained thing. Why should it be
>> in lists.erl because it "feels right".
>>
>> Strings are lists, so why do we have two modules lists.erl and
>> string.erl how should I decide in which module my new string/list
>> processing function should go.
>>
>> To avoid all mental anguish when I need a small function that
>> should be somewhere else and isn't I stick it in
>> a module elib1_misc.erl.
>>
>> My elib1_misc exports the following:
>>
>> added_files/2                 make_challenge/0
>> as_bits/1                     make_response/2
>> as_bits_test/0                make_response_test/0
>> bdump/2                       make_test_strings/1
>> bin2hex/1                     make_test_strings_test/0
>> bin2hex_test/0                make_tmp_filename/2
>> check_io_list/1               merge_kv/1
>> collect_atom/1                merge_kv_test/0
>> collect_atom_test/0           mini_shell/0
>> collect_int/1                 module_info/0
>> collect_int_test/0            module_info/1
>> collect_string/1              ndots/1
>> collect_string_test/0         nibble_to_hex_char/1
>> collect_word/1                nibble_to_hex_char_test/0
>> complete/2                    odd/1
>> complete_test/0               on_exit/2
>> dos2unix/1                    out_of_date/2
>> downcase_char/1               outfile/2
>> dump/2                        padd/2
>> dump_tmp/2                    perms/1
>> duplicates/1                  perms_test/0
>> ensure_started/2              pmap/2
>> eval_file/1                   pmap1/2
>> eval_file_test/0              pmap1_test/0
>> eval_string/1                 pmap_test/0
>> eval_string_test/0            priority_receive/0
>> every/3                       random_seed/0
>> expand_env_vars/1             random_string/1
>> expand_file_template/3        random_string/2
>> expand_string_template/2      read_at_most_n_lines/2
>> expand_tabs/1                 read_at_most_n_lines_test/0
>> expand_tabs_test/0            remove_duplicates/1
>> expand_template/2             remove_duplicates_test/0
>> extract_attribute/2           remove_leading_and_trailing_whitespace/1
>> extract_attribute_test/0      remove_leading_and_trailing_whitespace_test/0
>> extract_prefix/2              remove_leading_whitespace/1
>> fetch/2                       remove_prefix/2
>> fetch_test/0                  remove_prefix_test/0
>> file2lines/1                  remove_trailing_whitespace/1
>> file2lines_test/0             replace/3
>> file2md5/1                    root_dir/0
>> file2numberedlines/1          rpc/2
>> file2numberedlines_test/0     safe/1
>> file2paras/1                  show_loaded/1
>> file2stream/1                 signed_byte_to_hex_string/1
>> file2string/1                 signed_byte_to_hex_string_test/0
>> file2template/1               skip_blanks/1
>> file2term/1                   skip_blanks_test/0
>> file_size_and_type/1          skip_to_nl/1
>> find_src/1                    skip_to_nl_test/0
>> first/1                       sleep/1
>> flatten_io_list/1             spawn_monitor/3
>> flush_buffer/0                split_at_char/2
>> for/3                         split_at_char_test/0
>> force/1                       split_list/2
>> foreach_chunk_in_file/3       split_list_test/0
>> foreach_word_in_file/2        string2exprs/1
>> foreach_word_in_string/2      string2exprs_test/0
>> forever/0                     string2html/1
>> get_erl_section/2             string2latex/1
>> get_line/1                    string2lines/1
>> get_line/2                    string2lines_test/0
>> have_common_prefix/1          string2stream/1
>> have_common_prefix_test/0     string2stream_test/0
>> hex2bin/1                     string2template/1
>> hex2bin_test/0                string2template_test/0
>> hex2list/1                    string2term/1
>> hex2list_test/0               string2term_test/0
>> hex_nibble2int/1              string2toks/1
>> hex_nibble2int_test/0         string2toks_test/0
>> id/1                          sub_binary/3
>> include_dir/0                 template2file/3
>> include_file/1                term2file/2
>> interleave/2                  term2string/1
>> is_alphanum/1                 test/0
>> is_blank_line/1               test1_test/0
>> is_prefix/2                   test_function_over_substrings/2
>> is_prefix_test/0              tex2pdf/1
>> is_response_correct/3         time_fun/2
>> keep_alive/2                  time_stamp/0
>> lines2para/1                  to_lower/1
>> list2frequency_distribution/1 to_lower_test/0
>> list2frequency_distribution_tetrim/1
>> longest_common_prefix/1       trim_test/0
>> longest_common_prefix_test/0  unconsult/2
>> lookup/2                      unsigned_byte_to_hex_string/1
>> lorem/1                       unsigned_byte_to_hex_string_test/0
>> ls/1                          which/1
>>                               which_added/1
>>
>> Now I find this very convenient when I write a new small utility function
>> I stick in in elib1_misc.erl - no mental anguish in choosing a module
>> name is involved.
>>
>> The observation that I find this very-convenient is telling me something
>> about modules - I like my elib1_misc it feels right.
>>
>> (aside - It seems many development projects have their own private
>> lib_miscs ...)
>>
>> Which brings me to the point of my question.
>>
>> Do we need module's at all? Erlang programs are composed of lots of small
>> functions, the only place where modules seem useful is to hide a letrec.
>>
>> The classic example is fibonacci. We want to expose fib/1 but hide the
>> helper function fib/3. Using modules we say
>>
>> -module(math).
>> -export([fib/1]).
>>
>> fib(N) ->
>>     fib(N, 1, 0).
>>
>> fib(N, A, B) when N < 2 -> A;
>> fib(N, A, B) -> fib(N-1, A+B, A).
>>
>> The downside is we have had to *invent* one module name math - whose *only*
>> purpose is to hide the definition of fib/3 which we don't want to be made
>> callable.
>>
>> If we put a second function into the module math, then this second function
>> could call fib/3 which breaks the encapsulation of fib/3.
>>
>> We could say:
>>
>> let fib = fun(N) -> fib(N, 1, 0) end
>> in
>>    fib(N, A, B) when N < 2 -> A;
>>    fib(N, A, B) -> fib(N-1, A+B, A).
>> end.
>>
>> I hardly dare suggest a syntax for this since I've been following
>> another thread in this forum where syntax discussion seem to encourage
>> much comment.
>>
>> ** Please do suggest alternative syntax's here - but do not comment on
>> other peoples suggestions ...
>>
>> I would like to just talk about why we have modules.
>>
>> Another question:
>>
>> Does the idea of a module come from the idea that functions have to be
>> stored somewhere, so we store them in a file, and we slurp the
>> file (as a unit) into the system, so the file becomes a module?
>>
>> If all the files were store by themselves in a database would this
>> change things.
>>
>> I am thinking more and more that if would be nice to have *all* functions in
>> a key_value database with unique names.
>>
>> lookup(foo,2) would get the definition foo foo/2 from a database.
>>
>> The unique names bit is interesting - is this a good idea. Qualified
>> names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good
>> idea but but when I'm programming I have to invent the xxx or the
>> a.b.c which is very difficult. It also involves the "decision problem"
>> if the namespaces xxx and a.b.c already exist I have to *choose* which
>> to put my new function in.
>>
>> I think there might be a case for alises here joe:foo/2 could be used
>> while developing "joe" would expand to a horrible random local string the
>> real name being ab123aZwerasch123123_foo/2  but I would not be able to
>> publish my code or make it available to a third_part before I had
>> chosen a sensible name.
>>
>> (( managing namespaces seems really tricky, a lot of peoople seem
>> to thing that the problem goes away by adding "." 's to the name
>> but managing a namespace with namees like foo.bar.baz.z is just as complex
>> as managing a namespace with names like foo_bar_baz_z or names like
>> 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic
>> name like www.a.b to a reference like 123.45.23.12 - but how do we discover
>> the initial name www.a.b? - there are two answers - a) we are given the name
>> (ie we click on a link) - we do not know the name but we search fo it ))
>>
>>
>> When programs are small we can live with "just the code" in "a few
>> modules" the ratio of code to meta data is high.
>>
>> When programs are large we need a lot of meta-data to understand them.
>>
>> I would like to see all functions with all meta-data in a data base.
>>
>> I'd like to say:
>>
>>    lookup(foo,2,Attribute) when Attribute =
>>
>>       code|source|documentation|type signatures|revision history|authors|...
>>
>> The more I think about it the more I think program development should
>> viewed as changing the state of a Key-Value database.
>>
>> So I imagine:
>>
>>     1) all functions have unique names
>>     2) there are no modules
>>     3) we discover the name of a function by searching metadata
>>        describing the function in a database
>>     4) all public functions (think open source) are in the same
>>        database
>>
>> We could make a system to do this.
>>
>> I think this would make open-source projects easier, since the
>> granularity of contribution goes down. You could contribute
>> a single function - not an entire application.
>>
>> (( A problem with GUT style open source projects is there is
>>    not one database of functions, I often what one function from
>>    this project, another function from another project -- the
>>    granularity of reusable parts should be the individual function.
>>
>>    functions are really easy to reuse
>>    modules are more difficult to reuse
>>    entire applications are very difficult to reuse
>>      (Unless there are isolated through a communication channel))
>>
>> Possible extensions.
>>
>>     1) Voting for promotion
>>     2) A review process
>>
>> Given a raw database will *all* functions in it - we could derive an
>> "approved" functions database.
>>
>> Popular functions could be moved to the approved database - the
>> review process would need to be discussed - so kind of peer-review/wiki
>> stuff.
>>
>> Comments?
>>
>> Volunteers?
>>
>> /Joe
>>
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
>
>
> --
> Gordon Guthrie
> CEO hypernumbers
>
> http://hypernumbers.com
> t: hypernumbers
> [phone]
>

--
Gordon Guthrie
CEO hypernumbers

http://hypernumbers.com
t: hypernumbers
[phone]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Kresten Krab Thorup (Trifork)]

Modules have many uses, but they are often mixed up.  Yesterday I made this drawing:


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Vlad Dumitrescu-2]

Hi,

On Tue, May 24, 2011 at 10:06, Joe Armstrong <[hidden email]> wrote:

Do we need module's at all? Erlang programs are composed of lots of small
functions, the only place where modules seem useful is to hide a letrec.

The classic example is fibonacci. We want to expose fib/1 but hide the
helper function fib/3. Using modules we say

-module(math).
-export([fib/1]).

fib(N) ->
    fib(N, 1, 0).

fib(N, A, B) when N < 2 -> A;
fib(N, A, B) -> fib(N-1, A+B, A).

The downside is we have had to *invent* one module name math - whose *only*
purpose is to hide the definition of fib/3 which we don't want to be made
callable.

If we put a second function into the module math, then this second function
could call fib/3 which breaks the encapsulation of fib/3.

We could say:

let fib = fun(N) -> fib(N, 1, 0) end
in
   fib(N, A, B) when N < 2 -> A;
   fib(N, A, B) -> fib(N-1, A+B, A).
end.

One reason where letrec might not be a good enough replacement for a helper function that is private to a module, is if this helper function is called from many places. I wouldn't want to have to repeat its definition every time I need it...

I also feel like Mazen that unless there is a big shift in the way we think about the programs, the problem will still be present at the prefix level.

If there is to be a big shift in the paradigms, then the question is whether the result will still be Erlang... 

regards,

Vlad

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Kresten Krab Thorup (Trifork)]

Sorry, my mail got strangled in flight.  Here we go again... in plain text.

The gen_server module is used for three different things:

- as a collection of API function (left)
- a place to put the code that controls the gen_server process (middle)
- a place to describe the abstract notion of a gen_server behavior (right).

Ideally, I think those three usages should be separate language constructs, but probably not in separate files (or other kinds of units).

When using a gen_server, it would be nice if you did not have to expose the behavior api to the "user" of your module.


Currently, the module is the unit of code loading.  I think we could use a bette mechanism, perhaps one that supports an STM-like thing in the VM to control the loading of a set of modules.  I don't have much deep experience with code loading in Erlang (but do have it on Java), and it just seems to me that being able to control atomic loading of .. say a new version of an Erlang application would be good to have supported in the VM.


Gilad Bracha (who's working on Newspeak) has an excellent blog on new notions of modues that you should read.  You can start here http://gbracha.blogspot.com/2009/06/ban-on-imports.html

One of the things he's arguing is that modules should be VALUES.  A "module" in newspeak is kind of a template (function), which take other module instances as arguments.  So you hook up a system by applying such templates (possibly using letrec).  


If we had a visual programming environment, then many things don't need a name.  Self is an example of this (Self, the video: http://video.google.com/videoplay?docid=5776880551404953752#)  

Kresten

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Xiaopong Tran]

On 05/24/2011 04:49 PM, Josh Johnston wrote:

>
>
> In a normal language this would be a problem, but in erlang we can get
> the desired behaviour by using pattern matching, right?
>
> So instead of
>
>
>     to_html()
>     resource_exists()
>     etc
>
>
> We have
>
> resource(to_html, ...)
> resource(exists, ...)
> etc
>
> I'm not disagreeing with you - we do need some good way to do this. But
> I don't think modules are necessary in this case.
>

Pattern matching might work here, but do away with modules and using
pattern matching would a heavier load on the programmer.

Modules or not, it does not have much effect on the VM, we can
just make the VM work without modules. But I think programmers
would have to work harder, distinguishing which function is which.

Although I don't really like the Erlang module, but I think it's
better with it than without.

Xiaopong
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Jesper Louis Andersen-2]

On Tue, May 24, 2011 at 11:05, Kresten Krab Thorup <[hidden email]> wrote:
>
> One of the things he's arguing is that modules should be VALUES.  A "module" in newspeak is kind of a template (function), which take other module instances as arguments.  So you hook up a system by applying such templates (possibly using letrec).
>

This is a great idea. Why? Because it is the path of existential types :-)

Notice that in Erlang, a module is a Value. It is possible to call
Mod:Fun(..). Second, notice that the parameterized module extension
allows us to parameterize a module over some other module.
Specifically we could have

-module(RBTree, [OrderMod]).

where OrderMod is a module which has a function

-opaque t() :: .....
-export_type([t/0]).
-spec order(t(), t()) -> lt | eq | gt.

and so we have generalized our RedBlack trees to be able to use any
ordering relation, not simply the one that is built it.

The style Gilad is advocating is the "Fully functorial style" from the
Standard ML community. A Functor[1] in SML terminology is a function
from module to module ("functor" stems from category theory where it
designates a "function"[2] one level up). In the style, you build up
programs by stitching together smaller modularized parts building
larger and larger modules in the process.

Example: The gen_server is a functor from a module (which obeys the
gen_server behaviour and has handle_* functions) to a specific
instance of a server.

It also gives you quite the path towards the concept of OOP, as can be
observed from the parameterized module extension and its uses in
various places.

In languages with static types, you can of course enforce that functor
applications are type-correct. If you allow modules as values in such
languages you get existential types which is a way to get something
OOP-like into those languages[3].

So after a little tour around: Erlang already has this :P

Modules are an essential part of building large systems. The very
notion that you can pack up functions in a module and ship them as a
unit is quite important. Also the notion you can replace a
module-component at once. I feel that the problem of function
placement in the right module is less of a problem compared to the
hell it would be with one large global namespace. And it is much more
than just a "bunch of functions thrown together".

Specifically, modules allows me to have an *abstract* view of a data
structure. In the above, I have no idea what t() is. I am only allowed
to compare two t()'s to each other with OrderMod:compare(T1, T2). That
is, the only algebraic operation I am allowed to do is this. Other
parts of the program may have a different view of t() and be able to
do more with it, but to me, it is an opaque term. It matters because
someone can go replace t(), and my code does not have to be changed at
all. This is impossible if t() leaks. And I am grateful the dialyzer
can find places where it is leaking.

Likewise, a module can hide the fact it is implemented with a
gen_server. So I can go replace it with a gen_fsm, and nobody has to
alter their code because the API i exported was exactly the API I will
keep stable.

Processes componentize the heap.
Modules componentize the code.



[1] Beware of connotating the word functor in programming with
something specific in computer science. The meaning is different in
Standard ML, in Haskell and in C++.

[2] I am lying. The basic notion is a morphism of which the function
is a special case when considering the category of Sets.

[3] I still don't get why you can't use static typing in Erlang. The
conservative approach is simply to use polymorphic variants for all
message passing and it would essentially work right out of the bat.

--
J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Tim Carpenter]

Hi all,

I am sticking to the request to not comment on other solutions but  
present my own thoughts...

1. Some units of code only make sense as a collection. who would use  
part of gen_server in isolation? Well, you could,but to have it  
separate exposes it to modification that could distort the focused  
purpose of the unit.

2. some functions have intentionally hidden aspects, private calls etc  
and these again are best left hidden or considered as a logical whole  
otherwise side effects or bloat may occur.

3. I see no reason why we cannot logically group functions into  
modules but not have the code in there...i.e. the module contains the  
references to functions that make it up. this allows logical handling  
of groups of functions for code management and can allow one function  
to appear in multiple modules without code duplication. it also allows  
one to add/delete/replace a function in a module knowing that the  
source code for the other functions have not been touched!

4. naming conventions should be given great thought for simplicity and  
intuitiveness. functions could be in n sets, so a strict hierarchy may  
not suit.

5. if functions are named as being in modules, the need to provide  
abstraction to the full name may be advantageous, so modulename.sort/3  
can be used and the module definition translates this to a unique  
longhand which can change over time. The programmer should also be  
able to use the longhand at will using the module as a convenient bulk  
import/compile vehicle that will break at compile time (one would  
hope). Some form of 'expects...' notation could allow a form of  
protection against inadvertent swapping of functions in shared modules  
altering behaviour.

6. we may want 3 elements to naming of functions: shorthand, longhand,  
and version, so one can use the latest version of a longhand named  
function, lock to a specific version of a longhand named function or  
use the shorthand latest version or other such combinations.

7.  n- tier modules - modules of modules? possibly.

Tim



On 24 May 2011, at 10:05, Kresten Krab Thorup wrote:

> Sorry, my mail got strangled in flight.  Here we go again... in  
> plain text.
>
> The gen_server module is used for three different things:
>
> - as a collection of API function (left)
> - a place to put the code that controls the gen_server process  
> (middle)
> - a place to describe the abstract notion of a gen_server behavior  
> (right).
>
> Ideally, I think those three usages should be separate language  
> constructs, but probably not in separate files (or other kinds of  
> units).
>
> When using a gen_server, it would be nice if you did not have to  
> expose the behavior api to the "user" of your module.
>
>
> Currently, the module is the unit of code loading.  I think we could  
> use a bette mechanism, perhaps one that supports an STM-like thing  
> in the VM to control the loading of a set of modules.  I don't have  
> much deep experience with code loading in Erlang (but do have it on  
> Java), and it just seems to me that being able to control atomic  
> loading of .. say a new version of an Erlang application would be  
> good to have supported in the VM.
>
>
> Gilad Bracha (who's working on Newspeak) has an excellent blog on  
> new notions of modues that you should read.  You can start here http://gbracha.blogspot.com/2009/06/ban-on-imports.html
>
> One of the things he's arguing is that modules should be VALUES.  A  
> "module" in newspeak is kind of a template (function), which take  
> other module instances as arguments.  So you hook up a system by  
> applying such templates (possibly using letrec).
>
>
> If we had a visual programming environment, then many things don't  
> need a name.  Self is an example of this (Self, the video: http://video.google.com/videoplay?docid=5776880551404953752#)
>
> Kresten
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

Tim Carpenter
Amphibian Ltd
London
ENGLAND
[phone]
[hidden email]



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Gary Hai-2]

I am working on state-oriented programming model, and I have the same question as you.
In the view of SOP, erlang process is a FSM, and suitable granularity of functions as states.
The finite set of states of a FSM just like erlang module, but it always cross modules boundaries.
So in SOP model, the module concept has been very vague, just for unique distinct name of function.
The model of a state activity is a function as bellow:
act(Entity, Input) -> {ok, Output} | {ok, Output, UpdatedEntity} | {error, Error} | {stop, Output, Entity}
The output of the state may be {NextState, Output}.
The NextState may be the next state activity function or just a directive of next state.
Then FSM engine matches the directive to a state activity then transfers the previous state to the new state with the previous output as input.
The state activity function may be from anywhere, function in a module, binary loaded from database, code dynamically generated by JIT.

--------
Gary Hai

On May 24, 2011, at 4:06 PM, Joe Armstrong wrote:

> Why do we need modules at all?
>
> This is a brain-dump-stream-of-consciousness-thing. I've been
> thinking about this for a while.
>
> I'm proposing a slightly different way of programming here
> The basic idea is
>
>     - do away with modules
>     - all functions have unique distinct names
>     - all functions have (lots of) meta data
>     - all functions go into a global (searchable) Key-value database
>     - we need letrec
>     - contribution to open source can be as simple as
>       contributing a single function
>     - there are no "open source projects" - only "the open source
>       Key-Value database of all functions"
>     - Content is peer reviewed
>
> These are discussed in no particular order below:
>
> Why does Erlang have modules?
>
> There's a good an bad side to modules:
>
> Good: Provides a unit of compilation, a unit of code
> distribution. unit of code replacement
>
> Bad: It's very difficult to decide which module to put an individual
> function in. Break encapsulation (see later)
>
> Aside: lib_misc.erl
>
> When I'm programming I often get to the point were I say there should
> a function foo/2 in lists.erl but their isn't. There should be but
> there isn't - foo/2 is a small self contained thing. Why should it be
> in lists.erl because it "feels right".
>
> Strings are lists, so why do we have two modules lists.erl and
> string.erl how should I decide in which module my new string/list
> processing function should go.
>
> To avoid all mental anguish when I need a small function that
> should be somewhere else and isn't I stick it in
> a module elib1_misc.erl.
>
> My elib1_misc exports the following:
>
> added_files/2                 make_challenge/0
> as_bits/1                     make_response/2
> as_bits_test/0                make_response_test/0
> bdump/2                       make_test_strings/1
> bin2hex/1                     make_test_strings_test/0
> bin2hex_test/0                make_tmp_filename/2
> check_io_list/1               merge_kv/1
> collect_atom/1                merge_kv_test/0
> collect_atom_test/0           mini_shell/0
> collect_int/1                 module_info/0
> collect_int_test/0            module_info/1
> collect_string/1              ndots/1
> collect_string_test/0         nibble_to_hex_char/1
> collect_word/1                nibble_to_hex_char_test/0
> complete/2                    odd/1
> complete_test/0               on_exit/2
> dos2unix/1                    out_of_date/2
> downcase_char/1               outfile/2
> dump/2                        padd/2
> dump_tmp/2                    perms/1
> duplicates/1                  perms_test/0
> ensure_started/2              pmap/2
> eval_file/1                   pmap1/2
> eval_file_test/0              pmap1_test/0
> eval_string/1                 pmap_test/0
> eval_string_test/0            priority_receive/0
> every/3                       random_seed/0
> expand_env_vars/1             random_string/1
> expand_file_template/3        random_string/2
> expand_string_template/2      read_at_most_n_lines/2
> expand_tabs/1                 read_at_most_n_lines_test/0
> expand_tabs_test/0            remove_duplicates/1
> expand_template/2             remove_duplicates_test/0
> extract_attribute/2           remove_leading_and_trailing_whitespace/1
> extract_attribute_test/0      remove_leading_and_trailing_whitespace_test/0
> extract_prefix/2              remove_leading_whitespace/1
> fetch/2                       remove_prefix/2
> fetch_test/0                  remove_prefix_test/0
> file2lines/1                  remove_trailing_whitespace/1
> file2lines_test/0             replace/3
> file2md5/1                    root_dir/0
> file2numberedlines/1          rpc/2
> file2numberedlines_test/0     safe/1
> file2paras/1                  show_loaded/1
> file2stream/1                 signed_byte_to_hex_string/1
> file2string/1                 signed_byte_to_hex_string_test/0
> file2template/1               skip_blanks/1
> file2term/1                   skip_blanks_test/0
> file_size_and_type/1          skip_to_nl/1
> find_src/1                    skip_to_nl_test/0
> first/1                       sleep/1
> flatten_io_list/1             spawn_monitor/3
> flush_buffer/0                split_at_char/2
> for/3                         split_at_char_test/0
> force/1                       split_list/2
> foreach_chunk_in_file/3       split_list_test/0
> foreach_word_in_file/2        string2exprs/1
> foreach_word_in_string/2      string2exprs_test/0
> forever/0                     string2html/1
> get_erl_section/2             string2latex/1
> get_line/1                    string2lines/1
> get_line/2                    string2lines_test/0
> have_common_prefix/1          string2stream/1
> have_common_prefix_test/0     string2stream_test/0
> hex2bin/1                     string2template/1
> hex2bin_test/0                string2template_test/0
> hex2list/1                    string2term/1
> hex2list_test/0               string2term_test/0
> hex_nibble2int/1              string2toks/1
> hex_nibble2int_test/0         string2toks_test/0
> id/1                          sub_binary/3
> include_dir/0                 template2file/3
> include_file/1                term2file/2
> interleave/2                  term2string/1
> is_alphanum/1                 test/0
> is_blank_line/1               test1_test/0
> is_prefix/2                   test_function_over_substrings/2
> is_prefix_test/0              tex2pdf/1
> is_response_correct/3         time_fun/2
> keep_alive/2                  time_stamp/0
> lines2para/1                  to_lower/1
> list2frequency_distribution/1 to_lower_test/0
> list2frequency_distribution_tetrim/1
> longest_common_prefix/1       trim_test/0
> longest_common_prefix_test/0  unconsult/2
> lookup/2                      unsigned_byte_to_hex_string/1
> lorem/1                       unsigned_byte_to_hex_string_test/0
> ls/1                          which/1
>                               which_added/1
>
> Now I find this very convenient when I write a new small utility function
> I stick in in elib1_misc.erl - no mental anguish in choosing a module
> name is involved.
>
> The observation that I find this very-convenient is telling me something
> about modules - I like my elib1_misc it feels right.
>
> (aside - It seems many development projects have their own private
> lib_miscs ...)
>
> Which brings me to the point of my question.
>
> Do we need module's at all? Erlang programs are composed of lots of small
> functions, the only place where modules seem useful is to hide a letrec.
>
> The classic example is fibonacci. We want to expose fib/1 but hide the
> helper function fib/3. Using modules we say
>
> -module(math).
> -export([fib/1]).
>
> fib(N) ->
>     fib(N, 1, 0).
>
> fib(N, A, B) when N < 2 -> A;
> fib(N, A, B) -> fib(N-1, A+B, A).
>
> The downside is we have had to *invent* one module name math - whose *only*
> purpose is to hide the definition of fib/3 which we don't want to be made
> callable.
>
> If we put a second function into the module math, then this second function
> could call fib/3 which breaks the encapsulation of fib/3.
>
> We could say:
>
> let fib = fun(N) -> fib(N, 1, 0) end
> in
>    fib(N, A, B) when N < 2 -> A;
>    fib(N, A, B) -> fib(N-1, A+B, A).
> end.
>
> I hardly dare suggest a syntax for this since I've been following
> another thread in this forum where syntax discussion seem to encourage
> much comment.
>
> ** Please do suggest alternative syntax's here - but do not comment on
> other peoples suggestions ...
>
> I would like to just talk about why we have modules.
>
> Another question:
>
> Does the idea of a module come from the idea that functions have to be
> stored somewhere, so we store them in a file, and we slurp the
> file (as a unit) into the system, so the file becomes a module?
>
> If all the files were store by themselves in a database would this
> change things.
>
> I am thinking more and more that if would be nice to have *all* functions in
> a key_value database with unique names.
>
> lookup(foo,2) would get the definition foo foo/2 from a database.
>
> The unique names bit is interesting - is this a good idea. Qualified
> names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good
> idea but but when I'm programming I have to invent the xxx or the
> a.b.c which is very difficult. It also involves the "decision problem"
> if the namespaces xxx and a.b.c already exist I have to *choose* which
> to put my new function in.
>
> I think there might be a case for alises here joe:foo/2 could be used
> while developing "joe" would expand to a horrible random local string the
> real name being ab123aZwerasch123123_foo/2  but I would not be able to
> publish my code or make it available to a third_part before I had
> chosen a sensible name.
>
> (( managing namespaces seems really tricky, a lot of peoople seem
> to thing that the problem goes away by adding "." 's to the name
> but managing a namespace with namees like foo.bar.baz.z is just as complex
> as managing a namespace with names like foo_bar_baz_z or names like
> 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic
> name like www.a.b to a reference like 123.45.23.12 - but how do we discover
> the initial name www.a.b? - there are two answers - a) we are given the name
> (ie we click on a link) - we do not know the name but we search fo it ))
>
>
> When programs are small we can live with "just the code" in "a few
> modules" the ratio of code to meta data is high.
>
> When programs are large we need a lot of meta-data to understand them.
>
> I would like to see all functions with all meta-data in a data base.
>
> I'd like to say:
>
>    lookup(foo,2,Attribute) when Attribute =
>
>       code|source|documentation|type signatures|revision history|authors|...
>
> The more I think about it the more I think program development should
> viewed as changing the state of a Key-Value database.
>
> So I imagine:
>
>     1) all functions have unique names
>     2) there are no modules
>     3) we discover the name of a function by searching metadata
>        describing the function in a database
>     4) all public functions (think open source) are in the same
>        database
>
> We could make a system to do this.
>
> I think this would make open-source projects easier, since the
> granularity of contribution goes down. You could contribute
> a single function - not an entire application.
>
> (( A problem with GUT style open source projects is there is
>    not one database of functions, I often what one function from
>    this project, another function from another project -- the
>    granularity of reusable parts should be the individual function.
>
>    functions are really easy to reuse
>    modules are more difficult to reuse
>    entire applications are very difficult to reuse
>      (Unless there are isolated through a communication channel))
>
> Possible extensions.
>
>     1) Voting for promotion
>     2) A review process
>
> Given a raw database will *all* functions in it - we could derive an
> "approved" functions database.
>
> Popular functions could be moved to the approved database - the
> review process would need to be discussed - so kind of peer-review/wiki
> stuff.
>
> Comments?
>
> Volunteers?
>
> /Joe
>
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Jesper Louis Andersen-2]

On Tue, May 24, 2011 at 10:06, Joe Armstrong <[hidden email]> wrote:
> Why do we need modules at all?

I think we need modules, but this post sparked a thought I have had
for some time now.

I'd like to have module renaming/namespacing!

Specifically, I want a mechanism by which I can clean up the naming
mess in the stdlib without affecting old programs. That is, I want to
be able to, at an application/module level, rename function calls in a
way such a way that can clean up parameter orders and so on. I don't
care about how the solution is. I am after the goal: Cleaning up the
stdlib mess.

You may argue this is a messy thing, but I think it is a necessity if
you want to move the standard library on:

* We can't alter existing functions because people rely on them.
* Existing functions are not consistent.

This is a social deadlock I want to break. In fact it is a deadlock I
think we can only break if we allow the coexistence of more than one
module with the same name (but stemming from different namespaces). I
don't want nesting. Just two levels, so I can say

-use_namespace(v2_modules).

in modules which use version two of the stdlib in which things are
sensibly named. At some point you can then flip around and compile old
code with a compatibility layer.


--
J.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Bengt Kleberg]

Greetings,

It would be very nice to have module renaming/namespacing that allowed
me to do the following:
The modules "misc_lib" and "misc_lib_helper" from developer A should be
able to call each other with their original names.
The modules "misc_lib" and "misc_lib_helper" from developer B should be
able to call each other with their original names.
I should be able to call all 4 at the same time with names that I have
create/renamed.

So, I need to load modules as a set/application/??? that renames them
outside of that set (it would be ok to have the same prefix added to all
of them), and still allows the files in the set/application/??? to keep
using to the old names.


When it comes to improving stdlibs I suggest that we create new, better
modules. Keep the old ones (for ever?) but print a big "deprecated" all
over the man pages.


bengt

On Tue, 2011-05-24 at 12:26 +0200, Jesper Louis Andersen wrote:

> On Tue, May 24, 2011 at 10:06, Joe Armstrong <[hidden email]> wrote:
> > Why do we need modules at all?
>
> I think we need modules, but this post sparked a thought I have had
> for some time now.
>
> I'd like to have module renaming/namespacing!
>
> Specifically, I want a mechanism by which I can clean up the naming
> mess in the stdlib without affecting old programs. That is, I want to
> be able to, at an application/module level, rename function calls in a
> way such a way that can clean up parameter orders and so on. I don't
> care about how the solution is. I am after the goal: Cleaning up the
> stdlib mess.
>
> You may argue this is a messy thing, but I think it is a necessity if
> you want to move the standard library on:
>
> * We can't alter existing functions because people rely on them.
> * Existing functions are not consistent.
>
> This is a social deadlock I want to break. In fact it is a deadlock I
> think we can only break if we allow the coexistence of more than one
> module with the same name (but stemming from different namespaces). I
> don't want nesting. Just two levels, so I can say
>
> -use_namespace(v2_modules).
>
> in modules which use version two of the stdlib in which things are
> sensibly named. At some point you can then flip around and compile old
> code with a compatibility layer.
>
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Tim Watson-5]

My 2 pennies worth: I don't think getting rid of modules is a good
plan. Having global (mangled) names is a path to misery. In the
languages that do mess with names (python, C++, etc) it creates a
bloody nightmare. I would steer clear of this. I like the idea of
having a global database of code, but not all of it floating in some
etherial top level namespace.

Most of what's cool about http://www.haskell.org/hoogle/ and
http://hackage.haskell.org/packages/hackage.html is made possible by
the combination of good standardised build tools and hosting
repositories, so I'd be more included to focus on improving (and
extending) things like rebar and agner/erlware to provide the same
kind of developer-experience without changing the language so much.

Introducing letrec is a bloody marvelous suggestion. Also making it
easier to do partial application and function composition would
simplify a lot of the re-use cases without messing with the module
system.

I'm not going to stray off too far into alternative-solution land, but
given the OTP team reads this list, I don't want to miss the
opportunity to mention some minor things about the module system that
could do with an overhaul.

- Fully supported parameterised modules

Not because I want to use them as a state bag, but rather that they
make it easy to implement something akin to a functor without having
to reside in a process. This means that all the tools need to support
this concept and it is documented, etc. Extending the module system so
that you can -extend a module to create a new concrete implementation
would be useful:

%% finder.erl
-module(finder).
-functor([M]).
-compile(export_all).

find() -> walk(fun M:match/3).

%% rx_finder.erl
-module(rx_finder).
-extends(finder, [re]).

Perhaps the syntax/approach there isn't quite right, but the point I'm
getting at is that what with parameterised modules and -extends, you
should be able to compose stuff easily.

- Better support for *import*

As the quantity of code you're dealing with increases, you need to be
able to import discrete units of it in simpler and more intuitive
ways. I'm not exactly a fan of Java, but you can see in that world
that the quantity of libraries (often open source ones) being used on
any project means you simply cannot avoid having to import things
(packages, classes, static methods/functions) without going mad.
Although Erlang is still (thankfully) simple enough not to have this
problem, I suspect as the number of projects blossoms it will
eventually approach the critical mass.

Also being able to support something like -import_alias(name, mod)
would be quite handy, although I realise this is fairly trivial with a
parse_transform.

- Fully supporting packages

Personally I think the <app>_<subsystem>_<thing> convention that most
of us follow is fine, but it does get rather tedious. Package,
namespaces or nested modules are all solution that are known to work
and are widely adopted by other languages. The package system in
erlang works fine, but not all the tools properly support it today
(cover, reltool and a few others are broken).
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Tim Watson-5]

On 24 May 2011 11:39, Bengt Kleberg <[hidden email]> wrote:

> Greetings,
>
> It would be very nice to have module renaming/namespacing that allowed
> me to do the following:
> The modules "misc_lib" and "misc_lib_helper" from developer A should be
> able to call each other with their original names.
> The modules "misc_lib" and "misc_lib_helper" from developer B should be
> able to call each other with their original names.
> I should be able to call all 4 at the same time with names that I have
> create/renamed.
>
> So, I need to load modules as a set/application/??? that renames them
> outside of that set (it would be ok to have the same prefix added to all
> of them), and still allows the files in the set/application/??? to keep
> using to the old names.
>
>

I was thinking about this in terms of some kind of special -import or
-alias directive. I think it's possible to do just with a
parse_transform, as long as the set can be fully determined at compile
time. That last point though - therein lies the rub. App-upgrades
change things, and those things can include modules. Dragons.
Nuff-said.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Masklinn]

On 2011-05-24, at 13:19 , Tim Watson wrote:
> In the languages that do mess with names (python […]

As far as I know, Python never "messes with names" unless asked explicitly via the "__" prefix, which is rarely used and generally only there to avoid naming conflicts in inheritance cases.

Do you have instances, besides this quite explicit one, where Python performs any name mangling?
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions