Quantcast

Erlang 19 callbacks aren't attributes?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Erlang 19 callbacks aren't attributes?

Ryan Stewart
There was a question about this on the list a few months back, but it was unanswered. It seems that in Erlang 19, a module's callbacks no longer appear in the module_info() as 'attributes'. They are, however, included in the abstract code (via beam_lib). I'm wondering if this was an intentional change or an oversight.

Are callbacks supposed to show up under a module's module_info attributes like they used to?

Ryan

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang 19 callbacks aren't attributes?

Ryan Stewart
Anybody? This seems like a moderately serious issue. We've found that it can cause issues with meck, which assumes callbacks appear in module_info attributes.

On Mon, Jan 16, 2017 at 1:08 PM Ryan Stewart <[hidden email]> wrote:
There was a question about this on the list a few months back, but it was unanswered. It seems that in Erlang 19, a module's callbacks no longer appear in the module_info() as 'attributes'. They are, however, included in the abstract code (via beam_lib). I'm wondering if this was an intentional change or an oversight.

Are callbacks supposed to show up under a module's module_info attributes like they used to?

Ryan

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang 19 callbacks aren't attributes?

Björn Gustavsson-4
In reply to this post by Ryan Stewart
On Mon, Jan 16, 2017 at 8:08 PM, Ryan Stewart <[hidden email]> wrote:
> There was a question about this on the list a few months back, but it was
> unanswered. It seems that in Erlang 19, a module's callbacks no longer
> appear in the module_info() as 'attributes'. They are, however, included in
> the abstract code (via beam_lib). I'm wondering if this was an intentional
> change or an oversight.

The change was intentional, but we did not foresee that it would
cause problems.

There is no formal documentation exactly which attributes should
appear in module_info(attributes). A long time ago, all attributes
except 'module', 'export', 'import', 'file', and 'record' were included.

When the dialyzer attributes 'type', 'spec', and 'opaque' were introduced
we decided not to include them to reduce the code size.

Since 'callback' and 'optional_callback' are attributes that
are used by dialyzer, we decided that they too should be
excluded (for consistency).


I don't know exactly what information meck needs,
but there is the behaviour_info/1 function can be used
to retrieve the names of callback functions. See:

http://erlang.org/doc/design_principles/spec_proc.html#id81417

If the meck will need the complete spec, the solution for now
at least is to extract from the abstract code.

/Bjorn


--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang 19 callbacks aren't attributes?

Ryan Stewart
On Mon, Jan 23, 2017 at 6:28 AM Björn Gustavsson <[hidden email]> wrote:
The change was intentional, but we did not foresee that it would
cause problems.

There is no formal documentation exactly which attributes should
appear in module_info(attributes). A long time ago, all attributes
except 'module', 'export', 'import', 'file', and 'record' were included.

Well, to that point, the following appears on the Modules reference page (http://erlang.org/doc/reference_manual/modules.html):

"Any module attribute can be specified. The attributes are stored in the compiled code and can be retrieved by calling Module:module_info(attributes), or by using the module beam_lib(3) in STDLIB."

The most straightforward understanding of that has to be that all attributes are equally accessible in both places.

When the dialyzer attributes 'type', 'spec', and 'opaque' were introduced
we decided not to include them to reduce the code size.

Since 'callback' and 'optional_callback' are attributes that
are used by dialyzer, we decided that they too should be
excluded (for consistency).

That makes sense in a way, but at the same time, it seems very odd for all of these things to be "attributes", and yet only some of them appear when you ask the language for the attributes of a module. Regardless of the intended use of an attribute, why would the platform lie about whether or not an attribute is present?

Also, what do you mean by "excluded"? As I pointed out in my post, the attributes are still present in the abstract code. I don't understand what's saved by having them in one place but not the other.

Ryan

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang 19 callbacks aren't attributes?

Björn Gustavsson-4
On Tue, Jan 24, 2017 at 4:04 AM, Ryan Stewart <[hidden email]> wrote:

> Also, what do you mean by "excluded"? As I pointed out in my post, the
> attributes are still present in the abstract code. I don't understand what's
> saved by having them in one place but not the other.
>

Everything returned by module_info() is loaded into memory along with
the code for the module. The abstract code will not loaded be loaded
into memory.

/Bjorn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang 19 callbacks aren't attributes?

Richard Carlsson-3
In reply to this post by Ryan Stewart
There are some distinctions that have not been well documented. I'll try to clarify them based on how things actually work.

First, an Erlang program is divided into "forms" - sequences of tokens that are terminated by a full stop. Forms are either function definitions (starting with a function name, i.e., an atom) or what's generally called "attributes", which start with a minus and an atom, as in '-module(..).'. Seems straightforward, right?

There are however three kinds of forms in Erlang that have the general shape of attributes:

* Preprocessor Directives - things like '-include("foo.hrl").' or '-ifdef(DEBUG)'. These only make sense to the preprocessor (epp), and are removed by the preprocessing step.

* Declarations - things like module name, export/import lists, record definitions, type specifications, type definitions, and behaviour callbacks. Also some more unknown special declarations like '-file(...).' that the preprocessor inserts to tell the compiler which parts of the code came from which files, or errors and warnings passed on from preprocessing. The common thing about Declarations is that they have meaning on the Erlang Language level, and typically have particular rules for how they may look (a module declaration must provide a single module name, an import declaration must provide a module name and a list of function/arity pairs, a record declaration must list fields and possibly types, and so on), and where in the file they may occur (before any function, or interspersed with functions). Since these are part of the language, they are not included in the 'attributes' list of the Beam file; the corresponding information is generally found in some other part of the file.

* "Wild" Attributes (as they were named in the draft Standard Erlang specification) - any attribute that is not one of the Declarations, and has the form '-Atom(Term)' - note that only one argument is allowed. These can occur anywhere in the file and have no special meaning except possibly to various tools. For example, '-compile(Options).' only has a meaning to the specific Erlang compiler that you're using; it's not part of the language. These are just annotations, and these are the ones that are listed in the 'attributes' section of the module info.

These distinctions have not been properly documented, but have been intuitively followed by the people working on the language and compiler. In some cases, the lines are blurred. For example, one could easily argue that '-behaviour(Name).'  was just another wild attribute which only had a meaning to the current version of the compiler, and not part of the language proper. On the other hand, since the -callback declarations were introduced, it's pretty clear that behaviours are a language feature.

Along these lines, I think that e.g. the type declarations of a module should be included in Beam files as standard chunks, so you don't need to extract them from the "abstract format" debug information, if present, but that's another can of pull requests.

        /Richard

2017-01-24 4:04 GMT+01:00 Ryan Stewart <[hidden email]>:
On Mon, Jan 23, 2017 at 6:28 AM Björn Gustavsson <[hidden email]> wrote:
The change was intentional, but we did not foresee that it would
cause problems.

There is no formal documentation exactly which attributes should
appear in module_info(attributes). A long time ago, all attributes
except 'module', 'export', 'import', 'file', and 'record' were included.

Well, to that point, the following appears on the Modules reference page (http://erlang.org/doc/reference_manual/modules.html):

"Any module attribute can be specified. The attributes are stored in the compiled code and can be retrieved by calling Module:module_info(attributes), or by using the module beam_lib(3) in STDLIB."

The most straightforward understanding of that has to be that all attributes are equally accessible in both places.

When the dialyzer attributes 'type', 'spec', and 'opaque' were introduced
we decided not to include them to reduce the code size.

Since 'callback' and 'optional_callback' are attributes that
are used by dialyzer, we decided that they too should be
excluded (for consistency).

That makes sense in a way, but at the same time, it seems very odd for all of these things to be "attributes", and yet only some of them appear when you ask the language for the attributes of a module. Regardless of the intended use of an attribute, why would the platform lie about whether or not an attribute is present?

Also, what do you mean by "excluded"? As I pointed out in my post, the attributes are still present in the abstract code. I don't understand what's saved by having them in one place but not the other.

Ryan

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang 19 callbacks aren't attributes?

Alex S.

> 25 янв. 2017 г., в 13:00, Richard Carlsson <[hidden email]> написал(а):
>
> There are some distinctions that have not been well documented. I'll try to clarify them based on how things actually work.
>
> First, an Erlang program is divided into "forms" - sequences of tokens that are terminated by a full stop. Forms are either function definitions (starting with a function name, i.e., an atom) or what's generally called "attributes", which start with a minus and an atom, as in '-module(..).'. Seems straightforward, right?
>
> There are however three kinds of forms in Erlang that have the general shape of attributes:
>
> * Preprocessor Directives - things like '-include("foo.hrl").' or '-ifdef(DEBUG)'. These only make sense to the preprocessor (epp), and are removed by the preprocessing step.
>
> * Declarations - things like module name, export/import lists, record definitions, type specifications, type definitions, and behaviour callbacks. Also some more unknown special declarations like '-file(...).' that the preprocessor inserts to tell the compiler which parts of the code came from which files, or errors and warnings passed on from preprocessing. The common thing about Declarations is that they have meaning on the Erlang Language level, and typically have particular rules for how they may look (a module declaration must provide a single module name, an import declaration must provide a module name and a list of function/arity pairs, a record declaration must list fields and possibly types, and so on), and where in the file they may occur (before any function, or interspersed with functions). Since these are part of the language, they are not included in the 'attributes' list of the Beam file; the corresponding information is generally found in some other part of the file.
>
> * "Wild" Attributes (as they were named in the draft Standard Erlang specification) - any attribute that is not one of the Declarations, and has the form '-Atom(Term)' - note that only one argument is allowed. These can occur anywhere in the file and have no special meaning except possibly to various tools. For example, '-compile(Options).' only has a meaning to the specific Erlang compiler that you're using; it's not part of the language. These are just annotations, and these are the ones that are listed in the 'attributes' section of the module info.
>
> These distinctions have not been properly documented, but have been intuitively followed by the people working on the language and compiler. In some cases, the lines are blurred. For example, one could easily argue that '-behaviour(Name).'  was just another wild attribute which only had a meaning to the current version of the compiler, and not part of the language proper. On the other hand, since the -callback declarations were introduced, it's pretty clear that behaviours are a language feature.
>
> Along these lines, I think that e.g. the type declarations of a module should be included in Beam files as standard chunks, so you don't need to extract them from the "abstract format" debug information, if present, but that's another can of pull requests.
Along those exact lines, I argue that -callback attributes are nothing more than a preprocessor directive that gets transformed into behaviour_info/1 function, and therefore has no place in the standard chunks.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Erlang 19 callbacks aren't attributes?

Kostis Sagonas-2
On 01/25/2017 11:22 AM, Alex S. wrote:
>> 25 янв. 2017 г., в 13:00, Richard Carlsson <[hidden email]> написал(а):
>>
>> There are some distinctions that have not been well documented. I'll try to clarify them based on how things actually work. ... <SNIP> ...
>> These distinctions have not been properly documented, but have been intuitively followed by the people working on the language and compiler. In some cases, the lines are blurred. For example, one could easily argue that '-behaviour(Name).'  was just another wild attribute which only had a meaning to the current version of the compiler, and not part of the language proper. On the other hand, since the -callback declarations were introduced, it's pretty clear that behaviours are a language feature.
>>
>> Along these lines, I think that e.g. the type declarations of a module should be included in Beam files as standard chunks, so you don't need to extract them from the "abstract format" debug information, if present, but that's another can of pull requests.
> Along those exact lines, I argue that -callback attributes are nothing more than a preprocessor directive that gets transformed into behaviour_info/1 function, and therefore has no place in the standard chunks.

You are simply wrong.  For starters, -callback attributes are NOT
related to the Erlang preprocessor (epp); the mechanism that creates the
behaviour_info/1 function is in sys_pre_expand, which is part of the
compiler.  More importantly, callbacks are manipulated by other tools
(e.g. dialyzer, also meck?), so clearly they _are_ "more than a
preprocessor directive", even if this manipulation happens via the
abstract code currently.


But I think that the main problem with your mail (and my reply above!)
is that it might unfocus the discussion about what is an attribute and
what isn't.  Richard Carlsson has written a wonderful mail which
explains why certain things are the way they are, but also brings up two
important (IMO) points that should not be lost:

  1. "These distinctions have not been properly documented..."
        I think they should be.

  2. Certain declarations need not be extracted from the "abstract
format" debug information but should be included in Beam files as
standard chunks.
        FWIW, I agree.

Kostis
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Loading...