Feedback wanted: Remove compilation times from BEAM files in OTP 19?

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Björn Gustavsson-4
As long as anyone can remember, the compilation time has been included
in BEAM files (and can be retrieved using module_info()).

As far as I know, the inclusion of the time has only caused problems.
I can't remember that anyone has found any use for the compilation
time.

So if we decide to remove the compilation time by default in OTP 19...

... would it cause any problem for someone in some workflow?

... would we need an option to include the compilation time?

/Bjorn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Max Lapshin-2
You want to achieve stable builds?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Björn Gustavsson-4
On Fri, Apr 8, 2016 at 1:05 PM, Max Lapshin <[hidden email]> wrote:
> You want to achieve stable builds?

Yes.

See:

http://erlang.org/pipermail/erlang-questions/2016-April/088717.html

That is just the last time I heard of problems with
the compilation times.

/Bjorn


--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Loïc Hoguin-3
In reply to this post by Björn Gustavsson-4
The build time is occasionally useful but the same can be achieved by
looking at the beam file creation or modification time.

Is it just the build time though?

Wouldn't these infos also be a problem?

* Auto generated vsn (it looks like repeated builds keep the same
version number if the source doesn't change, but is that guaranteed?)

      [{vsn,[192941001008994618371492498907572068968]},

* Compiler version (if two different compiler versions produce the same
result except for the compiler version, this is a waste; if compiler
version of a build is desirable it should probably be kept separate)

      {version,"6.0.1"},

* Compile options with paths

      {outdir,"/build/erlang/src/otp/lib/ssl/src/../ebin"},
      {i,"/build/erlang/src/otp/lib/kernel/src"},

* Source file name

      {source,"/build/erlang/src/otp/lib/ssl/src/ssl.erl"}]},

On 04/08/2016 12:32 PM, Björn Gustavsson wrote:

> As long as anyone can remember, the compilation time has been included
> in BEAM files (and can be retrieved using module_info()).
>
> As far as I know, the inclusion of the time has only caused problems.
> I can't remember that anyone has found any use for the compilation
> time.
>
> So if we decide to remove the compilation time by default in OTP 19...
>
> ... would it cause any problem for someone in some workflow?
>
> ... would we need an option to include the compilation time?
>
> /Bjorn
>

--
Loïc Hoguin
http://ninenines.eu
Author of The Erlanger Playbook,
A book about software development using Erlang
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Matthias Rieber
In reply to this post by Björn Gustavsson-4
On Fri, 8 Apr 2016, Björn Gustavsson wrote:

> As long as anyone can remember, the compilation time has been included
> in BEAM files (and can be retrieved using module_info()).
>
> As far as I know, the inclusion of the time has only caused problems.
> I can't remember that anyone has found any use for the compilation
> time.
>
> So if we decide to remove the compilation time by default in OTP 19...
>
> ... would it cause any problem for someone in some workflow?

I couldn't resist:

https://xkcd.com/1172/

Matthias

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Eric Pailleau

Hi,
Beam files are separated in different chunks.
All dynamic information should be placed in a separate chunck Mho.
And sha1sum of code part, or checksum of its canonical abstract code maybe available in a chunk, existing or new one.
This could allow to know that a beam, even without the same checksum on the file globally,  run the same code than another Beam file, and so be able to do some Nix like distribution.

Compiler version allow to estimate what Erlang version compiled the beam. This is something to keep I think.

Regards

"Envoyé depuis mon mobile " Eric



---- Matthias Rieber a écrit ----

On Fri, 8 Apr 2016, Björn Gustavsson wrote:

> As long as anyone can remember, the compilation time has been included
> in BEAM files (and can be retrieved using module_info()).
>
> As far as I know, the inclusion of the time has only caused problems.
> I can't remember that anyone has found any use for the compilation
> time.
>
> So if we decide to remove the compilation time by default in OTP 19...
>
> ... would it cause any problem for someone in some workflow?

I couldn't resist:

https://xkcd.com/1172/

Matthias

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Serge Aleynikov-3
In reply to this post by Björn Gustavsson-4
I've been relying on module's compilation time for figuring out the changed modules (see below).

In the absence of {time, CompileTime}, would there be a way to determine the modification time stamp of the file at the time it was loaded by code loader?

Serge

modified_modules() ->
    [M || {M, _} <-  code:all_loaded(), module_modified(M) == true].

module_modified(Module) ->
    case code:is_loaded(Module) of
    {file, preloaded} ->
        false;
    {file, Path} ->
        CompileOpts = proplists:get_value(compile, Module:module_info()),
        CompileTime = proplists:get_value(time, CompileOpts),
        Src = proplists:get_value(source, CompileOpts),
        module_modified(Path, CompileTime, Src);
    _ ->
        false
    end.

module_modified(Path, PrevCompileTime, PrevSrc) ->
    case find_module_file(Path) of
    false ->
        false;
    ModPath ->
        {ok, {_, [{_, CB}]}} = beam_lib:chunks(ModPath, ["CInf"]),
        CompileOpts =  binary_to_term(CB),
        CompileTime = proplists:get_value(time, CompileOpts),
        Src = proplists:get_value(source, CompileOpts),
        not (CompileTime == PrevCompileTime) and (Src == PrevSrc)
    end.


On Fri, Apr 8, 2016 at 6:32 AM, Björn Gustavsson <[hidden email]> wrote:
As long as anyone can remember, the compilation time has been included
in BEAM files (and can be retrieved using module_info()).

As far as I know, the inclusion of the time has only caused problems.
I can't remember that anyone has found any use for the compilation
time.

So if we decide to remove the compilation time by default in OTP 19...

... would it cause any problem for someone in some workflow?

... would we need an option to include the compilation time?

/Bjorn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Loïc Hoguin-3
In reply to this post by Eric Pailleau
If you don't have hash(file1) = hash(file2) that means the usual tooling
doesn't work as well as it could: packages will do file changes that
don't change the program, rsync will do similar with syncs, and so on.

It's also a bit dubious to require Erlang to verify that the Erlang
files (including the compiler itself) were compiled as expected.

On 04/08/2016 02:11 PM, Éric Pailleau wrote:

> Hi,
> Beam files are separated in different chunks.
> All dynamic information should be placed in a separate chunck Mho.
> And sha1sum of code part, or checksum of its canonical abstract code
> maybe available in a chunk, existing or new one.
> This could allow to know that a beam, even without the same checksum on
> the file globally,  run the same code than another Beam file, and so be
> able to do some Nix like distribution.
>
> Compiler version allow to estimate what Erlang version compiled the
> beam. This is something to keep I think.
>
> Regards
>
> "Envoyé depuis mon mobile " Eric
>
>
>
> ---- Matthias Rieber a écrit ----
>
> On Fri, 8 Apr 2016, Björn Gustavsson wrote:
>
>  > As long as anyone can remember, the compilation time has been included
>  > in BEAM files (and can be retrieved using module_info()).
>  >
>  > As far as I know, the inclusion of the time has only caused problems.
>  > I can't remember that anyone has found any use for the compilation
>  > time.
>  >
>  > So if we decide to remove the compilation time by default in OTP 19...
>  >
>  > ... would it cause any problem for someone in some workflow?
>
> I couldn't resist:
>
> https://xkcd.com/1172/
>
> Matthias
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email] <mailto:[hidden email]>
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

--
Loïc Hoguin
http://ninenines.eu
Author of The Erlanger Playbook,
A book about software development using Erlang
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Martin Bjorklund
In reply to this post by Serge Aleynikov-3
Serge Aleynikov <[hidden email]> wrote:
> I've been relying on module's compilation time for figuring out the changed
> modules (see below).

I also use this function all the time
(I.e, in user_default: lm() -> [c:l(M) || M <- modified_modules()].
 super convenient!)

Anyway, if there's an alternative way to do the same thing that would
be fine.

> In the absence of {time, CompileTime}, would there be a way to determine
> the modification time stamp of the file at the time it was loaded by code
> loader?


/martin


>
> Serge
>
> modified_modules() ->
>     [M || {M, _} <-  code:all_loaded(), module_modified(M) == true].
>
> module_modified(Module) ->
>     case code:is_loaded(Module) of
>     {file, preloaded} ->
>         false;
>     {file, Path} ->
>         CompileOpts = proplists:get_value(compile, Module:module_info()),
>         CompileTime = proplists:get_value(time, CompileOpts),
>         Src = proplists:get_value(source, CompileOpts),
>         module_modified(Path, CompileTime, Src);
>     _ ->
>         false
>     end.
>
> module_modified(Path, PrevCompileTime, PrevSrc) ->
>     case find_module_file(Path) of
>     false ->
>         false;
>     ModPath ->
>         {ok, {_, [{_, CB}]}} = beam_lib:chunks(ModPath, ["CInf"]),
>         CompileOpts =  binary_to_term(CB),
>         CompileTime = proplists:get_value(time, CompileOpts),
>         Src = proplists:get_value(source, CompileOpts),
>         not (CompileTime == PrevCompileTime) and (Src == PrevSrc)
>     end.
>
>
> On Fri, Apr 8, 2016 at 6:32 AM, Björn Gustavsson <[hidden email]> wrote:
>
> > As long as anyone can remember, the compilation time has been included
> > in BEAM files (and can be retrieved using module_info()).
> >
> > As far as I know, the inclusion of the time has only caused problems.
> > I can't remember that anyone has found any use for the compilation
> > time.
> >
> > So if we decide to remove the compilation time by default in OTP 19...
> >
> > ... would it cause any problem for someone in some workflow?
> >
> > ... would we need an option to include the compilation time?
> >
> > /Bjorn
> >
> > --
> > Björn Gustavsson, Erlang/OTP, Ericsson AB
> > _______________________________________________
> > erlang-questions mailing list
> > [hidden email]
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Björn Gustavsson-4
In reply to this post by Loïc Hoguin-3
On Fri, Apr 8, 2016 at 1:16 PM, Loïc Hoguin <[hidden email]> wrote:
> The build time is occasionally useful but the same can be achieved by
> looking at the beam file creation or modification time.
>
> Is it just the build time though?
>
> Wouldn't these infos also be a problem?

Yes, some of them are potential problems.

> * Auto generated vsn (it looks like repeated builds keep the same version
> number if the source doesn't change, but is that guaranteed?)
>

Yes, it is guaranteed as long as the compiler
generates the code in the same way. That is,
new optimizations could change it (but in that
case, there would be other changes in the BEAM
file as well).

>      [{vsn,[192941001008994618371492498907572068968]},
>
> * Compiler version (if two different compiler versions produce the same
> result except for the compiler version, this is a waste; if compiler version
> of a build is desirable it should probably be kept separate)
>
>      {version,"6.0.1"},
>
> * Compile options with paths
>
>      {outdir,"/build/erlang/src/otp/lib/ssl/src/../ebin"},
>      {i,"/build/erlang/src/otp/lib/kernel/src"},
>
> * Source file name
>
>      {source,"/build/erlang/src/otp/lib/ssl/src/ssl.erl"}]},
>

/Bjorn
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Eric Pailleau
In reply to this post by Loïc Hoguin-3

I agree Loïc,  but native code is in another chunk named with a mnemo of architecture.
So a same code, compiled native or not will create Beam files with different checksum anyway.
It is a hard problem to solve, with also the constraints to try to keep backward compatibility... Actually I m not sure it is possible to achieve this, or code part have to be in another independent file. This break compatibility...

"Envoyé depuis mon mobile " Eric



---- Loïc Hoguin a écrit ----

If you don't have hash(file1) = hash(file2) that means the usual tooling
doesn't work as well as it could: packages will do file changes that
don't change the program, rsync will do similar with syncs, and so on.

It's also a bit dubious to require Erlang to verify that the Erlang
files (including the compiler itself) were compiled as expected.

On 04/08/2016 02:11 PM, Éric Pailleau wrote:

> Hi,
> Beam files are separated in different chunks.
> All dynamic information should be placed in a separate chunck Mho.
> And sha1sum of code part, or checksum of its canonical abstract code
> maybe available in a chunk, existing or new one.
> This could allow to know that a beam, even without the same checksum on
> the file globally,  run the same code than another Beam file, and so be
> able to do some Nix like distribution.
>
> Compiler version allow to estimate what Erlang version compiled the
> beam. This is something to keep I think.
>
> Regards
>
> "Envoyé depuis mon mobile " Eric
>
>
>
> ---- Matthias Rieber a écrit ----
>
> On Fri, 8 Apr 2016, Björn Gustavsson wrote:
>
>  > As long as anyone can remember, the compilation time has been included
>  > in BEAM files (and can be retrieved using module_info()).
>  >
>  > As far as I know, the inclusion of the time has only caused problems.
>  > I can't remember that anyone has found any use for the compilation
>  > time.
>  >
>  > So if we decide to remove the compilation time by default in OTP 19...
>  >
>  > ... would it cause any problem for someone in some workflow?
>
> I couldn't resist:
>
> https://xkcd.com/1172/
>
> Matthias
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email] <mailto:[hidden email]>
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

--
Loïc Hoguin
http://ninenines.eu
Author of The Erlanger Playbook,
A book about software development using Erlang

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Björn Gustavsson-4
In reply to this post by Serge Aleynikov-3
On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov <[hidden email]> wrote:
> I've been relying on module's compilation time for figuring out the changed
> modules (see below).
>
> In the absence of {time, CompileTime}, would there be a way to determine the
> modification time stamp of the file at the time it was loaded by code
> loader?

Yes. Use Mod:module_info(md5) to calculate MD5 for
the loaded module and compare it to beam_lib:md5(Mod).

Example:

13> c:module_info(md5).
<<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>>
14> beam_lib:md5(code:which(c)).
{ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138,
         190,214,118>>}}

/Bjorn
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Serge Aleynikov-3
​Thank you!


On Fri, Apr 8, 2016 at 8:54 AM, Björn Gustavsson <[hidden email]> wrote:
On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov <[hidden email]> wrote:
> I've been relying on module's compilation time for figuring out the changed
> modules (see below).
>
> In the absence of {time, CompileTime}, would there be a way to determine the
> modification time stamp of the file at the time it was loaded by code
> loader?

Yes. Use Mod:module_info(md5) to calculate MD5 for
the loaded module and compare it to beam_lib:md5(Mod).

Example:

13> c:module_info(md5).
<<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>>
14> beam_lib:md5(code:which(c)).
{ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138,
         190,214,118>>}}

/Bjorn


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Gleb Vinogradov
In reply to this post by Björn Gustavsson-4
May be we just need compiler option for creating stable builds? --stable-build , --pure-build or similar
So no build time, and no paths will be included in *.beam file. It will not break workflow for anyone, and may be later, in OTP 21 non-stable builds will be deprecated.

Gleb.

2016-04-08 16:32 GMT+06:00 Björn Gustavsson <[hidden email]>:
As long as anyone can remember, the compilation time has been included
in BEAM files (and can be retrieved using module_info()).

As far as I know, the inclusion of the time has only caused problems.
I can't remember that anyone has found any use for the compilation
time.

So if we decide to remove the compilation time by default in OTP 19...

... would it cause any problem for someone in some workflow?

... would we need an option to include the compilation time?

/Bjorn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Douglas Rohrer
I know it's not exactly the "principle of least surprise" but close enough - I'd agree that not changing the default, but providing a simple way to accomplish the goal, is probably the best way forward.

Doug

On Apr 8, 2016, at 9:45 AM, Gleb Vinogradov <[hidden email]> wrote:

May be we just need compiler option for creating stable builds? --stable-build , --pure-build or similar
So no build time, and no paths will be included in *.beam file. It will not break workflow for anyone, and may be later, in OTP 21 non-stable builds will be deprecated.

Gleb.

2016-04-08 16:32 GMT+06:00 Björn Gustavsson <[hidden email]>:
As long as anyone can remember, the compilation time has been included
in BEAM files (and can be retrieved using module_info()).

As far as I know, the inclusion of the time has only caused problems.
I can't remember that anyone has found any use for the compilation
time.

So if we decide to remove the compilation time by default in OTP 19...

... would it cause any problem for someone in some workflow?

... would we need an option to include the compilation time?

/Bjorn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Richard Carlsson-3
In reply to this post by Serge Aleynikov-3
Hi Serge (and everyone else)! I've actually got an improved version of that module change detection (i.e., based on md5) that I was going to clean up and submit to OTP as a standard part of the shell. I think the original snippets have been drifting around on the interwebs for aeons, and I'm not sure who wrote them originally. We've been using them at Klarna forever. But we needed something more reliable (that works even if you have made a completely new build of every module, but not actually changing more than a few), so I started by pushing some patches to OTP that made the md5 of loaded modules easily available in module_info. Then based on that I wrote an improved shell function like you just did - we've now tried it out for a while and found it stable, so it's a good time to push this as well.

I think my only hesitation is where to put the core "module changed" functionality if I'm to submit it as part of OTP: the code module? beam_lib? somewhere else? Any opinions out there?

        /Richard

2016-04-08 15:37 GMT+02:00 Serge Aleynikov <[hidden email]>:
​Thank you!


On Fri, Apr 8, 2016 at 8:54 AM, Björn Gustavsson <[hidden email]> wrote:
On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov <[hidden email]> wrote:
> I've been relying on module's compilation time for figuring out the changed
> modules (see below).
>
> In the absence of {time, CompileTime}, would there be a way to determine the
> modification time stamp of the file at the time it was loaded by code
> loader?

Yes. Use Mod:module_info(md5) to calculate MD5 for
the loaded module and compare it to beam_lib:md5(Mod).

Example:

13> c:module_info(md5).
<<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>>
14> beam_lib:md5(code:which(c)).
{ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138,
         190,214,118>>}}

/Bjorn


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Serge Aleynikov-3
Hi Richard,

It seems to me that code module would be the proper place for it, since that detection is directly related to code loading, which in turn could call the beam_lib's helper functions.

Serge

On Fri, Apr 8, 2016 at 4:42 PM, Richard Carlsson <[hidden email]> wrote:
Hi Serge (and everyone else)! I've actually got an improved version of that module change detection (i.e., based on md5) that I was going to clean up and submit to OTP as a standard part of the shell. I think the original snippets have been drifting around on the interwebs for aeons, and I'm not sure who wrote them originally. We've been using them at Klarna forever. But we needed something more reliable (that works even if you have made a completely new build of every module, but not actually changing more than a few), so I started by pushing some patches to OTP that made the md5 of loaded modules easily available in module_info. Then based on that I wrote an improved shell function like you just did - we've now tried it out for a while and found it stable, so it's a good time to push this as well.

I think my only hesitation is where to put the core "module changed" functionality if I'm to submit it as part of OTP: the code module? beam_lib? somewhere else? Any opinions out there?

        /Richard

2016-04-08 15:37 GMT+02:00 Serge Aleynikov <[hidden email]>:
​Thank you!


On Fri, Apr 8, 2016 at 8:54 AM, Björn Gustavsson <[hidden email]> wrote:
On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov <[hidden email]> wrote:
> I've been relying on module's compilation time for figuring out the changed
> modules (see below).
>
> In the absence of {time, CompileTime}, would there be a way to determine the
> modification time stamp of the file at the time it was loaded by code
> loader?

Yes. Use Mod:module_info(md5) to calculate MD5 for
the loaded module and compare it to beam_lib:md5(Mod).

Example:

13> c:module_info(md5).
<<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>>
14> beam_lib:md5(code:which(c)).
{ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138,
         190,214,118>>}}

/Bjorn


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Björn Gustavsson-4
In reply to this post by Richard Carlsson-3
On Fri, Apr 8, 2016 at 10:42 PM, Richard Carlsson
<[hidden email]> wrote:
[...]
>
> I think my only hesitation is where to put the core "module changed"
> functionality if I'm to submit it as part of OTP: the code module? beam_lib?
> somewhere else? Any opinions out there?
>

I think the code module is the best place.

/Bjorn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Joe Armstrong-2
I said this in another thread but I'll repeat it here:

I think we need repeatable builds so that if we compile the same
module several timeswith the same macros and library versions we
should get
a bit identical beam file (so we can hash the beam content and use the hash
as the key in a version control system)

What I don't know is how we should interpret the statement "the same
module" - if I correct a typo in a comment is the result "the same
module"?
I would argue that it is is not. I think a hash (md5, sha1 etc) of the source
and all the macros etc should be added to the beam file. That way
we can do a post hoc analysis of the beam code and tie together exactly
which version of the source was used.

/Joe

On Sat, Apr 9, 2016 at 8:48 AM, Björn Gustavsson <[hidden email]> wrote:

> On Fri, Apr 8, 2016 at 10:42 PM, Richard Carlsson
> <[hidden email]> wrote:
> [...]
>>
>> I think my only hesitation is where to put the core "module changed"
>> functionality if I'm to submit it as part of OTP: the code module? beam_lib?
>> somewhere else? Any opinions out there?
>>
>
> I think the code module is the best place.
>
> /Bjorn
>
> --
> Björn Gustavsson, Erlang/OTP, Ericsson AB
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Feedback wanted: Remove compilation times from BEAM files in OTP 19?

Jesper Louis Andersen-2

On Sat, Apr 9, 2016 at 12:45 PM, Joe Armstrong <[hidden email]> wrote:
I think we need repeatable builds so that if we compile the same
module several timeswith the same macros and library versions we
should get
a bit identical beam file (so we can hash the beam content and use the hash
as the key in a version control system)

Other valuable things you can do:

* Catch compilers which have been built incorrectly or are executing on faulty hardware
* Verify that two different compiler installations are the same. Sometimes this can lead to situations where you are hunting a bug only to learn later different compilers are used
* Catch maliciously altered compilers

The same model is what drives BitTorrent. A BitTorrent file is essentially a map:

#{ info => ..., trackers => [{tracker, ...}, ...] }

The "infohash" is the SHA1 checksum of the "info" block, but certain parts are left out of that block, notably the tracker list. It allows operators to alter the tracker list and add their own tracker, without changing the content of the torrent file. In other words, integrity is provided where it matter, but freedom is given to alter the parts around the integrity-protected parts if necessary. 


--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
12