Compile Question

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Compile Question

Michael.K.Schmidt-2

I have been chasing a compile error:

** (CompileError) elixir_compiler_1: function '__MODULE__'/1+17:

  An implementation limit was reached.

  Try reducing the complexity of this function.

 

  Instruction: {move,{x,0},{y,1555}}

 

    (stdlib) lists.erl:1338: :lists.foreach/2

    lib/absinthe/phase/schema/compile.ex:51: Absinthe.Phase.Schema.Compile.run/2

    lib/absinthe/pipeline.ex:283: Absinthe.Pipeline.run_phase/3

    lib/absinthe/schema.ex:214: Absinthe.Schema.__after_compile__/2

    (stdlib) lists.erl:1263: :lists.foldl/3

    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6

    (elixir) lib/kernel/parallel_compiler.ex:206: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6

 

I realize this is an Elixir / Absinthe issue, but my question is this:  How is lists:foreach() able to trigger this? 

https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1332

 

Poking around, there is indeed a list of 776 items that generates 2 function defs each, which gives us a final AST list of approx 1552.  I assume this is related to the {y,1555} in Error.

 

Can lists:each() not handle lists longer than 1024?  What am I missing?

 

Thanks!

Mike

 

 

 

 

 


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Compile Question

Fred Hebert-2


On Tue, Dec 4, 2018 at 1:44 PM Michael Schmidt <[hidden email]> wrote:

I have been chasing a compile error:

** (CompileError) elixir_compiler_1: function '__MODULE__'/1+17:

  An implementation limit was reached.

  Try reducing the complexity of this function.

 

  Instruction: {move,{x,0},{y,1555}}

...

Poking around, there is indeed a list of 776 items that generates 2 function defs each, which gives us a final AST list of approx 1552.  I assume this is related to the {y,1555} in Error.

 

Can lists:each() not handle lists longer than 1024?  What am I missing?

 



The limit is on the number of funs that can be exported by a single function. Each function gets an id like `#Fun<mod.9.54861464>` where the second integer (9) is a counter of the number of anonymous functions/closures within a single function. You probably overflow that counter.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Compile Question

Michał Muskała
In reply to this post by Michael.K.Schmidt-2
The issue seems to be that the compile-time function created by the module is huge and tries to allocate a compile-time list with 1555 elements. The lists:foreach call is probably misleading - that's just somewhere on the compiler stack. The good thing is that it's probably just a matter of refactoring  macros in the Absinthe.Phase.Schema.Compile module. It shouldn't need to build such huge lists (I've reviewed the module briefly).

Michał.
On 4 Dec 2018, 19:44 +0100, Michael Schmidt <[hidden email]>, wrote:

I have been chasing a compile error:

** (CompileError) elixir_compiler_1: function '__MODULE__'/1+17:

  An implementation limit was reached.

  Try reducing the complexity of this function.

 

  Instruction: {move,{x,0},{y,1555}}

 

    (stdlib) lists.erl:1338: :lists.foreach/2

    lib/absinthe/phase/schema/compile.ex:51: Absinthe.Phase.Schema.Compile.run/2

    lib/absinthe/pipeline.ex:283: Absinthe.Pipeline.run_phase/3

    lib/absinthe/schema.ex:214: Absinthe.Schema.__after_compile__/2

    (stdlib) lists.erl:1263: :lists.foldl/3

    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6

    (elixir) lib/kernel/parallel_compiler.ex:206: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6

 

I realize this is an Elixir / Absinthe issue, but my question is this:  How is lists:foreach() able to trigger this? 

https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1332

 

Poking around, there is indeed a list of 776 items that generates 2 function defs each, which gives us a final AST list of approx 1552.  I assume this is related to the {y,1555} in Error.

 

Can lists:each() not handle lists longer than 1024?  What am I missing?

 

Thanks!

Mike

 

 

 

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Compile Question

Michael.K.Schmidt-2
In reply to this post by Fred Hebert-2

Thanks!  I think you might be onto something.  We use a lot of funs in our resolve functions.  Is the limit per-function?  Or per-function-head?

 

From: Fred Hebert [mailto:[hidden email]]
Sent: Tuesday, December 4, 2018 1:06 PM
To: Michael Schmidt <[hidden email]>
Cc: Erlang <[hidden email]>
Subject: Re: [erlang-questions] Compile Question

 

[External email: Use caution with links and attachments]


 

 

On Tue, Dec 4, 2018 at 1:44 PM Michael Schmidt <[hidden email]> wrote:

I have been chasing a compile error:

** (CompileError) elixir_compiler_1: function '__MODULE__'/1+17:

  An implementation limit was reached.

  Try reducing the complexity of this function.

 

  Instruction: {move,{x,0},{y,1555}}

...

Poking around, there is indeed a list of 776 items that generates 2 function defs each, which gives us a final AST list of approx 1552.  I assume this is related to the {y,1555} in Error.

 

Can lists:each() not handle lists longer than 1024?  What am I missing?

 

 

 

The limit is on the number of funs that can be exported by a single function. Each function gets an id like `#Fun<mod.9.54861464>` where the second integer (9) is a counter of the number of anonymous functions/closures within a single function. You probably overflow that counter.


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Compile Question

Michael.K.Schmidt-2
In reply to this post by Michał Muskała

Thanks! That makes sense that the issue is not the lists:foreach(), but a layer deeper

 

I am open on how to address this—I need to test Fred’s hypothesis real quick.

 

In the mean time, this is as deep as I got:

def build_types(types) do

    IO.puts "build_types #{types |> length() |> inspect}"

    for type <- types do

      if !type.definition,

        do:

          raise("""

          No definition set!

          #{inspect(type)}

          """)

 

      ast = Macro.escape(type, unquote: true)

 

      quote do

        def __absinthe_type__(unquote(type.identifier)) do

          unquote(ast)

        end

 

        def __absinthe_type__(unquote(type.name)) do

          unquote(ast)

        end

      end

    end

    |> Enum.concat([

      quote do

        def __absinthe_type__(_type) do

          nil

       end

      end

    ])

  end

 

 

The types list is 776 long.  If I comment out the 2 “def __absinthe_type()” clauses the issue does not occur, so I assume this is the list referenced in the error message.

 

 

From: Michał Muskała [mailto:[hidden email]]
Sent: Tuesday, December 4, 2018 1:14 PM
To: [hidden email]; Michael Schmidt <[hidden email]>
Subject: Re: [erlang-questions] Compile Question

 

[External email: Use caution with links and attachments]


 

The issue seems to be that the compile-time function created by the module is huge and tries to allocate a compile-time list with 1555 elements. The lists:foreach call is probably misleading - that's just somewhere on the compiler stack. The good thing is that it's probably just a matter of refactoring  macros in the Absinthe.Phase.Schema.Compile module. It shouldn't need to build such huge lists (I've reviewed the module briefly).


Michał.

On 4 Dec 2018, 19:44 +0100, Michael Schmidt <[hidden email]>, wrote:

I have been chasing a compile error:

** (CompileError) elixir_compiler_1: function '__MODULE__'/1+17:

  An implementation limit was reached.

  Try reducing the complexity of this function.

 

  Instruction: {move,{x,0},{y,1555}}

 

    (stdlib) lists.erl:1338: :lists.foreach/2

    lib/absinthe/phase/schema/compile.ex:51: Absinthe.Phase.Schema.Compile.run/2

    lib/absinthe/pipeline.ex:283: Absinthe.Pipeline.run_phase/3

    lib/absinthe/schema.ex:214: Absinthe.Schema.__after_compile__/2

    (stdlib) lists.erl:1263: :lists.foldl/3

    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6

    (elixir) lib/kernel/parallel_compiler.ex:206: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6

 

I realize this is an Elixir / Absinthe issue, but my question is this:  How is lists:foreach() able to trigger this? 

https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1332

 

Poking around, there is indeed a list of 776 items that generates 2 function defs each, which gives us a final AST list of approx 1552.  I assume this is related to the {y,1555} in Error.

 

Can lists:each() not handle lists longer than 1024?  What am I missing?

 

Thanks!

Mike

 

 

 

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Compile Question

Michael.K.Schmidt-2

Just to close the loop, this was the fix:

https://github.com/absinthe-graphql/absinthe/pull/645

 

Thanks again Michał and Fred for the help

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Michael Schmidt
Sent: Tuesday, December 4, 2018 1:24 PM
To: Michał Muskała <[hidden email]>; [hidden email]
Subject: Re: [erlang-questions] Compile Question

 

[External email: Use caution with links and attachments]


 

Thanks! That makes sense that the issue is not the lists:foreach(), but a layer deeper

 

I am open on how to address this—I need to test Fred’s hypothesis real quick.

 

In the mean time, this is as deep as I got:

def build_types(types) do

    IO.puts "build_types #{types |> length() |> inspect}"

    for type <- types do

      if !type.definition,

        do:

          raise("""

          No definition set!

          #{inspect(type)}

          """)

 

      ast = Macro.escape(type, unquote: true)

 

      quote do

        def __absinthe_type__(unquote(type.identifier)) do

          unquote(ast)

        end

 

        def __absinthe_type__(unquote(type.name)) do

          unquote(ast)

        end

      end

    end

    |> Enum.concat([

      quote do

        def __absinthe_type__(_type) do

          nil

       end

      end

    ])

  end

 

 

The types list is 776 long.  If I comment out the 2 “def __absinthe_type()” clauses the issue does not occur, so I assume this is the list referenced in the error message.

 

 

From: Michał Muskała [[hidden email]]
Sent: Tuesday, December 4, 2018 1:14 PM
To: [hidden email]; Michael Schmidt <[hidden email]>
Subject: Re: [erlang-questions] Compile Question

 

[External email: Use caution with links and attachments]


 

The issue seems to be that the compile-time function created by the module is huge and tries to allocate a compile-time list with 1555 elements. The lists:foreach call is probably misleading - that's just somewhere on the compiler stack. The good thing is that it's probably just a matter of refactoring  macros in the Absinthe.Phase.Schema.Compile module. It shouldn't need to build such huge lists (I've reviewed the module briefly).


Michał.

On 4 Dec 2018, 19:44 +0100, Michael Schmidt <[hidden email]>, wrote:

I have been chasing a compile error:

** (CompileError) elixir_compiler_1: function '__MODULE__'/1+17:

  An implementation limit was reached.

  Try reducing the complexity of this function.

 

  Instruction: {move,{x,0},{y,1555}}

 

    (stdlib) lists.erl:1338: :lists.foreach/2

    lib/absinthe/phase/schema/compile.ex:51: Absinthe.Phase.Schema.Compile.run/2

    lib/absinthe/pipeline.ex:283: Absinthe.Pipeline.run_phase/3

    lib/absinthe/schema.ex:214: Absinthe.Schema.__after_compile__/2

    (stdlib) lists.erl:1263: :lists.foldl/3

    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6

    (elixir) lib/kernel/parallel_compiler.ex:206: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6

 

I realize this is an Elixir / Absinthe issue, but my question is this:  How is lists:foreach() able to trigger this? 

https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1332

 

Poking around, there is indeed a list of 776 items that generates 2 function defs each, which gives us a final AST list of approx 1552.  I assume this is related to the {y,1555} in Error.

 

Can lists:each() not handle lists longer than 1024?  What am I missing?

 

Thanks!

Mike

 

 

 

 

 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions