number of modules limit?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

number of modules limit?

Karlo Kuna
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Dániel Szoboszlay
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Karlo Kuna
oh wow this limit is rather small !!! 
are there any suggestions how to work around it ?


On Sun, May 7, 2017 at 9:19 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Gianfranco Alongi-2
You could purge unused modules and just keep the minimum necessary set at all times? 
Otherwise create the needed functions during runtime as funs and keep them in an ets table?


On May 7, 2017 20:33, "Karlo Kuna" <[hidden email]> wrote:
oh wow this limit is rather small !!! 
are there any suggestions how to work around it ?


On Sun, May 7, 2017 at 9:19 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Karlo Kuna
permise is that all are used 

and i would rather have them already compiled then in funs but that is one options

On Sun, May 7, 2017 at 9:49 PM, Gianfranco Alongi <[hidden email]> wrote:
You could purge unused modules and just keep the minimum necessary set at all times? 
Otherwise create the needed functions during runtime as funs and keep them in an ets table?


On May 7, 2017 20:33, "Karlo Kuna" <[hidden email]> wrote:
oh wow this limit is rather small !!! 
are there any suggestions how to work around it ?


On Sun, May 7, 2017 at 9:19 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Xavier Noria
Is the limit was per node, perhaps sharding them could be a workaround. (But I do not know if the premise holds.)

On Sun, 7 May 2017 at 21:53, Karlo Kuna <[hidden email]> wrote:
permise is that all are used 

and i would rather have them already compiled then in funs but that is one options

On Sun, May 7, 2017 at 9:49 PM, Gianfranco Alongi <[hidden email]> wrote:
You could purge unused modules and just keep the minimum necessary set at all times? 
Otherwise create the needed functions during runtime as funs and keep them in an ets table?


On May 7, 2017 20:33, "Karlo Kuna" <[hidden email]> wrote:
oh wow this limit is rather small !!! 
are there any suggestions how to work around it ?


On Sun, May 7, 2017 at 9:19 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
--
Sent from Gmail Mobile

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Karlo Kuna
so anyone knows if this limit is per node? if it is then distributing modules on different node would be great option.

On Sun, May 7, 2017 at 10:14 PM, Xavier Noria <[hidden email]> wrote:
Is the limit was per node, perhaps sharding them could be a workaround. (But I do not know if the premise holds.)

On Sun, 7 May 2017 at 21:53, Karlo Kuna <[hidden email]> wrote:
permise is that all are used 

and i would rather have them already compiled then in funs but that is one options

On Sun, May 7, 2017 at 9:49 PM, Gianfranco Alongi <[hidden email]> wrote:
You could purge unused modules and just keep the minimum necessary set at all times? 
Otherwise create the needed functions during runtime as funs and keep them in an ets table?


On May 7, 2017 20:33, "Karlo Kuna" <[hidden email]> wrote:
oh wow this limit is rather small !!! 
are there any suggestions how to work around it ?


On Sun, May 7, 2017 at 9:19 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
--
Sent from Gmail Mobile


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Dániel Szoboszlay
Yes, this is a per-node limit (just like almost all other VM limits). Distributing modules on different nodes would work.

Be aware though that by default the nodes would try to form a fully connected mesh, so if you would like to run them all on the same host (and same IP) the next thing you will run out of will be TCP ports. So start these nodes as hidden and/or try to distribute them over multiple IP-s/hosts.

Cheers,
Daniel

On Mon, 8 May 2017 at 13:09 Karlo Kuna <[hidden email]> wrote:
so anyone knows if this limit is per node? if it is then distributing modules on different node would be great option.

On Sun, May 7, 2017 at 10:14 PM, Xavier Noria <[hidden email]> wrote:
Is the limit was per node, perhaps sharding them could be a workaround. (But I do not know if the premise holds.)

On Sun, 7 May 2017 at 21:53, Karlo Kuna <[hidden email]> wrote:
permise is that all are used 

and i would rather have them already compiled then in funs but that is one options

On Sun, May 7, 2017 at 9:49 PM, Gianfranco Alongi <[hidden email]> wrote:
You could purge unused modules and just keep the minimum necessary set at all times? 
Otherwise create the needed functions during runtime as funs and keep them in an ets table?


On May 7, 2017 20:33, "Karlo Kuna" <[hidden email]> wrote:
oh wow this limit is rather small !!! 
are there any suggestions how to work around it ?


On Sun, May 7, 2017 at 9:19 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
--
Sent from Gmail Mobile

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: number of modules limit?

Karlo Kuna
thank you for useful info 
this will be interesting to deal with  

On Mon, May 8, 2017 at 1:26 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, this is a per-node limit (just like almost all other VM limits). Distributing modules on different nodes would work.

Be aware though that by default the nodes would try to form a fully connected mesh, so if you would like to run them all on the same host (and same IP) the next thing you will run out of will be TCP ports. So start these nodes as hidden and/or try to distribute them over multiple IP-s/hosts.

Cheers,
Daniel

On Mon, 8 May 2017 at 13:09 Karlo Kuna <[hidden email]> wrote:
so anyone knows if this limit is per node? if it is then distributing modules on different node would be great option.

On Sun, May 7, 2017 at 10:14 PM, Xavier Noria <[hidden email]> wrote:
Is the limit was per node, perhaps sharding them could be a workaround. (But I do not know if the premise holds.)

On Sun, 7 May 2017 at 21:53, Karlo Kuna <[hidden email]> wrote:
permise is that all are used 

and i would rather have them already compiled then in funs but that is one options

On Sun, May 7, 2017 at 9:49 PM, Gianfranco Alongi <[hidden email]> wrote:
You could purge unused modules and just keep the minimum necessary set at all times? 
Otherwise create the needed functions during runtime as funs and keep them in an ets table?


On May 7, 2017 20:33, "Karlo Kuna" <[hidden email]> wrote:
oh wow this limit is rather small !!! 
are there any suggestions how to work around it ?


On Sun, May 7, 2017 at 9:19 PM, Dániel Szoboszlay <[hidden email]> wrote:
Yes, there are limits for both the number of modules (65536) and the number of exported functions (524288 = 8 functions/module for the maximum number of modules).

See erlang:system_info(info) for details, and in particular look for the limits under the =index_table:module_code and =index_table:export_list sections.

Cheers,
Daniel

On Sun, 7 May 2017 at 19:34 Karlo Kuna <[hidden email]> wrote:
I am currently working on app that has potential to contain large number of modules > 10 000 
these are mostly implementations of behaviours that are generated in runtime. 

My question is, what is most modules you have encountered in production in running erlang application, and is there practical limit on number of number of modules in system?? 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
--
Sent from Gmail Mobile

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions