Very long hang during OTP app startup

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Very long hang during OTP app startup

Mark Geib-2

Using Erlang R18 and rebar3. This is an application, or release, built with rebar3 and using generated management script to start and stop the application. I see very strange, and random behavior at startup in that when I start the application with "../<app name> console" I will see erlang “PROGRESS REPORT” messages as included applications start, like amps_client. It appears my OTP supervisor and children start normally. In fact one of the children opens a socket and handshakes with an external system. Once the handshake completes I expect to start receiving data from the external system, but nothing else happens until I see the following in the terminal where I started the application:

=PROGRESS REPORT==== 4-May-2018::13:21:40 ===
          supervisor: {local,kernel_safe_sup}
             started: [{pid,<0.563.0>},
                       {id,timer_server},
                       {mfargs,{timer,start_link,[]}},
                       {restart_type,permanent},
                       {shutdown,1000},
                       {child_type,worker}]

At this point the app receives messages on the tcp socket and everything appears completely normal. However, sometimes it is hours before I see this last “PROGRESS REPORT” and normal operation starts, other times maybe a minute or less. I have this feeling that a supervisor or child in another tree, not mine, is hung in an init() callback or something. I just am at a loss on how to debug this problem. I have never seen this type of behavior before.

My release includes my application of course, but also lager, amqp_client and rabbit_common, and xmerl.

Any suggestions would be greatly appreciated.

Thanks,

Mark Geib
Principal Engineer
Cheyenne Software Engineering
[hidden email] / 335-215

"A long habit of not thinking a thing wrong gives it a superficial appearance of being right.”
-Thomas Paine

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions