Mnesia crashes and restarts the Erlang VM repeatedly

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Mnesia crashes and restarts the Erlang VM repeatedly

ARUN P
Hi,

    I am facing some serious issues with mnesia database. We have an erlang application using mnesia database as back-end. The application runs on the hardware and the IP address of the hardware will be changing based on its connectivity and so the erlang node name. Some times as soon as the node changes am getting an error log as given below and the VM restarts repeatedly. But if I delete the mnesia related files and restart again, application runs smoothly. For me data reliability and persistence are the prime concern so I can not go with the above mentioned method. Can somebody kindly assist me how to resolve this issue with out deleting mnesia schema.

Current working directory:/root/rel-3/galaxy/lib/galaxy-1.0.0/priv
 Release number as string:3
PHY: mdio@24520:01 - Link is Up - 100/Full
36524 01999.282  2001391.0     71.4  532699753591315.4  11871.3         0
Exec: /usr/lib/erlang/erts-8.0/bin/erlexec -boot /root/rel-3/galaxy/releases/0.0.1/galaxy -mode embedded -boot_var ERTS_LIB_DIR /usr/lib/erlang/erts-8.0/../lib -config /root/rel-3/galaxy/releases/0.0.1/sys.config -args_file /root/rel-3/galaxy/vm.args -- console
Root: /root/rel-3/galaxy
/root/rel-3/galaxy
Erlang/OTP 19 [erts-8.0] [source] [async-threads:10] [kernel-poll:false]


=INFO REPORT==== 17-Nov-2016::12:42:59 ===
disk_log: repairing "/root/mnesia_database/DECISION_TAB.LOG" ...
Mnesia('[hidden email]'): Data may be missing, Corrupt logfile deleted: "/root/mnesia_database/LATEST.LOG", {not_a_log_file,
                                                                                                                  "/root/mnesia_database/LATEST.LOG"}

=ERROR REPORT==== 17-Nov-2016::12:43:00 ===
Mnesia('[hidden email]'): ** ERROR ** (core dumped to file: [hidden email])
 ** FATAL ** Failed to merge schema: Bad cookie in table definition configuration_table: '[hidden email]' = {cstruct,configuration_table,set,[],['[hidden email]'],[],[],0,read_write,false,[],[],false,gen_record,[key,value,time_stamp],[],[],[],{{1479494951549325042,-134217623,1},'[hidden email]'},{{2,0},[]}}, '[hidden email]' = {cstruct,configuration_table,set,[],['[hidden email]'],[],[],0,read_write,false,[],[],false,gen_record,[key,value,time_stamp],[],[],[],{{1479575314787900797,-134217651,1},'[hidden email]'},{{2,0},[
 ]}}


=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_monitor terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,<0.411.0>,[],[],true,[],undefined,[],[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_recover terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,<0.411.0>,undefined,undefined,undefined,0,
                               false,true,[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_snmp_sup terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,
                            {local,mnesia_snmp_sup},
                            simple_one_for_one,
                            [{child,undefined,mnesia_snmp_sup,
                                 {mnesia_snmp_hook,start,[]},
                                 transient,3000,worker,
                                 [mnesia_snmp_sup,mnesia_snmp_hook,
                                  supervisor]}],
                            undefined,0,86400000,[],0,mnesia_snmp_sup,[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_subscr terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,<0.411.0>,12307}
** Reason for termination ==
** {{aborted,{node_not_running,'[hidden email]'}},
    [{mnesia,abort,1,[{file,"mnesia.erl"},{line,318}]},
     {mnesia_lib,del,2,[{file,"mnesia_lib.erl"},{line,485}]},
     {mnesia_subscr,do_handle_exit,1,[{file,"mnesia_subscr.erl"},{line,508}]},
     {mnesia_subscr,handle_exit,2,[{file,"mnesia_subscr.erl"},{line,502}]},
     {mnesia_subscr,do_prepare_stop,2,[{file,"mnesia_subscr.erl"},{line,526}]},
     {mnesia_subscr,terminate,2,[{file,"mnesia_subscr.erl"},{line,304}]},
     {gen_server,try_terminate,3,[{file,"gen_server.erl"},{line,629}]},
     {gen_server,terminate,7,[{file,"gen_server.erl"},{line,795}]}]}

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
Mnesia('[hidden email]'): ** ERROR ** mnesia_event got unexpected event: {'EXIT',
                                                                               <0.413.0>,
                                                                               {aborted,
                                                                                {node_not_running,
                                                                                 '[hidden email]'}}}

=INFO REPORT==== 17-Nov-2016::12:43:10 ===
    application: mnesia
    exited: {{shutdown,{failed_to_start_child,mnesia_kernel_sup,killed}},
             {mnesia_sup,start,[normal,[]]}}
    type: permanent
{"Kernel pid terminated",application_controller,"{application_start_failure,mnesia,{{shutdown,{failed_to_start_child,mnesia_kernel_sup,killed}},{mnesia_sup,start,[normal,[]]}}}"}

Crash dump is being written to: erl_crash.dump...
 
Thanks in advance,
Arun

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Mnesia crashes and restarts the Erlang VM repeatedly

Chandru-4
Hi Arun,

You can start your erlang node with an explicit name such as scu_plus@localhost or scu_plus@hostname which will prevent it from crashing whenever the IP address changes. If you are using something like relx to build your release, then you should be able to specify the node name in the $ROOTDIR/releases/<VSN>/vm.args file.

cheers,
Chandru


On 14 December 2016 at 11:18, Arun <[hidden email]> wrote:
Hi,

    I am facing some serious issues with mnesia database. We have an erlang application using mnesia database as back-end. The application runs on the hardware and the IP address of the hardware will be changing based on its connectivity and so the erlang node name. Some times as soon as the node changes am getting an error log as given below and the VM restarts repeatedly. But if I delete the mnesia related files and restart again, application runs smoothly. For me data reliability and persistence are the prime concern so I can not go with the above mentioned method. Can somebody kindly assist me how to resolve this issue with out deleting mnesia schema.

Current working directory:/root/rel-3/galaxy/lib/galaxy-1.0.0/priv
 Release number as string:3
PHY: mdio@24520:01 - Link is Up - 100/Full
36524 01999.282  2001391.0     71.4  532699753591315.4  11871.3         0
Exec: /usr/lib/erlang/erts-8.0/bin/erlexec -boot /root/rel-3/galaxy/releases/0.0.1/galaxy -mode embedded -boot_var ERTS_LIB_DIR /usr/lib/erlang/erts-8.0/../lib -config /root/rel-3/galaxy/releases/0.0.1/sys.config -args_file /root/rel-3/galaxy/vm.args -- console
Root: /root/rel-3/galaxy
/root/rel-3/galaxy
Erlang/OTP 19 [erts-8.0] [source] [async-threads:10] [kernel-poll:false]


=INFO REPORT==== 17-Nov-2016::12:42:59 ===
disk_log: repairing "/root/mnesia_database/DECISION_TAB.LOG" ...
Mnesia('[hidden email]'): Data may be missing, Corrupt logfile deleted: "/root/mnesia_database/LATEST.LOG", {not_a_log_file,
                                                                                                                  "/root/mnesia_database/LATEST.LOG"}

=ERROR REPORT==== 17-Nov-2016::12:43:00 ===
Mnesia('[hidden email]'): ** ERROR ** (core dumped to file: [hidden email])
 ** FATAL ** Failed to merge schema: Bad cookie in table definition configuration_table: '[hidden email]' = {cstruct,configuration_table,set,[],['[hidden email]'],[],[],0,read_write,false,[],[],false,gen_record,[key,value,time_stamp],[],[],[],{{1479494951549325042,-134217623,1},'[hidden email]'},{{2,0},[]}}, '[hidden email]' = {cstruct,configuration_table,set,[],['[hidden email]'],[],[],0,read_write,false,[],[],false,gen_record,[key,value,time_stamp],[],[],[],{{1479575314787900797,-134217651,1},'[hidden email]'},{{2,0},[
 ]}}


=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_monitor terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,<0.411.0>,[],[],true,[],undefined,[],[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_recover terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,<0.411.0>,undefined,undefined,undefined,0,
                               false,true,[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_snmp_sup terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,
                            {local,mnesia_snmp_sup},
                            simple_one_for_one,
                            [{child,undefined,mnesia_snmp_sup,
                                 {mnesia_snmp_hook,start,[]},
                                 transient,3000,worker,
                                 [mnesia_snmp_sup,mnesia_snmp_hook,
                                  supervisor]}],
                            undefined,0,86400000,[],0,mnesia_snmp_sup,[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
** Generic server mnesia_subscr terminating
** Last message in was {'EXIT',<0.411.0>,killed}
** When Server state == {state,<0.411.0>,12307}
** Reason for termination ==
** {{aborted,{node_not_running,'[hidden email]'}},
    [{mnesia,abort,1,[{file,"mnesia.erl"},{line,318}]},
     {mnesia_lib,del,2,[{file,"mnesia_lib.erl"},{line,485}]},
     {mnesia_subscr,do_handle_exit,1,[{file,"mnesia_subscr.erl"},{line,508}]},
     {mnesia_subscr,handle_exit,2,[{file,"mnesia_subscr.erl"},{line,502}]},
     {mnesia_subscr,do_prepare_stop,2,[{file,"mnesia_subscr.erl"},{line,526}]},
     {mnesia_subscr,terminate,2,[{file,"mnesia_subscr.erl"},{line,304}]},
     {gen_server,try_terminate,3,[{file,"gen_server.erl"},{line,629}]},
     {gen_server,terminate,7,[{file,"gen_server.erl"},{line,795}]}]}

=ERROR REPORT==== 17-Nov-2016::12:43:10 ===
Mnesia('[hidden email]'): ** ERROR ** mnesia_event got unexpected event: {'EXIT',
                                                                               <0.413.0>,
                                                                               {aborted,
                                                                                {node_not_running,
                                                                                 '[hidden email]'}}}

=INFO REPORT==== 17-Nov-2016::12:43:10 ===
    application: mnesia
    exited: {{shutdown,{failed_to_start_child,mnesia_kernel_sup,killed}},
             {mnesia_sup,start,[normal,[]]}}
    type: permanent
{"Kernel pid terminated",application_controller,"{application_start_failure,mnesia,{{shutdown,{failed_to_start_child,mnesia_kernel_sup,killed}},{mnesia_sup,start,[normal,[]]}}}"}

Crash dump is being written to: erl_crash.dump...
 
Thanks in advance,
Arun

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions