Monday, February 27, 2012

RabbitMQ startup and "Too short cookie string"

While setting up a RabbitMQ cluster through the Opscode Chef cookbookI ended up in a situation where rabbitmq-server wouldn't start even without a config file.  The error logs showed:

> /var/log/rabbitmq/startup_err
Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}})

> /var/log/rabbitmq/startup_log
{error_logger,{{2012,2,27},{18,8,41}},"Too short cookie string",[]}
{error_logger,{{2012,2,27},{18,8,41}},crash_report,[[{initial_call,{auth,init,['Argument__1']}},{pid,<0.19.0>},{registered_name,[]},{error_info,{exit,{"Too short cookie string",[{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]},[{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},{ancestors,[net_sup,kernel_sup,<0.9.0>]},{messages,[]},{links,[<0.17.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,987},{stack_size,24},{reductions,822}],[]]}
{error_logger,{{2012,2,27},{18,8,41}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{"Too short cookie string",[{auth,init_cookie,0},{auth,init,1},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},{offender,[{pid,undefined},{name,auth},{mfargs,{auth,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2012,2,27},{18,8,41}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2012,2,27},{18,8,41}},std_info,[{application,kernel},{exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"} 

The solution was simple. Checking in the data folder I saw the .erlang.cookie was of size zero:
# ls -al /var/lib/rabbitmq
total 272
drwxr-xr-x  3 rabbitmq rabbitmq   4096 Feb 27 18:05 .
drwxr-xr-x 28 root     root       4096 Feb 27 18:04 ..
-r--------  1 rabbitmq rabbitmq      0 Feb 23 16:45 .erlang.cookie
-rw-r-----  1 rabbitmq rabbitmq 213540 Feb 27 18:08 erl_crash.dump
drwxr-xr-x  4 rabbitmq rabbitmq   4096 Feb 23 16:44 mnesia

Simply remove the cookie file and start rabbitmq-server again and it succeeds.
# rm /var/lib/rabbitmq/.erlang.cookie