How to reset RabbitMQ node configuration

Reset RabbitMQ node configuration.

Reset node configuration

Stop the RabbitMQ application.

$ sudo rabbitmqctl stop_app
Stopping rabbit application on node rabbit@buster ...

Reset RabbitMQ node configuration.

$ sudo rabbitmqctl reset
Resetting node rabbit@buster ...

Start the RabbitMQ application.

$ sudo rabbitmqctl start_app
Starting node rabbit@buster ...

RabbitMQ application will start using the default configuration.

$ sudo rabbitmqctl -n rabbit cluster_status
Cluster status of node rabbit@buster ...
[{nodes,[{disc,[rabbit@buster]}]},
{running_nodes,[rabbit@buster]},
{cluster_name,<<"rabbit@buster.localdomain">>},
{partitions,[]},
{alarms,[{rabbit@buster,[]}]}]

Forcefully reset node configuration

A simple reset operation is not always possible, as you cannot remove the last disc node from the cluster.

$ sudo rabbitmqctl -n rabbit1 cluster_status
Cluster status of node rabbit1@buster ...
[{nodes,[{disc,[rabbit1@buster]},{ram,[rabbit3@buster,rabbit2@buster]}]},
{running_nodes,[rabbit2@buster,rabbit3@buster,rabbit1@buster]},
{cluster_name,<<"rabbit1@buster.localdomain">>},
{partitions,[]},
{alarms,[{rabbit2@buster,[]},{rabbit3@buster,[]},{rabbit1@buster,[]}]}]

Stop the RabbitMQ application.

$ sudo rabbitmqctl -n rabbit1 stop_app
Stopping rabbit application on node rabbit1@buster ...

Try to reset the RabbitMQ node configuration.

$ sudo rabbitmqctl -n rabbit1 reset
Resetting node rabbit1@buster ...
Error:
{:resetting_only_disc_node, 'You cannot reset a node when it is the only disc node in a cluster. Please convert another node of the cluster to a disc node first.'}

Also, you cannot perform reset operation when there are no online nodes present.

Resetting node rabbit@buster...
Error:
{:no_running_cluster_nodes, 'You cannot leave a cluster if no online nodes are present.'}

Forcefully reset RabbitMQ node configuration.

$ sudo rabbitmqctl -n rabbit1 force_reset
Forcefully resetting node rabbit1@buster ...

Try to rejoin RabbitMQ to the cluster.

$ sudo rabbitmqctl -n rabbit1 join_cluster --disc rabbit2
Clustering node rabbit1@buster with rabbit2
Error:
{:inconsistent_cluster, 'Node rabbit2@buster thinks it\'s clustered with node rabbit1@buster, but rabbit1@buster disagrees'}

$ sudo rabbitmqctl -n rabbit2 cluster_status
Cluster status of node rabbit2@buster ...
[{nodes,[{disc,[rabbit1@buster]},{ram,[rabbit3@buster,rabbit2@buster]}]},
{running_nodes,[rabbit3@buster,rabbit2@buster]},
{cluster_name,<<"rabbit1@buster.localdomain">>},
{partitions,[]},
{alarms,[{rabbit3@buster,[]},{rabbit2@buster,[]}]}]

Remove it from the cluster.

$ sudo rabbitmqctl -n rabbit2 forget_cluster_node rabbit1@buster
Removing node rabbit1@buster from the cluster

Rejoin RabbitMQ to the cluster.

$ sudo rabbitmqctl -n rabbit1 join_cluster --disc rabbit2
Clustering node rabbit1@buster with rabbit2

Start the RabbitMQ application.

$ sudo rabbitmqctl -n rabbit1 start_app
Starting node rabbit1@buster ...
completed with 3 plugins.

It will restore configuration as long as there is at least one other node running.

$ sudo rabbitmqctl -n rabbit1 cluster_status
Cluster status of node rabbit1@buster ...
[{nodes,[{disc,[rabbit1@buster]},{ram,[rabbit3@buster,rabbit2@buster]}]},
{running_nodes,[rabbit2@buster,rabbit3@buster,rabbit1@buster]},
{cluster_name,<<"rabbit1@buster.localdomain">>},
{partitions,[]},
{alarms,[{rabbit2@buster,[]},{rabbit3@buster,[]},{rabbit1@buster,[]}]}]

When to use force reset?

rabbitmqctl manual page states that …

reset Returns a RabbitMQ node to its virgin state.

      Removes the node from any cluster it belongs to, removes all data from the management database, such as configured users and vhosts, and deletes all persistent
      messages.

      For reset and force_reset to succeed the RabbitMQ application must have been stopped, e.g. with stop_app.

      For example, to resets the RabbitMQ node:

            rabbitmqctl reset

force_reset
      Forcefully returns a RabbitMQ node to its virgin state.

      The force_reset command differs from reset in that it resets the node unconditionally, regardless of the current management database state and cluster configura‐
      tion.  It should only be used as a last resort if the database or cluster configuration has been corrupted.

      For reset and force_reset to succeed the RabbitMQ application must have been stopped, e.g. with stop_app.

      For example, to reset the RabbitMQ node:

            rabbitmqctl force_reset

… but the real difference is visible in src/rabbit_mnesia.erl source code.

[...]
%% return node to its virgin state, where it is not member of any
%% cluster, has no cluster configuration, no local database, and no
%% persisted messages

-spec reset() -> 'ok'.

reset() ->
    ensure_mnesia_not_running(),
    rabbit_log:info("Resetting Rabbit~n", []),
    reset_gracefully().

-spec force_reset() -> 'ok'.

force_reset() ->
    ensure_mnesia_not_running(),
    rabbit_log:info("Resetting Rabbit forcefully~n", []),
    wipe().

reset_gracefully() ->
    AllNodes = cluster_nodes(all),
    %% Reconnecting so that we will get an up to date nodes.  We don't
    %% need to check for consistency because we are resetting.
    %% Force=true here so that reset still works when clustered with a
    %% node which is down.
    init_db_with_mnesia(AllNodes, node_type(), false, false, _Retry = false),
    case is_only_clustered_disc_node() of
        true  -> e(resetting_only_disc_node);
        false -> ok
    end,
    leave_cluster(),
    rabbit_misc:ensure_ok(mnesia:delete_schema([node()]), cannot_delete_schema),
    wipe().

wipe() ->
    %% We need to make sure that we don't end up in a distributed
    %% Erlang system with nodes while not being in an Mnesia cluster
    %% with them. We don't handle that well.
    [erlang:disconnect_node(N) || N <- cluster_nodes(all)],
    %% remove persisted messages and any other garbage we find
    ok = rabbit_file:recursive_delete(filelib:wildcard(dir() ++ "/*")),
    ok = rabbit_node_monitor:reset_cluster_status(),
    ok.
[...]

This is so cool!