nodetool drain¶

Drains the node by flushing all memtables, stopping acceptance of writes, and preparing for shutdown.

Synopsis¶

nodetool [connection_options] drain

Description¶

nodetool drain gracefully prepares a node for shutdown by:

Flushing all memtables to SSTables
Stopping native transport (CQL connections)
Stopping gossip
Stopping accepting writes
Completing in-flight requests

This ensures no data loss when stopping Cassandra.

When to Use¶

Before Stopping Cassandra¶

Always Drain Before Shutdown

nodetool drain
sudo systemctl stop cassandra

Without drain, Cassandra must replay commit logs on restart, which takes longer.

Before Upgrades¶

nodetool drain
sudo systemctl stop cassandra
# Perform upgrade
sudo systemctl start cassandra

Before Maintenance¶

nodetool drain
# Node is now safe to work on

When NOT to Use¶

For Quick Restart¶

If Cassandra needs to restart quickly and data persistence is not critical:

# Faster but commit log replay on startup
sudo systemctl stop cassandra

However, this is rarely recommended.

When Node Will Be Removed¶

If decommissioning the node:

# Use decommission instead
nodetool decommission

Drain Process¶

Step	Action	Description
1	Stop accepting new connections	No new client connections accepted
2	Complete in-flight requests	Allow current requests to finish
3	Flush all memtables to disk	Ensures all data in memory is written to SSTables
4	Stop gossip	Node stops communicating with cluster
5	Stop native transport	CQL connections closed
6	Enter DRAINED state	Node is safe to stop

After Drain

Node is now safe to stop. The Cassandra process is still running but not accepting work.

After Drain¶

Node State¶

nodetool info
# Shows: Mode: DRAINED

Behavior¶

Feature	State
CQL connections	Refused
Gossip	Stopped
Writes	Rejected
Reads	May still work briefly
JMX	Still available

Stopping Cassandra¶

# After drain completes
sudo systemctl stop cassandra

# Or
pkill -f CassandraDaemon

Examples¶

Standard Shutdown¶

nodetool drain
sudo systemctl stop cassandra

Shutdown Script¶

#!/bin/bash
echo "Draining node..."
nodetool drain

echo "Waiting for drain to complete..."
while nodetool info 2>/dev/null | grep -q "Mode: NORMAL"; do
    sleep 1
done

echo "Stopping Cassandra..."
sudo systemctl stop cassandra
echo "Done"

Remote Drain¶

ssh 192.168.1.101 "nodetool drain"
ssh 192.168.1.101 'sudo systemctl stop cassandra'

Monitoring Drain¶

Check Progress¶

# Watch memtable flush
nodetool tpstats | grep -i flush

# Check mode
nodetool info | grep Mode

Drain Complete¶

When drain completes:

Mode: DRAINED

Logs¶

tail -f /var/log/cassandra/system.log | grep -i drain

Common Issues¶

Drain Times Out¶

If drain takes too long:

Check for large memtables
Check disk I/O performance
Check for pending tasks

nodetool tpstats
nodetool compactionstats

"Cannot drain - already drained"¶

Node was already drained:

# Just stop the process
sudo systemctl stop cassandra

Connections Still Active After Drain¶

Some connections may linger briefly. Drain stops new connections but allows in-flight requests to complete.

Stuck in Draining State¶

If drain doesn't complete:

# Check what's blocking
nodetool tpstats
nodetool info

# Last resort: force stop
sudo systemctl stop cassandra

Force Stop Risk

Force stopping without drain completing requires commit log replay on restart.

Drain vs. Other Commands¶

Command	Purpose	Continues Running
`drain`	Prepare for shutdown	Yes (drained state)
`stopdaemon`	Stop Cassandra immediately	No
`disablebinary`	Stop CQL only	Yes (operational)
`disablegossip`	Stop gossip only	Yes (isolated)

When to Use Each¶

Scenario	Command
Graceful shutdown	`drain` then stop
Emergency stop	`stopdaemon` (last resort)
Maintenance (keep running)	`disablebinary`
Network troubleshooting	`disablegossip`

Recovery from Drained State¶

To return a drained node to service:

# Must restart Cassandra
sudo systemctl restart cassandra

There is no "undrain" command. The node must be restarted.

Best Practices¶

Drain Guidelines

Always drain before shutdown - Prevents data loss
Wait for completion - Check mode is DRAINED
Include in scripts - Automate shutdown procedures
Monitor during drain - Catch any issues
Plan for duration - Large memtables take time to flush

Shutdown Checklist¶

[ ] Check node status: nodetool status
[ ] Drain the node: nodetool drain
[ ] Verify drained: nodetool info | grep Mode
[ ] Stop service: sudo systemctl stop cassandra
[ ] Verify stopped: pgrep -f CassandraDaemon

Command	Relationship
flush	Flushes memtables only
disablebinary	Disables CQL only
disablegossip	Disables gossip only
info	Check drain state