Skip to content

Kafka Migration

Migration guides for Apache Kafka clusters.

Migration Types

Migration Complexity Downtime
Version upgrade Low-Medium Zero (rolling)
ZooKeeper to KRaft Medium Zero (rolling)
Cluster migration High Depends on approach
Cloud migration High Depends on approach

ZooKeeper to KRaft Migration

Overview

uml diagram

Migration Steps

1. Prepare cluster

# Verify all brokers are healthy
kafka-broker-api-versions.sh --bootstrap-server kafka:9092

# Check for under-replicated partitions
kafka-topics.sh --bootstrap-server kafka:9092 \
  --describe --under-replicated-partitions

2. Deploy KRaft controllers

# controller.properties
process.roles=controller
node.id=100
controller.quorum.voters=100@ctrl1:9093,101@ctrl2:9093,102@ctrl3:9093
listeners=CONTROLLER://0.0.0.0:9093
controller.listener.names=CONTROLLER

3. Enable migration mode on controllers

# Add to controller.properties
zookeeper.connect=zk1:2181,zk2:2181,zk3:2181
zookeeper.metadata.migration.enable=true

4. Start migration

# Format controller storage
kafka-storage.sh format -t $(kafka-storage.sh random-uuid) \
  -c controller.properties

# Start controllers
kafka-server-start.sh controller.properties

5. Roll brokers to enable migration

# Add to server.properties
controller.quorum.voters=100@ctrl1:9093,101@ctrl2:9093,102@ctrl3:9093
controller.listener.names=CONTROLLER

6. Complete migration

# Verify migration status
kafka-metadata.sh --snapshot /var/kafka/__cluster_metadata-0/*.log \
  --command "migration-state"

# Disable ZooKeeper on controllers
# Remove: zookeeper.metadata.migration.enable=true
# Restart controllers

7. Remove ZooKeeper dependency from brokers

# Remove from server.properties
# zookeeper.connect=...

# Rolling restart brokers

Version Upgrades

Rolling Upgrade Process

uml diagram

Pre-Upgrade Checklist

  • [ ] Review release notes for breaking changes
  • [ ] Verify client compatibility
  • [ ] Backup configurations
  • [ ] Test upgrade in non-production
  • [ ] Plan rollback procedure

Upgrade Commands

# Set protocol version before upgrade
# server.properties
inter.broker.protocol.version=3.6
log.message.format.version=3.6

# Upgrade each broker
for broker in broker1 broker2 broker3; do
  # Stop
  ssh $broker "sudo systemctl stop kafka"

  # Install new version
  ssh $broker "sudo yum install kafka-3.7"

  # Start
  ssh $broker "sudo systemctl start kafka"

  # Wait for recovery
  sleep 60
  kafka-topics.sh --bootstrap-server kafka:9092 \
    --describe --under-replicated-partitions
done

# Update protocol version
# server.properties
inter.broker.protocol.version=3.7
log.message.format.version=3.7

# Rolling restart

Cluster Migration

Using MirrorMaker 2

uml diagram

Migration Steps

1. Set up target cluster

# Create topics on target with same configuration
kafka-topics.sh --bootstrap-server target:9092 \
  --create --topic my-topic \
  --partitions 12 --replication-factor 3

2. Deploy MirrorMaker 2

# mm2.properties
clusters=source,target
source.bootstrap.servers=source-kafka:9092
target.bootstrap.servers=target-kafka:9092

source->target.enabled=true
source->target.topics=.*

sync.group.offsets.enabled=true
emit.checkpoints.enabled=true

3. Verify replication

# Check lag
kafka-consumer-groups.sh --bootstrap-server target:9092 \
  --describe --group mm2-source-target

# Verify data integrity
kafka-console-consumer.sh --bootstrap-server target:9092 \
  --topic my-topic --from-beginning --max-messages 10

4. Migrate consumers

# Translate offsets
# Consumers can read checkpoints topic to find equivalent offset

5. Migrate producers

# Update bootstrap.servers to target cluster
# Deploy updated configuration

6. Decommission source

# Stop MirrorMaker
# Verify no traffic to source
# Decommission source cluster

Rollback Procedures

Version Rollback

# Stop broker
sudo systemctl stop kafka

# Install previous version
sudo yum downgrade kafka-3.6

# Ensure protocol version matches
# server.properties
inter.broker.protocol.version=3.6

# Start broker
sudo systemctl start kafka

Migration Rollback

# For ZK to KRaft migration:
# 1. Re-enable ZooKeeper on brokers
# 2. Rolling restart brokers
# 3. Shut down KRaft controllers
# 4. Verify cluster operates with ZooKeeper