Monitor Kafka Consumer Lag
Kafma provides Kafka consumer lag monitoring at the group, topic, partition, and member levels. Start with the group total, then narrow the investigation to the partition where progress has slowed.
What is Kafka consumer lag?
Kafka consumer lag is the difference between a partition's end offset, its high watermark, and the consumer group's committed offset. Lag increases when new records arrive faster than the group advances its committed position.
Kafma measures lag from committed offsets to the high watermark, the same as kafka-consumer-groups.sh. After a consumer processes records, lag decreases only when the group commits a new offset.
Lag belongs to a group, topic, and partition—not to a topic alone. Two consumer groups reading the same partition can therefore report different lag.
A partition without a committed offset is shown as Not committed. In group details, total and max lag include only partitions with committed offsets. Not committed or Partial indicates that other partitions do not yet have a reliable lag value.
Read Kafka consumer lag metrics
Open a group to compare three signals:
| Metric | Use it to |
|---|---|
| Total lag | See the combined committed lag across the group |
| Max lag | Find the committed partition with the largest lag |
| Trend | See whether recent lag samples are rising or falling |
The trend uses recent lag samples. Kafma keeps the history when you close and reopen group details during the same app session; restarting Kafma clears it.
Keep lag history and monitor in the background
While group details are open, Kafma samples lag about every three seconds. Closing the details stops sampling unless monitoring is on. The history stays available when you reopen the group, and the time without samples appears as a gap in the chart. Without monitoring, a group keeps only its most recent samples, roughly the last ten minutes of sampling. Kafma keeps this history for up to 50 such groups; beyond that, it drops the ones you opened least recently.
Turn on the switch beside Total Lag to keep sampling after you close the details. Background sampling runs about every 15 seconds; opening the details switches back to the faster interval.
Monitored groups keep up to one hour of history, which Kafma doesn't drop, and show a MONITORING badge in the group list. Disconnecting the cluster turns monitoring off for its groups but keeps the history collected so far; turn the switch on again after reconnecting. History is kept in memory for this app session, not saved as a long-term record.
Deleting a consumer group through Kafma stops its monitoring and clears its lag history. Deleting a saved cluster also clears its groups' history.
Find lag by topic, partition, and member
Expand Topic Assignments to compare topic totals and individual partitions. Each partition shows its committed offset, lag, and whether a member is assigned. An Unassigned partition has no current member, but its saved offset can still fall behind as new records arrive.
Member lag is the sum of lag across that member's committed partitions. Expand a member to find which assigned partition contributes the backlog. A partition can be assigned to only one member of the group at a time.
Once Kafma has seen an unfinished transaction block a partition for more than 10 seconds, the partition shows how many offsets it holds back, including transaction markers, and the topic shows how many of its partitions are blocked. Hover over the label to see the offset where the transaction starts and how long Kafma has seen it blocking. read_committed consumers cannot read past that offset until the transaction commits or aborts.
Diagnose increasing Kafka consumer lag
Use these signals to decide what to check next:
| Signal | Check next |
|---|---|
| Lag rises across many assigned partitions | Consumer throughput, processing time, or downstream dependencies |
| Lag is concentrated in one partition | Key distribution, partition traffic, or repeated processing failures |
| Lag appears on an unassigned partition | Whether the group is rebalancing or no active member subscribes to the topic |
| Group is Empty and lag rises | Whether the consumer application is stopped or failing |
| A partition shows records blocked by a transaction | Which producer left the transaction open. It ends when the producer commits or aborts, or when transaction.timeout.ms passes. |
Kafka retention can move a partition's earliest available offset forward. Records already removed by retention cannot be recovered by the consumer or by resetting its offsets.
Inspect lagging messages in the Console
From an expanded topic in the group drawer, open the Console in Watch Group mode. For an assigned partition, hover over the partition and select Inspect to watch only that partition. Kafma shows a bounded window around the committed position of each partition without joining the group or changing its offsets.
The view separates records before the committed position from pending records at the front of the backlog. Use it to inspect the keys and values near a stalled position.