AxonOps Coordinator Dashboard Metrics Mapping¶

This document maps the metrics used in the AxonOps Coordinator dashboard.

Dashboard Overview¶

The Coordinator dashboard monitors coordinator-level request handling in Cassandra. When a client sends a request, the coordinator node handles the request and coordinates with replica nodes. This dashboard tracks latency and throughput broken down by consistency level, providing insights into how different consistency levels impact performance.

Metrics Mapping¶

Client Request Metrics¶

Dashboard Metric	Description	Attributes
`cas_ClientRequest_Latency`	Request latency at coordinator level	`scope` (Read/Write/RangeSlice), `function` (percentiles/count), `dc`, `rack`, `host_id`
`cas_Client_connectedNativeClients`	Number of connected native protocol clients	`dc`, `rack`, `host_id`
`cas_CommitLog_WaitingOnSegmentAllocation`	Time waiting for commit log segment allocation	`function` (percentile), `dc`, `rack`, `host_id`

Scope Patterns¶

Read Operations¶

Read - Simple read (no consistency level)
Read-ALL - Read with ALL consistency
Read-ONE - Read with ONE consistency
Read-TWO - Read with TWO consistency
Read-THREE - Read with THREE consistency
Read-QUORUM - Read with QUORUM consistency
Read-LOCAL_QUORUM - Read with LOCAL_QUORUM consistency
Read-EACH_QUORUM - Read with EACH_QUORUM consistency
Read-SERIAL - Read with SERIAL consistency
Read-LOCAL_SERIAL - Read with LOCAL_SERIAL consistency
Read-LOCAL_ONE - Read with LOCAL_ONE consistency

Write Operations¶

Write - Simple write (no consistency level)
Write-ALL - Write with ALL consistency
Write-ANY - Write with ANY consistency
Write-ONE - Write with ONE consistency
Write-TWO - Write with TWO consistency
Write-THREE - Write with THREE consistency
Write-QUORUM - Write with QUORUM consistency
Write-LOCAL_QUORUM - Write with LOCAL_QUORUM consistency
Write-EACH_QUORUM - Write with EACH_QUORUM consistency
Write-LOCAL_ONE - Write with LOCAL_ONE consistency

Range Operations¶

RangeSlice - Range query operations (SELECT with ranges)

Query Examples¶

Coordinator Read Distribution (Pie Chart)¶

sum(cas_ClientRequest_Latency{axonfunction='rate',scope='Read*',scope!='Read',function='Count',function!='Min|Max',dc=~'$dc',rack=~'$rack',host_id=~'$host_id'}) by (scope)

Coordinator Write Distribution (Pie Chart)¶

sum(cas_ClientRequest_Latency{axonfunction='rate',scope='Write*',scope!='Write',function='Count',function!='Min|Max',dc=~'$dc',rack=~'$rack',host_id=~'$host_id'}) by (scope)

Read Latency by Consistency Level¶

cas_ClientRequest_Latency{scope='Read.*$consistency',function='$percentile',function!='Min|Max',dc=~'$dc',rack=~'$rack',host_id=~'$host_id'}

Range Read Latency¶

cas_ClientRequest_Latency{scope='RangeSlice',function='$percentile',function!='Min|Max',dc=~'$dc',rack=~'$rack',host_id=~'$host_id'}

Write Latency by Consistency Level¶

cas_ClientRequest_Latency{scope='Write.*$consistency',function='$percentile',function!='Min|Max',dc=~'$dc',rack=~'$rack',host_id=~'$host_id'}

Read Throughput by Consistency¶

sum(cas_ClientRequest_Latency{axonfunction='rate',scope='Read.*$consistency',function='Count',function!='Min|Max',dc=~'$dc',rack=~'$rack', host_id=~'$host_id'}) by ($groupBy)

Commit Log Waiting Time¶

cas_CommitLog_WaitingOnSegmentAllocation{dc=~'$dc',rack='$rack',host_id=~'$host_id',function='$percentile'}

Panel Organization¶

Consistency Distribution Section¶

Coordinator Reads distribution - Pie chart showing read request distribution by consistency level
Coordinator Writes distribution - Pie chart showing write request distribution by consistency level

Latency Statistics By Node Section¶

Coordinator Read $consistency Latency - $percentile - Line chart for read latency at selected consistency
Coordinator Range Read Request Latency - $percentile - Line chart for range query latency
Coordinator Write $consistency Latency - $percentile - Line chart for write latency at selected consistency

Throughput Statistics Section¶

Coordinator Read Throughput Per $groupBy ($consistency) - Count Per Second - Read operations per second
Coordinator Range Read Request Throughput - Count Per Second - Range queries per second
Coordinator Write Throughput Per $groupBy ($consistency) - Count Per Second - Write operations per second

Connections Section¶

Number of Native Connections per host - Line chart showing client connections

Commitlog Statistics Section¶

Waiting on Segment Allocation - Time spent waiting for commit log segments

Filters¶

data center (dc) - Filter by data center
rack - Filter by rack
node (host_id) - Filter by specific node
groupBy - Dynamic grouping (dc, rack, host_id)
percentile - Select latency percentile (50th, 75th, 95th, 98th, 99th, 999th)
consistency - Filter by consistency level (ALL, ANY, ONE, TWO, THREE, SERIAL, QUORUM, etc.)

Consistency Levels¶

Strong Consistency¶

ALL - All replicas must respond
QUORUM - Majority of replicas must respond
LOCAL_QUORUM - Majority in local datacenter
EACH_QUORUM - Quorum in each datacenter

Weak Consistency¶

ONE - Only one replica must respond
TWO - Two replicas must respond
THREE - Three replicas must respond
ANY - Any node can accept write (including hints)
LOCAL_ONE - One replica in local datacenter

Serial Consistency¶

SERIAL - Linearizable consistency
LOCAL_SERIAL - Linearizable in local datacenter

Important Considerations¶

Latency vs Consistency Trade-off:

Higher consistency levels increase latency
Monitor percentiles to understand impact
Consider LOCAL variants for multi-DC

Throughput Patterns:

Distribution shows application consistency preferences
Imbalanced distribution may indicate issues
Monitor for consistency level changes

Coordinator Load:

Each node can be a coordinator
High coordinator load impacts performance
Balance using client-side load balancing

Range Queries:

Typically more expensive than point reads
Monitor separately from regular reads
Consider pagination for large ranges

Units and Display¶

Latency: microseconds
Throughput: reads/writes per second (rps/wps)
Connections: count (short)
Legend Format: $dc - $host_id or $groupBy for aggregated views