Skip to content

AxonOps Cassandra agent installation

This agent will enable metrics, logs and events collection with adaptive repairs and backups for Cassandra.

Available versions

  • Apache Cassandra 3.11
  • Apache Cassandra 3.0

Step 1 - Installation

Make sure that the version of your Cassandra and Cassandra agent are compatible from the compatibility matrix.

CentOS / RedHat installer

example with Apache Cassandra 3.11.x:

curl -1sLf 'https://repo.axonops.com/public/yum/cfg/setup/bash.rpm.sh' | sudo bash
sudo yum install axon-cassandra3.11-agent

Debian / Ubuntu installer

example with Apache Cassandra 3.11.x:

curl -1sLf 'https://repo.axonops.com/public/apt/cfg/setup/bash.deb.sh' | sudo bash
sudo apt-get install axon-cassandra3.11-agent

Note: This will install AxonOps Cassandra agent and it's dependency: axon-agent

Step 2 - Agent Configuration

Update the following highlighted lines from /etc/axonops/axon-agent.yml:

axon-server:
    hosts: "axon-server_endpoint" # Specify axon-server IP axon-server.mycompany.
    port: 1888

axon-agent:
    org: "my-company-test" # Specify your organisation name
    human_readable_identifier: "axon_agent_ip" # one of the following: 
    # # "axon_agent_ip"
    # # "env_<env_variable_name>"
    # # "awsec2_<variable_name>"
    # # "gcs_<variable_name>"

    # # environment  examples:
    # #   "env_hostname" 
    # # Amazon Web Services EC2 examples:
    # #   "awsec2_id"
    # #   "awsec2_name"
    # #   "awsec2_hostname"
    # # Google Cloud examples:
    # #   "gcs_id"
    # #   "gcs_name"
    # #   "gcs_hostname"

    # tls:
    #   mode: "disabled" # disabled, TLS
    #   skipVerify: false
    #   caFile: "***.crt"
    #   certFile: "***.crt"
    #   keyFile: "***.key"

NTP:
    host: "set_NTP_server" # Specify a NTP to determine a NTP offset 

cassandra:
  tier0: # metrics collected every 5 seconds
      metrics:
          jvm_:
            - "java.lang:*"
          cas_:
            - "org.apache.cassandra.metrics:*"
            - "org.apache.cassandra.net:type=FailureDetector"

  tier1:
      frequency: 300 # metrics collected every 300 seconds (5m)
      metrics:
          cas_:
            - "org.apache.cassandra.metrics:name=EstimatedPartitionCount,*"

  #tier2:
  #    frequency: 3600 # 1h

  #tier3:
  #    frequency: 86400 # 1d

  blacklist: # You can blacklist metrics based on Regex pattern. Hit the agent on http://agentIP:9916/metricslist to list JMX metrics it is collecting
    - "org.apache.cassandra.metrics:type=ColumnFamily.*" # duplication of table metrics
    - "org.apache.cassandra.metrics:.*scope=Repair#.*" # ignore each repair instance metrics
    - "org.apache.cassandra.metrics:.*name=SnapshotsSize.*" # Collecting SnapshotsSize metrics slows down collection
    - "org.apache.cassandra.metrics:.*Max.*"
    - "org.apache.cassandra.metrics:.*Min.*"
    - ".*999thPercentile|.*50thPercentile|.*FifteenMinuteRate|.*FiveMinuteRate|.*MeanRate|.*Mean|.*OneMinuteRate|.*StdDev"

  JMXOperationsBlacklist:
    - "getThreadInfo"
    - "getDatacenter"
    - "getRack"

  DMLEventsWhitelist: # You can whitelist keyspaces / tables (list of "keyspace" and/or "keyspace.table" to log DML queries. Data is not analysed.
  # - "system_distributed"

  DMLEventsBlacklist: # You can blacklist keyspaces / tables from the DMLEventsWhitelist (list of "keyspace" and/or "keyspace.table" to log DML queries. Data is not analysed.
  # - system_distributed.parent_repair_history

  logSuccessfulRepairs: false # set it to true if you want to log all the successful repair events.

  warningThresholdMillis: 200 # This will warn in logs when a MBean takes longer than the specified value.

  logFormat: "%4$s %1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS,%1$tL %5$s%6$s%n"

human_readable_identifier details

  • If you want to identify your nodes by their IP, you must set human_readable_identifier to "axon_agent_ip". axon-agent will resolve the IP and pass it to the server as its humanly readable identifier

  • If you want to use the hostname of the system, specify human_readable_identifier to "env_hostname"

  • If the node is a EC2 instance and you want to use AWS EC2 instance-id metadata, set it to "awsec2_id"

  • If the node is a Google Cloud instance and you want its Google Cloud ID, set it to "gcs_id"

Note: the log format will only be used by AxonOps Cassandra agent

Step 3 - Configure Cassandra

Edit cassandra-env.sh usually located in your Cassandra install path such as /<Cassandra Installation Directory>/conf/cassandra-env.sh and append the following line at the end of the file:

JVM_OPTS="$JVM_OPTS -javaagent:/usr/share/axonops/axon-cassandra{version}-agent.jar=/etc/axonops/axon-agent.yml"

example with Cassandra agent version 3.11:

JVM_OPTS="$JVM_OPTS -javaagent:/usr/share/axonops/axon-cassandra3.11-agent.jar=/etc/axonops/axon-agent.yml"

Make sure that this configuration will not get overridden by an automation tool.

Step 4 - Add axonops user to Cassandra user group and Cassandra user to axonops group

sudo usermod -aG <your_cassandra_group> axonops
sudo usermod -aG axonops <your_cassandra_user>

Step 5 - Start Cassandra

Step 6 - Start axon-agent

sudo service axon-agent start

(Optional) Step 7 - Cassandra Remote Backups or Restore Prerequisites

For security reasons, the default configuration of the AxonOps agent user (axonops) on the Cassandra server does not have read/write access to the Cassandra data directories. When using AxonOps to remotly backup data, the axonops user will need read access to the Cassandra data directories and when restoring a backup, write access is required.

The exact way this access can be managed is dependent on the security requirements of the cluster being managed. If there are no specific requirements to restrict the AxonOps agent users data folder access then adding the user to the Cassandra DB group would typically grant the user read/write access to the Cassandra data directories e.g usermod -a -G cassandra axonops

Cassandra agent Package details

  • Configuration: /etc/axonops/axon-agent.yml
  • Binary: /usr/share/axonops/axon-cassandra{version}-agent.jar
  • Version number: /usr/share/axonops/axon-cassandra{version}-agent.version
  • Copyright : /usr/share/doc/axonops/axon-cassandra{version}-agent/copyright
  • Licenses : /usr/share/axonops/licenses/axon-cassandra{version}-agent/

axon-agent Package details (dependency of Cassandra agent)

  • Configuration: /etc/axonops/axon-agent.yml
  • Binary: usr/share/axonops/axon-agent
  • Logs : /var/log/axonops/axon-agent.log
  • Systemd service: /usr/lib/systemd/system/axon-agent.service