Monitoring¶

This guide explains how to expose Neo4j metrics for Prometheus and where the operator wires things for you. For a complete end-to-end setup with Grafana dashboards and alerting, see the Prometheus and Grafana Setup Guide. To send Neo4j's metrics and logs into an OpenTelemetry pipeline instead, see OpenTelemetry.

Enable metrics via `spec.monitoring`¶

The operator uses Neo4j's built-in Prometheus endpoint (see https://neo4j.com/docs/operations-manual/current/monitoring/metrics/expose/). When spec.monitoring.enabled is true, the operator:

Enables the Neo4j Prometheus endpoint (server.metrics.prometheus.enabled=true).
Binds it to 0.0.0.0:2004 (safe in Kubernetes — pod network isolation prevents external access).
Exposes port 2004 on the Neo4j container.
Adds prometheus.io/* annotations for scrape-based setups.

Independently of spec.monitoring, the operator always disables the JMX and CSV metrics reporters (server.metrics.jmx.enabled=false, server.metrics.csv.enabled=false) on every Neo4j deployment — JMX is an unauthenticated management surface and CSV writes one file per metric into the pod's ephemeral filesystem. These hardening defaults are emitted regardless of whether monitoring is enabled; re-enable via spec.config if you genuinely need them.

Cluster example¶

spec:
  monitoring:
    enabled: true

For clusters, the operator also creates:

A Service named <cluster>-metrics (port 2004)
A ServiceMonitor named <cluster>-monitoring (if Prometheus Operator CRDs are available)

Standalone example¶

spec:
  monitoring:
    enabled: true

For standalone deployments, the metrics port is added to the <standalone>-client Service and a ServiceMonitor named <standalone>-monitoring is created automatically.

Full configuration example¶

spec:
  monitoring:
    enabled: true
    slowQueryThreshold: "5s"       # Log queries slower than this (maps to db.logs.query.threshold)
    queryLogLevel: "INFO"          # OFF, INFO, or VERBOSE (maps to db.logs.query.enabled)
    obfuscateLiterals: true        # Mask literal values in query logs (recommended in production)
    explainPlan: false             # Include execution plan in logs (performance impact — avoid in production)
    metricsFilter: "*"             # Enable all Neo4j metrics (default: subset only)
    metricsPrefix: "neo4j"         # Custom prefix for metric names

Prometheus scraping¶

Prometheus Operator¶

The operator auto-creates ServiceMonitor resources for both cluster and standalone deployments when monitoring.enabled: true. These target the metrics Service on port metrics (2004) with a 30-second scrape interval. No manual ServiceMonitor creation is needed.

Standard Prometheus¶

Add a scrape config that targets the metrics Service:

scrape_configs:
  - job_name: neo4j
    metrics_path: /metrics
    static_configs:
      - targets:
          - <cluster>-metrics.<namespace>.svc.cluster.local:2004

Note: for standalone deployments, scrape <standalone>-client.<namespace>.svc.cluster.local:2004 (the metrics port is added to the same Service that serves Bolt/HTTP). <standalone>-service also resolves, but that alias is deprecated — use -client.

Operator's own metrics (TokenReview-authenticated)¶

The Neo4j operator exposes its own Prometheus metrics on port 8080 of the operator pod. The operator's metrics endpoint defaults to HTTPS with TokenReview-based authentication (metrics.secure=true in the helm chart) — anonymous scrapes are rejected with 401/403. To scrape the operator's own metrics:

Bind a ServiceAccount to the metrics-reader ClusterRole the operator ships:

kubectl create clusterrolebinding prom-metrics-reader \
    --clusterrole=metrics-reader \
    --serviceaccount=monitoring:prometheus

On Kubernetes 1.24+, manually create a long-lived token Secret for that SA:

apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  name: prometheus-metrics-token
  namespace: monitoring
  annotations:
    kubernetes.io/service-account.name: prometheus

Enable the operator's ServiceMonitor (off by default) and tell it to use that token by setting --set metrics.serviceMonitor.enabled=true --set metrics.serviceMonitor.bearerTokenSecret.name=prometheus-metrics-token at install time. With metrics.secure=true the chart refuses to render the ServiceMonitor without bearerTokenSecret.name.

The ServiceMonitor template wires scheme: https, tlsConfig.insecureSkipVerify: true (controller-runtime serves a self-signed cert by default; swap for a cert-manager bundle in production), and the bearer token reference automatically when metrics.secure=true.

Set metrics.secure=false in helm values to serve plain HTTP without authn — this drops the authentication above, but is sometimes required for legacy scrapers that can't carry bearer tokens.

Customizing metrics settings¶

Metrics filter¶

By default, Neo4j only exposes a subset of its metrics. To enable all metrics or select specific categories:

spec:
  monitoring:
    enabled: true
    metricsFilter: "*"  # Enable all metrics

Or select specific categories with glob patterns:

spec:
  monitoring:
    enabled: true
    metricsFilter: "*bolt*,*transaction*,*page_cache*,*cluster.raft*"

Query log security¶

In production environments, enable literal obfuscation to prevent sensitive data (passwords, PII) from appearing in query logs:

spec:
  monitoring:
    enabled: true
    obfuscateLiterals: true
    queryLogLevel: "INFO"           # Log only slow queries, not all queries
    slowQueryThreshold: "2s"

Version-specific metrics¶

Neo4j metric names are identical across 5.26.x and 2025.x+ CalVer releases, but some metrics are only available in newer versions:

Metric Category	5.26.x	2025.x+
Core metrics (Bolt, transactions, page cache, JVM)	Yes	Yes
CPU usage (`vm.cpu_load.*`)	No	2025.01+
Raft snapshot metrics (`cluster.raft.snapshot_*`)	No	2025.01+
Virtual threads (`vm.threads.virtual`)	No	2025.05+
Raft election/queue metrics	No	2025.02+
Store copy download metrics	No	2025.02+
Deadlock rollback counter	No	2026.01+
Page cache async IO metrics	No	2026.02+
Discovery v1 metrics (`cluster.discovery.cluster.*`)	Deprecated	Removed

Prometheus metric naming¶

Neo4j converts metric names for Prometheus: dots become underscores, counter metrics get a _total suffix. The # HELP comment preserves the original name.

Examples:

neo4j.dbms.bolt.connections_running → neo4j_dbms_bolt_connections_running
neo4j.database.<db>.transaction.committed → neo4j_database_<db>_transaction_committed_total
neo4j.page_cache.hit_ratio → neo4j_page_cache_hit_ratio

Aura Fleet Management (cloud monitoring)¶

For a hosted monitoring experience, you can register your deployment with Neo4j Aura Fleet Management. This lets you view topology, status, and metrics for all self-managed Neo4j instances alongside your Aura-managed instances in the Aura console.

The operator handles plugin installation and token registration automatically. See the Aura Fleet Management Guide for setup instructions.

Complete Metrics Reference¶

The operator registers the following Prometheus metrics. All metrics use the prefix neo4j_operator_.

Build metadata¶

Metric	Type	Labels	Description
`neo4j_operator_build_info`	Gauge	`version`, `vcs_ref`, `build_date`, `go_version`	Always `1` — the information is in the labels. Which operator build is running.

Exposed from the moment the manager starts, before any Neo4j deployment exists, so it is also the quickest way to confirm a rollout actually landed:

neo4j_operator_build_info
# neo4j_operator_build_info{version="v1.15.0", vcs_ref="191ef05",
#   build_date="2026-09-14T09:12:41Z", go_version="go1.27.0"} 1

Released images carry a real version and commit. A binary built outside the release pipeline reports what it can — go install leaves version as the module version Go recorded, a go build from a working tree reports development — and every label falls back to unknown rather than an empty string, so a missing value is visible instead of silently absent.

Two uses worth wiring up:

Alert on version skew across namespaces if you run more than one operator: count(count by (version) (neo4j_operator_build_info)) > 1.
Include it in bug reports. The version and vcs_ref pair identifies the exact build far more reliably than an image tag, which can be overwritten.

Cluster metrics¶

Metric	Type	Labels	Description
`neo4j_operator_cluster_healthy`	Gauge	`cluster_name`, `namespace`	`1` in phase `Ready` (also while a server is down, within the grace period); `0` in `Forming`, `Degraded` or `Failed` Every series of a cluster is removed when the cluster is deleted.
`neo4j_operator_cluster_replicas_total`	Gauge	`cluster_name`, `namespace`, `role` (`desired`/`ready`)	Server counts: `desired` = `spec.topology.servers`, `ready` = StatefulSet ready replicas (set when the cluster reaches `Ready`)
`neo4j_operator_cluster_phase`	Gauge	`cluster_name`, `namespace`, `phase`	`1` for the current phase, `0` for all others (phases: `Pending`, `Forming`, `Ready`, `Failed`, `Degraded`, `Upgrading`)
`neo4j_operator_split_brain_detected_total`	Counter	`cluster_name`, `namespace`	Total split-brain detection events
`neo4j_operator_server_health`	Gauge	`cluster_name`, `namespace`, `server_name`, `server_address`, `k8s_cluster`	`1` = Enabled+Available; `0` = degraded. Collected once the cluster has formed, including while a server is down; a down server keeps the last address it reported, and a server the cluster no longer lists has its series removed. `k8s_cluster` is empty unless the operator runs with `--kubernetes-cluster-name` (Helm `kubernetesClusterName`) — see Multi-cluster

Which metrics are populated. Every metric listed here is registered, but a family exports a series only once a controller has recorded it. A row marked registered, not currently populated has no recorder wired up yet and exports nothing; do not build dashboards or alerts on it.

Reconcile metrics¶

Metric	Type	Labels	Description
`neo4j_operator_reconcile_total`	Counter	`cluster_name`, `namespace`, `operation`, `result` (`success`/`failure`)	Total reconciliation attempts. Recorded by the `Neo4jEnterpriseCluster` controller only (`operation="cluster"`)
`neo4j_operator_reconcile_duration_seconds`	Histogram	`cluster_name`, `namespace`, `operation`	Reconciliation loop duration (same scope as above)

Upgrade metrics¶

Metric	Type	Labels	Description
`neo4j_operator_upgrade_total`	Counter	`cluster_name`, `namespace`, `result` (`success`/`failure`)	Total upgrade attempts
`neo4j_operator_upgrade_duration_seconds`	Histogram	`cluster_name`, `namespace`, `phase` (`Staging`/`Rolling`/`Stabilizing`/`Verifying`)	Time a rolling upgrade spent in each phase, observed once when the phase ends. `Rolling` is the whole walk across all servers, not one server. The step that leaves the chain is observed too: `Verifying` when the upgrade completes, or whichever phase was active when it failed or paused (a `Rolling` timeout appears as a `Rolling` observation of about `upgradeTimeout`). `Completed`, `Failed` and `Paused` are terminal and not timed. Measured from the persisted `status.upgradeStatus.phaseStartTime`, so an operator restart mid-phase is included in that phase's wall-clock time and no phase is observed twice. Resolution is one second; buckets run from 5s to 1h

Cross-cluster replication metrics¶

Recorded by the Neo4jReplicaDatabase and Neo4jReplicaPromotion controllers (see Cross-cluster replication). cluster_name is the downstream cluster that hosts the replica.

Metric	Type	Labels	Description
`neo4j_operator_replica_lag_transactions`	Gauge	`namespace`, `cluster_name`, `replica`, `database`, `k8s_cluster`	How many transactions (not seconds) the replica is behind its upstream, as `SHOW DATABASES` reports it. `replica` is the `Neo4jReplicaDatabase` name and `database` the Neo4j database name (`spec.name`, defaulting to the CR name). `k8s_cluster` is empty unless the operator runs with `--kubernetes-cluster-name` (Helm `kubernetesClusterName`), exactly as for `neo4j_operator_server_health`. Set while the replica is observed replicating. The series is removed whenever the operator can no longer vouch for it, so a dashboard never shows a frozen last value: when the CR is deleted or the replica is promoted, when the database has vanished from the server, and while the downstream cluster is missing or not `Ready` or the lag cannot be read (Neo4j unreachable, `SHOW DATABASES` failing, or the spec rejected). It is set again by the first reconcile that reads the lag, so an absent series means "unknown", never "caught up". This is the data loss a promotion right now would make permanent
`neo4j_operator_replica_promotions_total`	Counter	`namespace`, `cluster_name`, `result` (`success`/`failure`), `k8s_cluster`	Replica promotions by terminal outcome, one increment per `Neo4jReplicaPromotion` when it reaches `Completed` (`success`) or `Failed` (`failure`). A promotion that is still retrying counts as neither, and reconciles after the terminal phase do not count again. `k8s_cluster` is populated as for the lag gauge

Backup metrics¶

Recorded for one-shot Neo4jBackup Jobs only (when the Job reaches a terminal state); scheduled (CronJob) runs are not counted. The cluster_name label carries the Neo4jBackup name, not the Neo4j cluster name.

Metric	Type	Labels	Description
`neo4j_operator_backup_total`	Counter	`cluster_name`, `namespace`, `result` (`success`/`failure`)	Total one-shot backup attempts
`neo4j_operator_backup_duration_seconds`	Histogram	`cluster_name`, `namespace`	Backup job duration

The operator does not export a backup-size metric. Measure the .backup artifacts in your storage backend, or read status.history[*].databaseArtifacts[*].size / shardArtifacts[*].size where the run populated them (status.history[*].stats.size is reserved and never populated).

Resource conflict metrics¶

Metric	Type	Labels	Description
`neo4j_operator_resource_version_conflicts_total`	Counter	`resource_type`, `namespace`	Total Kubernetes resource version conflicts encountered
`neo4j_operator_conflict_retry_attempts`	Histogram	`resource_type`, `namespace`	Retry attempts needed to resolve each conflict
`neo4j_operator_conflict_retry_duration_seconds`	Histogram	`resource_type`, `namespace`	Time spent retrying due to resource version conflicts

Aura orchestration metrics¶

Emitted by the Aura orchestration controllers for every request the operator makes to the Aura Platform API. The operation label is a normalized route template (resource IDs collapsed to {id}, e.g. POST /instances/{id}/upgrade) so cardinality stays bounded regardless of how many Aura instances you manage.

Metric	Type	Labels	Description
`neo4j_operator_aura_api_requests_total`	Counter	`operation`, `result` (`success`/`failure`)	Total Aura Platform API requests. `result=failure` counts any non-2xx response, including the 404s that idempotent deletes treat as success.
`neo4j_operator_aura_api_request_duration_seconds`	Histogram	`operation`	Latency of Aura Platform API requests, by route.

Live Cluster Diagnostics¶

Once the cluster has formed — and while it is degraded with a server down — the operator collects live diagnostics by running SHOW SERVERS and SHOW DATABASES against the cluster. Collection is on by default — it runs when spec.monitoring is omitted or spec.monitoring.enabled: true, and is skipped only when you set spec.monitoring.enabled: false. Results are written to status.diagnostics and two Kubernetes conditions without requiring kubectl exec into pods.

Prerequisites¶

None. Setting spec.monitoring.enabled: true is only needed for the Prometheus metrics endpoint and the other monitoring resources described above.

spec:
  monitoring:
    enabled: true

The operator connects for diagnostics once the cluster has formed. No extra configuration is needed — diagnostics are collected on every reconcile (every 30 seconds on a quiet Ready deployment, --ready-poll-interval) and written to status when something in them changes, or at least every 5 minutes.

Collection carries on during a rolling upgrade, at most once per --ready-poll-interval, so status.diagnostics, the two conditions and the server_health metric follow the servers as they restart: a server that is down when a sample is taken shows Unavailable, and ServersHealthy turns False. A server is usually back within 30 seconds of its restart, so a sample can miss it. The example alert below waits 5 minutes, far longer than a server takes to restart, so a normal upgrade does not fire it; a server that does not come back does.

Viewing Diagnostic Status¶

# View the full diagnostics sub-object
kubectl get neo4jenterprisecluster <name> -o json | jq '.status.diagnostics'

# Quick server state overview
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{range .status.diagnostics.servers[*]}{.name}{"\t"}{.state}{"\t"}{.health}{"\n"}{end}'

# Quick database status overview
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{range .status.diagnostics.databases[*]}{.name}{"\t"}{.status}{"\n"}{end}'

# When diagnostics were last collected
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{.status.diagnostics.lastCollected}'

# Any collection error
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{.status.diagnostics.collectionError}'

Diagnostics Status Fields¶

Field	Type	Description
`status.diagnostics.servers[]`	Array	One entry per server from `SHOW SERVERS`
`status.diagnostics.servers[].name`	string	Server display name
`status.diagnostics.servers[].address`	string	Bolt address
`status.diagnostics.servers[].state`	string	Lifecycle state: `Enabled`, `Cordoned`, `Deallocating`
`status.diagnostics.servers[].health`	string	Health status: `Available` or `Unavailable`
`status.diagnostics.servers[].hostingDatabases`	int	Number of databases hosted
`status.diagnostics.databases[]`	Array	One entry per database from `SHOW DATABASES`
`status.diagnostics.databases[].name`	string	Database name
`status.diagnostics.databases[].status`	string	Current status: `online`, `offline`, `quarantined`
`status.diagnostics.databases[].requestedStatus`	string	Desired status
`status.diagnostics.databases[].role`	string	Role on last-contacted server: `primary`, `secondary`
`status.diagnostics.databases[].default`	bool	True for the default database
`status.diagnostics.users[]`	Array	Bounded summary of users from `SHOW USERS` (user, roles, suspended, homeDatabase)
`status.diagnostics.userCount`	int	Total users observed, even when the `users` list is truncated
`status.diagnostics.roles[]`	Array	Bounded summary of roles from `SHOW ROLES` (role, immutable)
`status.diagnostics.roleCount`	int	Total roles observed, even when the `roles` list is truncated
`status.diagnostics.lastCollected`	RFC3339	Timestamp of last successful collection
`status.diagnostics.collectionError`	string	Error from last collection attempt; empty on success

Diagnostic Conditions¶

The operator maintains two standard Kubernetes conditions on the cluster resource:

Condition	`True` when	`False` when	`Unknown` when
`ServersHealthy`	All servers are `state=Enabled` and `health=Available`	Any server is Cordoned, Deallocating, or Unavailable	Diagnostics cannot be collected
`DatabasesHealthy`	All user databases have `status=online`	Any database has `requestedStatus=online` but `status≠online`	Diagnostics cannot be collected

Note: The system database is excluded from the DatabasesHealthy check because it has special internal lifecycle behavior.

Check conditions directly:

# Check ServersHealthy condition
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{.status.conditions[?(@.type=="ServersHealthy")]}'

# Check DatabasesHealthy condition
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{.status.conditions[?(@.type=="DatabasesHealthy")]}'

# Watch all conditions
kubectl get neo4jenterprisecluster <name> -o jsonpath='{.status.conditions}' | jq .

Prometheus Server Health Metric¶

The operator exposes a per-server health gauge when diagnostics are enabled:

Metric	Labels	Value
`neo4j_operator_server_health`	`cluster_name`, `namespace`, `server_name`, `server_address`, `k8s_cluster`	`1` = Enabled+Available; `0` = degraded

Example PrometheusRule alert:

groups:
  - name: neo4j-operator
    rules:
      - alert: Neo4jServerDegraded
        expr: neo4j_operator_server_health == 0
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Neo4j server {{ $labels.server_name }} in cluster {{ $labels.cluster_name }} is degraded"
          description: "Server {{ $labels.server_name }} at {{ $labels.server_address }} has been unhealthy for 5 minutes"

Enable the ServiceMonitor to scrape this metric (see Prometheus scraping above).

Troubleshooting Diagnostics Collection¶

status.diagnostics.collectionError is set:

This means the operator could not reach the cluster via Bolt. Common causes:

Cause	Check
Cluster has not formed yet	Diagnostics run once the cluster has formed (been `Ready` at least once), and keep running while it is degraded; check the cluster phase
Auth secret missing or wrong	Check `spec.auth.adminSecret`; verify the secret exists
Network policy blocking Bolt	Verify the operator pod can reach port 7687 of the cluster service
Cluster overloaded	The Bolt client uses a 10s timeout; check Neo4j pod resource usage

ServersHealthy=False:

# Get details on which servers are degraded
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{.status.conditions[?(@.type=="ServersHealthy")].message}'

# Check server pods directly
kubectl get pods -l neo4j.com/cluster=<name>

DatabasesHealthy=False:

# Get details on which databases are offline
kubectl get neo4jenterprisecluster <name> \
  -o jsonpath='{.status.conditions[?(@.type=="DatabasesHealthy")].message}'

# Exec into a pod to investigate
kubectl exec <cluster-name>-server-0 -c neo4j -- \
  cypher-shell -u neo4j -p <password> "SHOW DATABASES"

Disabling Diagnostics¶

Set spec.monitoring.enabled: false explicitly. Omitting the monitoring section does not disable diagnostics — it leaves them on (and also leaves the Prometheus endpoint off). The status.diagnostics field will remain at its last-known value but will not be updated.