Back up PMM Server Docker container¶

Regular backups of your PMM Server are essential for protecting your monitoring configuration and historical data, especially before migrations and upgrades.

Back up Grafana plugins¶

Grafana plugins have been moved to the /srv directory since PMM 2.23.0. So if you are upgrading PMM from a version before 2.23.0 and have installed additional plugins, you’ll need to reinstall them after the upgrade.

To check used Grafana plugins:

docker exec -t pmm-server ls -l /var/lib/grafana/plugins

Backup procedure¶

To back up your PMM Server container, follow the backup instructions for your deployment type.

Step 1: Back up PMM Server container¶

Identify your deployment type and storage method since different PMM Server deployments store data differently:

Docker with named volumesDocker with host directoriesPodman with SystemDKubernetes

Your PMM data is stored in a Docker-managed volume (like pmm-data). You’ll need to copy this volume’s contents to create a backup.

Command to check mount configuration

docker inspect pmm-server | grep -A 10 '"Mounts"'

Expected output

"Mounts": [
    {
        "Type": "volume",
        "Name": "pmm-data",
        "Source": "/var/lib/docker/volumes/pmm-data/_data",
        "Destination": "/srv",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    }
]

Your PMM data is stored in a directory on your host machine that’s mounted into the container. You’ll back up this directory using standard file system tools.

Command to check mount configuration

docker inspect pmm-server | grep -A 10 '"Mounts"'

Expected output

"Mounts": [
    {
        "Type": "bind",
        "Source": "/home/user/srv",
        "Destination": "/srv",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
    }
]

Your PMM runs as a SystemD service using Podman. You’ll need to stop the service, back up the volume, and restart the service.

Command to check if PMM service is running

systemctl --user is-active pmm-server

Expected output

active

Your PMM runs in a Kubernetes cluster. You’ll use volume snapshots or persistent volume backups.

Command to check if PMM pods are running

kubectl get pods -l app.kubernetes.io/name=pmm

Expected output:

NAME    READY   STATUS    RESTARTS   AGE
pmm-0   1/1     Running   0          2d

Command to check persistent volume claims

kubectl get pvc -l app.kubernetes.io/name=pmm

Expected output

NAME               STATUS   VOLUME                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pmm-storage-pmm-0  Bound    pvc-abc123-def4-5678-9012  8Gi        RWO            gp2            2d

Step 2: Choose a backup method¶

Choose the appropriate backup method based on your PMM Server deployment:

Docker with named volumeDocker with host directoryPodman with systemDKubernetes (Helm)Universal container copy

This is the most common deployment pattern and is ideal for migrations, as it preserves the entire pmm-data volume structure required for a successful transfer. Use this method if your PMM Server is deployed with a named Docker volume:

Example deployment

docker run --detach --restart always \
  --publish 443:8443 \
  --volume pmm-data:/srv \
  --name pmm-server \
  percona/pmm-server:3

Create volume backup

# Stop PMM Server
docker stop pmm-server

# Create backup volume with timestamp
BACKUP_VOLUME="pmm-data-backup-$(date +%Y%m%d-%H%M%S)"
docker volume create $BACKUP_VOLUME 1> /dev/null

# Copy data from current volume to backup volume
docker run --rm -v pmm-data:/from -v $BACKUP_VOLUME:/to alpine ash -c 'cp -av /from/. /to'

# Verify backup
docker run --rm -v $BACKUP_VOLUME:/backup alpine ls -la /backup

# Restart PMM Server
docker start pmm-server

# Note backup volume name for recovery
echo "Backup volume created: $BACKUP_VOLUME"

Alternative: Export volume to archive:

# Create compressed backup archive
mkdir -p pmm-volume-backups
docker run --rm -v pmm-data:/volume -v $(pwd)/pmm-volume-backups:/backup alpine tar czf /backup/pmm-data-backup-$(date +%Y%m%d-%H%M%S).tar.gz -C /volume .

Use this method if your PMM Server mounts a host directory:

Example deployment

docker run --detach --restart always \
  --publish 443:8443 \
  --volume /home/user/srv:/srv \
  --name pmm-server \
  percona/pmm-server:3

Create host directory backup

# Stop PMM Server
docker stop pmm-server

# Create backup directory
BACKUP_DIR="pmm-directory-backup-$(date +%Y%m%d-%H%M%S)"
mkdir -p $BACKUP_DIR

# Copy mounted directory (adjust path to match your deployment)
cp -r /home/user/srv $BACKUP_DIR/

# Or use rsync for incremental backup
rsync -av /home/user/srv/ $BACKUP_DIR/

# Restart PMM Server
docker start pmm-server

Use this method if your PMM Server runs with Podman and SystemD:

Example deployment via SystemD service

# Podman with named volume via SystemD
systemctl --user status pmm-server

Create Podman volume backup

# Stop PMM Server service
systemctl --user stop pmm-server

# Create backup volume with timestamp
BACKUP_VOLUME="pmm-data-backup-$(date +%Y%m%d-%H%M%S)"
podman volume create $BACKUP_VOLUME

# Copy data from current volume to backup volume
podman run --rm -v pmm-server:/from -v $BACKUP_VOLUME:/to alpine ash -c 'cp -av /from/. /to'

# Verify backup
podman run --rm -v $BACKUP_VOLUME:/backup alpine ls -la /backup

# Restart PMM Server service
systemctl --user start pmm-server

# Note backup volume name for restoration
echo "Backup volume created: $BACKUP_VOLUME"

Use this method if your PMM Server runs on Kubernetes via Helm. Requires StorageClass and VolumeSnapshotClass that support snapshots. Check with your Kubernetes provider for availability.

Example deployment

helm install pmm percona/pmm

Create Kubernetes volume snapshot

# Check available storage classes and snapshot classes
kubectl get storageclass
kubectl get volumesnapshotclass

# Create volume snapshot
cat <<EOF | kubectl apply -f -
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: pmm-backup-$(date +%Y%m%d-%H%M%S)
  labels:
    app.kubernetes.io/name: pmm
spec:
  source:
    persistentVolumeClaimName: pmm-storage-pmm-0
  volumeSnapshotClassName: your-snapshot-class
EOF

# Verify snapshot creation
kubectl get volumesnapshot -l app.kubernetes.io/name=pmm

While this method works universally, the deployment-specific methods above are more efficient and preserve storage structures better. This method works for all deployment types as a fallback option when you’re unsure about your deployment type or you need a quick backup without determining volume setup.

Stop the running PMM Server container:
```
docker stop pmm-server
```
Rename the container to preserve it as a backup source:
```
docker rename pmm-server pmm-server-backup
```

Create a backup subdirectory and navigate to it:

mkdir pmm-data-backup-$(date +%Y%m%d-%H%M%S) && cd pmm-data-backup-$(date +%Y%m%d-%H%M%S)

Back up the data:
```
docker cp pmm-server-backup:/srv/. .
```
Verify the backup was created successfully:
```
ls -la .
```

Create new container from original image:

docker run -d -v pmm-data:/srv -p 443:8443 --name pmm-server --restart always percona/pmm-server:3

Step 3: Verify the integrity of the backup¶

Volume backupsDirectory backupsPodman volumesKubernetes snapshots

For backups created using the Named Volume method:

# Check backup volume contents
docker run --rm -v $BACKUP_VOLUME:/backup alpine ls -la /backup

# Verify critical directories
docker run --rm -v $BACKUP_VOLUME:/backup alpine \
  ash -c 'ls -la /backup/grafana /backup/prometheus /backup/clickhouse 2>/dev/null || echo "Some directories may not exist in older versions"'

For directory-based backups:

# Check backup directory
ls -la $BACKUP_DIR/

# Verify critical subdirectories
ls -la $BACKUP_DIR/{grafana,prometheus,clickhouse} 2>/dev/null || echo "Some directories may not exist in older versions"

For backups created using Podman volumes:

# Check backup volume contents
podman run --rm -v $BACKUP_VOLUME:/backup alpine ls -la /backup

# Verify critical directories exist
podman run --rm -v $BACKUP_VOLUME:/backup alpine ls -la /backup/grafana /backup/prometheus /backup/clickhouse 2>/dev/null || echo "Some directories may not exist in older versions"

For backups created using Kubernetes volume snapshots:

# Check snapshot status
kubectl get volumesnapshot -l app.kubernetes.io/name=pmm

# Verify snapshot is ready
kubectl get volumesnapshot pmm-backup-YYYYMMDD-HHMMSS -o jsonpath='{.status.readyToUse}'

# Check snapshot size and source
kubectl describe volumesnapshot pmm-backup-YYYYMMDD-HHMMSS

Next steps after backup¶

After creating your backup, you have two options:

Resume normal operations if you were creating a routine backup, restart your original container.
Upgrade or restore the container if you were backing up before an upgrade or restoration.

Backup storage recommendations¶

Store backups in a location separate from the PMM Server host
Implement automated rotation of backups to manage disk space
Consider encrypting backups containing sensitive monitoring data
Test restores periodically to verify backup integrity