User Tools

Site Tools


operations:monitoring

This is an old revision of the document!


Monitoring Setup

How to add hosts to monitoring.

Stack

Component Purpose
Prometheus Metrics collection
Grafana Visualization
node_exporter Host metrics (CPU, RAM, Disk, Network)
Uptime Kuma Status page + Alerts

Current Hosts

All hosts run node_exporter on port 9100:

Host IP
hv-04 (Proxmox) 10.100.160.100
traefik (CT 102) 10.100.161.102
openclaw (CT 103) 10.100.160.249
postgres (CT 110) 10.100.161.110
n8n (CT 111) 10.100.161.111
vaultwarden (CT 112) 10.100.161.112
dokuwiki (CT 113) 10.100.161.113
uptime-kuma (CT 114) 10.100.161.114
monitoring (CT 115) 10.100.161.115

Adding a New Host

1. Install node_exporter

cd /tmp
wget https://github.com/prometheus/node_exporter/releases/download/v1.9.0/node_exporter-1.9.0.linux-amd64.tar.gz
tar xzf node_exporter-*.tar.gz
cp node_exporter-*/node_exporter /usr/local/bin/
 
cat > /etc/systemd/system/node_exporter.service << 'EOF'
[Unit]
Description=Node Exporter
After=network.target
 
[Service]
Type=simple
ExecStart=/usr/local/bin/node_exporter
Restart=always
 
[Install]
WantedBy=multi-user.target
EOF
 
systemctl daemon-reload
systemctl enable --now node_exporter

2. Add to Prometheus

Edit /etc/prometheus/prometheus.yml on CT 115:

  - job_name: "node"
    static_configs:
      - targets:
        - "NEW_HOST_IP:9100"  # new host

Reload: kill -HUP $(pgrep prometheus)

Troubleshooting

# Test node_exporter locally
curl http://localhost:9100/metrics | head
 
# Test from Prometheus server
curl http://HOST_IP:9100/metrics | head
 
# Check Prometheus targets
curl http://10.100.161.115:9090/api/v1/targets | jq '.data.activeTargets[] | {instance, health}'
operations/monitoring.1770425255.txt.gz ยท Last modified: by 127.0.0.1