User Tools

Site Tools


services:prometheus

This is an old revision of the document!


Prometheus + Grafana

Infrastructure monitoring with Prometheus and Grafana.

Access

Service URL Port
Grafana https://monitoring.home.miskam.xyz 3000
Prometheus https://prometheus.home.miskam.xyz 9090
Container CT 115
IP 10.100.161.115

Default Login: admin / admin (change on first login!)

Installation

Container

pct create 115 local:vztmpl/debian-12-standard_12.12-1_amd64.tar.zst \
  --hostname monitoring \
  --memory 1024 --swap 512 \
  --cores 2 \
  --net0 name=eth0,bridge=vmbr0,ip=10.100.161.115/23,gw=10.100.161.254 \
  --features nesting=1 \
  --unprivileged 1 \
  --onboot 1

Prometheus

useradd --no-create-home --shell /bin/false prometheus
mkdir -p /etc/prometheus /var/lib/prometheus
 
cd /tmp
wget https://github.com/prometheus/prometheus/releases/download/v3.2.1/prometheus-3.2.1.linux-amd64.tar.gz
tar xzf prometheus-*.tar.gz
cd prometheus-*/
cp prometheus promtool /usr/local/bin/
cp prometheus.yml /etc/prometheus/
chown -R prometheus:prometheus /etc/prometheus /var/lib/prometheus

Systemd service: /etc/systemd/system/prometheus.service

Grafana

apt install -y apt-transport-https software-properties-common
wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" > /etc/apt/sources.list.d/grafana.list
apt update && apt install -y grafana
systemctl enable --now grafana-server

Prometheus Config

/etc/prometheus/prometheus.yml:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "node"
    static_configs:
      - targets:
        - "10.100.161.102:9100"  # traefik
        - "10.100.160.100:9100"  # hv-04
        - "10.100.160.249:9100"  # openclaw
        - "10.100.161.110:9100"  # postgres
        - "10.100.161.111:9100"  # n8n
        - "10.100.161.112:9100"  # vaultwarden
        - "10.100.161.113:9100"  # dokuwiki
        - "10.100.161.114:9100"  # uptime-kuma
        - "10.100.161.115:9100"  # monitoring

Node Exporter

Installiert auf allen 9 Hosts (Port 9100).

wget https://github.com/prometheus/node_exporter/releases/download/v1.9.0/node_exporter-1.9.0.linux-amd64.tar.gz
tar xzf node_exporter-*.tar.gz
cp node_exporter-*/node_exporter /usr/local/bin/
# + systemd service
systemctl enable --now node_exporter

Grafana Setup

  1. Data Source: Configuration → Data Sources → Prometheus → URL: http://localhost:9090
  2. Dashboard: + → Import → ID 1860 (Node Exporter Full)

Management

# Status
pct exec 115 -- systemctl status prometheus grafana-server
 
# Logs
pct exec 115 -- journalctl -u prometheus -f
pct exec 115 -- journalctl -u grafana-server -f
 
# Prometheus config reload
pct exec 115 -- kill -HUP $(pgrep prometheus)
services/prometheus.1770460780.txt.gz · Last modified: by admin