kube-prometheus-stack 抓取节点metrics

kube-prometheus-stack 添加Prometheus scrape配置 采用标准的 Prometheus服务发现 获取Kubernetes对象的监控数据。不过,生产环境也有很多服务器并不属于Kubernetes机群,而是独立部署的物理主机,有各种应用自己开发输出的 Metrics ,也需要通过Prometheus采集。

以下是一个简单的配置案例,监控一种分布式存储不同角色(master/worker)输出的metrics,指定 scrape 目标以及通过 labels 方式为服务器分组,这样后续在 Prometheus 和 Grafana通用可视分析平台 可以根据 group 这个label进行查询:

指定服务器抓取metrics
    additionalScrapeConfigs:
    - job_name: 'store-metrics-9999'
      scrape_interval: 30s
      metrics_path: /metrics/prometheus
      static_configs:
      - targets:
        - "192.168.1.8:9999"
        - "192.168.1.7:9999"
        - "192.168.1.6:9999"
        labels:
          group: 'store-master-1'
      - targets:
        - "192.168.1.200:9999"
        - "192.168.1.197:9999"
        - "192.168.1.201:9999"
        labels:
          group: 'store-master-2'
      ...
    - job_name: 'store-metrics-10000'
      scrape_interval: 30s
      metrics_path: /metrics/prometheus
      static_configs:
      - targets:
        - "192.168.44.250:10000"
        - "192.168.44.253:10000"
        - "192.168.44.248:10000"
        - "192.168.1.3:10000"
        - "192.168.1.0:10000"
        labels:
          group: 'store-worker-1'
      - targets:
        - "192.168.1.236:10000"
        - "192.168.2.12:10000"
        - "192.168.1.251:10000"
        - "192.168.1.237:10000"
        - "192.168.1.255:10000"
        labels:
          group: 'store-worker-2'
      ...
    - job_name: 'store-metrics-10001'
      scrape_interval: 30s
      metrics_path: /metrics/prometheus
      static_configs:
      - targets:
        - "192.168.2.4:10001"
        - "192.168.2.5:10001"
        - "192.168.1.254:10001"
        - "192.168.1.253:10001"
        - "192.168.1.248:10001"
        labels:
          group: 'store-worker-1'
      - targets:
        - "192.168.1.236:10001"
        - "192.168.2.12:10001"
        - "192.168.1.251:10001"
        - "192.168.1.237:10001"
        - "192.168.1.255:10001"
        labels:
          group: 'store-worker-2'
      ...

这里采用了多个Targes配置

参考