prometheus-webhook-dingtalk 模版

在部署了 prometheus-webhook-dingtalk 并且正确收到钉钉通知:

../../../../_images/alert_dingtalk.png

但是你会发现有些不足:

  • 没有展示出属于哪个集群(需要维护多个集群告警)

  • 所有label都是英文(如果能够定制一些中文更好)

定制模版

prometheus-webhook-dingtalk 新版本在 config.yml 中提供了指定模版的方法,所以对于 prometheus-webhook-dingtalk 中我所采用 containerd运行时(runtime) 运行容器,可以直接将模版配置映射进容器

配置使用自定义模板不生效 #86 有人提供了一个模版文件,大致能看出修改的方法,不过这个模版是文本模式,对于Markdown格式发送消息后格式不佳

我的需求比较简单,就是在默认的模版上再增加一个集群的文字说明,当然如果能够将模版中的一些标签修改成中文就更好。

通过 检查Docker镜像 可以提取出 prometheus-webhook-dingtalk 镜像中默认的模版文件 default.tmpl ,然后简单添加和集群相关的标记信息(中文),结合 config.yml 指定使用自定义模版:

template.tmpl 中添加标记集群信息(中文)
{{ define "__subject" }}[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}{{ end }}
{{ define "__alertmanagerURL" }}{{ .ExternalURL }}/#/alerts?receiver={{ .Receiver }}{{ end }}

{{ define "__text_alert_list" }}{{ range . }}
**Labels**
{{ range .Labels.SortedPairs }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}
**Annotations**
{{ range .Annotations.SortedPairs }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}
**Source:** [{{ .GeneratorURL }}]({{ .GeneratorURL }})
{{ end }}{{ end }}

{{ define "default.__text_alert_list" }}{{ range . }}
#### \[{{ .Labels.severity | upper }}\] {{ .Annotations.summary }}

**Description:** {{ .Annotations.description }}

**Graph:** [📈]({{ .GeneratorURL }})

**Details:**
{{ range .Labels.SortedPairs }}{{ if and (ne (.Name) "severity") (ne (.Name) "summary") }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}{{ end }}
{{ end }}{{ end }}

{{/* Default */}}
{{ define "default.title" }}{{ template "__subject" . }}{{ end }}
{{ define "default.content" }}#### \[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}\] **[{{ index .GroupLabels "alertname" }}]({{ template "__alertmanagerURL" . }})**
{{ if gt (len .Alerts.Firing) 0 -}}
# Cloud Atlas监控告警
**Alerts Firing**
{{ template "default.__text_alert_list" .Alerts.Firing }}
{{ range .AtMobiles }}@{{ . }}{{ end }}
{{- end }}
{{ if gt (len .Alerts.Resolved) 0 -}}
# Cloud Atlas监控告警
**Alerts Resolved**
{{ template "default.__text_alert_list" .Alerts.Resolved }}
{{ range .AtMobiles }}@{{ . }}{{ end }}
{{- end }}
{{- end }}

{{/* Legacy */}}
{{ define "legacy.title" }}{{ template "__subject" . }}{{ end }}
{{ define "legacy.content" }}#### \[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}\] **[{{ index .GroupLabels "alertname" }}]({{ template "__alertmanagerURL" . }})**
{{ template "__text_alert_list" .Alerts.Firing }}
{{- end }}

{{/* Following names for compatibility */}}
{{ define "ding.link.title" }}{{ template "default.title" . }}{{ end }}
{{ define "ding.link.content" }}{{ template "default.content" . }}{{ end }}
config.yml 中引用定制模版
## Request timeout
# timeout: 5s

## Uncomment following line in order to write template from scratch (be careful!)
#no_builtin_template: true

## Customizable templates path
#templates:
#  - contrib/templates/legacy/template.tmpl
#  这里在主机上准备好 template.tmpl 然后通过 -v $PWD/template.tmpl:/etc/prometheus-webhook-dingtalk/template.tmpl 映射进容器
templates:
  - /etc/prometheus-webhook-dingtalk/template.tmpl

## You can also override default template using `default_message`
## The following example to use the 'legacy' template from v0.3.0
#default_message:
#  title: '{{ template "legacy.title" . }}'
#  text: '{{ template "legacy.content" . }}'

## Targets, previously was known as "profiles"
targets:
  cloud_atlas_alert:
    url: https://oapi.dingtalk.com/robot/send?access_token=zzzzzzzzzzzz
    mention:
      mobiles: ['136xxxxxxxxx']
  sre_team_1:
    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
    mention:
      mobiles: ['136xxxx8827', '139xxxx8325']
  sre_team_2:
    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
    mention:
      mobiles: ['156xxxx8827', '189xxxx8325']
  • 执行以下命令采用定制模版运行:

结合指定模版nerdctl( containerd运行时(runtime) )方式运行 prometheus-webhook-dingtalk
nerdctl run -d --restart always -p 8060:8060 -v $PWD/config.yml:/etc/prometheus-webhook-dingtalk/config.yml \
    -v $PWD/template.tmpl:/etc/prometheus-webhook-dingtalk/template.tmpl \
    timonwong/prometheus-webhook-dingtalk --config.file=/etc/prometheus-webhook-dingtalk/config.yml \
    --web.listen-address=0.0.0.0:8060 --web.enable-ui

此时收到的告警就会带上标记的信息(中文),方便知晓对应的集群信息

../../../../_images/alert_dingtalk_clusterinfo.png

参考