Friday, July 10, 2020

Zabbix Slack Integration

Have been using Zabbix Slack AlertScript for sending alert to Slack happily on Zabbix 3.4. On a new install of Zabbix 4.4, tried the built-in Slack Media Type utilizing new Bot User instead of custom Incoming Webhook. After testing, I went back to install AlertScript on Zabbix 4.4 to get everything working as expected, although AlertScript documentation says Zabbix 4.4 should use the built-in Slack Media Type. Zabbix documentation indicates that custom alertscripts is still an option in later version of Zabbix.

Below are issues I found with built-in Slack Media Type:
  1. Concurrent sessions setting in Slack Media Type must be higher than the number of Slack recipients of a single alert.
  2. With multiple Slack recipients, often some recipients miss recovery notification.
  3. With a single Slack recipient, recovery notification could also fail. The error message on Zabbix is "channel not found".
  4. The notification content template is a custom script instead of standard Zabbix Action Operations template. The custom script is difficult to customize.
The missing recovery notification is probably caused by the new behaviour of recovery notification,which, instead of sending a new one of recovery, it searches for the previous problem notification and replace it with a recovery one. The search often fails, probably due to permission and Slack's upgrade of its internal working. hence the failure of recovery notification. Another downside of replacing previous notification with recovery one is the original notification is lost. As alert history, it is important to see when the original alert happened.

No comments: