Machine Learning In Zabbix 6.0 LTS: Anomaly Detection And Baselines

10d ago
12 Views
0 Downloads
2.42 MB
36 Pages
Last View : 2d ago
Last Download : n/a
Upload by : Francisco Tran
Transcription

MACHINE LEARNING IN ZABBIX 6.0 LTS: ANOMALY DETECTION AND BASELINES Aleksandrs Kalimulins Developer

MODERN MONITORING CHALLENGES More devices, VMs, servers and applications More monitored entities means more metrics IT environments are changing rapidly New concepts emerge frequently

MODERN MONITORING CHALLENGES Less time to keep track of what is normal Hard to get right signal-to-noise ratio

MACHINE LEARNING: ZABBIX APPROACH "Field of study that gives computers the ability to learn without being explicitly programmed“ - Arthur Samuel (computer scientist, machine learning pioneer)

01 MACHINE LEARNING: ZABBIX APPROACH EASY AND TRANSPARENT:

WHAT IS MACHINE LEARNING? EASY AND TRANSPARENT: Simple configuration Easy to understand Easy to verify

FOCUS AREAS Smart triggers Anomaly detection Analyse hystorical data Find outliers in analysis results

FOCUS AREAS Smart triggers Anomaly detection Analyse hystorical data Find outliers in analysis results Baselines Calculate averages in past calendar periods Find how far are current values

ANOMALY DETECTION

ANOMALY DETECTION Works when the majority is normal data Long-term analytics, works with trends Zabbix uses STL decomposition

STL DECOMPOSITION

STL DECOMPOSITION

STL DECOMPOSITION Yt Tt St Rt 1. Apply smoothing to the original curve, get Tt 2. Subtract result from the original curve, split into seasons 3. Apply averaging to seasons, get seasonal curve St 4. Subtract Tt and St, get residue Rt

STL DECOMPOSITION

STL DECOMPOSITION

DEVIATIONS Deviation is a measure of data variability How “far” values are from average?

DEVIATIONS Standard and median deviations in Zabbix stddevpop(), stddevsamp(), mad() Also supported in anomaly function

ANOMALY DETECTION ALGORITHM Get trend values for the period Decompose values, get remainder Calculate deviation for values in remainder Select values with deviations threshold

ANOMALY DETECTION FUNCTION trendstl(/host/key,period:time shift,detection period,season,deviations,dev algorithm) Returns 0 number 1 (ratio anomaly count / value count)

ANOMALY DETECTION FUNCTION trendstl(/host/key,period:time shift,detection period,season,deviations,dev algorithm) Parameters /host/key - item period:time shift - evaluation period (for decomposition) detection period – report anomalies in this period season – season’s length for decomposition deviations, dev algorithm

ANOMALY DETECTION FUNCTION ��mad”) 0.1 Decompose last 30 days Report anomalies within last 7 days Use season 12 hours Count points 3 median deviations Same as: trendstl(/Web/net.if.out[en0],30d:now/d,7d,12h) 0.1

CAVEATS trendstl() Long term analytics, works only with trends Usable only if data has seasonality Season parameter is seconds

BASELINES

02 WHAT IS BASELINE? “BASELINE IS A VALUE DERIVED FROM AN AVERAGE OVER MULTIPLE CALENDAR PERIODS OF THE SAME LENGTH” – Zabbix (best monitoring solution)

PERIODS AND SEASONS Periods and seasons Average from past calendar periods E.g., every Monday of the past 4 weeks Monday is a period, week is a season Periods linked to current time If today is Wednesday, then periods are Tuesdays

PERIODS VS SEASONS

BASELINE FUNCTIONS baselinewma(/host/key,period :time shift ,seasons) Returns baseline by averaging data periods in seasons Uses Weighted Moving Average algorithm (WMA)

BASELINE FUNCTIONS baselinewma(/host/key,period :time shift ,seasons) Returns baseline by averaging data periods in seasons Uses Weighted Moving Average algorithm (WMA)

BASELINE FUNCTIONS baselinedev(/host/key,period :time shift ,seasons) Returns number of standard deviations Compares last period to periods before within seasons

BASELINE FUNCTIONS baselinedev(/host/key,period :time shift ,seasons) Returns number of standard deviations Compares last period to periods before within seasons

BASELINE FUNCTIONS baselinedev(/Zabbix server/system.cpu.load,1h,10d) 3 Check if load for last hour 3 deviations away from mean Use 10 one-hour periods over last 10 days

BASELINE FUNCTIONS baselinewma(/Zabbix server/nginx.requests.total.rate,1d,12w)*2 trendavg(/Zabbix server/nginx.requests.total.rate,1d:now/d) Check if web traffic yesterday is 2x higher than WMA on the same weekdays over last 12 weeks

CAVEATS Baselines “remember” problems Abnormal values included in calculations Time units are not interchangeable 7d 1w

TECHNICAL CONSIDERATIONS Maintain trend storage intervals Set reasonable TrendCacheSize Set reasonable intervals for calculated items

WHAT TO CHOOSE? Suitable only for long term analytics trendstl() heavier on resources Calendar periods in baselinewma/dev() trendstl() works best with few anomalus points

Thank you! www.zabbix.com

ANOMALY DETECTION. Works when the majority is normal data Long-term analytics, works with trends Zabbix uses STL decomposition. STL DECOMPOSITION. STL DECOMPOSITION. STL DECOMPOSITION. Yt Tt St Rt. 1. Apply smoothing to the original curve, get Tt. 2.

Related Documents:

dikirimkan dari tindakan Zabbix contohnya "Zabbix server is unreachable for 5 minutes - Zabbix server (127.0.0.1)". Output notifikasi dapat bekerja secara optimal yang sebelumnya dilakukan dengan pembuatan user slack dan menambahkan alert script pada file Zabbix, kemudian media yang berada di aplikasi Zabbix ditambahkan email admin dan

Zabbix instances. A user also can download the templates from Zabbix website Link to download Morningstar SNMP templates ( For Zabbix 5.2 and higher). Template is a .YAML file. Link to download Morningstar SNMP templates ( For Zabbix 5.0 and higher. Template is a.XML file. Click on the "download" button

Graphite key: mesos-masterx.mesos-master.gauge.master_elected Zabbix Host: mesos-masterx Zabbix key: graphite[mesos-master.gauge.master_elected] Pros: Possible to use Graphite functions in zabbix requests (zabbix key) 21

Starting monitoring MySQL with Zabbix is really easy 1. Install Zabbix agent2 2. Create a MySQL user with monitoring privileges (for example GRANT USAGE,REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW ON *.* TO 'zbx_monitor'@'%'; ) 3. Create host with the Zabbix agent interface 4. Assign the MySQL template to the host 5.

The Zabbix Configuration white paper covered how to use the Zabbix web frontend to configure and administer Zabbix. This white paper will cover how to use the Zabbix web frontend to view the item data that has been collected and the problems and actions that have been identified and taken. Not all of the sections will be covered.

Zabbix : Solution de supervision libre Zabbix : Détection des pannes Deux types de vues de remontée des alertes Une vue type tableau Présentation des déclencheurs à plusieurs niveaux - Pour tout le réseau - Par groupe - Par élément Présentation des alertes en cours Réactivité des équipes

Talking about network traffic . Company network Internal Services Active Directory DNS Intranet FTP POP IMAP The 5-tuple: Source IP x Source port y . There is no limit to monitor when using Zabbix if out-of-the-box features are not enough, just extend Zabbix

Realizar testes do sistema de monitoramento Zabbix. Analisar as informações obtidas da ferramenta Zabbix. 1.2 JUSTIFICATIVA O monitoramento ambiente de rede, contribui para um ambiente com baixa indisponibilidade, os alertas são essenciais para o administrador de rede, pois,