Homeโ€บ๐Ÿš€ Production Skillsโ€บModule 131 min read ยท 14/16

Alert Creation

Hands-on2 exercises

Alert Structure

Dynatrace has two alert systems. Classic metric events (Gen2) and Gen3 Davis anomaly detectors. Both work, but anomaly detectors are more powerful.

Gen3: Davis Anomaly Detectors (recommended)

{
  "schemaId": "builtin:davis.anomaly-detectors",
  "value": {
    "enabled": true,
    "title": "[P1] Device CPU Critical",
    "analyzer": {
      "name": "dt.statistics.ui.anomaly_detection.StaticThresholdAnomalyDetectionAnalyzer",
      "input": [
        {"key": "query", "value": "timeseries avg(custom.device.cpu), interval:1m"},
        {"key": "threshold", "value": "90"},
        {"key": "alertCondition", "value": "ABOVE"},
        {"key": "violatingSamples", "value": "3"},
        {"key": "slidingWindow", "value": "5"},
        {"key": "dealertingSamples", "value": "5"}
      ]
    }
  }
}

Three analyzer models: StaticThreshold (fixed number), AutoAdaptive (learns baseline), SeasonalBaseline (daily/weekly patterns).

Classic: Metric Events (still works)

event_template:
  title: "[P3] {dims:dt.entity.my_ext:device.name} - CPU High"
  event_type: CUSTOM_ALERT
  davis_merge: false

model_properties:
  type: STATIC_THRESHOLD
  alert_condition: ABOVE
  threshold: 70
  samples: 35
  violating_samples: 3
  dealerting_samples: 5
  alert_on_no_data: false

Priority Levels

PrioritySeverityTypical ThresholdTitle Prefix
P1Severe / Criticalโ‰ฅ 90%[P1]
P2Critical / Highโ‰ฅ 80%[P2]
P3Warning / Lowโ‰ฅ 70%[P3]

Key Rules

โš ๏ธ eventEntityDimensionKey = PARENT entity type. Problems are raised on the parent for ticket generation.

SettingValueWhy
alertOnNoDatafalsePrevents false alerts when device is unreachable
davisMergefalseKeeps extension alerts separate from Davis AI
samples35~35 minutes of data at 1-min polling
violatingSamples33 consecutive breaches before alerting
dealertingSamples55 consecutive OK before clearing
๐Ÿ›  Try it

This alert extension is missing role: default on the parent entity. Add it so alerts attach to the device.

extension.yamlYAML
Loading...

Title Placeholders

{dims:dt.entity.my_ext:device.name}     โ†’ device name
{dims:dt.entity.my_ext:interface.name}  โ†’ interface name
{dims:if.name}                          โ†’ raw dimension value

Deployment

# Deploy via unified script
python3 alerts/deploy_alerts.py my-extension
python3 alerts/deploy_alerts.py my-extension --dry-run  # preview

# Or via Settings API v2 directly
curl -X POST "$BASE/api/v2/settings/objects" \
  -H "Authorization: Api-Token $TOKEN" \
  -H "Content-Type: application/json" \
  -d @alert-payload.json

๐Ÿ’ก Terraform may fail with "No schema with topic identifier" on some environments. Settings API v2 always works.

๐Ÿ›  Try itExercise 2

The CPU metric is missing its unit. Set it to Percent so alert thresholds work correctly.

extension.yamlYAML
Loading...

What's Next

Module 14 โ€” Validate your extension against requirements and catch bugs before delivery.