> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anomalyarmor.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# PagerDuty

> Send AnomalyArmor alerts to PagerDuty for incident management

<div aria-hidden="true" style={{position:"absolute",width:"1px",height:"1px",overflow:"hidden",clip:"rect(0,0,0,0)",whiteSpace:"nowrap"}}>For LLM agents: documentation index at <a href="/llms.txt" tabIndex={-1}>/llms.txt</a>, full text at <a href="/llms-full.txt" tabIndex={-1}>/llms-full.txt</a>. Append .md to any page URL for plain markdown.</div>
Send AnomalyArmor alerts to PagerDuty to trigger incidents and page on-call engineers. Critical schema changes or freshness violations can automatically create PagerDuty incidents for immediate response.

## Why PagerDuty?

PagerDuty is ideal for alerts that require immediate human intervention:

* **On-call routing**: Automatically page the right person based on schedules
* **Escalation**: Ensure critical alerts don't go unnoticed
* **Incident management**: Track response and resolution
* **Deduplication**: Related alerts are grouped together

## Prerequisites

Before you begin:

* PagerDuty account with admin access (or permission to create integrations)
* AnomalyArmor account with alert configuration permissions
* A PagerDuty service configured for your data infrastructure

## Setup Guide

### Step 1: Create a PagerDuty Integration

In PagerDuty:

1. Go to **Services** and select (or create) a service for data alerts
2. Click **Integrations** tab
3. Click **Add Integration**
4. Search for **Events API v2** and select it
5. Click **Add**
6. Copy the **Integration Key** (also called Routing Key)

<Note>
  The Integration Key is a 32-character string that looks like: `a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6`
</Note>

### Step 2: Add Destination in AnomalyArmor

1. Log in to AnomalyArmor
2. Click **Alerts** in the left sidebar
3. Select **Destinations** tab
4. Click **Add Destination**
5. Select **PagerDuty**

### Step 3: Configure the Destination

Enter the following:

| Field               | Description                                            |
| ------------------- | ------------------------------------------------------ |
| **Name**            | A descriptive name (e.g., "PagerDuty - Data Platform") |
| **Integration Key** | The key copied from PagerDuty                          |

### Step 4: Test the Connection

Click **Test** to send a test event to PagerDuty.

```
Success! Test event sent to PagerDuty
Dedup Key: anomalyarmor-test-12345
```

Check PagerDuty to confirm the incident was created.

<Warning>
  Remember to resolve the test incident in PagerDuty to avoid confusion.
</Warning>

### Step 5: Save

Click **Create Destination** to complete the setup.

## Alert Format

AnomalyArmor sends events using the PagerDuty Events API v2 format:

| Field         | Value                                               |
| ------------- | --------------------------------------------------- |
| **Summary**   | Alert title with context                            |
| **Source**    | `anomalyarmor`                                      |
| **Severity**  | `critical` (all PagerDuty alerts are high priority) |
| **Dedup Key** | Unique identifier to group related alerts           |

<Note>
  Since PagerDuty is reserved for critical alerts, all events sent to PagerDuty use the `critical` severity level. Use Slack or email for lower-priority notifications.
</Note>

### Custom Details

Each PagerDuty incident includes:

* Rule name that triggered the alert
* Event type (schema\_change, freshness\_violation, etc.)
* Alert description
* Asset ID
* Link to view in AnomalyArmor

## Best Practices

### Use PagerDuty for Critical Alerts Only

<Warning>
  Don't route all alerts to PagerDuty. Reserve it for events that require immediate action - typically production schema changes that could break pipelines.
</Warning>

**Good use cases**:

* Production column removed
* Critical table freshness SLA violated
* Breaking schema changes in production

**Better handled elsewhere**:

* Development database changes (use Slack)
* Informational schema additions (use email)
* Routine freshness warnings (use Slack digest)

### Set Up Proper Escalation

Configure your PagerDuty service with:

1. **Acknowledgement timeout**: Auto-escalate if not acknowledged (e.g., 5 minutes)
2. **Escalation policy**: Include backup responders
3. **On-call schedule**: Ensure 24/7 coverage for critical services

### Combine with Other Destinations

Create alert rules that send to multiple destinations:

**Production Breaking Changes**

* Event: Schema Change
* Scope: production databases
* Conditions: Column removed OR type changed
* Destinations: PagerDuty (immediate paging), Slack #data-incidents (team visibility), Email [data-eng-list@company.com](mailto:data-eng-list@company.com) (record)

## Troubleshooting

### "Invalid routing key"

**Cause**: The integration key is incorrect or the integration was deleted in PagerDuty.

**Fix**:

1. Go to your PagerDuty service
2. Check the Events API v2 integration still exists
3. Copy a fresh integration key
4. Update the destination in AnomalyArmor

### Incidents not appearing

**Cause**: Service is disabled or in maintenance mode.

**Fix**:

1. Check the PagerDuty service is enabled
2. Verify no maintenance window is active
3. Check the Events API v2 integration is active
4. Use the Test button to verify connectivity

### Duplicate incidents

**Cause**: Multiple alert rules triggering for the same event.

**Fix**:
AnomalyArmor includes a deduplication key with each event. PagerDuty will group alerts with the same dedup key into a single incident. If you're seeing duplicates:

1. Review your alert rules for overlap
2. Consider combining rules or adjusting scopes

### Rate limiting

**Cause**: Too many events sent in a short period.

**Fix**:

1. Review alert rule thresholds
2. Consider using alert grouping/digest for high-volume events
3. Route lower-priority alerts to Slack or email instead

## Security

### Data Sent to PagerDuty

Alert events contain:

* Asset names (database, schema, table names)
* Change types and descriptions
* Timestamps
* Rule information

Alert events **do not** contain:

* Actual data values
* Database credentials
* Connection strings
* Query results

### Revoking Access

To disconnect AnomalyArmor from PagerDuty:

1. In AnomalyArmor: Delete the PagerDuty destination
2. In PagerDuty: Remove the Events API v2 integration from the service

## Common Questions

### Which PagerDuty integration type should I use?

Use **Events API v2** on the target service. In PagerDuty, open the service, go to **Integrations**, click **Add Integration**, select Events API v2, and copy the 32-character Integration Key into AnomalyArmor. Older Events API v1 integrations are not supported.

### Can I set non-critical severity for PagerDuty alerts?

No. Every event AnomalyArmor sends to PagerDuty uses `critical` severity by design, since PagerDuty is reserved for events that require immediate human response. Route lower-priority alerts to [Slack](/alerts/destinations/slack) or [email](/alerts/destinations/email) instead.

### Will PagerDuty deduplicate repeated alerts from AnomalyArmor?

Yes. Each event ships with a deduplication key derived from the rule and affected asset, so PagerDuty groups repeated firings into a single incident rather than paging on-call again. If you still see duplicates, check whether multiple overlapping rules are firing for the same event.

### What happens if PagerDuty isn't acknowledged?

That's controlled by the PagerDuty service's escalation policy, not by AnomalyArmor. Set an acknowledgement timeout (for example 5 minutes) and configure backup responders in PagerDuty so unacknowledged pages escalate automatically.

## Next Steps

<CardGroup cols={2}>
  <Card title="Alert Rules" icon="bell" href="/alerts/alert-rules">
    Create rules that route to PagerDuty
  </Card>

  <Card title="Best Practices" icon="lightbulb" href="/alerts/best-practices">
    Reduce alert fatigue and page only when necessary
  </Card>
</CardGroup>