> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anomalyarmor.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Freshness Monitoring

> Set SLAs on data freshness and detect stale data before it impacts your business

<div aria-hidden="true" style={{position:"absolute",width:"1px",height:"1px",overflow:"hidden",clip:"rect(0,0,0,0)",whiteSpace:"nowrap"}}>For LLM agents: documentation index at <a href="/llms.txt" tabIndex={-1}>/llms.txt</a>, full text at <a href="/llms-full.txt" tabIndex={-1}>/llms-full.txt</a>. Append .md to any page URL for plain markdown.</div>
Freshness monitoring tracks when your data was last updated and alerts you when it becomes stale. Stale data can be just as damaging as wrong data. If your dashboard shows yesterday's numbers, decisions made today could be wrong.

<Note>
  **Prerequisites**: Before setting up freshness monitoring, you need:

  * A [connected data source](/data-sources/overview) with discovery completed
  * Assets with timestamp columns (`created_at`, `updated_at`, or similar)
</Note>

<Frame caption="Freshness timeline showing expected vs actual data updates">
  <img src="https://mintcdn.com/anomalyarmor/un2W3qlHEQ29uwyl/images/diagrams/freshness-timeline-light.svg?fit=max&auto=format&n=un2W3qlHEQ29uwyl&q=85&s=0f50ca067d6dc638efe0cfcea44db4ed" alt="Timeline showing expected update time, warning threshold, and violation with alert trigger" className="block dark:hidden" width="900" height="220" data-path="images/diagrams/freshness-timeline-light.svg" />

  <img src="https://mintcdn.com/anomalyarmor/CZXBGa_D1aE9spAI/images/diagrams/freshness-timeline-dark.svg?fit=max&auto=format&n=CZXBGa_D1aE9spAI&q=85&s=b4ce84bb98426ae871a996574b770cdb" alt="Timeline showing expected update time, warning threshold, and violation with alert trigger" className="hidden dark:block" width="900" height="220" data-path="images/diagrams/freshness-timeline-dark.svg" />
</Frame>

## Why Freshness Matters

Data has an expected update cadence. When that cadence breaks, something is wrong:

<img src="https://mintcdn.com/anomalyarmor/-pFpKEip0ftEEXe9/images/diagrams/without-with-monitoring-light.svg?fit=max&auto=format&n=-pFpKEip0ftEEXe9&q=85&s=39217edf473d7ae8da2060d58b9698d9" alt="Stale data problem with and without monitoring" className="block dark:hidden" width="800" height="380" data-path="images/diagrams/without-with-monitoring-light.svg" />

<img src="https://mintcdn.com/anomalyarmor/OSEzjlRMQ1RGruVN/images/diagrams/without-with-monitoring-dark.svg?fit=max&auto=format&n=OSEzjlRMQ1RGruVN&q=85&s=cb94f236b794ef39a09be74a57a5b1dd" alt="Stale data problem with and without monitoring" className="hidden dark:block" width="800" height="380" data-path="images/diagrams/without-with-monitoring-dark.svg" />

## How Freshness Monitoring Works

AnomalyArmor tracks freshness by monitoring timestamp columns:

1. **You specify** which column indicates "when data was updated"
2. **Discovery queries** the maximum value of that column
3. **We compare** against your configured SLA
4. **Alert fires** if data is older than allowed

```sql theme={null}
-- What AnomalyArmor checks
SELECT MAX(created_at) FROM orders;
-- Result: 2024-01-15 07:58:32

-- Your SLA: Updated within 1 hour
-- Current time: 2024-01-15 08:30:00
-- Data age: 32 minutes
-- Status: OK Within SLA
```

## Setting Up Freshness Monitoring

<Steps>
  <Step title="Navigate to Asset">
    Go to **Assets** and click on the table you want to monitor.
  </Step>

  <Step title="Open Freshness Tab">
    Click the **Freshness** tab in the asset details.
  </Step>

  <Step title="Select Timestamp Column">
    Choose the column that best represents data recency:

    | Column Type      | Best For                            |
    | ---------------- | ----------------------------------- |
    | `created_at`     | Append-only tables (events, logs)   |
    | `updated_at`     | Tables with updates (user profiles) |
    | `loaded_at`      | ETL destination tables              |
    | `_etl_timestamp` | Warehouse staging tables            |

    <Warning>
      Choose a column that updates when **new data arrives**. For tables with updates, use `updated_at` instead of `created_at` to track the most recent changes.
    </Warning>
  </Step>

  <Step title="Set Expected Frequency">
    Define how often you expect new data:

    * Every 15 minutes
    * Hourly
    * Every 6 hours
    * Daily
    * Weekly
    * Custom (specify hours)
  </Step>

  <Step title="Configure Alert Threshold">
    Set when to trigger alerts:

    **Alert Configuration:**

    * **Warning threshold**: 80% of expected (optional)
    * **Violation threshold**: 100%+ of expected

    **Example (hourly expectation):**

    * **Warning**: Data is 48+ minutes old
    * **Violation**: Data is 60+ minutes old
  </Step>

  <Step title="Save Configuration">
    Click **Save** to activate freshness monitoring.
  </Step>
</Steps>

## SLA Strategies

### Start Conservative

Set SLAs with buffer room to avoid false positives:

| Actual Cadence | Recommended SLA | Why                            |
| -------------- | --------------- | ------------------------------ |
| Every hour     | 2 hours         | Allows for occasional delays   |
| Every day      | 36 hours        | Accounts for timing variations |
| Every 15 min   | 30 minutes      | Double the expected window     |

<Tip>
  Start lenient and tighten over time. It's easier to make SLAs stricter than to deal with alert fatigue from SLAs that are too tight.
</Tip>

### By Data Criticality

| Criticality | SLA Approach   | Example                             |
| ----------- | -------------- | ----------------------------------- |
| Critical    | Expected + 25% | Revenue: 1hr expected → 75min SLA   |
| High        | Expected + 50% | Orders: 1hr expected → 90min SLA    |
| Medium      | Expected × 2   | Analytics: 1hr expected → 2hr SLA   |
| Low         | Expected × 3   | Reports: daily expected → 3 day SLA |

### By Update Pattern

<Tabs>
  <Tab title="Streaming/Real-time">
    For data expected within minutes:

    | Setting       | Value             |
    | ------------- | ----------------- |
    | **Expected**  | Every 5 minutes   |
    | **Warning**   | 10 minutes        |
    | **Violation** | 15 minutes        |
    | **Alert**     | Slack + PagerDuty |
  </Tab>

  <Tab title="Batch/Hourly">
    For hourly ETL jobs:

    | Setting       | Value      |
    | ------------- | ---------- |
    | **Expected**  | Every hour |
    | **Warning**   | 80 minutes |
    | **Violation** | 2 hours    |
    | **Alert**     | Slack      |
  </Tab>

  <Tab title="Daily">
    For daily batch loads:

    | Setting       | Value             |
    | ------------- | ----------------- |
    | **Expected**  | Daily by 6am      |
    | **Warning**   | 8am (2hr buffer)  |
    | **Violation** | 12pm (6hr buffer) |
    | **Alert**     | Email             |
  </Tab>
</Tabs>

## Freshness Alerts

### Setting Up Alert Rules

Create rules to notify you of freshness violations:

| Field            | Value                    |
| ---------------- | ------------------------ |
| **Rule**         | Critical Table Freshness |
| **Event**        | Freshness Violation      |
| **Assets**       | orders, payments, users  |
| **Destinations** | Slack, PagerDuty         |

### Alert Content

Freshness alerts include:

```
[!] Freshness Violation

Asset: production.public.orders
Column: created_at

Expected: Updated every 1 hour
Last update: 3 hours 15 minutes ago
Data age: 3h 15m (SLA: 1h)

Detected: January 15, 2024 at 8:00 AM UTC

[View Asset] [View Dashboard]
```

## Handling Special Cases

### Weekends and Holidays

Some data doesn't update on weekends:

**Options:**

1. **Longer weekend SLA**: Set different thresholds for Saturday/Sunday
2. **Pause monitoring**: Temporarily disable freshness checks
3. **Adjust expectations**: Set SLA to "72 hours" to cover full weekends

### Maintenance Windows

During planned maintenance:

1. **Disable rules**: Toggle OFF freshness alert rules for affected assets
2. **Document**: Note expected staleness
3. **Re-enable**: Toggle rules back ON after maintenance

### Infrequently Updated Tables

Some tables legitimately update rarely:

| Table Type         | Update Pattern | SLA Approach                |
| ------------------ | -------------- | --------------------------- |
| Reference/lookup   | Monthly        | 45-day SLA or no monitoring |
| Historical archive | Never          | Don't monitor freshness     |
| Dimension tables   | Daily/weekly   | Match actual pattern        |

## Viewing Freshness Status

### Asset List View

In the **Assets** list, freshness status appears as indicators:

| Indicator | Meaning                   |
| --------- | ------------------------- |
| Green     | Within SLA                |
| Yellow    | Warning threshold reached |
| Red       | SLA violated              |
| Gray      | Freshness not configured  |

### Asset Detail View

Click an asset to see:

* Current freshness status
* Last update timestamp
* Freshness history over time
* SLA configuration

The detail view shows a status card with the current freshness state, last update timestamp, data age, and a progress bar indicating how much of your SLA window has been consumed. Below that, a 7-day history shows freshness status over time.

**Example freshness status:**

| Field            | Value                           |
| ---------------- | ------------------------------- |
| Status           | ✓ Within SLA                    |
| Timestamp Column | `created_at`                    |
| Last Update      | January 15, 2024 at 2:15 PM UTC |
| Data Age         | 45 minutes                      |
| SLA Threshold    | 2 hours                         |
| Progress         | 37% of SLA (45 min / 2 hr)      |

**Freshness History:** All checks in the last 7 days were within SLA.

## Troubleshooting

<AccordionGroup>
  <Accordion title="False positive alerts">
    **Problem**: Freshness alerts fire when data is actually fine.

    **Solutions**:

    1. Verify timestamp column choice. Is it the right one?
    2. Loosen SLA threshold
    3. Check timezone handling (UTC vs. local)
    4. Review discovery schedule vs. SLA timing
  </Accordion>

  <Accordion title="Missed violations">
    **Problem**: Data was stale but no alert fired.

    **Solutions**:

    1. Verify freshness monitoring is enabled for the asset
    2. Check alert rule configuration
    3. Confirm discovery is running frequently enough
    4. Verify timestamp column has recent values
  </Accordion>

  <Accordion title="Wrong timestamp column">
    **Problem**: Freshness showing incorrect values.

    **Solutions**:

    1. Review column choice
    2. For tables with updates, use `updated_at` not `created_at`
    3. For ETL tables, use the load timestamp column
    4. Ensure column is always populated (no NULLs)
  </Accordion>

  <Accordion title="Timezone issues">
    **Problem**: Freshness calculations seem off by hours.

    **Solutions**:

    1. Check timestamp column timezone
    2. AnomalyArmor normalizes to UTC
    3. Ensure consistent timezone handling in your ETL
  </Accordion>
</AccordionGroup>

## Best Practices

### 1. Start with Critical Tables

Focus monitoring on:

* Revenue-impacting tables
* Customer-facing dashboard sources
* Compliance-required data

### 2. Align with Business Needs

Ask: "When would stale data cause a problem?"

| Scenario               | Acceptable Staleness | SLA      |
| ---------------------- | -------------------- | -------- |
| Real-time dashboard    | Minutes              | 15 min   |
| Daily executive report | Hours                | Same-day |
| Monthly compliance     | Days                 | 1 week   |

### 3. Coordinate with ETL Schedules

| Phase               | Time                    |
| ------------------- | ----------------------- |
| **ETL Schedule**    | 2:00 AM daily           |
| **Processing time** | \~30 minutes            |
| **Data available**  | \~2:30 AM               |
| **SLA**             | 4:00 AM (90 min buffer) |

### 4. Use Warning Thresholds

Configure two levels:

1. **Warning**: "Heads up, getting stale" → Slack
2. **Violation**: "SLA breached" → PagerDuty

## Common Questions

### Which timestamp column should I pick?

Pick the column that updates when new data arrives. Use `created_at` for append-only tables like events and logs, `updated_at` for tables that get updated in place, and the load timestamp for ETL destination tables. Avoid columns that can be NULL.

### What's a sensible starting SLA?

Start lenient, around double the expected cadence. If data is expected hourly, set a 2-hour SLA. It's easier to tighten thresholds than to fight alert fatigue from SLAs set too tight on day one.

### How do I handle tables that don't update on weekends?

Either extend the SLA to cover the weekend (for example, 72 hours), set different thresholds for weekend days, or pause the freshness rule during known quiet periods. Choose based on how critical the data is. For row count and metric monitors on the same table, [operating-period awareness](/data-quality/metrics#operating-period-awareness) handles weekends automatically by baselining active and dormant periods separately.

### Why does the freshness calculation look off by several hours?

Almost always a timezone issue. AnomalyArmor normalizes to UTC, so a timestamp column stored in local time will appear skewed. Confirm your ETL writes timestamps consistently and check the column's declared timezone.

### What's the difference between a warning and a violation?

A **warning** fires before the SLA is breached (often at 80% of the threshold) as an early heads-up. A **violation** fires once the data is officially past SLA. Route them separately, warnings to Slack, violations to PagerDuty.

### Do I need freshness monitoring on every table?

No. Focus on tables that power dashboards, revenue reporting, customer-facing products, or compliance processes. Reference tables and historical archives usually don't need freshness checks.

## What's Next

<CardGroup cols={2}>
  <Card title="Set Up Metrics" icon="chart-line" href="/data-quality/metrics">
    Track row counts, null percentages, and detect anomalies
  </Card>

  <Card title="Configure Alerts" icon="bell" href="/alerts/alert-rules">
    Get notified when freshness SLAs are violated
  </Card>

  <Card title="Report Badges" icon="shield-check" href="/data-quality/report-badges">
    Embed freshness status in dashboards
  </Card>

  <Card title="Best Practices" icon="lightbulb" href="/alerts/best-practices">
    Reduce alert fatigue
  </Card>
</CardGroup>