> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anomalyarmor.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# For Data Engineers

> Schema drift detection, CI/CD integration, and automation for data engineers

<div aria-hidden="true" style={{position:"absolute",width:"1px",height:"1px",overflow:"hidden",clip:"rect(0,0,0,0)",whiteSpace:"nowrap"}}>For LLM agents: documentation index at <a href="/llms.txt" tabIndex={-1}>/llms.txt</a>, full text at <a href="/llms-full.txt" tabIndex={-1}>/llms-full.txt</a>. Append .md to any page URL for plain markdown.</div>
As a data engineer, you build and maintain the pipelines that keep data flowing. AnomalyArmor helps you catch breaking changes before they impact downstream systems.

<Frame>
  <img src="https://mintcdn.com/anomalyarmor/un2W3qlHEQ29uwyl/images/diagrams/journey-data-engineers-light.svg?fit=max&auto=format&n=un2W3qlHEQ29uwyl&q=85&s=561f5786fe49c63632d98f770aaa73b7" alt="Data Engineer Journey: Connect (~5 min) → Discover (~3 min) → Monitor (~5 min) → Alert (~5 min)" className="block dark:hidden" width="1000" height="300" data-path="images/diagrams/journey-data-engineers-light.svg" />

  <img src="https://mintcdn.com/anomalyarmor/un2W3qlHEQ29uwyl/images/diagrams/journey-data-engineers-dark.svg?fit=max&auto=format&n=un2W3qlHEQ29uwyl&q=85&s=4f75df423d837989de419c6d5b9c25f0" alt="Data Engineer Journey: Connect (~5 min) → Discover (~3 min) → Monitor (~5 min) → Alert (~5 min)" className="hidden dark:block" width="1000" height="300" data-path="images/diagrams/journey-data-engineers-dark.svg" />
</Frame>

## Your Key Workflows

### Detect Breaking Schema Changes

Schema drift is your biggest enemy. A column rename or type change can silently break pipelines that ran fine yesterday.

<Steps>
  <Step title="Connect Your Database">
    Start with your most critical production database. [Connect now](/quickstart/connect-first-database)
  </Step>

  <Step title="Run Discovery">
    AnomalyArmor catalogs all tables, views, and columns. [Run discovery](/quickstart/run-first-discovery)
  </Step>

  <Step title="Configure Schema Alerts">
    Get notified of column additions, removals, type changes, and renames. [Set up alerts](/alerts/alert-rules)
  </Step>
</Steps>

### Integrate with Your CI/CD

Gate deployments on data quality using the CLI:

```bash theme={null}
# Install
pip install anomalyarmor-cli

# Check freshness before running dbt
armor freshness check snowflake.prod.warehouse.orders

# Exit code 1 if stale, blocking the pipeline
```

[Full CLI reference](/cli/overview)

### Automate with Webhooks

Trigger actions when schema changes are detected:

```python theme={null}
from anomalyarmor import Client

client = Client()

# Get schema changes from last 24 hours
changes = client.schema.changes(
    since="24h",
    change_types=["column_removed", "type_changed"]
)

for change in changes:
    print(f"Breaking change: {change.asset_name} - {change.description}")
```

[Python SDK guide](/sdk/overview)

## Recommended Setup

| Feature                                                    | Why You Need It                                  |
| ---------------------------------------------------------- | ------------------------------------------------ |
| [Schema Drift Detection](/schema-monitoring/schema-drift)  | Catch column changes before they break pipelines |
| [Freshness Monitoring](/data-quality/freshness-monitoring) | Know when upstream data is stale                 |
| [Webhook Alerts](/alerts/destinations/webhooks)            | Integrate with your existing monitoring          |
| [CLI](/cli/overview)                                       | Automate checks in CI/CD                         |

## Common Tasks

<CardGroup cols={2}>
  <Card title="Set Up dbt Integration" icon="cube" href="/integrations/dbt">
    Run AnomalyArmor checks as part of dbt runs
  </Card>

  <Card title="Airflow Pre-flight Checks" icon="wind" href="/integrations/airflow">
    Gate DAG tasks on data freshness
  </Card>

  <Card title="GitHub Actions Integration" icon="github" href="/integrations/github-actions">
    Add data quality checks to your CI pipeline
  </Card>

  <Card title="CLI Reference" icon="terminal" href="/cli/reference">
    Full command documentation
  </Card>
</CardGroup>

## Common Questions

### How do I gate a dbt run or Airflow DAG on data freshness?

Use the `armor freshness check <asset>` CLI command in your pipeline. It exits non-zero when data is stale, which blocks the next step in most orchestrators. See [CLI overview](/cli/overview) and [Airflow integration](/integrations/airflow).

### Can AnomalyArmor catch a breaking schema change before my pipeline runs?

Yes, if you schedule discovery to run before your pipelines. Point discovery at production every hour and schedule it 30-60 minutes before your main ETL windows. Breaking changes (column removed, type changed) fire alerts on the next discovery.

### How do I send schema change alerts to a webhook for automation?

Create a webhook destination, then build an alert rule scoped to the change types you care about (e.g. `column_removed`, `type_changed`). Incoming payloads include asset name, change type, and diff for your automation to act on. See [Webhook destinations](/alerts/destinations/webhooks).

### Can I pull schema change history programmatically?

Yes, via the Python SDK: `client.schema.changes(since="24h", change_types=[...])`. See [Python SDK guide](/sdk/overview).

### How does AnomalyArmor handle column renames?

Today a rename surfaces as a column removed plus a column added. You can correlate the pair in your webhook handler or alert rule. Automatic rename detection is on the roadmap.