## The Solution
With AnomalyArmor, you'll know about schema changes before your pipelines run:
## Setup Guide
### Step 1: Connect Your Source Databases
Connect the databases that your pipelines read , not just your warehouse.
**Common sources to monitor:**
* Production application databases (the ones your dbt reads from)
* Third-party data sources
* Shared data lakes
For each source, follow the [connection guide](/data-sources/overview).
### Step 2: Schedule Frequent Discovery
For pipeline-critical databases, run discovery frequently:
| Database Type | Recommended Schedule | Why |
| --------------------- | -------------------- | -------------------------- |
| Application databases | Hourly | Changes can happen anytime |
| Shared warehouses | Every 6 hours | Less frequent changes |
| Third-party sources | Daily | Usually stable |
Configure in: **Data Sources → \[Your Connection] → Settings → Discovery Schedule**
### Step 3: Create Breaking Change Alerts
Set up alerts specifically for changes that break pipelines:
**Rule: Breaking Schema Changes (Production)**
| Field | Value |
| ---------------- | -------------------------------------------------------- |
| **Event** | Schema Change Detected |
| **Data Source** | `production-app-db` |
| **Schema** | `public` |
| **Assets** | All (or list specific tables) |
| **Change Type** | Column Removed, Table Removed, Type Changed |
| **Destinations** | Slack `#data-engineering`, Email `data-team@company.com` |
### Step 4: Time Alerts Before Pipeline Runs
If your dbt runs at 3 AM, schedule discovery at 2 AM:
## Advanced: Pre-dbt Validation
### Option 1: Webhook Integration
Use webhooks to fail your pipeline early if breaking changes are detected:
1. Set up a webhook destination in AnomalyArmor
2. Point it at a validation endpoint in your orchestrator
3. If webhook fires, block the dbt run
4. **AnomalyArmor Alert** fires on schema change
5. **Webhook** sent to Airflow/Dagster
6. **Set flag**: `schema_changes_detected = true`
7. **dbt task** checks flag before running
8. **If flag = true**: Fail fast with meaningful error
### Option 2: Discovery Schedule Alignment
Align discovery with your orchestration schedule:
```python theme={null}
# In your Airflow DAG
discovery_check = SimpleHttpOperator(
task_id='check_for_schema_changes',
http_conn_id='anomalyarmor',
endpoint='/api/v1/discoveries/latest',
method='GET',
)
run_dbt = BashOperator(
task_id='run_dbt',
bash_command='dbt run',
)
discovery_check >> run_dbt
```
## What to Do When Alerts Fire
### Immediate Actions
1. **Acknowledge the alert**: Let your team know you're investigating
2. **Check the change details**: View in AnomalyArmor: what changed, when, and on which asset
3. **Assess impact**: Which models/dashboards use this table?
### If the Change is Breaking
1. **Pause affected pipelines** (if possible before they run)
2. **Update your dbt models** to handle the change
3. **Test locally** with the new schema
4. **Deploy the fix** before the next scheduled run
### If the Change is Expected
1. **Document it**: Note in AnomalyArmor or your team wiki
2. **Update downstream**: Ensure all dependents are updated
3. **Consider communication**: Should you announce to stakeholders?
## Model Dependency Mapping
Know which models depend on which tables:
**Source Table: `production.orders`**
* `stg_orders` (staging model)
* `int_orders_enriched` (intermediate)
* `fct_orders` (fact table)
* monthly\_revenue (dashboard)
* customer\_lifetime\_value (analytics)
* `rpt_daily_orders` (report)
* `dim_order_status` (dimension)
When `production.orders` changes, all of these are potentially impacted.