Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.anomalyarmor.ai/llms.txt

Use this file to discover all available pages before exploring further.

AnomalyArmor can visualize how your data flows from source to destination, but it needs lineage data to work with. This guide covers three ways to get lineage into AnomalyArmor:
  1. Upload a dbt manifest.json file (most common)
  2. Sync from dbt Cloud automatically
  3. Define lineage manually via the API

​
Option 1: Upload a dbt manifest.json

If you use dbt, the fastest way to populate lineage is uploading your manifest.json file. This file contains your full DAG, including all models, sources, seeds, and their dependencies.

​
Generate the manifest

Run one of these dbt commands to produce target/manifest.json: 
# Either of these generates a manifest.json in target/
dbt parse      # Fastest, parses without compiling
dbt compile    # Compiles SQL, also generates manifest
dbt run        # Full run, also generates manifest

​
Upload via the API

curl -X POST \
  "https://api.anomalyarmor.ai/api/v1/assets/{asset_id}/lineage/upload" \
  -H "Authorization: Bearer aa_live_xxx" \
  -F "file=@target/manifest.json" \
  -F "sync_to_catalog=true"

​
What gets imported

AnomalyArmor parses the nodes and parent_map from your manifest to extract:
  • Models (transformations in your dbt project)
  • Sources (raw tables dbt reads from)
  • Seeds (CSV files loaded by dbt)
  • Parent-child relationships between all of the above
The sync_to_catalog parameter (default: true) also triggers a asset discovery job so your dbt models appear as assets in the catalog.

​
Response

{
  "data": {
    "asset_id": "550e8400-e29b-41d4-a716-446655440000",
    "sync_stats": {
      "nodes_created": 42,
      "nodes_updated": 8,
      "edges_created": 67,
      "edges_updated": 3
    },
    "manifest_metadata": {
      "generated_at": "2025-03-15T10:30:00Z",
      "dbt_version": "1.7.4",
      "project_name": "my_analytics"
    },
    "catalog_sync_job_id": "job-uuid-here"
  }
}

​
Automate uploads in CI/CD

Add a manifest upload step after your dbt run completes: 
# .github/workflows/dbt.yml
jobs:
  dbt-run:
    steps:
      - name: Run dbt
        run: dbt run

      - name: Upload lineage to AnomalyArmor
        run: |
          curl -X POST \
            "https://api.anomalyarmor.ai/api/v1/assets/$ASSET_ID/lineage/upload" \
            -H "Authorization: Bearer $ARMOR_API_KEY" \
            -F "file=@target/manifest.json"
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
          ASSET_ID: ${{ vars.ARMOR_ASSET_ID }}

​
Option 2: Sync from dbt Cloud

If you use dbt Cloud, AnomalyArmor can fetch the manifest directly from your dbt Cloud account. No file upload needed. 
curl -X POST \
  "https://api.anomalyarmor.ai/api/v1/assets/{asset_id}/lineage/dbt-cloud/sync" \
  -H "Authorization: Bearer aa_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "account_id": "12345",
    "api_token": "dbtc_your_token_here",
    "job_id": "67890"
  }'

​
Finding your dbt Cloud credentials

ParameterWhere to find it
account_iddbt Cloud URL: cloud.getdbt.com/deploy/**12345**/...
api_tokendbt Cloud > Account Settings > API Access > Service tokens
job_iddbt Cloud > Jobs > select your job > ID in the URL
Use a service token with at least the “Read artifacts” permission. Personal tokens work but are tied to individual users.

​
Option 3: Define lineage manually

For data sources that are not managed by dbt, you can define lineage nodes and edges directly via the API.

​
Create a lineage node

curl -X POST \
  "https://api.anomalyarmor.ai/api/v1/assets/{asset_id}/lineage/nodes" \
  -H "Authorization: Bearer aa_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "unique_id": "source.crm.customers",
    "name": "customers",
    "resource_type": "source",
    "schema": "crm",
    "database": "production"
  }'

​
Create a lineage edge

curl -X POST \
  "https://api.anomalyarmor.ai/api/v1/assets/{asset_id}/lineage/edges" \
  -H "Authorization: Bearer aa_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "parent_unique_id": "source.crm.customers",
    "child_unique_id": "model.analytics.dim_customers",
    "relationship_type": "derives_from"
  }'

​
Updating lineage

When your dbt project changes, re-upload the manifest. AnomalyArmor handles updates intelligently:
  • New nodes and edges are created
  • Existing nodes are updated with new metadata
  • Relationships that no longer exist are removed
For a clean reset, delete all lineage from a source first: 
curl -X DELETE \
  "https://api.anomalyarmor.ai/api/v1/assets/{asset_id}/lineage/source/dbt" \
  -H "Authorization: Bearer aa_live_xxx"

​
Limits

ConstraintValue
Max manifest file size50 MB
File formatJSON (UTF-8 encoded)
Required manifest keysnodes, parent_map

​
Common Questions

​
Where do I find my dbt manifest.json file?

It lives under target/manifest.json after you run dbt parse, dbt compile, or dbt run. dbt parse is the fastest option because it does not compile SQL or hit your warehouse. See Generate the manifest.

​
Do I need dbt to upload lineage?

No. Option 3 lets you define lineage manually via the API by creating nodes and edges directly. This is how you model lineage for sources outside dbt, like Fivetran pipelines, Airflow tasks, or bespoke ETL jobs.

​
How do I sync lineage from dbt Cloud instead of uploading a file?

Call POST /api/v1/assets/{asset_id}/lineage/dbt-cloud/sync with your dbt Cloud account_id, api_token, and job_id. AnomalyArmor pulls the latest manifest from the run artifacts directly. Use a service token with “Read artifacts” permission, not a personal token. See Option 2.

​
How often should I re-upload the manifest?

Re-upload any time your dbt project changes, typically as a CI step after dbt run in your deploy pipeline. AnomalyArmor diffs the new manifest against the existing graph: new nodes and edges are created, updated ones are updated, and relationships that no longer exist are removed.

​
What is the max manifest size AnomalyArmor accepts?

50 MB, UTF-8 encoded JSON, with nodes and parent_map keys required. Most dbt projects fit well under this; if you’re hitting it, check whether you’re accidentally uploading a run_results.json instead of manifest.json.

​
Next steps

Query Lineage

Explore upstream and downstream dependencies

Impact Analysis

Check downstream impact before schema changes

dbt Integration

Add quality gates to your dbt workflows

AI Agent: Lineage

Ask natural language questions about data flow