Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.anomalyarmor.ai/llms.txt

Use this file to discover all available pages before exploring further.

The Validity API enables programmatic management of data validity rules. Use it to enforce data quality constraints, detect invalid records, and integrate validation into your pipelines.

Endpoints

MethodEndpointDescription
GET/api/v1/sdk/validity/{asset_id}/summaryGet validity summary for an asset
GET/api/v1/sdk/validity/{asset_id}List validity rules for an asset
GET/api/v1/sdk/validity/{asset_id}/{rule_id}Get validity rule details
POST/api/v1/sdk/validity/{asset_id}Create a new validity rule
PATCH/api/v1/sdk/validity/{asset_id}/{rule_id}Update a validity rule
DELETE/api/v1/sdk/validity/{asset_id}/{rule_id}Delete a validity rule
POST/api/v1/sdk/validity/{asset_id}/{rule_id}/checkTrigger validity check
GET/api/v1/sdk/validity/{asset_id}/{rule_id}/resultsList check results

Rule Types

TypeDescriptionConfiguration
NOT_NULLColumn must not contain null valuesNone
UNIQUEColumn values must be uniqueNone
REGEXValues must match a regex patternrule_config.pattern
RANGENumeric values must be within rangerule_config.min, rule_config.max
ENUMValues must be in allowed setrule_config.allowed_values
DATE_FORMATValues must match date formatrule_config.format
CUSTOM_SQLCustom SQL expressionrule_config.sql_expression

Get Validity Summary

GET /api/v1/sdk/validity/{asset_id}/summary
Returns aggregate validity statistics for an asset.
curl -H "Authorization: Bearer aa_live_xxx" \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000/summary"

Response

{
  "data": {
    "total_rules": 12,
    "passing": 10,
    "failing": 1,
    "error": 1
  }
}

List Validity Rules

GET /api/v1/sdk/validity/{asset_id}

Query Parameters

ParameterTypeDefaultDescription
rule_typestring-Filter by type (e.g., NOT_NULL, REGEX)
is_activeboolean-Filter by active status
limitinteger50Max results (max: 100)
offsetinteger0Results to skip
curl -H "Authorization: Bearer aa_live_xxx" \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000?rule_type=NOT_NULL"

Response

{
  "data": {
    "items": [
      {
        "id": 123,
        "uuid": "v_550e8400-e29b-41d4-a716-446655440001",
        "table_path": "snowflake.prod.warehouse.orders",
        "column_name": "customer_email",
        "rule_type": "NOT_NULL",
        "name": "Customer Email Required",
        "severity": "critical",
        "is_active": true,
        "check_interval": "daily",
        "created_at": "2024-12-01T10:00:00Z"
      }
    ]
  },
  "pagination": {
    "total": 12,
    "limit": 50,
    "offset": 0,
    "has_more": false
  }
}

Get Validity Rule Details

GET /api/v1/sdk/validity/{asset_id}/{rule_id}
curl -H "Authorization: Bearer aa_live_xxx" \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000/v_550e8400-e29b-41d4-a716-446655440001"

Response

{
  "data": {
    "id": 123,
    "uuid": "v_550e8400-e29b-41d4-a716-446655440001",
    "table_path": "snowflake.prod.warehouse.orders",
    "column_name": "customer_email",
    "rule_type": "NOT_NULL",
    "rule_config": null,
    "name": "Customer Email Required",
    "description": "Email address must not be null for valid orders",
    "severity": "critical",
    "is_active": true,
    "alert_threshold_percent": 1.0,
    "treat_null_as_valid": false,
    "check_interval": "daily"
  }
}

Create Validity Rule

POST /api/v1/sdk/validity/{asset_id}
Requires read-write or admin scope.

Request Body

FieldTypeRequiredDescription
rule_typestringYesRule type (see table above)
table_pathstringYesFull table path (catalog.schema.table)
column_namestringFor column rulesColumn name
rule_configobjectFor some typesRule-specific configuration
namestringNoHuman-readable rule name
descriptionstringNoRule description
severitystringNoinfo, warning, critical (default: warning)
error_messagestringNoCustom error message
alert_threshold_percentfloatNoAlert when invalid % exceeds this
treat_null_as_validbooleanNoWhether nulls pass (default: false)
check_intervalstringNohourly, daily, weekly (default: daily)

Examples

curl -X POST -H "Authorization: Bearer aa_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "rule_type": "NOT_NULL",
    "table_path": "snowflake.prod.warehouse.orders",
    "column_name": "customer_email",
    "name": "Customer Email Required",
    "severity": "critical"
  }' \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000"

Response

{
  "data": {
    "id": 124,
    "uuid": "v_550e8400-e29b-41d4-a716-446655440002",
    "table_path": "snowflake.prod.warehouse.orders",
    "column_name": "customer_email",
    "rule_type": "NOT_NULL",
    "name": "Customer Email Required",
    "severity": "critical",
    "is_active": true,
    "check_interval": "daily",
    "created_at": "2024-12-04T10:30:00Z"
  }
}

Update Validity Rule

PATCH /api/v1/sdk/validity/{asset_id}/{rule_id}
Requires read-write or admin scope.

Request Body

FieldTypeDescription
is_activebooleanWhether rule is active
namestringRule name
descriptionstringRule description
severitystringSeverity level
alert_threshold_percentfloatAlert threshold
treat_null_as_validbooleanNull handling
check_intervalstringCheck interval
curl -X PATCH -H "Authorization: Bearer aa_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{"severity": "critical", "alert_threshold_percent": 0.5}' \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000/v_550e8400-e29b-41d4-a716-446655440001"

Delete Validity Rule

DELETE /api/v1/sdk/validity/{asset_id}/{rule_id}
Requires read-write or admin scope.
curl -X DELETE -H "Authorization: Bearer aa_live_xxx" \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000/v_550e8400-e29b-41d4-a716-446655440001"

Trigger Validity Check

POST /api/v1/sdk/validity/{asset_id}/{rule_id}/check
Requires read-write or admin scope.

Request Body

FieldTypeDefaultDescription
sample_limitinteger10Max invalid samples to collect
curl -X POST -H "Authorization: Bearer aa_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{"sample_limit": 20}' \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000/v_550e8400-e29b-41d4-a716-446655440001/check"

Response

{
  "data": {
    "id": 456,
    "validity_rule_id": 123,
    "status": "fail",
    "total_rows": 10000,
    "invalid_count": 25,
    "invalid_percent": 0.25,
    "invalid_samples": {
      "samples": [
        {"row_id": 1001, "value": null},
        {"row_id": 1042, "value": null}
      ]
    },
    "execution_duration_ms": 1250,
    "checked_at": "2024-12-04T10:35:00Z"
  }
}

List Check Results

GET /api/v1/sdk/validity/{asset_id}/{rule_id}/results

Query Parameters

ParameterTypeDefaultDescription
limitinteger100Max results
offsetinteger0Results to skip
curl -H "Authorization: Bearer aa_live_xxx" \
  "https://api.anomalyarmor.ai/api/v1/sdk/validity/550e8400-e29b-41d4-a716-446655440000/v_550e8400-e29b-41d4-a716-446655440001/results?limit=30"

Response

{
  "data": {
    "items": [
      {
        "id": 456,
        "validity_rule_id": 123,
        "status": "fail",
        "total_rows": 10000,
        "invalid_count": 25,
        "invalid_percent": 0.25,
        "execution_duration_ms": 1250,
        "checked_at": "2024-12-04T10:35:00Z"
      },
      {
        "id": 455,
        "validity_rule_id": 123,
        "status": "pass",
        "total_rows": 9975,
        "invalid_count": 0,
        "invalid_percent": 0.0,
        "execution_duration_ms": 1100,
        "checked_at": "2024-12-03T10:35:00Z"
      }
    ]
  },
  "pagination": {
    "total": 60,
    "limit": 30,
    "offset": 0,
    "has_more": true
  }
}

Use Case: Validate Email Format

Ensure all customer emails match a valid format:
from anomalyarmor import Client

client = Client()
asset_id = "550e8400-e29b-41d4-a716-446655440000"

# Create email validation rule
rule = client.validity.create(
    asset_id,
    rule_type="REGEX",
    table_path="snowflake.prod.warehouse.customers",
    column_name="email",
    rule_config={"pattern": r"^[\w.-]+@[\w.-]+\.\w{2,}$"},
    name="Valid Email Format",
    description="Validates email addresses match standard format",
    severity="warning",
    alert_threshold_percent=1.0,  # Alert if > 1% invalid
)

# Run initial check
result = client.validity.check(asset_id, rule.uuid, sample_limit=20)

if result.status == "fail":
    print(f"Warning: {result.invalid_count} invalid emails found")
    print(f"Invalid rate: {result.invalid_percent:.2f}%")
    for sample in (result.invalid_samples or {}).get("samples", [])[:5]:
        print(f"  - {sample}")
else:
    print("All emails are valid!")

Error Responses

Rule Not Found (404)

{
  "error": {
    "code": "RULE_NOT_FOUND",
    "message": "Validity rule not found",
    "details": {"rule_id": "v_invalid-uuid"}
  }
}

Invalid Rule Configuration (400)

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid regex pattern in rule_config",
    "details": {"field": "rule_config.pattern", "error": "Invalid regex syntax"}
  }
}

Forbidden (403)

{
  "error": {
    "code": "FORBIDDEN",
    "message": "Insufficient permissions. Required scope: read-write",
    "details": {"current_scope": "read-only", "required_scope": "read-write"}
  }
}

Common Questions

Which validity rule type should I pick?

Use NOT_NULL for required columns, UNIQUE for primary-key-like invariants, REGEX for string formats (emails, IDs), RANGE for numeric bounds, ENUM for finite allowed sets, DATE_FORMAT for date strings, and CUSTOM_SQL when no built-in type fits. Each type’s required rule_config is listed in the Rule Types table at the top of this page.

How do I inspect rows that failed a validity rule?

Every check response includes invalid_samples.samples, up to sample_limit rows (default 10, configurable per check). Bump sample_limit up to 100 when debugging a broken ingest. The response also returns invalid_count and invalid_percent so you can report a failure rate even when individual samples aren’t needed.

What does the alert_threshold_percent field control?

alert_threshold_percent is the invalid-row percentage that flips a check result from pass to fail. Set it to 0 if any single invalid row should page you. Use higher values (e.g. 1.0) on rules where a small amount of invalid data is tolerated and you only want to catch systemic regressions.

How is treat_null_as_valid different from using NOT_NULL?

treat_null_as_valid governs how a non-NULL-type rule (REGEX, RANGE, ENUM, etc.) handles NULLs. When true, NULL rows are skipped; when false, NULLs count as invalid. Combine with a separate NOT_NULL rule when you need to enforce both non-null and format at once - they surface as two distinct checks you can alert on independently.