databricks

auto-cdc

@databricks/auto-cdc
databricks
280
131 forks
Updated 1/18/2026
View on GitHub

Apply Change Data Capture (CDC) with apply_changes API in Spark Declarative Pipelines. Use when user needs to process CDC feeds from databases, handle upserts/deletes, maintain slowly changing dimensions (SCD Type 1 and Type 2), synchronize data from operational databases, or process merge operations.

Installation

$skills install @databricks/auto-cdc
Claude Code
Cursor
Copilot
Codex
Antigravity

Details

Repositorydatabricks/cli
Pathexperimental/aitools/lib/skills/pipelines/auto-cdc/SKILL.md
Branchmain
Scoped Name@databricks/auto-cdc

Usage

After installing, this skill will be available to your AI coding assistant.

Verify installation:

skills list

Skill Instructions

name: auto-cdc description: Apply Change Data Capture (CDC) with apply_changes API in Spark Declarative Pipelines. Use when user needs to process CDC feeds from databases, handle upserts/deletes, maintain slowly changing dimensions (SCD Type 1 and Type 2), synchronize data from operational databases, or process merge operations.

Auto CDC (apply_changes) in Spark Declarative Pipelines

The apply_changes API enables processing Change Data Capture (CDC) feeds to automatically handle inserts, updates, and deletes in target tables.

Key Concepts

Auto CDC in Spark Declarative Pipelines:

  • Automatically processes CDC operations (INSERT, UPDATE, DELETE)
  • Supports SCD Type 1 (update in place) and Type 2 (historical tracking)
  • Handles ordering of changes via sequence columns
  • Deduplicates CDC records

Language-Specific Implementations

For detailed implementation guides:

Note: The API is also known as applyChanges in some contexts.