---
title: "Declarative AI Infrastructure: Define Pipelines, Not Plumbing"
description: "Replace thousands of lines of orchestration code with declarative computed columns. Pixeltable handles execution, dependencies, caching, and incremental updates automatically."
keywords:
  - declarative ai data infrastructure
  - declarative data processing
  - ai pipeline automation
  - declarative ai pipelines
  - reduce infrastructure code
complexity: "beginner"
estimated_time: "15 min"
url: "https://pixeltable.com/use-cases/declarative-ai-data-infrastructure"
---

# Declarative AI Infrastructure: Define Pipelines, Not Plumbing

Replace thousands of lines of orchestration code with declarative computed columns. Pixeltable handles execution, dependencies, caching, and incremental updates automatically.

## Prerequisites

- Basic Python programming
- Understanding of AI workflows

## The Problem

Traditional AI infrastructure requires imperative orchestration: loading data, managing state, scheduling jobs, handling retries, tracking dependencies. This code is brittle, hard to test, and grows faster than the actual application logic.

## The Solution

Pixeltable's declarative model lets you define what to compute, not how. Computed columns express your pipeline logic. The engine handles execution order, dependency tracking, incremental updates, caching, and error recovery.

## Implementation

### Declarative Columns

Express your entire pipeline as computed columns, no orchestrator needed.

```python
import pixeltable as pxt
from pixeltable.functions import openai

# Define what you want, not how to get it
images = pxt.create_table('app.images', {
    'image': pxt.Image,
    'title': pxt.String,
})

# AI inference as a computed column
images.add_computed_column(
    description=openai.chat_completions(
        model='gpt-4o-mini',
        messages=[{
            'role': 'user',
            'content': [images.image, 'Describe this image.']
        }]
    ).choices[0].message.content
)

# Dependent columns run in correct order automatically
images.add_computed_column(
    tags=openai.chat_completions(
        model='gpt-4o-mini',
        messages=[{
            'role': 'user',
            'content': images.description.apply(
                lambda d: f"Extract 5 tags from: {d}"
            )
        }]
    ).choices[0].message.content
)
```

Pixeltable tracks dependencies between columns and executes them in the right order. No DAG configuration files.


### Incremental Processing

Only changed data triggers recomputation, saving compute and money.

```python
# Insert 1000 images: all are processed
images.insert([...image_batch])  # Processes 1000 images

# Add 1 more: only the new one is processed
images.insert([{'image': 'new.jpg', 'title': 'Latest'}])
# ✅ Only 1 image processed, not 1001

# Add a new computed column: processes all existing rows
images.add_computed_column(
    sentiment=openai.chat_completions(
        model='gpt-4o-mini',
        messages=[{
            'role': 'user',
            'content': images.description.apply(
                lambda d: f"Sentiment of: {d}"
            )
        }]
    ).choices[0].message.content
)
# Processes 1001 rows for the new column only
# description and tags are NOT recomputed
```

Incremental processing saves up to 95% of compute costs on iterative development workflows.


## Benefits

- 90% reduction in infrastructure code
- Automatic dependency tracking and execution ordering
- Incremental updates: only recompute what changed
- Built-in caching, error handling, and retry logic
- Full data lineage and versioning

## Use Cases

- ML model training data pipelines
- Automated data enrichment workflows
- AI application backends
- Research experimentation platforms

## Performance


| Metric | Value | Description |

| --- | --- | --- |

| Code Reduction | 90% | Less infrastructure code |

| Dev Speed | 10x faster | From concept to production |

## Requirements

- Python 3.9+
- API keys for AI providers

## Resources

- [Declarative AI Pipelines: Open Standard](https://pixeltable.com/blog/declarative-ai-pipelines-open-standard) - Why we built Pixeltable open source
- [Dependency Graph Magic](https://pixeltable.com/blog/dependency-graph-magic-computed-columns) - How computed columns maintain data consistency