---
title: "AI Functions vs. Pipelines: The Pixeltable Declarative Approach"
date: "2024-10-28"
author: "Pierre Brunelle"
tags:
  - AI Functions
  - Pixeltable
  - Declarative AI
description: "Move beyond traditional AI pipelines. Pixeltable's AI Functions offer a declarative, flexible, efficient alternative for complex workloads. Learn more."
url: "https://pixeltable.com/blog/ai-functions-vs-pipelines"
---

# AI Functions vs. Pipelines: The Pixeltable Declarative Approach

## The Bottleneck: Wrestling with Traditional Data Pipelines

 
In today's AI landscape, developers often find themselves spending more time battling complex data pipelines than building innovative AI solutions. As companies race to integrate [multimodal AI](/blog/building-multimodal-apps) capabilities (from LLMs processing text to computer vision models analyzing images and video), the complexity of managing data transformations, model inference, and versioning has reached a breaking point. This is where a new paradigm emerges: **AI Functions**.

 
## The Problem with Pipelines in the AI Era

 
Traditional data pipelines, often built with sequential scripts and disparate tools, weren't designed for the dynamic and iterative nature of modern AI development. They typically struggle with:

 

 - **Complex Data Types:** Efficiently managing video frames, audio clips, large documents, embeddings, and structured metadata requires intricate custom code and specialized storage.

 - **Costly Recomputations:** Minor changes to source data, processing logic, or a model version often trigger full, expensive dataset reprocessing, hindering rapid iteration.

 - **Lost Lineage:** Tracing a model's prediction or an LLM's response back through the pipeline to the specific source data and processing steps used becomes a difficult, time-consuming detective mission.

 - **Development-Production Gap:** Code and workflows developed locally often require significant refactoring, optimization, and infrastructure changes to run reliably and efficiently in production.

 

 
These challenges force teams to invest heavily in building and maintaining brittle infrastructure, diverting focus from their primary goal: AI model development and application innovation.

 
## Introducing AI Functions: A Declarative Revolution

 
AI Functions represent a fundamental shift. Instead of imperatively defining *how* data should flow through a series of steps, developers [declaratively define](/blog/declarative-multimodal-incremental) *what* computations should happen using functions that operate directly on data structures like tables and views. Pixeltable is built around this concept.

 
Think of AI Functions as operations (like feature extraction, model inference, data transformation) applied as computed columns or transformations within a data management system that understands AI workloads.

 
## AI Functions in Action: Pixeltable Example ([Video Processing](/blog/video-analysis-guide))

 
Consider processing videos to detect objects in frames. Compare the traditional approach with Pixeltable's AI Function approach:

 
**Traditional Pipeline (Conceptual Python):**

 
```python

# Requires manual implementation of:
# - Video loading & frame extraction loop
# - Frame storage/caching logic
# - Error handling for processing individual frames
# - Batching for model inference
# - Storing/linking results

def process_videos_manually(video_paths):
 all_results = []
 # Assume existence of these helper functions needing complex implementation
 # from frame_extractor import extract_frames
 # from model_loader import load_detection_model, run_inference
 # from storage import save_frame_detection

 # model = load_detection_model()

 for video_path in video_paths:
 try:
 frames_data = extract_frames(video_path, fps=1)
 for frame_num, frame_image in frames_data:
 # Manual batching might be needed here for efficiency
 detections = run_inference(model, frame_image)
 if detections:
 result = {
 'video': video_path,
 'frame': frame_num,
 'detections': detections
 }
 save_frame_detection(result)
 all_results.append(result)
 except Exception as e:
 print(f"Error processing {video_path}: {e}")
 # Manual error handling and potential retries
 return all_results

# This is highly simplified and omits versioning, incremental updates, lineage etc.
 
```

 
**With Pixeltable AI Functions:**

 
```python

import pixeltable as pxt
from pixeltable.functions.video import frame_iterator
# Import a specific object detection function (adjust path/name as needed)
# from pixeltable.functions.vision import yolox # Or another specific detector
# Or assume it's registered e.g., via client.register_managed_function()

# Connect to Pixeltable
# client = pxt.Client()

# 1. Define the table for videos
# Use current type name: pxt.Video
videos = pxt.create_table('videos', {'video': pxt.Video})

# 2. Create a view using frame_iterator
# Pixeltable automatically extracts/caches frames incrementally
frames = pxt.create_view(
 'frames',
 videos,
 iterator=frame_iterator(video=videos.video, fps=1)
)

# 3. Apply the object detection function using add_computed_column
# Pixeltable manages execution, storage, incremental updates, and lineage.
frames.add_computed_column(detections=yolox(frames.frame))

# Example usage:
# videos.insert([{'video': 'path/to/video.mp4'}])
# results = frames.select(frames.detections).show()
 
```

 
This concise definition handles the entire workflow, including incremental updates. You can similarly apply functions for [video similarity search](/blog/video-similarity-search) using embeddings.

 
## Why This Changes Everything

 
The difference is more than just lines of code. The Pixeltable (AI Functions) approach fundamentally changes the workflow:

 

 - **Eliminates Boilerplate:** No need to manually manage frame extraction loops, data loading, storage, caching, or basic error handling.

 - **Updates Incrementally:** Pixeltable automatically processes only new or changed data. Update a video? Only its frames are reprocessed. Add a video? Only that video is processed.

 - **Maintains Lineage:** Every computed value (like detections) has a traceable lineage back to the input data (frame) and the function (pxt.functions.vision.yolox) used.

 - **Works Everywhere:** The same declarative code defines the workflow in development, staging, and production environments.

 

 
## Real-World Impact & Benefits

 
Teams adopting Pixeltable and the AI Function paradigm report significant improvements:

 

 - **Compute Cost Reduction:** Often 70% or more saved by avoiding unnecessary recomputations through incremental processing.

 - **Reduced Maintenance Time:** Up to 90% less time spent building, debugging, and maintaining complex pipeline code.

 - **Faster Iteration:** Quickly swap models or adjust parameters by changing the AI Function; Pixeltable handles the updates efficiently.

 - **Consistency & Reproducibility:** Built-in lineage and versioning ensure results are understandable and reproducible.

 - **Focus on Innovation:** Engineers spend more time on core AI logic and less on infrastructure plumbing.

 

 
## Beyond Basic Transformations: Multimodal & RAG

 
The power of AI Functions extends to complex, multi-stage, and multimodal workflows:

 
**Multimodal Processing (Video -> Audio -> Transcript -> Summary):**

 
```python

import pixeltable as pxt
# Use specific function references where possible
from pixeltable.functions.video import extract_audio
from pixeltable.functions.audio import whisper # Example transcription
from pixeltable.functions.openai import chat_completions # Example LLM call

# Use current type name: pxt.Video
content = pxt.create_table('content', {
 'video': pxt.Video,
})

# Chain AI Functions across modalities using add_computed_column
# Pixeltable handles dependencies and incremental updates automatically
content.add_computed_column(audio=extract_audio(content.video))
content.add_computed_column(transcript=whisper(content.audio))

# Define the prompt structure dynamically referencing the transcript column
messages_expr = [
 {
 "role": "user",
 # Concatenate the fixed prompt string with the transcript text column
 "content": "Summarize the following transcript:
" + content.transcript['text']
 }
]
# Apply the chat_completions function
content.add_computed_column(
 summary=chat_completions(
 messages=messages_expr, # Pass the expression that builds the messages list
 model="gpt-3.5-turbo"
 )
)
 
```

 
**[RAG Applications](/blog/production-rag-data-centric) (Document Chunking, Embedding, Indexing):**

 
```python

import pixeltable as pxt
from pixeltable.functions.document import document_splitter
# Use a specific embedding function reference
from pixeltable.functions import sentence_transformer

# Use current type name: pxt.Document
docs = pxt.create_table('knowledge_base', {'document': pxt.Document})

# View for automatic document chunking
# Updates incrementally when 'docs' table changes
chunks = pxt.create_view(
 'chunks',
 docs,
 iterator=document_splitter(document=docs.document, separators='token_limit', limit=300)
)

# Add embedding index directly using add_embedding_index
# Pixeltable manages embedding generation and index updates.
chunks.add_embedding_index(
 'text', # Column containing the text to embed
 embed=sentence_transformer.using( # Specify the embedding function
 text_feat=chunks.text,
 model_id='sentence-transformers/all-MiniLM-L12-v2' # Example model
 )
)

# Example similarity search:
# First, embed the query text separately if using sentence_transformer directly
# query_embedding = sentence_transformer.encode("User query here")
# Perform search using the pre-computed query embedding
# results = chunks.select(chunks.text).nearest(query=query_embedding, limit=5)
 
```

 
## The Future is Declarative

 
As AI becomes more deeply integrated into applications, the need for simplified, maintainable, and efficient AI workflows is paramount. AI Functions, as implemented in Pixeltable, represent more than just a technical improvement. They enable a new generation of complex AI applications that would be impractical or prohibitively expensive to build and maintain using traditional pipeline approaches.

 
## Getting Started with AI Functions in Pixeltable

 
Transitioning doesn't require an overnight switch. Teams can start incrementally:

 

 - Identify a bottleneck or a complex pipeline causing maintenance headaches.

 - Model that workflow in Pixeltable using tables, views, and AI functions (computed columns).

 - Measure the impact on development velocity, compute costs, and maintainability.

 - Gradually expand the use of Pixeltable to other workflows.

 

 
## Conclusion: Embrace the Shift

 
The rise of AI Functions marks a turning point. By abstracting the complexity of data orchestration, they allow teams to focus on building value, not infrastructure. The future of AI development is declarative, efficient, and focused on outcomes.

 
Ready to transform your AI workflows?

 

 - **[Try Pixeltable on GitHub](https://github.com/pixeltable/pixeltable)**

 - **[Check out the Getting Started Guide](/docs/getting-started)**

 - **[AI Transformations Belong in the Schema](/blog/ai-transformations-in-the-schema)**: why computed columns replace pipeline orchestration

 - **[Join our Discord Community](https://discord.gg/QPyqFYx2UN)**