---
title: "5 Surprising Truths That Will Change How You Build AI Applications"
date: "2026-01-13"
author: "Pixeltable Team"
tags:
  - AI Infrastructure
  - Best Practices
  - Data Management
  - Declarative AI
  - Multimodal AI
  - ML Engineering
description: "Discover why 80% of AI development time goes to infrastructure instead of innovation, and learn five counter-intuitive principles that can transform your approach to building multimodal AI applications."
url: "https://pixeltable.com/blog/5-surprising-truths-building-ai-applications"
---

# 5 Surprising Truths That Will Change How You Build AI Applications

The journey of building an AI application often begins with a spark of excitement. The prospect of creating intelligent systems that can analyze video, understand documents, or power sophisticated agents is a powerful motivator. But for many teams, this initial enthusiasm quickly gives way to a frustrating reality: wrestling with a tangled web of infrastructure.
 

 

 Before any real AI work can begin, you're stuck stitching together a chaotic mix of databases, object storage, vector indexes, and pipeline orchestrators. This is the **"80% tax on AI innovation"**: a phenomenon where engineering teams spend the vast majority of their time on data plumbing instead of building the valuable AI features that drive business value.
 

 

 The constant need to write custom ETL scripts, manage data synchronization between systems, and manually re-process entire datasets for minor changes becomes a significant drag on productivity and a major source of cost. What if there was a simpler path?
 

 

 By analyzing the architecture of modern AI applications, we've uncovered five counter-intuitive but powerful truths. These truths challenge common assumptions about AI infrastructure and reveal a more streamlined, efficient, and intuitive way to build.
 

 
## 1. The "80% Infrastructure Tax" is Real, and Optional

 

 The common approach to building AI infrastructure is a fragmented one. Teams typically find themselves stitching together a collection of specialized, single-purpose tools:
 

 

 - **S3 or other cloud storage** for raw files

 - **Postgres or data warehouses** for structured data

 - **Pinecone or other vector databases** for embeddings

 - **Airflow or Prefect** for orchestration

 - **Custom ETL scripts** to connect everything

 

 

 This fragmented stack is the direct cause of the 80% infrastructure tax. The counter-intuitive truth is that you can [replace this entire complex stack](/blog/unified-multimodal-ai-infrastructure-pixeltable) with a single, unified platform.
 

 

 Instead of writing 200+ lines of imperative boilerplate code just to ingest multimodal data, a [declarative approach](/blog/declarative-vs-imperative-ai-pipelines) allows you to achieve the same result (ingesting video, image, and audio into a single, unified, and queryable table) in as little as 15 lines:
 

 
```python
import pixeltable as pxt

# Create a unified multimodal table
pxt.create_dir('media_analysis', if_exists='ignore')
media = pxt.create_table('media_analysis.content', {
 'source_id': pxt.String,
 'video': pxt.Video,
 'audio': pxt.Audio,
 'image': pxt.Image,
 'document': pxt.Document,
 'metadata': pxt.Json,
})

# That's it. No S3 buckets, no separate databases, 
# no ETL pipelines to connect them.
```

 

 By adopting a unified data platform, you aren't just reducing lines of code; you are eliminating entire categories of work. You stop paying the infrastructure tax and can finally refocus your team's energy on what truly matters: [innovation and building next-generation AI features](/blog/building-multimodal-apps).
 

 
## 2. Your Database Shouldn't Be Dumb About Data

 

 In the traditional database model, tables are simple containers for scalar values. Rich media like images, videos, and audio are treated as opaque binary large objects (BLOBs). The database can store them, but it has no understanding of their content. This forces all the intelligent processing to happen in a separate application layer.
 

 

 The surprising truth is that a modern AI data platform treats the table as a **"universal interface"** where [multimodal data (images, videos, audio, and documents) are first-class citizens](/blog/traditional-tables-to-multimodal-ai). This is more than just storage; it means the data platform itself can compute on and understand the content.
 

 

 You can integrate AI model calls directly into the data layer by adding a "pluggable virtual column" using a [User-Defined Function (UDF)](/blog/python-udfs-pixeltable). This UDF can contain arbitrary logic, such as calling a vision model on an image or transcribing an audio file:
 

 
```python
import pixeltable as pxt
from pixeltable.functions.openai import chat_completions

# Create a table for competitor analysis
analysis = pxt.get_table('media_analysis.content')

# Use vision model on competitor screenshots
@pxt.udf
def build_vision_prompt(img: pxt.Image) -> list[dict]:
 return [{
 'role': 'user',
 'content': [
 {'type': 'text', 'text': 'Analyze this competitor pricing screenshot. Extract key pricing tiers and features.'},
 {'type': 'image_url', 'image_url': {'url': img.url}}
 ]
 }]

# Add AI-powered computed column
analysis.add_computed_column(
 vision_prompt=build_vision_prompt(analysis.image)
)

analysis.add_computed_column(
 competitor_analysis=chat_completions(
 model='gpt-4o',
 messages=analysis.vision_prompt
 ).choices[0].message.content
)
```

 

 This represents a major paradigm shift. The data platform is no longer a passive container for data; it's an active participant in understanding and enriching it, turning raw media into structured, queryable insights. Learn more about [integrating vision models](/blog/working-with-gemini) and [multimodal AI providers](/blog/claude-anthropic-multimodal-integration-pixeltable).
 

 
## 3. The Most Expensive Compute is the One You Run Twice

 

 A common and incredibly wasteful practice in AI development is re-processing entire datasets when a small change is made. Whether you update a model, tweak a parameter, or fix a bug in your transformation logic, the standard response is often to re-run the entire pipeline, incurring significant costs in both time and API fees.
 

 

 The impactful truth is that with [incremental computation](/blog/dependency-graph-magic-computed-columns), you can avoid this waste entirely. Consider the scenario of updating a document chunking strategy for a RAG system with 10,000 documents:
 

 
| Traditional Approach | Incremental Approach |
| --- | --- |
| Re-chunk all 10,000 documents | Update chunking parameters once |
| Re-generate all embeddings (API costs + hours) | System identifies only affected data |
| Delete old vector index | Re-computes only that portion |
| Re-upload everything | Untouched results remain cached |

 

 This is possible because the platform maintains a [dependency graph](/blog/context-graphs-decision-traces-agentic-ai) of all computations, allowing it to precisely trace the impact of any change. This isn't a minor optimization. Incremental processing can reduce reprocessing costs by **up to 70%** and eliminate hours of manual work and pipeline reruns.
 

 

 In video analysis, this approach can reduce the number of frames you need to analyze by **90% or more** by processing only [keyframes](/blog/video-keyframe-extraction-pixeltable). This shifts the focus from merely optimizing re-computation to eliminating it entirely, proving that the most expensive compute is the one you never needed to run again.
 

 
```python
# When you add new documents, only new embeddings are computed
docs_table.insert([
 {'doc_path': '/new/document.pdf'},
 {'doc_path': '/another/document.pdf'},
])

# Existing 10,000 documents? Their embeddings are untouched.
# Only the 2 new documents get processed.

# Update chunking strategy? Only affected chunks recompute.
docs_table.drop_column('chunks')
docs_table.add_computed_column(
 chunks=document_splitter(
 document=docs_table.document,
 separators='token_limit',
 limit=500 # Changed from 300
 )
)
# Pixeltable automatically recomputes only what changed
```

 
## 4. AI Agents Need a Record of Decisions, Not Just Data

 

 As we build more sophisticated AI agents, a high-level truth is emerging: the next generation of enterprise software will be built not on systems of record for objects, but on **"systems of record for decisions."**
 

 
> 
 "Context graphs represent a living record of decision traces stitched across entities and time so precedent becomes searchable."
 Foundation Capital on the trillion-dollar opportunity in agentic AI
 

 

 This concept of a "decision trace" is critical for [building robust and auditable AI agents](/blog/practical-guide-building-agents). It's not enough to have access to data; the agent needs the full context of *why* a decision was made, including the inputs, the model version, the prompt used, and the final response.
 

 

 The surprising implementation detail is that a [dependency graph within an intelligent data platform is a context graph](/blog/context-graphs-decision-traces-agentic-ai). Every computed column is a node in the graph, representing a step in the decision-making process. Every row in the table becomes a complete, versioned, and queryable decision trace.
 

 
```python
# Every row is a complete decision trace
agent_decisions = pxt.create_table('ops.support_routing', {
 'ticket_id': pxt.String,
 'customer_tier': pxt.String,
 'ticket_content': pxt.String,
 'policy_version': pxt.String, # Captured at decision time
})

# Add reasoning steps as computed columns
agent_decisions.add_computed_column(
 intent_classification=classify_intent(agent_decisions.ticket_content)
)

agent_decisions.add_computed_column(
 priority_score=calculate_priority(
 agent_decisions.intent_classification,
 agent_decisions.customer_tier
 )
)

agent_decisions.add_computed_column(
 routing_decision=route_request(
 agent_decisions.intent_classification,
 agent_decisions.priority_score
 )
)

# Query decisions with full context: "Why did we route this ticket to tier-3?"
# The dependency graph shows: ticket_content → intent → priority → decision
```

 

 This infrastructure-level approach captures the full lineage of every decision automatically, providing the foundation needed to build [stateful, memory-powered AI agents](/blog/building-memory-powered-ai-stateful-agents-pixeltable). With [time travel capabilities](/blog/pixeltable-versioning-time-travel), you can even replay the exact state at any decision point.
 

 
## 5. You Can Stop Manually Orchestrating Your Pipelines

 

 Data pipelines are typically built using an imperative approach. With tools like [Airflow](/blog/pixeltable-vs-airflow-ml-orchestration), you must manually define every step, manage dependencies between tasks, and specify the exact order of execution. This often results in hundreds of lines of code dedicated to managing the "how" of the pipeline, rather than the "what."
 

 

 The surprising alternative is the [declarative approach](/blog/declarative-ai-pipelines-open-standard). Instead of writing a script that says "first extract frames, then run object detection, then generate embeddings," you simply declare the desired end state: *"this column contains object detections from that frame column."*
 

 
```python
# Declarative: Define WHAT you want, not HOW to get it
video_analysis = pxt.create_table('analysis.videos', {
 'video': pxt.Video,
})

# Declare frame extraction
video_analysis.add_computed_column(
 frames=frame_iterator(video=video_analysis.video, fps=1)
)

# Declare object detection on frames
video_analysis.add_computed_column(
 detections=yolox(video_analysis.frames.frame)
)

# Declare embeddings on detected objects
video_analysis.add_computed_column(
 object_embeddings=clip_embedding(video_analysis.frames.frame)
)

# The system automatically:
# ✓ Discovers dependencies
# ✓ Determines execution order
# ✓ Parallelizes independent work
# ✓ Handles updates when data changes
```

 

 The system automatically discovers the dependencies, determines the correct execution order, parallelizes independent work, and handles all updates when the underlying data changes. This is why teams are [switching from complex orchestration tools](/blog/teams-switching-pixeltable-vector-databases) to declarative platforms.
 

 

 When you publish a dataset built this way, you're sharing everything as one complete unit:
 

 

 - **Raw data**: The original videos, images, documents

 - **Computed results**: All transformations and model outputs

 - **Embedding indexes**: Ready for similarity search

 - **Schema and metadata**: Complete reproducibility

 

 

 This declarative model frees developers from the tedious, error-prone work of pipeline orchestration. It allows them to focus their expertise on the core AI logic, dramatically accelerating the development cycle from prototype to production. See [data sharing in action](/blog/pixeltable-data-sharing-launch) for more details.
 

 
## Conclusion: The Future of AI is Declarative and Unified

 

 The five truths we've explored point to a clear and powerful conclusion: the chaotic, fragmented era of AI infrastructure is giving way to a new paradigm. This future is centered on **unified, declarative, and intelligent data platforms** that manage complexity automatically, allowing teams to move faster and build more ambitious applications.
 

 

 Let's recap what we've learned:
 

 

 - **The infrastructure tax is optional:** unified platforms eliminate 80% of data plumbing

 - **Databases should understand your data:** multimodal types and computed columns enable AI-native workflows

 - **Incremental computation saves massive costs:** dependency graphs eliminate redundant processing

 - **Decision traces enable auditable AI:** context graphs capture the "why" behind every decision

 - **Declarative beats imperative:** focus on what you want, not how to orchestrate it

 

 

 By eliminating the manual, imperative work of data plumbing, we can finally focus on the true goal of AI development: building applications that deliver real value.
 

 

 **If you could eliminate 80% of your data plumbing, what truly innovative AI features would you build next?**
 

 

 Ready to experience these truths firsthand? Start with our [10-minute tutorial](/blog/your-first-pixeltable-project) or explore the [interactive playground](/playground) to see declarative AI infrastructure in action.