Scaffold locally · backend
Full-stack showcase
Complete reference app: Gemini, DETR, Whisper, React UI. Scaffold locally; Cloud deploy is routes-only later.
Scaffold locally
uvx pixeltable-new --template full-stack-showcase my-full-stack-showcaseSame starter-kit files Cloud uses. Local UI (static HTML in some templates) is for uvx, not for Cloud. Cloud deploys schema + insert routes via pxt serve.
Secrets
Set these on the database before calling model-backed routes: GEMINI_API_KEY
app.py
"""FastAPI backend for the full-stack showcase.
Serves the React frontend (if built) and exposes REST endpoints for:
- Video upload, list, detail, delete
- Cross-modal search (text, image, video, audio)
- Browse (frames, segments, scenes, audio, detections)
- Dashboard (stats, alerts, activity)
uv run uvicorn app:app --reload # dev server (full API + React UI)
uv run pxt serve sitewatch # headless API-only subset (ingest + list; see pyproject.toml)
"""
import logging
from contextlib import asynccontextmanager
from pathlib import Path
import config
import pixeltable as pxt
import schema # noqa: F401 -- triggers schema init on import
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, JSONResponse
from routers import browse, dashboard, search, videos
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s [%(name)s] %(message)s")
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
try:
pxt.get_table(f"{config.NAMESPACE}.videos")
logger.info("Connected to Pixeltable schema")
except Exception:
logger.warning("Pixeltable schema not initialized. Run 'python schema.py' first.")
yield
app = FastAPI(
title="SiteWatch — Full-Stack Showcase",
description="Video intelligence platform powered by Pixeltable",
version="1.0.0",
lifespan=lifespan,
)
app.add_middleware(
CORSMiddleware,
allow_origins=config.CORS_ORIGINS,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(videos.router)
app.include_router(search.router)
app.include_router(browse.router)
app.include_router(dashboard.router)
@app.get("/api/health")
def health():
return {"status": "ok"}
STATIC_DIR = Path(__file__).resolve().parent / "static"
@app.get("/{full_path:path}")
def spa_fallback(full_path: str):
if not STATIC_DIR.is_dir():
return JSONResponse(
{"detail": "Frontend not built. Run: cd frontend && npm run build"},
status_code=404,
)
requested = (STATIC_DIR / full_path).resolve()
# Containment guard: reject paths that escape STATIC_DIR (e.g. "../../etc/passwd").
if requested.is_relative_to(STATIC_DIR) and requested.is_file():
return FileResponse(requested)
return FileResponse(STATIC_DIR / "index.html")
def _find_port(default: int = 8000) -> int:
import os
import socket
port = int(os.environ.get("PORT", default))
while port < default + 100:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
if s.connect_ex(("localhost", port)) != 0:
return port
port += 1
return default
if __name__ == "__main__":
import uvicorn
port = _find_port()
print(f"Starting server at http://localhost:{port}")
uvicorn.run("app:app", host="0.0.0.0", port=port, reload=True)
README
Full-Stack Showcase
The complete Pixeltable reference implementation — a production-grade video intelligence platform that exercises every core primitive in one codebase.
| Layer | What it demonstrates |
|---|---|
| Schema | Multimodal tables (video, image, audio, text), computed columns (Gemini, DETR, Whisper), views with iterators, multimodal embedding indexes |
| Backend | FastAPI routers, thread-safe pxt.get_table(), cross-modal search (text→video, image→video, audio→video), alerting, dashboard |
| Frontend | React + TypeScript + Tailwind — 5 pages: Operations, Inspections, Browse, Investigate, Alerts |
Quick Start
uvx pixeltable-new --template full-stack-showcase myapp
cd myapp && cp .env.example .env # add your GEMINI_API_KEY
uv sync && uv run python app.py # API at http://localhost:8000
# Frontend dev server (new terminal)
cd frontend && npm install && npm run dev # UI at http://localhost:5173
In development, the Vite dev server (:5173) proxies /api to the backend (:8000). For a single-port production build, run cd frontend && npm run build (outputs to static/), then python app.py serves the built UI and the API together on :8000.
API-only mode (no UI)
uv sync
uv run python schema.py
uv run pxt serve sitewatch # http://localhost:8000/docs
Do not run both pxt serve and app.py at the same time — they bind to the same port.
AI Stack
| Component | Model | Purpose |
|---|---|---|
| Video Analysis | Gemini 2.5 Flash | Whole-video summary, segment condition/severity/PPE |
| Audio Transcription | Whisper (local) | Speech-to-text on extracted audio chunks |
| Multimodal Embeddings | Gemini Embedding 2 | Text, image, audio, video in one semantic space |
| Object Segmentation | DETR ResNet-50 Panoptic | Per-frame panoptic segmentation + overlay |
| Scene Detection | PySceneDetect | Content-based scene boundaries |
Project Structure
├── schema.py # Pixeltable schema (tables, views, indexes, computed columns)
├── config.py # Configuration and Gemini prompts
├── functions.py # Shared helpers (Gemini response parsing)
├── app.py # FastAPI application
├── models.py # Pydantic response models
├── routers/
│ ├── videos.py # Upload, list, detail, delete, frames, scenes
│ ├── search.py # Cross-modal search + related events
│ ├── browse.py # Multi-medium browsing + DETR detections
│ └── dashboard.py # ROI metrics, alerts, activity feed
├── frontend/
│ ├── src/
│ │ ├── App.tsx
│ │ ├── components/ # dashboard, videos, browse, search, alerts
│ │ ├── lib/api.ts
│ │ └── types/
│ └── package.json
├── pyproject.toml
└── .env.example
Without GEMINI_API_KEY
The template works without an API key — you still get:
- Video upload and metadata
- Frame extraction with DETR panoptic segmentation
- Audio extraction + Whisper transcription
- Scene detection
Cross-modal search and LLM analysis require GEMINI_API_KEY.