Scaffold locally · backend

Full-stack showcase

Complete reference app: Gemini, DETR, Whisper, React UI. Scaffold locally; Cloud deploy is routes-only later.

Scaffold locally

uvx pixeltable-new --template full-stack-showcase my-full-stack-showcase

Same starter-kit files Cloud uses. Local UI (static HTML in some templates) is for uvx, not for Cloud. Cloud deploys schema + insert routes via pxt serve.

Secrets

Set these on the database before calling model-backed routes: GEMINI_API_KEY

app.py

"""FastAPI backend for the full-stack showcase.

Serves the React frontend (if built) and exposes REST endpoints for:
  - Video upload, list, detail, delete
  - Cross-modal search (text, image, video, audio)
  - Browse (frames, segments, scenes, audio, detections)
  - Dashboard (stats, alerts, activity)

    uv run uvicorn app:app --reload        # dev server (full API + React UI)
    uv run pxt serve sitewatch             # headless API-only subset (ingest + list; see pyproject.toml)
"""

import logging
from contextlib import asynccontextmanager
from pathlib import Path

import config
import pixeltable as pxt
import schema  # noqa: F401 -- triggers schema init on import
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, JSONResponse
from routers import browse, dashboard, search, videos

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s [%(name)s] %(message)s")
logger = logging.getLogger(__name__)


@asynccontextmanager
async def lifespan(app: FastAPI):
    try:
        pxt.get_table(f"{config.NAMESPACE}.videos")
        logger.info("Connected to Pixeltable schema")
    except Exception:
        logger.warning("Pixeltable schema not initialized. Run 'python schema.py' first.")
    yield


app = FastAPI(
    title="SiteWatch — Full-Stack Showcase",
    description="Video intelligence platform powered by Pixeltable",
    version="1.0.0",
    lifespan=lifespan,
)

app.add_middleware(
    CORSMiddleware,
    allow_origins=config.CORS_ORIGINS,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

app.include_router(videos.router)
app.include_router(search.router)
app.include_router(browse.router)
app.include_router(dashboard.router)


@app.get("/api/health")
def health():
    return {"status": "ok"}


STATIC_DIR = Path(__file__).resolve().parent / "static"


@app.get("/{full_path:path}")
def spa_fallback(full_path: str):
    if not STATIC_DIR.is_dir():
        return JSONResponse(
            {"detail": "Frontend not built. Run: cd frontend && npm run build"},
            status_code=404,
        )
    requested = (STATIC_DIR / full_path).resolve()
    # Containment guard: reject paths that escape STATIC_DIR (e.g. "../../etc/passwd").
    if requested.is_relative_to(STATIC_DIR) and requested.is_file():
        return FileResponse(requested)
    return FileResponse(STATIC_DIR / "index.html")


def _find_port(default: int = 8000) -> int:
    import os
    import socket

    port = int(os.environ.get("PORT", default))
    while port < default + 100:
        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
            if s.connect_ex(("localhost", port)) != 0:
                return port
        port += 1
    return default


if __name__ == "__main__":
    import uvicorn

    port = _find_port()
    print(f"Starting server at http://localhost:{port}")
    uvicorn.run("app:app", host="0.0.0.0", port=port, reload=True)

README

Full-Stack Showcase

The complete Pixeltable reference implementation — a production-grade video intelligence platform that exercises every core primitive in one codebase.

Layer What it demonstrates
Schema Multimodal tables (video, image, audio, text), computed columns (Gemini, DETR, Whisper), views with iterators, multimodal embedding indexes
Backend FastAPI routers, thread-safe pxt.get_table(), cross-modal search (text→video, image→video, audio→video), alerting, dashboard
Frontend React + TypeScript + Tailwind — 5 pages: Operations, Inspections, Browse, Investigate, Alerts

Quick Start

uvx pixeltable-new --template full-stack-showcase myapp
cd myapp && cp .env.example .env   # add your GEMINI_API_KEY
uv sync && uv run python app.py    # API at http://localhost:8000

# Frontend dev server (new terminal)
cd frontend && npm install && npm run dev   # UI at http://localhost:5173

In development, the Vite dev server (:5173) proxies /api to the backend (:8000). For a single-port production build, run cd frontend && npm run build (outputs to static/), then python app.py serves the built UI and the API together on :8000.

API-only mode (no UI)

uv sync
uv run python schema.py
uv run pxt serve sitewatch         # http://localhost:8000/docs

Do not run both pxt serve and app.py at the same time — they bind to the same port.

AI Stack

Component Model Purpose
Video Analysis Gemini 2.5 Flash Whole-video summary, segment condition/severity/PPE
Audio Transcription Whisper (local) Speech-to-text on extracted audio chunks
Multimodal Embeddings Gemini Embedding 2 Text, image, audio, video in one semantic space
Object Segmentation DETR ResNet-50 Panoptic Per-frame panoptic segmentation + overlay
Scene Detection PySceneDetect Content-based scene boundaries

Project Structure

├── schema.py            # Pixeltable schema (tables, views, indexes, computed columns)
├── config.py            # Configuration and Gemini prompts
├── functions.py         # Shared helpers (Gemini response parsing)
├── app.py               # FastAPI application
├── models.py            # Pydantic response models
├── routers/
│   ├── videos.py        # Upload, list, detail, delete, frames, scenes
│   ├── search.py        # Cross-modal search + related events
│   ├── browse.py        # Multi-medium browsing + DETR detections
│   └── dashboard.py     # ROI metrics, alerts, activity feed
├── frontend/
│   ├── src/
│   │   ├── App.tsx
│   │   ├── components/  # dashboard, videos, browse, search, alerts
│   │   ├── lib/api.ts
│   │   └── types/
│   └── package.json
├── pyproject.toml
└── .env.example

Without GEMINI_API_KEY

The template works without an API key — you still get:

  • Video upload and metadata
  • Frame extraction with DETR panoptic segmentation
  • Audio extraction + Whisper transcription
  • Scene detection

Cross-modal search and LLM analysis require GEMINI_API_KEY.