Find a Video from an Image with Pixeltable

Summary: Want an AI video finder by image—upload a still and jump to matching moments in a library? With Pixeltable you extract frames, add a CLIP embedding index, and call .similarity() with an image or a text string. No separate vector database required. Try the hosted experience at PixelSearch, or build the pipeline below.

What You Are Building#

People search for this as image to video search, find video from image, similar video finder, and AI video search. The product capability is the same:

• Index video content at the frame level
• Query with an image (visual similarity) or text (cross-modal CLIP)
• Return the parent video + timestamp / frame position

Architecture in Four Steps#

1. Videos table — store pxt.Video paths or URLs
2. Frames view — frame_iterator at a chosen FPS
3. Embedding index — CLIP on the frame column
4. Query — frames.frame.similarity(image=...) or string=...

Working Example#

Same Index, Text Query#

FPS Tradeoffs#

FPS	Best for	Cost
0.5	Long archives, coarse recall	Fewer embeddings
1	General video finder	Balanced
2+	Fast action / short clips	More storage + compute

Where frames and indexes live on disk: VideoRAG storage guide.

Use PixelSearch Instead of Building#

PixelSearch is a multimodal search UI on Pixeltable—upload media, search by image or text, hybrid signals. Deep dive: Build your own multimodal search engine. Hard-way comparison: Video frame similarity search.

Production Notes#

• Incremental: New videos only embed new frames—indexes stay in sync.
• Persistence: Mount PIXELTABLE_HOME in Docker so the catalog survives restarts (Docker guide).
• Serve: Expose search via pxt serve / FastAPI (Starter Kit patterns).

FAQ#

Can I embed whole videos instead of frames?#

Frame-level CLIP is the usual image→moment path. Video-native embedders (e.g. Twelve Labs) are available for segment-level search—see provider docs.

Do I need Pinecone?#

No. Pixeltable embedding indexes cover this workload for most libraries. Scale media to object storage when needed.

Can this find a movie from a screenshot?#

Technically similar retrieval works on any indexed frames. Respect copyright and only index content you have rights to search.

See Also#

• PixelSearch
• Video frame similarity search
• Multimodal search engine guide
• Where VideoRAG stores indexes
• Video intelligence pipeline
• Docs: Embedding indexes