Yan Ma (@ManTle_Ma) “Video foundation models (e.g., Seedance 1.0 → 1.5 → 2.0; V-JEPA 1.0 → 2.0 → 2.1)” — TopicDigg

Video foundation models (e.g., Seedance 1.0 → 1.5 → 2.0; V-JEPA 1.0 → 2.0 → 2.1) keep getting stronger, while public descriptions of how their training data is built keep getting shorter. ⚒️ We built VidaForge, an open-source, five-stage video data pipeline that turns raw video collections into training-ready datasets. In an academic lab, studying video data recipes begins with a lot of tedious engineering: handling broken videos and transcode failures, keeping large jobs resumable, tracking every clip, and packaging the result for training. Across ingestion, segmentation, selection, annotation, and training dataset packaging, data moves through: raw videos → standardized videos → clips → curated clips → annotated clips → training datasets At every step, we can inspect what happened to each video or clip. This lets us see which samples changed under a data recipe and trace a training dataset back through the pipeline. VidaForge currently connects processed data to two video foundation model pretraining paths: Wan video generation through NeMo-AutoModel, and self-supervised video representation learning through the official V-JEPA2 repository. We ran VidaForge end to end on 200K videos, producing over 700K clips. From these clips, we built Selected-200K, Mixed-200K, and Rejected-200K datasets for Wan2.1-1.3B and V-JEPA2.1-1B pretraining. In these early runs, the three datasets produced different training behavior: data selection appeared in Wan2.1-1.3B eval curves and V-JEPA2.1-1B training stability. Pipeline, experiments, open data, and project resources in the thread below ↓