Project information

  • Company: ORamaVR
  • Categories: Data Engineering, MLOps, 3D Deep Learning
  • Main technologies: Python, PyTorch, HuggingFace Datasets, Kaolin, Metaflow

Summary

Training generative 3D models requires massive, well-structured datasets. OVR-Datasets was built as the foundational data engineering pipeline to handle 3D data at scale.

I developed an automated processing pipeline capable of standardizing and pre-processing millions of 3D meshes for deep learning workflows. A core feature of the project was creating a streamlined codebase to efficiently store, version, and retrieve these large-scale 3D datasets directly from HuggingFace.

By orchestrating the workflow with Metaflow and utilizing NVIDIA's Kaolin library for 3D operations, I ensured that massive datasets could be seamlessly and efficiently injected into heavy PyTorch training pipelines.

The Corpora

Public 3D datasets vary wildly in scale, orientation, face count and cleanliness, and carry no usable text annotations. The pipeline standardized across several of them:

  • ShapeNet (55k meshes)
  • MedShapeNet (100k, medical anatomy)
  • Objaverse (800k)
  • ShapeTalk, used separately for mesh editing, pairing a source and target mesh with an edit prompt such as "the arms are longer and the legs shorter."

Processing

Each mesh went through unit-box scaling, decimation that reduces face count while preserving surface properties, degenerate-face removal and general cleaning, plus multi-angle rendering to produce the paired image data needed for image-to-3D. Built with the Blender API, pymeshlab, trimesh and Kaolin, parallelised with Metaflow.

Every processed source was published as a versioned HuggingFace dataset with a custom loader, so a training run could select which datasets to use and filter down to individual categories straight from its config file.

Captioning: Adapting Cap3D

Text-conditioned generation needs captions, and hand-labelling a corpus this size was never an option. I implemented the Cap3D methodology, which renders each mesh from multiple views, uses CLIP to pick the best caption of five per view, and has an LLM summarize across views.

Rather than the paper's BLIP and ChatGPT stages, I substituted a locally hosted vision-language model served through Ollama, starting with LLaVA and moving to Gemma 3 on release for better caption quality. This removed the per-caption API cost entirely.

Throughput and Limits

The full pipeline, from mesh loading through cleaning, rendering, captioning and saving, ran at roughly 30 seconds per mesh and parallelised cleanly. Meshes with millions of faces, such as landscape scans, were the one category that could not be usefully processed.