- Company: ORamaVR
- Categories: Generative AI, Spatial Computing, Large Language Models
- Main technologies: Python, ChatGPT API, Geometric Algebra, USD (Universal Scene Description)
Summary
Designing virtual environments manually is a highly time-consuming process. Shenlong addresses this by functioning as an AI-powered 3D scene generator that blends Large Language Models with spatial mathematics.
Starting with a database of individual 3D assets (e.g., TVs, couches, tables), users can simply ask the assistant to "create a living room." The system queries the ChatGPT API to determine the logical objects required for the scene. It then retrieves those assets and applies Geometric Algebra to calculate realistic spatial placements and orientations in a 3D coordinate system.
The tool also supports iterative, natural language edits, allowing users to intuitively refine the space (e.g., "move the couch closer to the wall"). Finished environments export ready to drop straight into Unity or Blender.
My Contribution: the Retrieval Layer
Asking an LLM which objects belong in a living room is the easy half. The harder half is finding the right assets in a corpus of hundreds of thousands of meshes, where the only searchable signals are the geometry itself and machine-generated captions.
I built that retrieval layer. Mesh renders were embedded with CLIP and their generated captions with BERT, then served as hybrid retrieval over both signals through PyVespa. Combining the visual and textual embeddings matters: captions capture what an object is, renders capture what it looks like, and a query usually cares about both.
The Geometric Algebra used for spatial placement was the work of a research scientist on the team, not mine. My part there was cleaning up that code and extending the scene export so generated environments could be used directly in Unity and Blender.