From Pixel to Mesh: Accelerating Game Asset Creation via a Semantically-Guided 2D-to-3D Generative Pipeline

Abstract

In independent game development, creating high-quality 3D assets remains a critical bottleneck, as traditional workflows require specialized skills that hinder non-expert designers. While Generative AI has democratized 2D concept art, translating these visions into game-ready 3D assets is technically demanding. To address this, we propose an automated pipeline that leverages semantic and geometric feature extraction to synthesize 3D meshes and PBR materials from AI-generated 2D images. Our system orchestrates a workflow bridging text-to-image diffusion models with advanced computer vision modules. It employs monocular depth estimation and intrinsic decomposition to interpret geometric structures and isolate material properties (albedo, roughness, metallic) from multi-view consistent concepts. These features are algorithmically processed to produce finalized .glb assets. A comparative user study with indie developers demonstrates that this pipeline reduces asset production time by approximately 80% compared to traditional modeling tools. By shifting the user’s role from manual vertex manipulation to high-level semantic curation, this research validates a novel workflow that empowers designers to rapidly populate immersive worlds, streamlining the future of interactive media design.

Keywords: Generative AI, Game Asset Creation, 3D Reconstruction, Human-computer Interaction, Interactive Media Design

DOI: 10.54941/ahfe1007640