LegoGPT: AI Turns Text into Physically Stable Lego Designs, Now Free on GitHub
Imagine turning your wildest Lego ideas into reality with just a text prompt. That's now possible with LegoGPT, a groundbreaking AI model developed by computer scientists at Carnegie Mellon University. This innovative tool transforms text inputs into physically stable Lego designs, opening up a new realm of possibilities for both hobbyists and professional builders.
Why is this important? Unlike other AI generators that might produce structurally unsound creations, LegoGPT ensures that its designs adhere to the laws of physics. This means you can actually build the models generated by the AI, making it a practical and exciting tool for Lego enthusiasts.
The team's research, available on GitHub, reveals that LegoGPT was trained on a massive dataset of over 47,000 Lego structures, encompassing 28,000 unique 3D objects. The result? An impressive 98% success rate in generating physically stable designs.
How does it work? The LegoGPT system utilizes a three-part process to bring your text-based Lego dreams to life. It can even incorporate texture and color! Want an "Electric guitar in metallic purple"? LegoGPT can generate it, assigning the appropriate colors to the bricks.
To validate their creations, the researchers employed a dual-robot arm system equipped with force sensors to assemble the AI-generated designs. Human testers also built the models by hand, proving the AI’s ability to create genuinely buildable structures. The team's paper emphasizes that LegoGPT delivers stable, diverse, and aesthetically pleasing Lego designs that closely align with the provided text prompts.
When compared to other 3D creation AI systems like LLaMA-Mesh, LegoGPT excels in structural integrity, producing a significantly higher percentage of stable structures.
While the current version operates within a 20x20x20 building space and utilizes a limited set of eight standard brick types, the researchers plan to expand the brick library and training dataset in future updates. They've also released their dataset, code, and models on their project website and GitHub, encouraging further development and collaboration.
Ready to build your dreams? LegoGPT is now freely available on GitHub. You can even upload pictures of your existing blocks to see what the AI suggests you build with what you have!
LegoGPT converts the text into a LEGO design, which is then converted into text tokens ordered from bottom to top. Instructions are then created to pair the structured LEGO bricks with annotations explaining the design, so that the AI will understand the relationships between the text prompt and the physical bricks.
What will you build with LegoGPT? Share your thoughts and creations in the comments below!
