Have you ever had an idea of something that looks good but doesn’t work well in practice? When it comes to designing things like decor and personal accessories, Generative Artificial Intelligence (GenAI) models may be relevant. They can produce creative and detailed 3D designs, but when you try to turn such blueprints into real-world objects, they are usually not durable for everyday use.
The underlying problem is that GenAI models often lack an understanding of physics. While tools like Microsoft’s Trellis system can create a 3D model from a text prompt or image, for example, its design for a chair may be unstable, or its parts may come apart. The model doesn’t fully understand what your intended object is designed to do, so even if your seat can be 3D printed, it will fall apart due to the pressure of someone sitting below.
In an effort to make these designs work in the real world, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are giving generative AI models a reality check. Their “PhysioOpt” system augments these tools with physics simulations, creating blueprints for individual objects such as cups, keyholders and bookends that function as intended when they are 3D printed. It rapidly tests whether the structure of your 3D model is viable, gradually modifying small shapes while ensuring that the overall form and function of the design is preserved.
You can simply type into PhysiOpt what you want to build and what it will be used for, or upload an image to the system’s user interface, and in about half a minute, you’ll have a realistic 3D object to build. For example, it inspired CSAIL researchers to create a “flamingo-shaped drinking glass,” which they 3D printed into a drinking glass with a handle and base resembling a tropical bird’s foot. As the design was completed, PhysiOpt made small refinements to ensure that the design was structurally sound.
“PhysioOpt combines GenAI and physics-based shape optimization, helping anyone create custom designs for unique accessories and decorations,” says MIT Electrical Engineering and Computer Science (EECS) PhD student and CSAIL researcher Xiao Hsin Zhan SM ’25, co-lead author of a paper presenting the work. “It’s an automated system that allows you to physically build a shape, given certain constraints. PhysiOpt can repeat your creations as many times as you want, without any additional training.”
This approach enables you to create a “smart design,” where an AI generator tailors your item based on users’ specifications while also considering functionality. You can plug in your favorite 3D generative AI model, and after typing in what you want to generate, you specify how much force or weight the object should handle. This is a good way to simulate real-world usage, such as predicting whether a hook will be strong enough to hold your coat. Users also specify what material they will build the item from (such as plastic or wood), and how it is supported – for example, a cup stands on the ground, while a bookend leans against a collection of books.
Given the specifications, PhysiOpt begins to iteratively optimize the object. Under the hood, it runs a physics simulation called “finite element analysis” to stress test the design. This comprehensive scan provides a heat map over your 3D model, indicating where your blueprint is not well supported. If you were building a birdhouse, you might find that the support beams under the house were bright red, meaning the house would collapse if it was not strengthened.
PhysiOpt can create even bolder pieces. Researchers saw this versatility firsthand when they created a steampunk (a style that blends Victorian and futuristic aesthetics) keyholder that featured complex, robotic-looking hooks and a “giraffe table” with a flat back on which you could place items. But how to know what “steampunk” is, or what such a unique piece of furniture should look like?
Notably, the answer is not extensive training – at least, not on the part of researchers. Instead, PhysiOpt uses a pre-trained model that has already seen thousands of shapes and objects. “Existing systems often require a lot of additional training to have a meaningful understanding of what you want to see,” says co-lead author Clément Jambon, who is also an MIT EECS PhD student and CSAIL researcher. “But we use a model with the experience you want to build already, so PhysiOpt is training-free.”
By working with pre-trained models, PhysiOpt can use “shape priors,” or knowledge of how shapes should look based on prior training, to generate what users want to see. It’s kind of like an artist recreating the style of a famous painter. Their expertise lies in closely studying a variety of artistic approaches, so they will likely be able to reflect that particular aesthetic. Similarly, being familiar with the shapes of a pre-trained model helps one in generating 3D models.
CSAIL researchers observed that PhysiOpt’s visual information helped it create 3D models more efficiently than “DiffIPC”, a comparable method that simulates and optimizes shapes. When both approaches were tasked with generating 3D designs for objects such as chairs, CSAIL’s system was about 10 times faster per iteration, producing more realistic objects.
PhysiOpt offers a potential bridge between ideas and real-world individual objects. For example, what you might consider a great idea for a coffee mug may soon make its way from your computer screen to your desk. And while PhysiOpt already does stress-testing for designers, it may soon be able to predict constraints like load and limits, rather than requiring users to provide those details. This more autonomous, common sense approach can be made possible by incorporating vision language models, which combine human language understanding with computer vision.
Additionally, Zane and Jambon intend to make the system even more physics-aware, removing the artifacts, or random pieces that sometimes appear in PhysiOpt’s 3D models. MIT scientists are also looking at how they can model more complex constraints for different manufacturing techniques, such as minimizing overhanging components for 3D printing.
Zan and Zambon wrote their paper with Kenny Ng ’89, SM ’90, PhD ’00, principal research scientist at the MIT-IBM Watson AI Lab, and two CSAIL colleagues: graduate researcher Evan Thompson and assistant professor Mina Konakovich Lukovic, who is a principal investigator in the lab.
The researchers’ work was supported, in part, by the MIT-IBM Watson AI Laboratory and Wistron Corp. He presented it at the Association for Computing Machinery’s SIGGRAPH conference and exhibition on computer graphics and interactive technologies in Asia in December.
