A new chip developed by MIT researchers could help small, low-power UAVs avoid obstacles as they move around tight corners inside an industrial HVAC system to investigate a gas leak.
The chip allows small autonomous robots and other battery-limited devices to create detailed 3D maps of their environments in real time using only as much power as an LED. A robot can use such a map to plan a collision-free path to reach its goal.
Typically, generating such depth maps requires power-consuming systems and large amounts of memory to create and store 3D representations of obstacles in the robot’s environment.
The MIT researchers took a different approach by combining a highly efficient mapping algorithm with specialized hardware designed to accelerate their workloads while also minimizing memory and power consumption.
This system-on-a-chip consumes only 6 milliwatts of power, a fraction of the power required by other systems.
This low-power operation could also make the chip suitable for lightweight augmented reality headsets that can be worn for extended periods of time for applications such as educational medical simulations or detailed repair and assembly work.
“This paper shows a prime example of how you can really leverage the co-design of algorithms and hardware to increase energy efficiency. While a lot of work has been done on compact 3D maps, what stands out about this work is that it also makes sure that the process of creating those maps is as efficient as possible. Our chip allows you to store very large maps in a very small space and do it in a very energy efficient way, ” says Professor Vivienne Saez of the Department of Electrical Engineering and Computer Science. (EECS), member of the Research Laboratory of Electronics (RLE) and senior author of a paper on the chip.
He is joined on the paper by co-lead authors and MIT graduate students Zih-Hsing Fu and Peter Xie Juan Li, as well as Sertak Karaman, professor of aeronautics and astronautics and director of LIDS. This work was recently presented at the IEEE Very Large-Scale Integrated Circuits Symposium.
A more concise map
For a robot, generating a 3D map that includes obstacles in its environment typically requires a lot of power because it must store the images captured by its camera, and process all the 3D pixels in each image multiple times.
Instead of representing the environment using 3D pixels, which are cubes called voxels, the MIT researchers used a technique that maps obstacles in space using ellipsoidal blobs called Gaussians.
The size, shape, and thickness of these ellipsoids can be easily customized, so they match the shapes of curved objects more efficiently than using rigid, cube-shaped voxels.
Importantly, the map captures obstacles and free space around the robot, and together these allow the robot to plan a safe, collision-free path. Mapping constraints and free space with voxels usually consumes a lot of memory, making traditional methods power-hungry. Because Gaussians can fit the geometry flexibly, a single elongated ellipsoid can represent a region that will take multiple voxels, so occupied surfaces and free space are more compactly captured.
For their new system-on-a-chip, called Glenmar, the researchers employed an algorithm called GMMap, developed in their lab, which efficiently generates a 3D map of the robot’s environment by using Gaussians to represent obstacles.
With the traditional approach, a robot would need to load and process each depth image multiple times to adjust to the shape and size of the ellipsoid. The system will typically construct a Gaussian by comparing all the pixels of an image to each other. But the amount of memory and power required to do this remains too high for many edge devices.
To solve this problem, MIT researchers invented a technique that can generate highly accurate Gaussians from depth images with just one pass, after which they can discard the images, so the chip never has to store the entire image at once.
Instead of comparing every pixel to every other pixel in the 3D image, their algorithm assumes that nearby pixels are in the same Gaussian, so it only needs to compare each pixel to its neighbors.
“At any given time, we only need to store a few pixels in memory, which significantly reduces the memory footprint required by our algorithm,” says Lee.
Leveraging Co-Design
But as the robot moves through space, it usually sees the same object from different perspectives. When it generates Gaussians, some will overlap because they represent the same object. This can cause the 3D map to become too large to store on an edge device.
Fusing makes overlapping Gaussian maps more compact, but doing so usually requires the algorithm to process many raw pixels stored in memory. The researchers developed a novel technique to perform this fusion process directly on overlapping Gaussians, without the need to revisit the original pixels. Since Gaussians are more compact than pixels, this significantly reduces memory and power requirements.
The same principle runs through their algorithms – most computations operate directly on compact Gaussians rather than native pixels, enabling energy efficiency.
The researchers used this principle to design a chip that keeps Gaussians working actively within small, fast on-chip memory, right next to the computational units. This is only possible because the Gaussian map is so dense.
The Gaussians the robot needs to work on next are waiting in on-chip memory units, so there is no need to fetch them from more distant, power-consuming, off-chip storage.
“By having a dedicated memory that stores the objects you viewed in the last few frames, you can access the data more efficiently,” Fu explains.
They tested the system-on-a-chip by recreating a series of diverse, pre-existing 3D environments. The chip can also reconstruct obstacles and free space directly from live data streamed from the iPhone camera.
Glenmar produced detailed 3D maps in real time while consuming about 6 milliwatts of power. It requires only 2.5 percent of the power of the best existing chips for map generation.
By reusing the compact Gaussian on the path it plans, the chip lets a robot chart a safe trajectory using only 20 percent of the energy it would otherwise need.
“We reduce memory consumption by making sure the algorithm is efficient. Then we speed up the workloads performed by that efficient algorithm, so in the end, our chip is as efficient as possible,” says Lee.
The researchers plan to further improve energy efficiency by moving the processing units on the chip closer to the sensors gathering environmental data. They may also explore additional applications, such as the use of Gaussians for schematic representation. This could help AI systems reason about complex blueprints more efficiently.
“Real-time 3D mapping has been the missing piece for small autonomous systems. A drone inspecting a pipeline or a pair of AR glasses navigating a room both need to understand the space around them – instantly, continuously, and with almost no power cost. Glenmar makes this possible for the first time in a chip you can hold between your fingers,” says Karman.
This work is supported in part by an MIT-Mathworks Fellowship, Amazon, the US National Science Foundation, and Intel.
