If you rotate an image of a molecular structure, a human can tell that the rotated image is still the same molecule, but a machine-learning model can think that it is a new data point. In computer science parlance, the molecule is “symmetrical”, which means that the original structure of that molecule remains the same if it passes through certain changes, such as rotation.
If a drug search model does not understand symmetry, it can make wrong predictions about molecular properties. But despite some empirical successes, it is not clear that there is a computationally skilled method to train a good model that is guaranteed to respect symmetry.
A new study by MIT researchers answers this question, and shows the first method for machine learning with symmetry that is efficient in the context of both calculation and data requirement.
These results clarify a fundamental question, and they can help researchers in the development of more powerful machine-learning models who are designed to handle symmetry. Such models will be useful in various types of applications, ranging from the discovery of new materials to the discovery of astronomical discrepancies to highlight the complex climate pattern.
“These symmetry are important because they are some types of information that nature is telling us about data, and we should keep it in our machine-learning model. We have now shown that it is possible to do machine-learning with symmetrical data in a efficient way,” an MIT graduate student and co-in-law writer of this study is called Behroz Tamsabi.
He has joined paper by co-Leid writer and MIT graduate student Ashkan Solemani; Stephanie Jegelka, an associate professor of Electrical Engineering and Computer Science (EECS) and Institute for Data, Systems and Society (IDSS) and Computer Science and Artificial Intelligence Laboratory (CSAIL); And senior writer Patrick Jail, The Dugged C. A prominent investigator in Jackson Professor of Electrical Engineering and Computer Science and Laboratory for Information and Decision Systems (LIDS). Research was recently presented at the International Conference on Machine Learning.
Synchronization
Symptoms appear in many domains, especially natural science and physics. A model that recognizes symmetry is able to identify an object like a car, no matter that the object is placed in an image, for example.
As long as a machine-learning model is designed to handle symmetry, it may be less accurate and may be the possibility of failure when faced with new symmetrical data in real-world conditions. On the other hand, models taking advantage of symmetry can be faster and training requires less data.
But training a model to process symmetrical data is not an easy task.
A common approach is called data growth, where researchers turn each symmetrical data point into multiple data points to help the model normalize the new data. For example, no one can rotate a molecular structure several times to produce new training data, but if researchers want the model to guarantee to respect symmetry, it can be computationally prohibitory.
An alternative approach is to encoded symmetry in the architecture of the model. A famous example of this is a graph neural network (GNN), which naturally handles symmetric data because it is designed.
“Graph neural networks are sharp and efficient, and they take great care of symmetry, but no one really knows what these models are learning or why they work. Understanding GNNs is a main inspiration for our work, so we started with a theoretical evaluation when the data is symmetrical,” says Tamesbi.
He discovered statistical-computational tradeoffs in machine learning with symmetrical data. This tradeoff means that methods that require less data can be more computationally expensive, so researchers need to find the right balance.
Construction on this theoretical evaluation, researchers designed a skilled algorithm for machine learning with symmetrical data.
Mathematical combination
To do this, he borrowed to shrink ideas from algebra and simplify the problem. Then, he improved the problem by using ideas from geometry that effectively catch symmetry.
Finally, they added algebra and geometry to an adaptation problem, which could be resolved efficiently, resulting in their new algorithm.
“Most principles and applications were focusing on either algebra or geometry. Here we combined them jointly,” called Tamasebi.
The algorithm requires low data samples for training compared to classical approaches, which will improve the accuracy of a model and the ability to adapt to new applications.
By proving that scientific symmetry can develop efficient algorithms for machine learning, and can showcase how it can be done, these results can lead to the development of new neural network architecture that can be more accurate and less resource-temperature than current models.
Scientists can also use this analysis as an early point to examine the internal functioning of the gNNs, and how their operations differ from the algorithm developed by the MIT researchers.
“Once we know that is better, we can design more interpretable, more strong and more efficient nerve network architecture,” says Solemani.
This research is funded by the National Research Foundation of Singapore, DSO National Laboratories of Singapore, US Office of Naval Research, US National Science Foundation, US National Science Foundation and an Alexander von Hmblet Professorship.
