The discovery of sharp algorithms for matrix multiplication remains an important discovery in computer science and numerical linear algebra. Since the leading contribution of Strassen and Vinograd in the late 1960s, it is assumed that the normal matrix products are already believed to be calculated with less multiplication with various strategies. These include gradient-based methods, heuristic technology, group-principle structures, graph-based random walk and deep reinforcement learning. However, matrix products with the underlying structure have focused significantly, such as when the second matrix is transport or similar for the first, or when the matriasis has sparsity or symmetry. This inspection is notable, given that manifestations such as AA^T appear in doses such as data, deep education and communication, represent important constructions such as grams and Kovrian Matris. In addition, large language models such as XX^t Muon and shampoo are repetitive in training algorithms.
Previous studies have detected structured matrix multiplication using various theoretical and machine learning-based methods. Representation theory and Cohn -umans structure have been employed to design efficient multiplication plans for structured matris. Strengthening Learning has also shown the promise – the model has learned to find or re -find a known algorithm such as Strassen. Recent work has focused on adapting the calculation of XX^T on finite regions and complex domains. Of these, the most efficient known method of real-valued XX^t is the algorithm of strassen, which recurrents the algorithm of Strassen on the 2 × 2 block matrices, effectively translates the effective structured problem back into the domain of general matrix multiplication.
Researchers at the Chinese University and the Shenzen Research Institute of Big Data have developed RXTX, efficiently an algorithm for computing XX^T where XR^N*M is related to. RXTX reduces the number of bold and additions to about 5% compared to current leading methods. Unlike many algorithms that only show benefits for large matris, RXTX also improves for small sizes (eg, n = 4). The algorithm was discovered through machine learning-based discovery and combinatorial optimization, which takes advantage of the specific structure of XX^T for continuous-function acceleration.
The RXTX algorithm improves matrix multiplication by reducing the number of operations compared to previous methods such as recurrence strasen and Strassen-Vinograd. It uses 26 general matrix multiplication and additional additional schemes, resulting in low total operation. Theoretical analysis suggests that RXTX performs low multiplication and joint operations, especially for large matris. Practical testing on 6144 × 6144 metrices using single-thread CPU show RXTX is about 9% faster than the standard blass routine, the speedup is observed in 99% runs. These results expose the efficiency of RXTX for large -scale symmetrical matrix products and its advantage on traditional and state -of -the -art recurrent algorithms.
The proposed functioning integrates RL with a two-tier mixed integer linear programming (MILP) pipeline to discover efficient matrix multiplication algorithm, especially for XX^t computing. The discovery of the RL -directed large neighborhood produces a large set of potential rank -1 billinier products, which are candidates expression. The MILP-A investigates all linear combinations of these products to express the target output, while MILP-B identifies the smallest that can represent all the goals. This setup reflects the alphatensor approach, but simplifies it by reducing the action space, focuses on low-dimensional tensor products and takes advantage of milpe solvers such as gurobi for rapid calculation.
For example, to calculate the XX^t for 2 × 2 matrix x, the target is to achieve expressions such as X_1^2 + x_2^2 or x_1x_3 + x_2x_4. The RL policy randomly samples thousands of bilinier products using coefficients from {, 1, 0, +1}. Milp-A finds a combination of these products that match the desired manifestations, and Milp-B selects the lowest required to cover all goals. This structure enables the discovery of RXTX, an algorithm that performs 5% less and overall operations than the previous methods. The RXTX is efficient for large and small matris and displays a successful fusion of ML-based search and combinatorial optimization.
See paper, All credit for this research goes to the researchers of this project. Also, feel free to follow us Twitter And don’t forget to join us 95K+ ML Subredit More membership Our newspaper,
Sana Hasan, a counseling intern and double degree student at Marktekpost in IIT Madras, is emotional about implementing technology and AI to resolve real -world challenges. With a keen interest in solving practical problems, he brings a new approach to the intersection of AI and real -life solutions.
