Ranging from Karpathy llm.c, I’m wondering myself “May I write this in Rust?” Listed below are the teachings I realized and the way I’m writing llm.rust. On this first article, let’s deal with the matrix multiplication downside.
18 hours in the past
Matrix multiplication could also be an important operation in Machine Studying. I nonetheless keep in mind once I was an engineering scholar, and in one of many first linear algebra classes, the trainer began to clarify matrices, eigenvectors, and foundation and orthonormal foundation. I used to be very confused, my head took a short while to start out understanding why we have been bothering a lot about matrices and foundation units, and what foundation implies for our world. From there, I all the time discovered linear algebra so fascinating, and, from a pure laptop science viewpoint, how wonderful all these algorithms that attempt to be an increasing number of environment friendly in dealing with matrices.
Specifically, we all know that the matrix-vector product is fairly easy, however issues are getting an increasing number of difficult when we have now matrices-matrices or tensors-tensors merchandise. From right here, many methodologies have been carried out to optimize the matrix multiplication. For instance, a very long time in the past I posted about DeepMind…