Estimating Matrix Rank With Neural Networks | Two Minute Papers #94

Estimating Matrix Rank With Neural Networks | Two Minute Papers #94


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. This piece of work is not meant to be a highly
useful application, only a tongue in cheek jab at the rising trend of trying to solve
simple problems using deep learning without carefully examining the problem at hand. As always, we note that all intuitive explanations
are wrong, but some are helpful, and the most precise way to express these thoughts can
be done by using mathematics. However, we shall leave that to the textbooks
and will try to understand these concepts by floating about on the wings of intuition. In mathematics, a matrix is a rectangular
array in which we can store numbers and symbols. Matrices can be interpreted in many ways,
for instance, we can think of them as transformations. Multiplying a matrix with a vector means applying
this transform to the vector, such as scaling, rotation or shearing. The rank of a matrix can be intuitively explained
in many ways. My favorite intuition is that the rank encodes
the information content of the matrix. For instance, in an earlier work on Separable
Subsurface Scattering, we recognized that many of these matrices that encode light scattering
inside translucent materials, are of relatively low rank. This means that the information within is
highly structured and it is not random noise. And from this low rank property follows that
we can compress and represent this phenomenon using simpler data structures, leading to
an extremely efficient algorithm to simulate light scattering within our skin. However, the main point is that finding out
the rank of a large matrix is an expensive operation. It is also important to note that we can also
visualize these matrices by mapping the numbers within to different colors. As a fun sidenote, the paper finds, that the
uglier the colorscheme is, the better suited it is for learning. This way, after computing the ranks of many
matrices, we can create a lot of input images and output ranks for the neural network to
learn on. After that, the goal is that we feed in an
unknown matrix in the form of an image, and the network would have to guess what the rank
is. It is almost like having an expert scientist
unleash his intuition on such a matrix, much like a fun guessing game for intoxicated mathematicians. And the ultimate question, as always is, how
does this knowledge learned by the neural network generalize? The results are decent, but not spectacular,
but they also offer some insights as to which matrices have surprising ranks. We can also try computing the products of
matrices, which intuitively translates to guessing the result after we have done one
transformation after the other. Like the output of scaling after a rotation
operation. They also tried to compute the inverse of
matrices, for which the intuition can be undoing the transformation. If it is a rotation to a given direction,
the inverse would be rotating back the exact same amount, or if we scaled something up,
then scaling it back down would be its inverse. Of course, these are not the only operations
that we can do with matrices, we only used these for the sake of demonstration. The lead author states on his website that
this paper shows that “linear algebra can be replaced with machine learning”. Talk about being funny and tongue in cheek. Also, I have linked the website of David in
the description box, he has a lot of great works and I am surely not doing him justice
by of all those great works, covering this one. Rufus von Woofles, graduate of the prestigious
Muddy Paws University was the third author of the paper, overlooking the entirety of
the work and making sure that the quality of the results is impeccable. As future work, I would propose replacing
the basic mathematical operators such as addition and multiplication by machine learning. Except that it is already done and is hilariously
fun, and it even supports division by zero. Talk about the almighty powers of deep learning. Thanks for watching, and for your generous
support, and I’ll see you next time!

Author: Kevin Mason

14 thoughts on “Estimating Matrix Rank With Neural Networks | Two Minute Papers #94

  1. Was this episode understandable? Were there terms that were not explained clearly? Or was everything fine in there? Let us know, we love to read your feedback! 🙂

  2. <Personal Opinion>This is typically a domain where the Deep-Learning should not be used ( :</Personal Opinion>
    We understand enough matrices why do we want an approximative result where the exact one exist?
    <Objection of the Personal Opinion>Exception of my senstence: When we want fast approximative result, I should test it on large matrix 10^5; 10^6 before to claim that :D</Objection of the Personal Opinion>

  3. Haha, that was great!
    I'm glad that you went a little more in-depth this time around. Ironically enough, you did that for a joke-paper.
    I think this is closer to what this channel should be than the last couple videos. – For me this was still too light to really learn something new (I know linear algebra and am familiar with ranks) but that's probably not necessarily the case for the majority of watchers.

  4. Karoly, I love your channel so far and look forward to every video. Have you heard about the recent paper "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"? The results are absolutely mind-blowing, and the 4x downscaled then upscaled image looks nearly indistinguishable from the original image. I know some people look down on anything that uses a GAN, but I think the results speak for themselves. I'm guessing you already saw it on /r/machinelearning, but here it is if you haven't seen it yet: http://arxiv.org/abs/1609.04802

  5. In the video, you're showing the web interface, which (if I understand correctly) can't use machine learning (that's why it says "real"). So you're just showing an ordinary calculator. You have to download and build the electron app to use the learned operations.

    However, I was unable to get anything but "real" mode to work by just following the readme. The "=" button seemed to just have no effect. A shame, because it's a really cute idea.

  6. I lold. I am myself trying to train a deep neural network on videos of me working so that I can take the day off, courtesy of machine learning.

  7. But why? XD It feels like calculating the rang of a matrix would be one of the last things that you would be able to teach a neural network.

Leave a Reply

Your email address will not be published. Required fields are marked *