The Story of Light ☀️


Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Whenever we look at these amazing research
papers on physical simulations, it is always a joy seeing people discussing them in the
comments section. However, one thing that caught my attention
is that some people comment about how things look, and not on how things move in these
papers. Which is fair enough, and to this end, I will
devote this episode to talk a little about a few amazing techniques used in light transport
simulations. But first things first: when talking about
physics simulations, we are talking about a technique that computes how things move. Then, we typically run a light simulation
program that computes how things look. The two are completely independent, which
means that it is possible that the physical behavior of bread breaking here is correct,
but the bread itself does not look perfectly realistic. This second part depends on the quality of
the light simulation and the materials used there. We can create such an image by simulating
the path of millions and millions of light rays. And initially, this image will look noisy,
and as we add more and more rays, this image will slowly clean up over time. If we don’t have a well-optimized program,
this can take from hours to days to compute. We can speed up this process by carefully
choosing where to shoot these rays, and this is a technique that is called importance sampling. But then, around 1993, an amazing paper appeared
by the name Bidirectional Path Tracing, that proposed that we don’t just start building
light paths from one direction, but two instead – one from the camera, and one from a light
source and then, connect them. This significantly improved the efficiency
of these light simulations, however, it opened up a new can of worms. There are many different ways of connecting
these paths which leads to mathematical difficulties. For instance, we have to specify the probability
of a light path forming, but what do we do if there are multiple ways of producing this
light path? There will be multiple probabilities. What do we do with all this stuff? To address this, Eric Veach described a magical
algorithm in this thesis, and thus, multiple importance sampling was born. I can say without exaggeration that this is
one of the most powerful techniques in all photorealistic rendering research. What multiple importance sampling, or from
now on, MIS in short does, is combine these multiple sampling techniques in a way that
accentuates the strengths of each of them. For instance, you can see the image created
by one sampling technique here, and the image from a different one here. Both of them are quite noisy, but if we combine
them with MIS, we get this instead in the same amount of time. A much smoother, less noisy image. In many cases, this can truly bring down the
computation times from several hours to several minutes. Absolute witchcraft. Later, even more advanced techniques appeared
to accelerate the speed of these light simulation programs. For instance, it is now not only possible
to compute light transport between points in space, but between a point and a beam instead. You see the evolution of an image using this
photon beam-based technique. This way, we can get rid of the point-based
noise and get a much, much more appealing rendering process. The lead author of this beam paper is Wojciech
Jarosz, who, three years later, ended up being the head of the Rendering group at the amazing
Disney Reseach lab. Around that time he also hired me to work
with him at Disney on a project I can’t talk about, which was an incredible and life-changing
experience and I will be forever grateful for his kindness. By the way, he is now a professor at the Darthmouth
University and just keeps pumping out one killer paper after another. So, as you might have guessed, if it is possible
to compute light transport between two points, a point and a beam, later, it became possible
to do this between two beams. None of these are for the faint of the heart,
but it works really well. But, there is a huge problem. These techniques work with different dimensionalities,
or in other words, they estimate the final result so differently, that they cannot be
combined with multiple importance sampling. That is indeed a problem, because all of these
have completely different strengths and weaknesses. And now, hold on to your papers, because we
have finally arrived to the main paper of this episode. It bears the name UPBP, which stands for unifying
points, beams and paths, and it formulates multiple importance sampling between all of
these different kinds of light transport simulations. Basically, what we can do with this is throw
every advanced simulation program we can think of together, and out comes a super powerful
version of them that combines all their strengths and nullifies nearly all of their weaknesses. It is absolutely unreal. Here you see four completely different algorithms
running, and as you can see, they are noisy and smooth at very different places. They are good at computing different kinds
of light transport. And now, hold on to your papers, because the
final result with the UPBP technique is this. Wow! Light transport on steroids. While we look at some more results, I will
note that in my opinion, this is one of the best papers ever written in light transport
research. The crazy thing is that I hardly ever hear
anybody talk about it. If any paper would deserve a bit more attention,
it is this one, so I hope this video will help with that. I would like to dedicate this video to Jaroslav
Krivanek, the first author of this absolutely amazing paper, who has tragically passed away
a few months ago. In my memories, I think of him as the True
King of Multiple Importance Sampling and I hope that now, you do too. Note that MIS is not limited to light transport
algorithms, it is a general concept that can be used together with a mathematical technique
called Monte Carlo integration, which is used pretty much everywhere, from finding out what
an electromagnetic field looks like, to financial modeling, and much, much more. If you have anything to do with Monte Carlo
integration, please read Eric Veach’s thesis and this paper, and if you feel that it is
a good fit, try to incorporate Multiple Importance Sampling into your system. You’ll be glad you did. Also, we have recorded my lectures of a Master-level
course on light transport simulations at the Technical University of Vienna. In this course, we write write such a light
simulation program from scratch, and it is available free of charge for everyone, no
strings attached, so make sure to click the link in the video description to get started. Additionally, I have implemented a small 1
dimensional example of MIS, if you wish to pick it up and try it, that’s also available
in the video description. While talking about the Technical University
of Vienna – we are hiring for a PhD and a PostDoc position. The call here about “Lighting Simulation
For Architectural Design” is advised by my PhD advisor, Michael Wimmer, who I highly
recommend. Apply now if you feel qualified, the link
is in the video description. Thanks for watching and for your generous
support, and I’ll see you next time!

Author Since: Mar 11, 2019

  1. Hey guys, i just have been wondering if there have been any methods that could generate MotionCapture data from just videos?

  2. One thing I learned from all this papers, I'll always choose "Our Method" whenever available in any software..! 😀
    – Thanks a lot for always bringing more and more amazing stuff to light Mr. Fehér !! Great content as always !!

  3. That's pretty incredible! Do we know if any 3D programs have implemented this into their renderer? Or is it still a DIY sort of thing?

  4. Interesting. Logically beams would be more appropriate than points since light behaves as a particle and wave at the same time, meaning it only really behaves as a particle at the very point it bounces off an object, and although that could sum up the points which comprise the many complex angles of the object, it is only a good half of what it really looks like.

    However, I'd like to suppose that Riemman's ζ function in Riemman's hypothesis could take this a step further and reducing the computational time by it's square root. After gandering at how these many beams interact with each other, (Feynman's experiment) you see that the intersecting beams form waves and the waves almost look entirely like circles overlapping each other left and right.

  5. Very very impressive! I can't wait until some of this research becomes available for experimentation in some commercial packages like Blender.

  6. How do you have time to find, understand, and produce a video with animated content, and with this quality of summarization?
    For all the due credit to the paper authors, your work has done them justice. Kudos.

  7. I'm gonna give that PhD job a shot! It sounds like exactly the sort of challenge I'd enjoy. What's PhD stand for? I guess I'll find out if I get the job. Wish me luck!

  8. You have to wonder about the mental states of people who are obsessed with faking reality. There's something off about them that's hard to put your finger on.

  9. You are doing amazing work here! Despite the fact I'm far away from this field, this is truly amazing and even I can appreciate the work and knowledge sharing you are doing! My best wishes and support in future!

  10. I have a question, as i was just watching old footage of the 1920's, is it possible to train AI so that it will generate sound for footage that has none?

  11. This channel is fun because it has character. I'm only interested in AI because it's complex, I don't study or use it or something, but I keep watching this channel because it's not only informative but also fun to watch/listen.

  12. Can you tell if upbp is integrated in such popular production render engines as octane, Arnold or redshift or other? I’m not very good at technical aspects, as far as I know they use qmc algorithm for sampling. Is upbp somewhat different or is it also based on qmc?

  13. This channel genuinely mind-blowing. Thank you so much Doctor! I love that series is free with no strings whatsoever!

Related Post