From 78dc1f96e6e5c1e6074797efa78539cff65baf66 Mon Sep 17 00:00:00 2001 From: CDaut Date: Tue, 27 Jun 2023 21:29:42 +0200 Subject: [PATCH] corrections stefan --- handout/diagrams/area_sampling.tex | 2 +- handout/diagrams/edge_sampling.tex | 2 +- handout/handout.tex | 28 ++++++++++++++++++---------- 3 files changed, 20 insertions(+), 12 deletions(-) diff --git a/handout/diagrams/area_sampling.tex b/handout/diagrams/area_sampling.tex index e23c2f9..37c1979 100644 --- a/handout/diagrams/area_sampling.tex +++ b/handout/diagrams/area_sampling.tex @@ -1,4 +1,4 @@ -\begin{tikzpicture}[scale = 0.6, every node/.style={scale=0.6}] +\begin{tikzpicture}[framed, scale = 0.55, every node/.style={scale=0.6}] %tri after movement \draw[ fill, diff --git a/handout/diagrams/edge_sampling.tex b/handout/diagrams/edge_sampling.tex index 2486b6b..826c0be 100644 --- a/handout/diagrams/edge_sampling.tex +++ b/handout/diagrams/edge_sampling.tex @@ -1,4 +1,4 @@ -\begin{tikzpicture}[scale = 0.6, every node/.style={scale=0.6}] +\begin{tikzpicture}[framed, scale = 0.55, every node/.style={scale=0.6}] %tri after movement \draw[ diff --git a/handout/handout.tex b/handout/handout.tex index 8496757..016894e 100644 --- a/handout/handout.tex +++ b/handout/handout.tex @@ -25,7 +25,7 @@ \title{Differentiable Monte Carlo Ray Tracing through Edge Sampling} \author{Clemens Dautermann} -\date{\today} % Put the date of your talk here +\date{July 26, 2023} % Put the date of your talk here % Bibliography setup \usepackage[citestyle=numeric,bibstyle=numeric,hyperref,backend=biber]{biblatex} @@ -39,7 +39,15 @@ \DeclareCaptionType{eqcap}[Equation][List of equations] \usepackage{tikz} -\usetikzlibrary{calc,patterns,angles,quotes,shapes,arrows, positioning,overlay-beamer-styles} +\usetikzlibrary{calc, +patterns, +angles, +quotes, +shapes, +arrows, +positioning, +overlay-beamer-styles, +backgrounds} \usepackage{subfig} @@ -53,7 +61,7 @@ \maketitle \begin{abstract} - Differentiable Programming is a technique frequently used to solve optimization problems by minimizing some kind of error function. This requires the error function to be differentiable with respect to the parameters that are to be optimised, a condition which usually does not hold with ray tracing. This report will explain why this problem occurs and present the method developed in~\cite{ACM:diffable_raytracing} to tackle it. + Differentiable Programming is a technique frequently used to solve optimization problems by minimizing some kind of error function. This requires the error function to be differentiable with respect to the parameters that are to be optimized, a condition which usually does not hold with ray tracing. This report will explain why this problem occurs and present the method developed in~\cite{ACM:diffable_raytracing} to tackle it. \end{abstract} \section{Introduction} @@ -74,12 +82,12 @@ To formalize the problem of photo realistic image synthesis, an equation has bee \notag \label{eq:rendering_integral} \end{equation} - \captionof{eqcap}{The rendering equation capturing physical light transport. It assigns a value to the the Intensity of light transported from a point $x$ to a point $x^\prime$. The geometry term $g$ will be discussed later. The term $\epsilon$ accounts for the emissivity of the material at point $x$. The integral term represents all light scattered from any other point in the scene towards the point $x$. The integral domain $S$ contains all points in the scene.} + \captionof{eqcap}{The rendering equation capturing physical light transport. It assigns a value to the intensity of light transported from a point $x$ to a point $x^\prime$. The geometry term $g$ will be discussed later. The term $\epsilon$ accounts for the emissivity of the material at point $x$. The integral term represents all light scattered from any other point in the scene towards the point $x$. The integral domain $S$ contains all points in the scene.} \end{eqcap} -This equation (equation~\ref{eq:rendering_integral}) is now widely recognized as ``the rendering integral``. It can not be solved analytically and is thus most commonly approximated using Monte-Carlo integration~-~i.e.\ ray tracing. Ray tracing works by backtracking light rays from the light sources in the scene and thus simulate physically realistic lighting.\\ -To do this, rays are cast from the camera, through each pixel in the camera frustum. The intersection point with the scene geometry $x$ is calculated for each ray and material properties (e.g.\ color, emissivity etc.) are taken into account to calculate the pixel color. From this point more rays are drawn towards each light source. If the light source is visible, its light contributes to the pixel color as well. To account for indirect lighting the ray ``bounces around', yielding a color for some of the points scattering lights towards $x$. This approximates the integral term in equation~\ref{eq:rendering_integral}.\\ +This equation (equation~\ref{eq:rendering_integral}) is now widely recognized as ``the rendering integral``. It can not be solved analytically and is thus most commonly approximated using Monte-Carlo integration~-~i.e.\ ray tracing. Ray tracing works by backtracking light rays from the light sources in the scene and thus simulates physically realistic lighting.\\ +To do this, rays are cast from the camera, through each pixel in the camera frustum. The intersection point with the scene geometry $x$ is calculated for each ray and material properties (e.g.\ color, emissivity etc.) are taken into account to calculate the pixel color. From this point more rays are drawn towards each light source. If the light source is visible, its light contributes to the pixel color as well. To account for indirect lighting, the ray ``bounces around', yielding a color for some of the points scattering lights towards $x$. This approximates the integral term in equation~\ref{eq:rendering_integral}.\\ Differentiable ray tracing is the task of calculating the gradient of this process with respect to \emph{any} scene parameter. \section{Problems with differentiability} @@ -96,7 +104,7 @@ The geometry term $g(x,x^\prime)$ in equation~\ref{eq:rendering_integral} is the \caption{The visibility of a point and the differentiation}\label{fig:visibility_term} \end{figure} -As illustrated in figure~\ref{fig:occlusion_rays} an infinitesimal angle change $d\omega$ can lead to the blocker obstructing the wall or the wall being visible. The geometry term is thus a Heaviside step function which when differentiated yields a Dirac delta functional (see figure~\ref{fig:visibility_term}). Since the Dirac delta functional only differs from 0 in $\omega_0$ one point, the probability of sampling it when using uniformly distributed Monte-Carlo integration is 0. +As illustrated in figure~\ref{fig:occlusion_rays}, an infinitesimal angle change $d\omega$ can lead to the blocker obstructing the wall or the wall being visible. The geometry term is thus a Heaviside step function which when differentiated yields a Dirac delta functional (see figure~\ref{fig:visibility_term}). Since the Dirac delta functional only differs from 0 in $\omega_0$ one point, the probability of sampling it when using uniformly distributed Monte-Carlo integration is 0. \section{Proposed solution: Edge sampling} The key observation is that these discontinuities only occur at the edges of meshes. @@ -117,7 +125,7 @@ Consider a situation as depicted in figure~\ref{fig:area_vs_edge_sampling}. When \subsection{Half spaces} -To understand why edge sampling correctly captures the gradient an explanation of what exactly happens at the point where a ray intersects an edge is required. +To understand why edge sampling correctly captures the gradient, an explanation of what exactly happens at the point where a ray intersects an edge is required. \begin{figure}[h] \centering @@ -125,8 +133,8 @@ To understand why edge sampling correctly captures the gradient an explanation o \caption{An edge separates the domain into two regions, whose contribution can be considered separately. The contribution of the upper half space increases while the contribution of the lower half space decreases with respect to the gray triangle moving right. Occlusion is handled correctly because there the contribution of either half space is equal.}\label{fig:halfspaces} \end{figure} -As seen in figure~\ref{fig:halfspaces} the edge separates the space into two half spaces. These two halfspaces can now be considered separately and a gradient can be calculated from the color difference. It is also evident that occlusion is handled correctly. This arises from the fact that where occlusion occurs, the half spaces' contribution to color is equal. From that it follows that the gradient will be 0 where occlusion occurs.\\ -Mathematically this yields that the Integral can be separated into two parts. One part that represents the discontinuous edges and one part that accounts for the original pixel integral over continuous regions. +As seen in figure~\ref{fig:halfspaces}, the edge separates the space into two half spaces. These two halfspaces can now be considered separately and a gradient can be calculated from the color difference. It is also evident that occlusion is handled correctly. This arises from the fact that where occlusion occurs, the half spaces' contribution to color is equal. From that it follows that the gradient will be 0 where occlusion occurs.\\ +This allows the integral to be separated into two parts: one part that represents the discontinuous edges and one part that accounts for the original pixel integral over continuous regions. \section{Results} The main application of differentiable ray tracing discussed in the paper is inverse rendering. That is: Given images of a scene, synthesize a 3D representation of that scene. To do this an error function $e: \Phi\mapsto\mathbb{R}$, that measures how close an approximate render is to a target image with respect to a set of scene parameters $\Phi$, is required.\\