corrections

This commit is contained in:
CDaut 2023-06-21 22:39:24 +02:00
parent 9dce9af2d6
commit 3796ef3b3d

View file

@ -51,12 +51,12 @@
\maketitle \maketitle
\begin{abstract} \begin{abstract}
Differentiable Programming is a technique frequently used to solve optimization problems by minimizing some kind of error function. To do this though the error function needs to be differentiable with respect to the parameters that are to be optimized. This is usually not the case with ray tracing. This report will explain why this problem occurs and present the method to tackle it developed in~\cite{ACM:diffable_raytracing}. Differentiable Programming is a technique frequently used to solve optimization problems by minimizing some kind of error function. To do this though the error function needs to be differentiable with respect to the parameters that are to be optimized which is usually not the case with ray tracing. This report will explain why this problem occurs and present the method to tackle it developed in~\cite{ACM:diffable_raytracing}.
\end{abstract} \end{abstract}
\section{Introduction} \section{Introduction}
One of the main tasks in Computer Graphics is image synthesis. This means ``given a 3D scene, output an image depicting the scene``. Often it is required for the image to be as realistic as possible, meaning as close to a picture of the scene as if it was set in the real world as possible. This is most commonly achieved using the ray tracing algorithm and is a well studied problem. Doing this in a differentiable way however is much less trivial. This stems from the fact that the rendering integral (equation~\ref{eq:rendering_integral}) is not differentiable in certain well defined places. Because being able to ray trace an image differentiably has numerous applications a solution to this problem has been proposed in~\cite{ACM:diffable_raytracing}. This report will go into how exactly this problem arises, how it can be mitigated and what some of the applications of differentiable ray tracing are. One of the main tasks in Computer Graphics is image synthesis. This means ``given a 3D scene, output an image depicting the scene``. Often it is required for the image to be as realistic as possible, meaning as close to a picture of the scene as if it was set in the real world as possible. This is most commonly achieved using the ray tracing algorithm and is a well studied problem. Doing this in a differentiable way however is much less trivial. This stems from the fact that the rendering integral (equation~\ref{eq:rendering_integral}) is not differentiable in certain well defined places. Because being able to ray trace an image differentiably has numerous applications a solution to this problem has been proposed in~\cite{ACM:diffable_raytracing}. This report will go into how exactly the problem arises, how it can be mitigated and what might be an applications of differentiable ray tracing.
\section{Ray Tracing} \section{Ray Tracing}
To formalize the problem of photo realistic image synthesis, an equation has been proposed by Kajiya in 1986~\cite{ACM:rendering_equation}. This equation captures physical light transport for a scene and if solved yields the color for a given point in the scene accounting for most physical light transport phenomena. To formalize the problem of photo realistic image synthesis, an equation has been proposed by Kajiya in 1986~\cite{ACM:rendering_equation}. This equation captures physical light transport for a scene and if solved yields the color for a given point in the scene accounting for most physical light transport phenomena.
@ -109,7 +109,7 @@ Thus the problem can be mitigated by not sampling a pixel uniformly, but importa
\caption{The figure depicts the sampling situation in a single pixel. The gradient with respect to the gray triangle moving up is the metric to be calculated. Area sampling does not account for this color change whereas edge sampling does. Primary occlusion is handled correctly.}\label{fig:area_vs_edge_sampling} \caption{The figure depicts the sampling situation in a single pixel. The gradient with respect to the gray triangle moving up is the metric to be calculated. Area sampling does not account for this color change whereas edge sampling does. Primary occlusion is handled correctly.}\label{fig:area_vs_edge_sampling}
\end{figure} \end{figure}
Consider a situation as depicted in figure~\ref{fig:area_vs_edge_sampling}. When the gray triangle moves up, the white area decreases and the gray area increases. Thus white should contribute less to the pixel color. Area sampling does not account for this change, since all samples land on an area with the same color before and after movement. Edge sampling does account for this change because especially areas of discontinuity i.e.\ the edges are sampled. The samples will have a different colors after the triangle moves up, correctly capturing the gradient. Primary occlusion is handled correctly because the sample that intersected the blocker before, will also intersect the blocker after movement as it is independent of the triangle movement. Consider a situation as depicted in figure~\ref{fig:area_vs_edge_sampling}. When the gray triangle moves up, the ratio of the white area decreases and the ratio of the gray area increases. Thus white should contribute less to the pixel color. Area sampling does not account for this change, since all samples land on an area with the same color before and after movement. Edge sampling does account for this change because especially areas of discontinuity i.e.\ the edges are sampled. The samples will have a different colors after the triangle moves up, correctly capturing the gradient. Primary occlusion is handled correctly because the sample that intersected the blocker before, will also intersect the blocker after movement as it is independent of the triangle movement.
\subsection{Half spaces} \subsection{Half spaces}
@ -118,15 +118,15 @@ To understand why edge sampling correctly captures the gradient an explanation o
\begin{figure}[h] \begin{figure}[h]
\centering \centering
\input{diagrams/halfspaces.tex} \input{diagrams/halfspaces.tex}
\caption{An edge separates the domain into two regions, whose contribution can be considered separately. The contribution of the upper half space increases while the contribution of the lower half space decreases with respect to the gray triangle moving right. Occlusion is handled correctly because the contribution of either half space is equal.}\label{fig:halfspaces} \caption{An edge separates the domain into two regions, whose contribution can be considered separately. The contribution of the upper half space increases while the contribution of the lower half space decreases with respect to the gray triangle moving right. Occlusion is handled correctly because there the contribution of either half space is equal.}\label{fig:halfspaces}
\end{figure} \end{figure}
As seen in figure~\ref{fig:halfspaces} the edge separates the space into two half spaces. These two halfspaces can now be considered separately and a gradient can be calculated from the color difference. It is also evident that occlusion is handled correctly. This arises from the fact that where occlusion occurs, the half spaces' contribution to color is equal. From that it follows that the gradient will be 0.\\ As seen in figure~\ref{fig:halfspaces} the edge separates the space into two half spaces. These two halfspaces can now be considered separately and a gradient can be calculated from the color difference. It is also evident that occlusion is handled correctly. This arises from the fact that where occlusion occurs, the half spaces' contribution to color is equal. From that it follows that the gradient will be 0 where occlusion occurs.\\
Mathematically this yields that the Integral can be separated into two parts. One part that represents the discontinuous edges and one part that accounts for the original pixel integral over continuous regions. Mathematically this yields that the Integral can be separated into two parts. One part that represents the discontinuous edges and one part that accounts for the original pixel integral over continuous regions.
\section{Results} \section{Results}
The main application of differentiable ray tracing discussed in the paper is inverse rendering. That is: Given images of a scene, synthesize a 3D representation of that scene. To do this an error function $e: \Phi\mapsto\mathbb{R}$ that measures how close an approximate render is to a target image with respect to a set of scene parameters $\Phi$ is required.\\ The main application of differentiable ray tracing discussed in the paper is inverse rendering. That is: Given images of a scene, synthesize a 3D representation of that scene. To do this an error function $e: \Phi\mapsto\mathbb{R}$, that measures how close an approximate render is to a target image with respect to a set of scene parameters $\Phi$, is required.\\
Given this function and a set of target images the inverse rendering problem can be solved using differentiable rendering. The first step is to render an approximation of the scene differentiably. The resulting images are then compared to the target images by calculating the error. Since the rendering process is differentiable the gradient of the error function can be calculated, allowing for a minimization of the error function using a process such as stochastic gradient descent. Given this function and a set of target images the inverse rendering problem can be solved using differentiable ray tracing. The first step is to render an approximation of the scene differentiably. The resulting images are then compared to the target images by calculating the error. Since the rendering process is differentiable the gradient of the error function can be calculated, allowing for a minimization of the error function using a process such as stochastic gradient descent.
\begin{figure}[h] \begin{figure}[h]
\centering \centering