Monoscopic 360 Video Optimization

A very important question centers on the size() of the Processing sketch and number of panels. If the sketch size is too small or there are not enough panels, the end result will be pixelated. The video size might be 4K but there won't be 4K of resolution. If the sketch size is too large or there are too many panels, performance will suffer as Camera3D and Processing will be doing unnecessary work that will not improve the final output.

As stated earlier, it is optimal to have one panel for each of the six camera orientations to minimize the number of calls to the draw() method. The optimal size for the Processing sketch is a square with height \(h_{f}\) and width \(w_{f}\) equal to \(\frac{w_{p}}{\pi}\). This cannot always be achieved for all computers and projection sizes \(w_{p}\), which is why this Generator supports multiple panels per camera orientation.

To obtain the optimal size \(\frac{w_{p}}{\pi}\) we can use trigonometry to find the minimum size that ensures that neighboring pixels in the equirectangular projection are derived from different pixels in the rendering panels. Since an equirectangular projection's resolution is not uniform throughout the entire resolution, this needs to be done in the area where resolution is weakest. It may not be obvious, but that area is along the projection's equator, at the four points that are at the center of the four panels that intersect the equator. This will become more clear later.

Our first step is to derive the link between the pixels in the equirectangular projection and the pixels in each panel.

Consider an equirectangular projection with pixel dimensions \(w_{e}\) and \(h_{e}\). Each pixel in the projection has a \(\theta\) and \(\phi\) value in polar coordinates. The horizontal and vertical step sizes from one pixel to the next are:

\begin{align*} d\theta &= \frac{2\pi}{w_{e}} \\ d\phi &= \frac{\pi}{h_{e}} \end{align*}

The cartesian coordinates of a pixel in the equirectangular projection belonging to the front panel mapped to a sphere with radius 0.5 is:

\begin{align*} x_{s} &= -0.5 \sin(\phi) \sin(\theta) \\ y_{s} &= -0.5 \cos(\phi) \\ z_{s} &= -0.5 \sin(\phi) \cos(\theta) \end{align*}

If we project that point onto the face of a unit cube centered at the origin, we can calculate the coordinates of the new point using an (x,y) coordinate space of that face with an origin in the corner.

\begin{align*} x_{f} &= 0.5 - \frac{x_{s}}{2z_{s}} \\ &= 0.5 - \frac{\sin(\theta)}{2\cos(\theta)} \\ &= 0.5 - 0.5 \tan(\theta) \\ y_{f} &= 0.5 - \frac{y_{s}}{2z_{s}} \\ &= 0.5 - \frac{\cos(\phi)}{2\sin(\phi)\cos(\theta)} \\ &= 0.5 - 0.5 \frac{\cot(\phi)}{\cos(\theta)} \\ \end{align*}

If the face of the unit cube is one of the Generator's panels, the pixel coordinates of the point would be:

\begin{align*} x_{p} &= w_{p} (0.5 - 0.5 \tan(\theta)) \\ y_{p} &= h_{p} \left(0.5 - 0.5 \frac{\cot(\phi)}{\cos(\theta)} \right) \\ \end{align*}

Our goal is to calculate the dimensions \((w_{p}, h_{p})\) of the panel in pixels such that

\begin{align*} w_{p} (0.5 - 0.5 \tan(\theta - d\theta)) &- w_{p} (0.5 - 0.5 \tan(\theta)) &= 1 \\ h_{p} \left(0.5 - 0.5 \frac{\cot(\phi - d\phi)}{\cos(\theta)} \right) &- h_{p} \left(0.5 - 0.5 \frac{\cot(\phi)}{\cos(\theta)} \right) &= 1 \\ \end{align*}

This can be simplified to:

\begin{align*} w_{p} &= \frac{2}{\tan(\theta) - \tan(\theta - d\theta)} \\ h_{p} &= \frac{2\cos(\theta)}{\cot(\phi) - \cot(\phi - d\phi)} \\ \end{align*}

We quickly observe that the calculated dimensions \((w_{p}, h_{p})\) are largest when \(\theta = 0\) and \(\phi = \frac{\pi}{2}\), found at the center of the panel. That means this is the critical point and we can simplify the equations further and substitute in our projection step sizes:

\begin{align*} w_{p} &= \frac{2}{\tan(\frac{2\pi}{w_{e}})} \\ h_{p} &= 2\tan \left(\phi - \frac{\pi}{h_{e}} \right) \\ \end{align*}

We can then calculate the correct panel dimensions are (1304, 1304) pixels for a 4K video.

In Camera3D's example code there is a resolution test sketch for this Generator called Monoscopic360ResolutionTest. You can empirically test these calculations using that utility.

Interestingly, we note that for large values of \(w_{e}\):

\begin{align*} w_{p} &= \frac{2}{\tan(\frac{2\pi}{w_{e}})} \approx \frac{w_{e}}{\pi} \\ \end{align*}

This is the case because

\begin{equation*} \lim_{x \to \infty} x \tan \left( \frac{2\pi}{x} \right) = 2\pi \end{equation*}

There's nothing special about \(\pi\) here as this happens to be true for any constant \(z\):

\begin{equation*} \lim_{x \to \infty} x \tan \left( \frac{z}{x} \right) = z \end{equation*}

We can see this by replacing our trigonometric functions with their respective infinite series, like so:

\begin{equation*} \lim_{x \to \infty} \frac{ x \sin \left( \frac{z}{x} \right)} { \cos \left( \frac{z}{x} \right)} \end{equation*}

\begin{align*} \lim_{x \to \infty} \frac{ x \left( \frac{z}{x} - \frac{(\frac{z}{x})^3}{3!} + \frac{(\frac{z}{x})^5}{5!} - \frac{(\frac{z}{x})^7}{7!} + \cdots \right) } { 1 - \frac{(\frac{z}{x})^2}{2!} + \frac{(\frac{z}{x})^4}{4!} - \frac{(\frac{z}{x})^6}{6!} + \cdots } \\ \end{align*}

Evaluate the numerator's product, positioning us to take the limit.

\begin{equation*} \lim_{x \to \infty} \frac{ z - \frac{x (\frac{z}{x})^3}{3!} + \frac{x (\frac{z}{x})^5}{5!} - \frac{x (\frac{z}{x})^7}{7!} + \cdots } { 1 - \frac{(\frac{z}{x})^2}{2!} + \frac{(\frac{z}{x})^4}{4!} - \frac{(\frac{z}{x})^6}{6!} + \cdots } \end{equation*}

Most of the terms drop out when \(x \to \infty\), proving that

\begin{equation*} \lim_{x \to \infty} x \tan \left( \frac{z}{x} \right) = \frac{z}{1} = z \end{equation*}

It gave me great joy to find an elegant solution to this problem.

My friend Andrey pointed out to me that this can also be reduced to the commonly known limit \(\lim_{x \to 0} \frac{sin(x)}{x} = 1\). Obviously I forgot about this from school, but happily he remembered:

\begin{equation*} \lim_{x \to \infty} x \tan \left( \frac{z}{x} \right) \end{equation*}

Substituting \(u = \frac{z}{x}\),

\begin{equation*} \lim_{u \to 0} \frac{z}{u} \tan(u) = \lim_{u \to 0} z \frac{\sin(u)}{u} \frac{1}{\cos(u)} = z \cdot 1 \cdot 1 \end{equation*}

Much simpler.