So you send months creating sequences in fulldome format... modeling, rendering, compositing... getting to where you want with whatever tricks here and there to get that image in what is truly a wonderful but strange format to work in. And finally you finish and someone latter asks "Could you get that for me in a 16x9 flat-screen version?" And of course it's expected to not cost anything...
INTRODUCTION:
Unless during production you render a specifically designed "flat-screen" version of your scenes (and of course, there was a budget for that too), in the end you have a fulldome circle master that has all these hours of effort and loads of pixel resolution, but no simple way to take that version and convert to an acceptable flat-screen movie with the proper story composition. While experimenting with ways to preview a fulldome show in the studio without the use of a dome (a workstation screen fulldome player, or an OculusTM headset interface), we began to see a possible route to making a conversion possible.
If you consider the way the audience views a fulldome show, by looking around at key points of interest and having their peripheral vision provide the immersive feel... and then compare how the film industry converted wide screen movies for display on rather square standard definition television sets... a pan&scan technique might be a route to converting an immersive fulldome view movie to a 16x9 aspect movie. We began to experiment with making this possible and in a somewhat automatic way.
A PATH TO ACCOMPLISH THIS:
To the fulldome audience, the view on the dome is providing an immersive look into the storytelling scene. Although the video frames start out as a polar orientated circle, when projected onto the dome surface and with a little "suspension of disbelief," the audience gets the feeling of looking into a real three dimensional environment. Part of the reason this works is because we have a brain behind our eyes. As the viewer's head rotates the eyes around to view what they want to look at...the brain interprets what we are seeing and makes it acceptable to what we consider a reality in our space.
Unfortunately, the fulldome pipeline creates a problem when we want to convert that "interpreted" view into a "framed" view... the "suspension of disbelief" for a framed kind of viewing is different and directly slicing out only part of the fulldome circle leaves enough clues to tell the brain that this is not what I should be seeing.
A single fulldome frame and a 16x9 crop out of the same image shows that a simple crop has undesirable problems.
You can obviously see that a straight crop will not work... in order to get a rectangle and have an entire view of interest contained, you may get some of the outside circle... and the fulldoem master has a polar distortion which, while it works on the dome, does not in the 16x9 format. Also taking a curved "pie slice" out of the circle to avoid seeing the black frame around the circle... and "bending" it into a rectangle creates other distortions and artifacts in both perspective and sampling... and if the area of interest crosses the dome zenith (center of the circle), even more distortion is created.
The pan&scan technique used to convert wide screen films to less than wide screen views requires a certain amount of design input to properly retain the filmmaker's composition efforts intended for the storytelling. In that perspective, panning the scan left and right allows the designer to keep what is necessary within the frame. Now, of course viewing the pan&scan result gets lots of film purest in an tissy, but since the fulldome audience is generally doing its own pan&scan as they look around the dome, possibly something similar for our needs may be quite effective for what our goal is.
There are plugins for Adobe AfterEffects that can do a similar kind of conversion... originally designed to go in the reverse direction of rectangular to polar. Some can take a single full view of a good part of the front area of the dome and create a rectangle from that area of the dome. But the view that the audience tends to "zoom into" with their brain becomes consistently "far away" and lessenes some of the intended storytelling composition design. To include the fuller capability of the motion picture pan&scan idea, a way is desired to also animate the viewed area of interest since there are times we will want to creatively do more in the reformatting of the imagery to enhance the storytelling intent than just take the whole front view of the dome.
During past production of fulldome content, we discovered that our Mental Ray fulldome camera rig at times wanted to have more or less than a 180 degree field of view (FOV) to achieve the expected look. So our technical director rewrote the interface to allow a slider to animate and keyframe the FOV as needed. Also, taking the old view-camera ability to offset the film-plane view to correct perspective, an offset ability was also included. We decided to work towards a similar interface to do conversion of fulldome polar masters into a more creatively controlled 16x9 format rectangle.
The concept is basically designed to place a standard film-based Autodesk Maya camera inside a simulated planetarium and animate the view point and look at positions, and FOV as needed, restraining it to proper views and limits to make it easy to get what you want creatively. Then after correctly UV mapping the fulldome polar frames onto the simulated dome we simply render out the camera's views in typical fashion on our render farm. A standard interface of animatable parameter sliders and numeric output is in the works sfor ease of use.
Screen-captures of our audience view camera rig and simulated planetarium dome setup, interactive interface, render view of a HD720 rendered frame and the original fulldome frame.
A sample movie of a short sequence illustrates what a final 16x9 framed output can look like when processed through the Maya interface is available below. Quality is high due to the advanced rendering sampling available in Mental Ray and render times are typically 10 seconds or less per frame.
With a little experience, an animator can take a fulldome sequence and create an acceptaable 16x9 framed movie with a more creative composition style than a straight front view approach.
That completes another Home Run Pictures' "Inside the Production." If you would like more information on the fulldome to 16x9 conversion process, just give me a call or drop me an email.
-TC
SAMPLE RENDERED SEQUENCE - PAN & SCAN AND 16x9
© 2015 H o m e R u n P i c t u r e s