Swisscom - Vivo Tutto
Virtual Production - Pixel accurate Previs to Shoot to Post pipeline.
Swisscom, a leading European telecommunications company, created a series of adverts based on the theme of miniature people working inside electronic equipment – as if the people were doing the work, not the technology.
Take4D’s CineBridge was used as the backbone of two of these adverts, doing the heavy lifting for the pre-visualisation, tech-visualisation and motion control shoot.
Stories (www.stories.ch) was engaged to create these adverts and wished them to be a single continuous camera movement in one take from start to finish with no edits. Everything was to appear as if it was “in-camera” shot in a single pass. The camera moved from one piece of equipment to another exposing the tiny people working inside them.
Previs Overlayed On Top of The Final Advert
Using CineBridge’s unique calibration and motion control tools, plates were shot with pixel accuracy such that no post-production tracking was necessary for the insertion of any of the foreground CG or miniature elements. The real-world camera moved exactly in a one-to-one pixel precise correlation to the pre-visualised planned camera.
In the video below, the pre-visualised camera move and wireframe models can be seen rendered on-top of the final advert. Note that changes were still possible – you’ll notice that the size of the television and the stand that it sits on were changed in the final version.
On-Set Real Time Virtual Production - Live Augmented Reality On Location
Virtual Production is a buzz phrase often used to mean LED volumes, real-time camera tracking and games engine technology, however CineBridge has been able to do all that and more of over two decades.
The Director used Take4D’s CineBridge as his main monitor for much of the shoot as he could see the CG elements live on set. In the video below, you can see, in the bottom right corner, a real-time on set composite being generated live for the Director and crew. This enabled everyone to be able to visualise the foreground elements that were to be inserted later in post-production. The real-time built-in render engine of CineBridge allowed the post-production elements to be made transparent giving the Director the better understand the final composition and the placement of practical props in the real world. Highly detailed texture maps and lighting were not needed at this stage of the production, allowing the team to concentrate on the framing and line-up rather than details that were going to be done at a later stage anyway.
Below, you can see a good example of why the real-time compositing technique was so powerful. Note the chair and the dog. During rehearsals the Director could spot that the chair needed to be turned around as the set-top box would obscure it in its original position. By using this live on set composite, he was also able to find exactly where to place the dog so that it would too would not be occluded by the foreground elements to be inserted later in post-production.
Background Plate
All foreground elements (phones, computers, television, and furniture) were computer generated using 3D photorealistic rendering to such a high quality that they appear indistinguishable from the real-world elements and the background plate. It was decided very early on that motion control was the best way to shoot these adverts and Stories called on Take4D to assist in the planning and shooting of the content staring from the very beginning of pre-production.
These adverts needed to be meticulously designed in advance to ensure that each element (scaled people) could be seamlessly incorporated into a single take of the background plate footage.
Richard Widgery was engaged to work with Stories’ director Tobias Fueter, VFX supervisor Denis Spycher and lead animator Roman Altenburg, to plan the camera movement using Take4D’s CineBridge ensuring that when scaled, the move was still within the physical capabilities of the motion control rig.
This preparation was an iterative process that was done over a week, working remotely in three different countries via the internet using Skype conference calling and FBX file sharing. In a world first at the time, all the miniature elements had to sit within a single continuous take for the whole advert – no margin for error since there would be no cuts or break in the single shot. What made this particularly challenging was the very small scale of the miniature people. In some parts, the camera move needed to be scaled up a 1:1000 times (1.5mm people scaled back up to 1.5m), for example: when the people ran down a network cable.
After the main and the scaled segments of the camera movement had been signed off as working on the motion control rig, shooting could begin in Bucharest, Romania. Richard worked with the motion control vendor on set to calibrate and align the rig, and all camera movements, to the physical sets and run CineBridge in Live Virtual Production mode to provide real-time visualisation of the CG elements against the live camera for the director.
Telephone People
There were two adverts. The first was a domestic setting with a home phone, laptop and a TV, where-as the second was a business office environment.
Two different set and two different women were used, one for each advert to represent the ring tone of the telephones. They needed to be 3mm tall, one playing a xylophone, the other playing a chime musical instrument.
The camera starts on the woman and pulls back, out through the (CG) phone into the room.
This tiny 1:1 camera movement needed to be scaled up 1:500 times to correctly make the actors 3mm in size.
(scale x500)
Network Cable People
From the phone, the camera moved down and over to a laptop which had a network cable attached. The camera passed directly through the center of the cable showing miniature people pushing images toward the computer – representing streaming a movie over the internet.
These people were only 1.5mm tall and tested the accuracy of the motion control rig (which passed with flying colours).
On the left you will see Take4D’s CineBridge being used on set in Live Virtual Production mode. The motion control data was being stream into CineBridge directly from the rig in real-time to generate a virtual 3D camera that precisely matched the movements of the real camera.
A 3D model of the set, including the network cable and representation of the miniature people was imported into CineBridge. The background plate video was live captured from the real film camera and mixed with the 3D virtual camera live.
In this example, you see the camera passing through the (non-existent) network cable live on set. It was extremely impressive to see the repeatability and accuracy of the rig as it passed through the same place in the network cable every time (estimate <0.1mm repeatable accuracy).
This Virtual Production technique was used throughout the shooting of the two adverts and was an essential tool for the director to be able to place set pieces and line up shots.
(scale x1000)
Mobile Phone People
From the network cable, the camera moved down and past a mobile phone. As it passed by, the side opened up to reveal Rocketman inside. Rocketman represents data traveling at high speed from one device to another.
Rocketman in this case was only 15mm tall, so the camera move needed to be scaled up 1:66 times. One challenge here was the speed at which the camera passes the phone. Too high, and the motion control rig would not be able to keep up with it when it was scale up, too slow, and it would not be dramatic enough for the shot to work.
It was decided to actually shoot this at half speed to enable the rig to fly past even faster and to give a slightly undercranked feel to the motion of the actor’s movements.
Another difficulty was that the camera descended from the network cable into the phone fly pass. This descent, whilst small in the real world became very significant when scaled up. Take4D’s CineBridge tools found the optimum curve into and out of the shot, using all of the rigs’ reach, velocity and acceleration capabilities.
On the left, you can see CineBridge in Live Virtual Production mode, this time using the built-in chroma-keyer to enable the Director to place the actor in the scene to ensure nothing poked through the top of the phone.
(scale x66)
Set-Top Box People
This was where the two adverts differ. In the living room version, the camera next moved over to the television and set-top-box. In the office version, it turned back around to look at a woman sitting at the desk.
The office version was very simple since there were no more miniature inserts. Take a look at the picture to the left. Everything other than the background set and the woman were computer generated (the table, laptop, phone, lamp…) – very nice work by Stories.ch.
The living room version presented a different challenge. The planned move could not be shot. The miniature people were 30mm high requiring a scaling of 33 times. However, the distance that the camera moved was too large.
From the moment that the miniature people first enter frame to the end of the shot would have required 100 meters of track with a camera height of 40 meters when scaled up. Even if recorded at half speed, it would have required the rig to travel down the track at 15 meters per second, which was not possible with a Milo Motion Control crane.
CineBridge has special tools to handle this kind of shot. It can modify the camera move whist maintaining the correct perspective to a size and speed that the rig can handle and then generate new data to help compositors reposition and scale the 2D plate back into its correct position during post-production.
In the picture to the left, everything except the background set, chair, miniature people and dog are computer generated.
Since CineBridge has a real-time render engine and knows what the original shot should look like, it took the live motion control data with the live video image, repositioning everything back into its correct position live in real-time. Hence the image on CineBridge’s monitor correctly showed how the final post-production image would look like.
Below that you will also see CineBridge in Live Virtual Production mode, this time, chroma-keying the set-top-box back-in against the live video image (laptop on left of shot – normal video signal on monitor on right of shot).
The virtual camera is being driven by the motion control rig’s data in real-time, so what-ever the rig does, the Director can see how it works against the 3D virtual set.
(scale x33)
