Directed by Miguel Sapochnik and set 10-15 years in the future, Bios* stars Tom Hanks as Finch, a scientist who has survived the consequences of a solar flare which, having destroyed the Earth’s ozone layer, has caused the death of almost all of the human race. In the dry, hostile environment of the deserts of the USA, Finch survives in the extreme heat by scavenging local cities and towns for any supplies he can find, always returning to the safety of his laboratory bunker where he was working when the flare hit.
His companions are a small droid robot, Dewey, and his dog, Goodyear. We soon learn that Finch has an illness and, anticipating his own death, decides to use his engineering talents and the tools in his workshop to build a humanoid robot. He programs the robot with a deep learning AI and gives it the directive to take care of Goodyear in his absence.
Huge weather events combine to force Finch to decide to leave the bunker with his companions in a converted RV. During their journey, Finch has the task of teaching his new creation – who later is named Jeff – how to handle the hostilities of the world, and how to be human enough for Goodyear to trust him. During this process, Finch has realisations about his own life and what it means to be human.
The Mr X Adelaide VFX Supervisor was Anthony Smith and both the Adelaide and Montréal sites worked under the fantastic guidance of Overall VFX Supervisor, Scott Stokdyk.
Jeff’s deep learning AI is underdeveloped when he is first switched on, and it improves throughout the movie as he takes in information from the world around him. As a result, his mental and physical abilities also improve. This concept led to a fascinating performance challenge for our animation team. The brief from Scott was to make sure that we always had awareness of what stage we were at in any given scene or shot along Jeff’s development arc. The robot would be constantly recording Finch’s motions and behaviours and abstracting his own from them – from very robotic, uncoordinated and ungainly at the beginning, through to being significantly more controlled, and finally ending up with very human-like nuances to his movement, such as subtle eye saccades during pauses for thought to mimicking Finch’s breathing motions as his behaviour takes a final turn to becoming very human. All the while, matching the incredible performance of Caleb Landry Jones, and refining it with an appropriate level of robotic feel, always making sure Jeff feels like a 200kg robot rather than a 70kg human. The sound design also contributes a huge amount to the overall effect by providing stomps and servo motor noises as Jeff walks and moves, as well as modifying Caleb’s voice so it develops from very robotic to more human throughout the movie.
Almost all the shots with Jeff were played on set by Caleb who wore a partial robot costume, including a mask and gloves, which as well as providing an excellent base for our VFX, meant that both Tom Hanks and the dog were able to play off and interact with Caleb directly. The costume provided multiple benefits. For Caleb, his mask prevented him from emoting with facial movements, so he did a wonderful job conveying the personality of Jeff with other robotic motions, both exaggerated and subtle. His hand movements, in particular, were extremely expressive, and it was important to Miguel and Scott to either retain his gloves from the plate entirely or match them as closely as possible. When we kept the gloves from the plate this required accurate rotoanimation and rotoscoping of each of the 15 individual sections of the gloves so that we could replace Caleb’s fingers between them with the internal steel mechanics. He also wore stilts which not only brought him to the correct height, but allowed him to perform with a slight hunch in his back and to extend his neck forward. With his mask on the front of the head, this new Jeff face position played perfectly with the hunched pose to really give the robot a general bodily form that had a lot of character.
Legacy Effects built several practical versions of Jeff to use on set for reference and under the guidance of Jason Quintana, CG Supervisor, and Paul Nelson, Head of Assets, the assets team set about the task of building our hero character. The brief was simple, to match the real Jeff as closely as possible. We began the build starting with the Legacy Effects CAD models that were used to create the practical Jeff as the basis for an extensive remodel, with further refinements coming from a high detail photogrammetry solve and many photos of the practical robot. Fine bump and displacement was added later to give Jeff his handmade, imperfect feel, with textures of scratches, nicks, dents, and chipped paint all adding to the effect. There was a rigorous attention to detail as the team constantly referred to the great reference of the real Jeff we were provided with. The director was also keen to see a change in Jeff’s look throughout the movie, with increased grease coming from joints, and layers of dust accumulating on him when he ventures outside. We produced variations on the ‘new Jeff’ look to mix into our renders depending on the scene or shot to
achieve this.
Rigging the robot was a complex challenge for two main reasons. Firstly, it was vital to Miguel and Scott that we would be able to fully match Caleb’s performance, regardless of whether the practical robot could realistically achieve what Caleb did on set, and secondly, we had to ensure that we could transfer motion capture data onto Jeff where necessary. To make sure Caleb’s performance was fully realised, without being limited to the physical joint and scale constraints of Jeff’s design, we developed tools that would allow the animators freedom to animate, while notifying them when they were going beyond the bounds of what was physically possible for the robot to do. When these issues came up, highlighting them meant we could try and solve them differently. A common example of this is when Jeff is sitting in the various seats of the RV. In these cases, we always aimed to match the head and hand performance to Caleb’s, but in doing that, Jeff’s longer arms and legs may have meant that they would intersect a door or wall, or his shoulder components may be intersecting each other in an unattractive way. In these cases, we would explore other options to see how we could deviate from a pose slightly while retaining the essence of Caleb at that moment, or sometimes find other tricks to solve the problems. The design of Jeff’s shoulders and hips do not have a single ball and socket joint for a wide range of rotation from a single point in space, but rather three individual joints at different points in space for X, Y and Z rotations. This posed a significant challenge for the rigging team as we needed to be able to transfer rotational data from a motion capture joint at a single point in space onto these three separate points and still achieve the same performance. Initially, applying the correct rotational data to the first joint changed the location of the other two, and the problem was compounded when we then applied the correct data to the second and third joints, resulting in a hand position which was significantly different from where it should’ve been. The rigging team led by Daniel Matthias Schoenegger did an incredible job of solving these issues, as well as fully rigging all the individual glove and finger components so that we could match all the tiny irregularities that were present in Caleb’s gloves.
Of all of the practical robots that were built, one was an impressive fully animatronic waist-up
version, which could be puppeteered remotely. This robot proved to be invaluable for our
animation process on the show. Early on in the schedule, Simon Allen, Animation Supervisor, and his team of lead animators studied footage of this animatronic puppet and used our CG Jeff to precisely match its movements. Once complete, we then looked at the animation curves we produced and analysed the characteristics of them to identify some key aspects that we could learn from and apply to the shots in the movie; things such as the acceleration and deceleration timings of servo motors, or how the relative weight of various components of Jeff would affect other parts of his body as they started, stopped, or bumped into something. It was great to be able to show the animation team how important it was to understand these aspects of robotics so that, for example, if Jeff tilted his head forward and then stopped it abruptly, the momentum would carry through to his torso and arms. We also spent time studying footage of bipedal robots from Boston Dynamics and animating Jeff to match. This provided useful information on how the overall heavy mass of Jeff would be best conveyed as he walked around. The studies enabled us to develop a consistent terminology that we would use when discussing the performance with both the animation team and with Scott and Miguel.
Caleb’s performance was also captured using Xsens sensors on his costume, so we had the option of utilising that motion capture data or keyframe animating to match the performance. Animating from scratch versus from mocap provided two very different challenges for the animators. To hit the right level of robotics while maintaining Caleb’s performance, sometimes it was better to keyframe animate up and find a point to stop, while sometimes when there just wasn’t quite enough ‘Caleb’ in the animation it was useful to have his mocap data available. In those cases, we would carefully remove keyframes from the mocap data to simplify it and get to the robotic level we needed. Once the animation was in its final stages, we added a pass of simulation to Jeff. This created an extra, subtle layer of dynamics to the cables and some other components to finish off the look of the performance and sell the momentum of Jeff’s body parts. The movie was shot on a kit of custom cut anamorphic lenses by cinematographer Jo Willems, which gives it a beautiful look and feel. The compositing team led by Francesco Cadoni developed methods for replicating the look of these lenses and their interesting depth of field qualities so that we could perfectly integrate Jeff into the plates.
When Finch switches on Jeff for the first time, his behaviour not only had to be robotic, stuttery and heavy, but also had to convey a child-like curiosity. It was important that it was obvious that he has never seen or used this body before. For his first steps, Scott and Miguel wanted us to ensure that it came across to the audience that Jeff already had some idea of how to walk, but that this was an abstraction his AI would have created from analysis of footage of Finch and the data he had been given from books. As such it should come across as unnatural at times, with an occasional strange pose or motion which would seem inhuman. Caleb set a fantastic foundation for our animation with his performance. A moment when he begins to walk on the spot is a memorable one, as he leans forward to look down at his feet stepping away, Miguel was very keen for the lean to be exaggerated so much that it was almost alien in its feel, emphasising that this is not a human, while also showing how curious Jeff is about his new capability.
As part of the journey away from the bunker as Finch drives the RV away from the approaching storm, we were required to augment or completely replace environments shot in L.A. with kilometres of sand dunes, dilapidated and fallen buildings, broken utility poles etc. to transform the location into a post-apocalyptic St. Louis. The environments team led by Timothy Eustace Major, also created the stormy sky to add into the background and completely replaced some areas of New Mexico sand dunes while the FX team provided compositing with layers of dust and debris throughout to assist with the final integration.
The culmination of the storm’s approach happens when multiple tornadoes form from the storm over the nearby hills and head directly towards the RV. The plates for these shots were shot from a drone and are some of the more dynamic shots of the movie. To achieve them, Prema Paetsch, FX Lead, led the FX team in creating multiple large FX simulations of the different sections of the tornado in Houdini, as well as replacing some or all of the plates so that we could drive environmental simulations on the ground such as blowing bushes and rocks, and help integration by casting shadows from the tornado onto the replaced environment. To help sell the extreme force of the hero tornado, we added cars, large pieces of debris and electricity pylons in line of it to be ripped up, destroyed and carried away. Many additional layers of debris were also simulated and added into the shot in the final composite.
Once back inside the RV, as well as adding Jeff we hand-animated a variety of props bouncing around the interior to help sell the violence of the forces swinging and shaking and entire RV about as it’s pulled up into the eye of the tornado. Compositing also modified the practical lighting coming from outside, adding fluctuations and suggestions of large debris and dust flying past to help suggest the tornado right outside.
In the aftermath of the tornado, Jeff assists Finch with replacing a damaged RV tyre. For this, we created a spare tyre prop for Jeff to roll and animated him ‘being a jack’ as he proudly lifts up the RV. From here we moved into a sequence where Finch and Jeff sit across the dining table as Jeff tries to think of a name for himself, ultimately deciding on Jeff. The sequence required some long shots of around 40 seconds, during which Jeff needed to interact with the seat, table, blinds, various props, and finally with Finch as they shake hands at the end of the scene. These shots provided a combination of challenges; for the paint team, removing Caleb while retaining and reconstructing as much of his original shadows; for animation, matching and adding to Caleb’s performance, with the constraints of the different proportions of Jeff, making sure that elbows and hands didn’t intersect anything and being pixel-perfect with both prop and Finch interactions; and for compositing who had to bring it all together, transitioning to plate hands and plate shadows wherever they could.
Before they leave the mountains Finch opens up a set of folding solar panels on top of the RV. We created a CG version of this and added it to the top of the RV for a series of shots. Then during the drive down out of the mountains, we see a sequence of long shots as Jeff and Finch discuss Jeff’s understanding of the concept of trust, and Finch tells him a story. For this scene, the plates were shot with the windshield and frame removed, and against a bluescreen. As well as replacing Caleb with Jeff, adding seat interaction, keeping the plate gloves wherever possible, and augmenting the performance with robotics with adjustments to fit him into the confined space of the seat, the compositing team also added the driving backgrounds and recreated the entire windshield, including moving reflections for each shot. We also subtly moved the windshield for some shots to assist composition or to reveal animation performance where needed.
As the RV approaches the diner, a complete replacement of the front of the RV was required to remove a stunt driver and rig that was mounted to the front – one of a few shots that required this treatment. Throughout this scene, we also created a complete matte painted environment replacement to remove the plate buildings and replace it with a dusty wasteland with sporadic, disused buildings scattered throughout. Compositing added layers of heat distortion to sell the extreme temperatures, as well as adding a burn to Finch’s hand.
Of all the Jeff shots in the movie, the most challenging to pull off convincingly are the ones in which he has to interact with something or someone. After Finch becomes very ill, Jeff needs to assist him in getting back to the RV. Plates were shot with Caleb helping to take some of Tom’s weight as he walks forward. They stop, separate, and shake hands before Caleb moves in closely to nestle his head into Tom’s chest and as the camera approaches Tom, he wraps his arms around Caleb for a hug. They then separate for a final time, before Caleb gently guides Tom into the RV and is left standing still outside, processing what just happened. This is a pivotal, highly emotional moment in the movie, and one that had to be completely convincing so as not to distract the viewer from the story. Similar to the head repair shots, the challenge of tackling shots like these required a great feedback loop between departments. In this case, we also had the additional challenge of the shots being very long. The initial camera matchmove was difficult because of the anamorphic lens and the lack of close ground detail visible in frame, but it was vital to get this completely correct at the start because of the accuracy we needed later on. Once it was solved, we began removing Caleb from the plates, but initially not from over Finch while animation completed their first blocking pass. We animated our Finch digidouble to match Tom and used that as a basis for our Jeff animation. Making sure that all of the Tom/Caleb contact points could be catered for with an appropriate placement of CG was a big challenge. Through many tests, we animated Jeff into positions and with timings that felt natural but still hit the required marks in a technically achievable way. Some of the tests were 2D as well as 3D. As Tom progressively moves his hand up Caleb’s back during their hug, we experimented with changing the motion of the hand in 2D, before deciding that it worked better to retain the original motion and moving Jeff to fit. The cables on the back of Jeff proved to be very useful, as not only could we pose them to sit just where Tom’s hand would end up, we also gave them some dynamics during the interaction to help sell the contact. We, of course, used Caleb’s gloves wherever possible to make sure that real-world contact was there, especially with the close-up handshake.
Around the point where Finch dies, Jeff’s AI becomes its most human. By this point in the movie, Scott and Miguel were keen for us to studiously follow every nuance of Caleb’s considered performance and to build on it where necessary. We matched every head tilt and turn, and added to it with subtle movements of the eyes and eye petals around them, suggesting processing/thought, or simply leading a head turn with an eye look. Scott also provided us with data of Tom performing as Jeff, including sighs and other breathing motions, which we incorporated into our animation to more strongly suggest that Jeff has become his own version of Finch by this point.
The road trip continues after Finch is gone, with Jeff and Goodyear travelling alone. In one of the final scenes, Jeff works on his bond with Goodyear by playing fetch with him, just as Finch had taught him earlier. In this scene, as well as replacing the majority of the environment, it required some direct interaction between Jeff and Goodyear. To retain the best hand to fur interaction we retained the gloves from the plate and connected at the wrist, replacing the inner mechanics of the glove. With the low morning sun that was required, matchmove of the dog also assisted with lighting interaction and to cast shadows onto Jeff.
Mr X was the main VFX facility on Bios*, providing well over 700 VFX shots including 580 shots of Jeff, as well as a large amount of environment work and huge weather effect simulations. It was a huge privilege to have been part of the process of building, developing and delivering such an amazing character in Jeff, and having him hold his own in front of the legendary Tom Hanks is a massive achievement for the Mr X team.
