To tell the story of one of our first-generation Mesmer, Jing Boran, is to wax lyrical about waxworks. Well, silicone works to be exact.
In 2017 Madame Tussauds tasked us with creating their first ever intelligent ‘wax’ figure to promote the launch of Chinese blockbuster film Monster Hunt 2. The idea was to make a lifelike replica of the film’s lead star, Chinese actor Jing Boran, and his monster companion from the film Wuba, to be installed at Madame Tussauds in Shanghai.
A star in the making
When our project manager, Morgan, was shipped off to Shanghai to install the robots he came face to face with the magnitude of how big the film was going to be in China.
“There was an advertising screen at a metro station,” he says, “and there was this video of Wuba with a mop in a toilet, advertising toilet cleaner. It was so strange.”
While this serves as testament to the surrealness of modern advertising, it was also testament to how huge Monster Hunt 2 was in China, and how important the task we’d been given was.
What was to separate Jing Boran from his waxwork counterparts – other than the fact that he was made from silicone – was that he was to be a fully operational robot that could move and interact with people, offering visitors a jaw-dropping experience.
Having skin in the game
In order to create a true to life Jing Boran robot, we had to start with a 3D model of the actor’s skull. This isn’t the sort of item you can grab at your local budget supermarket, so we had to create one from scratch.
Being a world famous movie star, Jing Boran has endured his fair share of photoshoots, but none as in depth as the hundreds of photos we took in order to get an accurate scan of his skull. The technique is called photogrammetry and uses lots of different photos from lots of different angles to build up one complete image. We then took this model and 3D printed it.
You can’t expect an esteemed actor to be photogenic without a strict skincare regime, so once we had the 3D model, we had to add the skin. At this point in our Mesmer journey we hadn’t made any robot skins ourselves. So while we handled the difficult task of creating the body, mechanics, interactions and behaviours, we let the skin experts at Madame Tussauds handle making Jing Boran look dermatologically delightful.
Since creating Jing Boran we’ve innovated new ways of creating silicone skin for our other Mesmer robots, which we make inhouse ourselves. The first Mesmer we created entirely ourselves was Fred, who changed the way we made realistic skin for our robots.
All eyes on you
Rather than just stand there and look strikingly pretty, we wanted Jing Boran to interact with people by tracking them and following them with his eyes. Getting him to detect where people’s faces were and ensure his eyes were following them was a big challenge. But not one we were going to let get in our way. We got to work and were able to achieve realistic eye movement using a series of external cameras.
As with the skin, we have since innovated and updated our way of working and have developed our own cameras that go inside the eyes of all of our Mesmer that give them real robot vision. This allows them to easily track and follow people, giving a more realistic feel to the robot-human interaction.
Jing Boran and Wuba then set up shop nestled amongst some atmospheric lighting and a large screen that displayed animation behind them.
But Jing Boran had a few other tricks up his sleeve. He was able to detect people’s facial expressions, changing the animations and the images on screen with him to match people’s mood. If someone was standing in front of the robot and frowning, thunder and lightning would explode on the screen and his monster friend Wuba would imitate their angry expression.
Similarly, if Jing Boran detected a smiling face, Wuba would match the person’s happiness. This gave the visitors a truly breathtaking, emotional experience. Let’s face it, everyone would love to see their mood mirrored in a robot.
Interfacing and integrating
Once Jing Boran was built, our Director of Operations Morgan Roe, who was project managing the project, headed out to Shanghai to install the robot, which had been shipped out in advance. Having ensured Jing Boran was screwed to his platform, set up and configured, Morgan tested him to make sure everything was working.
Having to interface and integrate our robot system with the existing Madame Tussauds system was a particularly complex hurdle to hop over. But after a tireless effort of sheer determination – combined with some gruelling 16-hour-long days at work – the instalment was finally in place.
The world of robotics moves on quickly and a lot of the technology and tools we used to create Jing Boran that were cutting edge at the time have now been superseded. But designing and creating Jing Boran taught us a lot and has helped us innovate even more with our subsequent robots.
Looking to entertain and mesmerise? Find out how the next generation of Mesmer can help you astound audiences.