From the People's Daily App.
This is Story in the Story.
Chinese netizens were amused and shocked by a Chinese costume drama that used Artificial Intelligence (AI) technology to replace an actress's face.
According to reports, the producers of the drama Sanqianyasha had to change the actress Liu Lu's face after Liu was reported to have affected the execution of official duties by railway law enforcement officers and disturbed public order.
Liu was detained for the offences at a police station for five days in September 2019.
Mango TV later issued a statement ending their contract with the actress for her behavior, and the drama company used AI technology to make Liu's face look like another actress, Zhang Dingding, due to limited funding and time.
Today’s Story in the Story looks at how AI technology in general is fast becoming a useful tool in television and video productions.
Promotional material of Chinese drama Sanqianyasha (Photo: Sina Weibo)
The problems for Mango TV didn’t end, however, as the distorted face, stiff expression and lack of pairing with the face and neck severely affected the drama's quality and aroused Chinese netizens' dissatisfaction. Many said the producer was irresponsible and that the drama should not be broadcast.
"The AI face looks so scary. I felt like I was watching a horror movie. It would be better to delete the related part or make a mosaic on the face directly," one Chinese netizen said on Weibo.
"I am going to give this drama a bad review. It seems that the popular technology finally has its practical application, but the effect is really bad," another Chinese netizen commented on Weibo.
The drama earned a score of 5.9/10 on review site Douban.com.
Shi Wenxue, a film critic living in Beijing, said that film and television companies face ethical problems when actors get into trouble over illegal behavior.
The producers of the film Monster Hunt chose to replace the actor and reshoot the film after the original actor Ko Chen-tung was arrested for drug abuse. And the producers of Sanqianyasha dealt with the issue by using AI after obtaining consent from Liu and Zhang, but still faced pressure over ethics.
"In order to save costs, the use of poor face-changing technology is a deception for the audience," Shi said.
He said that the time for using AI technology to change faces has yet to come in China but is already mature in Hollywood which has tested the method for more than 10 years. Ang Lee's recent film Gemini Man raised this technology to a high level, Shi noted.
Some effects in the music video (Screenshots of Chris Lee's music video)
Films like Fast & Furious 7, The Curious Case of Benjamin Button and The Irishman also used the technology, but the premise is that these movies are designed to use this kind of technology in advance, including actors appearing as stand-in actors, using motion capture and Computer-generated imagery special effects to complete the final visual effect.
"When facing this kind of situation, the ideal thing for a production company is to use mature and professional technology when shooting and using it afterwards, without violating ethics or laws," Shi said.
Technology giant Intel created its first commercial music video using artificial intelligence (AI) technology in 2017.
The video featuring Chinese pop star Chris Lee, also known as Li Yuchun, was for the songstress’ single “Rainy Day, But We Are Together.”
During the half-a-year production, Intel projected dramatic effects on Lee’s face to "express her inner world." The effects included flying bubbles, twinkling stars and a koi swimming across the bridge of the singer's nose.
The effects on Lee were created with real-time 3D face reconstruction using a machine learning algorithm.
It did so by studying Lee’s facial structures and expressions in still photos, which helped Intel to layer the effects on Lee without asking the singer to wear tracking markers or the filming crew to record in particular angles.
The combination of AI technology and filmmaking is good news for cinematographers, according to International Business Insider, as AI may help "shape their artistic vision and produce such videos at a lower cost."
(Produced by Nancy Yan Xu, Brian Lowe, Lance Crayon and Da Hang. Music by bensound.com. Text from Global Times and CGTN.)