Ji Hengshan, an engineer working in Beijing, trains machines to identify "kissing scenarios" on August 3, 2018. (Photo: CFP)

A scantily clad female in sexy lingerie was jiggling her breasts on a Chinese live streaming app, a scenario which was immediately identified by an AI porn-identification system. The system promptly raised the alarm for a manual check by a human to decide whether to close the account immediately. 
However, the AI system failed to filter out a different stream featuring a woman who was not wearing anything but a necktie, letting the naughty video stream slip through the censor's net.

This is an example of China's fledgling AI porn-identification systems, which keep improving based on their algorithms. This nascent systems are greatly helping China's increasingly stringent crackdown on pornographic content across live-streaming short video platforms.

Content moderating is becoming one of the fastest-growing tasks in China's online videos and news sector. Chinese live-video streaming start-up Kuaishou announced in April that it would recruit 3,000 additional content moderators, bolstering its current 2,000. 

More and more contractors are providing AI content moderation services to cope with skyrocketing demand. But when even humans can't decide what is pornographic and what is acceptable, how can a machine be expected to do it?

Good and bad moans  

China will soon be able to crack down on explicit, violent and pornographic sounds and voices, in addition to pictures and texts. This is the vision of China's e-commerce giant, Alibaba, which recently revealed its voiceprint technology to identify online pornographic content.

The AI technology can identify multiple languages, including Japanese, Russian and English, as well as the Hunan and Sichuan dialects. It can identify the groan sound as well, according to a statement Alibaba sent to the Global Times in August. 

Offline and real-time voices can be converted to words. By matching content on its database, Alibaba's AI system is able to recognize speech containing illegal and objectionable information, such as pornography.

At the beginning, the research group collected over 13 million high-quality porn images from nearly 2,000 websites to teach the AI system, said Wei Shi, the senior algorithm engineer of Alibaba's security department. The group also used a large number of Cantonese TV series to make the AI familiar with local dialects.

The intensity of the official crackdown on pornography calls for AI's efficacy in the handling of pornographic audio.

Compared with the detection of images and text, identifying pornographic audio remains in a fledgling stage, Wei told the Global Times, noting that challenges include how to reduce the interference of background noise when AI examines a voice.

Tuputech, a Guangzhou-based leading AI company in online porn identification, offers thorough filtering of online photos based on its self-developed deep learning system and AI technology. The company claims that its AI system can review at least 1 billion photos every day.

Tuputech's AI moderation system uses deep learning to assign an image into three categories - normal, sexual, pornographic.

Its AI moderator is now able to automatically detect explicitly erotic pictures, such as nipples, sexual behavior, sexual connotation in certain occasions, and anime porn. It can also promptly classify images as "pornographic," and then wait for human intervention in the next stage of censorship.

Jiang Zerong, operations director of Tuputech, said that some AI systems now have the ability to identify sexual organs outlined in a comic book image. 

"It is kind of a protective effort for teenagers who are addicted to comics," Jiang said.

"With the improved training and evolution of deep learning algorithms, the AI moderators are now able to automatically classify an unhealthy image with greater precision," said Jiang.

Challenges remain

However, the results of AI systems on identifying pornographic content are not always satisfactory. Even machine-learning systems of developed tech companies are not immune to mistakes.

The media has reported that porn-hunting AI systems cannot yet tell the difference between certain deserts and nudes.  

AI can make errors such as categorizing sand dunes as pornographic. Those errors are straightforward to correct. Things get more complex when AI must consider nuanced points around context and intent, according to Jiang.

Jiang believes that the degree of openness to sexual culture may also affect such judgments. In the early days, for example, the company classified the well-known marble sculpture "David" by Michelangelo as pornographic. 

Jiang thought David's genitals were too lifelike and might be eroticized. "We are different from Westerners who have a more open mind to sex and greater acceptance of nude art. The statue holds a sexual connotation for some groups, such as young teens, in certain situations," Jiang said. "The decisions must be culturally contextualized."

"Clients normally have their own standards for pornography for different intentions. For clients in the field of arts and culture, the standard is relatively loose. But for live-streaming platforms the rules are absolutely stricter," Jiang said.

"One of our clients defined the standard for pornography as depending on whether it is artistic enough," Jiang said. "Such a boundary is not clear to algorithms. It instead requires intuition and human judgment."

Chinese netizens also gave their concern over the introduction of AI in detecting the eroticized content on internet. The early system blocked more content than necessary - filtering pictures of sumo wrestling, swimming or Western frescos in churches that have nudity.

"Based on a mechanized standard, the AI may indiscriminately sweep much gentle content that could survive a manual review. The ways that people use to satisfy their desire, aesthetic or sexual, would all be blindly blocked," an anonymous Weibo user commented.

However, Wang Sixin, a professor of internet regulation from Communication University of China, disagreed. He suggests that AI's intelligent distinction between erotic art and pornography can be improved by technological advances and further efforts to define a standard.

"The standard between erotic art and pornography is set by humans, and we should not blame machines for not identifying it. Even professionals find it impossible to establish a well-accepted standard when making decisions. Humans aren't perfect, and neither are machines. The problem is not exclusively the machines'," he said. 

Wang believes that the AI porn-hunting will be the future trend for cleaning up cyberspace.

Jiang told the Global Times that giving a clear classification standard is always the first step in designing the system, but the definition and standard need to be continuously revised and advanced when applied in practice. 

He emphasized that his company's AI model is highly flexible and can be adjusted in real time according to user requirements.

When is a banana just a banana?

Despite its disadvantages on distinguishing erotic art and pornographic contents, applying AI technologies in censoring improper content for the internet is still on the increase.  

Jiang, who has more than four years of experience in filtering porn, thanks the improving AI system for releasing him from identifying disturbing pictures.

A trained moderator can examine about 10,000 pictures a day, but an AI program can do 100,000 times as much as humans, for an annual fee of only 1.2 million yuan ($175,000), Wei noted.  

More and more live streaming platforms have now started to seek help from AI - which can quickly detect suspected unhealthy images.  

Both Alibaba and Tuputech admitted that the best censorship mechanism is a combination of AI tech and human labor at the moment.

Before AI was introduced, Jiang always felt embarrassed when asked about his career over fear of potential prejudice and misunderstanding. But now he is much more confident, announcing he works with artificial intelligence.

The future is limitless, said Jiang, who  is currently teaching AI to recognize images that contain soft pornographic content, such as props or fruit that stimulate sexual organs, and even expressions of sexual attraction.