News Analysis: As Chinese universities use AI detectors to screen theses, questions arise over accuracy
BEIJING, Nov. 22 (Xinhua) -- After five rounds of revisions, one student still had a thesis flagged as 30 percent AI-generated by an artificial intelligence (AI) detector. Such results have become a source of anxiety for many graduating students in China.
As AI detection tools become more widely used in thesis reviews, some universities have made the "AI rate" a formal requirement alongside conventional plagiarism checks. A paper that passes the plagiarism check can still be barred from thesis defense if its AI rate exceeds the permitted threshold.
But the reliability of this technology is increasingly in question. Some detection platforms have flagged passages from classic Chinese literary works, including Zhu Ziqing's "Moonlight over the Lotus Pond" and writings by Lu Xun, as more than 80 percent AI-generated.
These cases highlight a problem facing universities as they try to regulate generative AI: a single percentage says little about how much intellectual work a student has actually done. Four scholars in the fields of law, communication and education, speaking to Xinhua, said the AI rate should not serve as a decisive standard.
WHY DETECTORS ARE INCONSISTENT
Commercial AI detectors use AI to identify AI-generated writing, said Zhang Linghan, head of the institute of AI law at China University of Political Science and Law. They analyze statistical features such as perplexity and burstiness, assuming that machine-generated prose is unusually fluent and standardized while human writing varies in rhythm. Given the probabilistic method, no tool is fully accurate, she said, and a percentage alone "cannot serve as reliable evidence of academic misconduct."
Wang Feng, dean of the School of Communication at East China Normal University, said AI detection differs fundamentally from plagiarism screening. Plagiarism checks compare papers against existing texts and identify sources of overlap. AI detectors, by contrast, rely on algorithmic rules rather than traceable source texts. Results can vary across platforms using different data and algorithms, and can even differ when the same paper is submitted to the same platform at different times, he said.
Classic works can be flagged because the stylistic conventions detectors treat as markers of AI writing largely derive from classic literature, Wang added.
Xiong Bingqi, head of the 21st Century Education Research Institute in Shenzhen, south China, noted that plagiarism checks, introduced in China more than 20 years ago, had never resolved questions of originality: students could evade them by rewording or rearranging borrowed text. AI detectors, he said, similarly cannot determine whether a paper is original or innovative.
WHERE THE LINE IS
The limits of detection matter because students may use AI to search academic literature, polish language, draft an outline or generate an entire paper -- activities with very different academic implications that current detection tools cannot distinguish.
Zhang said the key question is whether the student's own contribution remains "substantive and dominant." Using AI for literature searches, language polishing or formatting generally should not be treated as misconduct and may need no or only brief disclosure, she said. AI-generated outlines are also generally acceptable, provided students check them for factual errors. But if AI contributes to a paper's core ideas or directly generates text used in its main body, such use must be disclosed, she said.
Wang, whose university was among the first in China to issue guidelines on students' use of AI, said East China Normal University initially set a 20-percent limit on directly generated content. The limit applied only to text used without subsequent changes, while text students revised and adapted to their research should not count as directly generated. He also cautioned against judging papers based on their so-called AI feel: scientific writing is often formulaic, and academic value should rest on originality, not prose style.
Chu Zhaohui, a research fellow at the China National Academy of Educational Sciences, stressed that AI cannot replace original research based on real circumstances, experiments and data. He cited two instances in which AI had fabricated material: a sentence generated by AI and circulated online under his name, and a student assignment citing a nonexistent article attributed to him, complete with a title, journal name and page numbers.
Despite these limitations, many Chinese universities have adopted percentage limits on the "AI rate" because they are relatively easy to administer and can serve as a deterrent, Zhang said. But she warned that the thresholds lack strong scientific support, risk penalizing legitimate AI use and have fueled a gray market in services promising to lower AI-detection scores.
Rigid scoring rules can also push students into meaningless rewriting just to lower their AI scores, Zhang said. A better approach, she argued, would allow AI use with transparent disclosure, strengthen process-based evaluation and adviser oversight, and gradually reduce reliance on a single quantitative measure.
WHAT A THESIS SHOULD MEASURE
The debate over AI detection ultimately raises a more fundamental question: what should a university thesis demonstrate?
For Chu, the central issue is not AI but evaluation itself. A good paper should address questions others have not explored or offer insights that advance existing understanding, he said. Theses built on repetition, piles of references or superficial formalities had little value even before AI, and the technology has only made their weaknesses more apparent. Universities, he said, should give greater weight to process-based evaluation while also assessing the final work for its research substance and by the standards of the academic community.
Zhang called for combining institutional rules with human judgment. Universities should assess students throughout the research process, from topic selection and research proposals to fieldwork, writing and the final defense, she said. Substantive judgments about a paper's originality, rigor and research value should be left to advisers and peer experts, with an appeals process for disputed decisions.
Xiong said closer supervision could also reduce reliance on detection tools. Advisers who follow students from topic selection through research and writing can identify many problems before a thesis is completed.
Wang emphasized the role of teachers in making this shift work. Rather than simply barring students from using AI, universities should invest more in improving teachers' AI literacy and exploring how the technology can be integrated into teaching and research. Teachers, he said, should help students explore appropriate uses of AI, guide them in its application and provide academic oversight.
The four scholars' views converge on one conclusion: academic integrity cannot be reduced to a numerical score. A thesis should be judged not by whether a student stays below an AI rate threshold, but by whether the student can pose meaningful questions, investigate them via genuine research and arrive at independent conclusions.