Japan Physical Society Trials AI Detection Software for Scientific Manuscripts

The Rise of AI in Academic Publishing and Japan's Proactive Response

The academic world is grappling with a profound shift as artificial intelligence tools like ChatGPT and its successors become ubiquitous in research workflows. In Japan, the Physical Society of Japan (JPS), a venerable organization founded in 1878 with over 15,000 members including researchers from top universities like the University of Tokyo and Kyoto University, has taken a decisive step. On April 13, 2026, JPS initiated a trial of proprietary AI detection software for screening manuscripts submitted to its English-language journals. This move addresses the growing influx of low-quality, AI-generated papers plaguing global scientific literature, particularly in physics.

Physics, a cornerstone of Japanese higher education and research, relies on rigorous peer review to maintain integrity. JPS publishes flagship journals such as the Journal of the Physical Society of Japan (JPSJ), a monthly peer-reviewed outlet for fundamental physics research, and Progress of Theoretical and Experimental Physics (PTEP), an open-access journal covering theoretical and experimental advancements. These platforms attract submissions from universities across Japan and internationally, making them prime targets for AI abuse.

Global Surge in AI-Generated Manuscripts: Statistics and Concerns

Globally, the proliferation of generative AI has led to an explosion in scientific output, but not always of high quality. According to recent analyses, AI-suspected retractions surged dramatically post-ChatGPT's 2022 release, with over 195 cases by 2026, a 22-fold increase from pre-2022 levels. In 2023 alone, 46.3% of AI-related retractions occurred, often due to fabricated data or citations. Physics journals have not been immune; controversies over poor-quality submissions suspected of AI origin have sparked debates on peer review efficiency.

In Japan, where physics research drives innovations in quantum computing, materials science, and superconductivity—fields led by institutions like RIKEN and Tohoku University—the stakes are high. Universities report increased workloads for reviewers sifting through suspicious texts, diverting focus from groundbreaking work like ultrafast optical switching at Waseda University or stem cell breakthroughs at Kyoto University.

• Over 200 AI-generated papers retracted in high-impact journals since 2023.
• Median time-to-retraction for AI cases: ~600 days, faster than average.
• China leads in volume, but Japan sees rising concerns in English submissions.

Unveiling the Technology: ATIO TEXT and Its Development

The software in question, dubbed ATIO TEXT, was developed by a team at the National Institute of Informatics (NII) led by Professor Isao Echizen, a leading expert in information security and multimedia forensics. Trained on abstracts from over 200,000 papers categorized into three groups—purely human-written, fully AI-generated, and human-edited AI text—it achieves over 95% accuracy. Crucially, it detects AI content even after 'humanizer' tools attempt to mask it by altering phrasing.

AI detectors like ATIO TEXT typically analyze linguistic patterns: perplexity (predictability of text), burstiness (sentence length variation), and stylistic inconsistencies humans rarely produce. Echizen's team emphasized offline functionality to safeguard confidential submissions, preventing data leaks during cloud processing. This aligns with JPS's March 2024 policy prohibiting referees from uploading manuscripts to external AI tools.

"Generative AI prioritizes fluent connections over factual accuracy, often fabricating citations," Echizen noted, highlighting a key vulnerability exploited by bad actors.

Implementation in JPS Workflow: From Submission to Peer Review

During the trial, ATIO TEXT screens abstracts upon submission to JPSJ and PTEP. High-risk flags trigger deeper scrutiny, quantifying AI involvement to aid editors and reviewers. JPS permits AI for polishing grammar or translating but mandates disclosure and holds authors accountable. Editor-in-chief Hiroshi Harima emphasized, "This will deter misconduct and streamline peer review."

Japan's universities, home to JPS members, benefit indirectly as flawed literature erodes trust in physics research ecosystems.

Stakeholder Perspectives: Universities, Researchers, and Critics

At the University of Tokyo, physics faculty welcome the tool amid rising AI use in student theses. Kyoto University researchers note AI aids simulations but warn against overreliance. Critics argue detectors aren't foolproof—false positives could burden honest authors. Yet, with retractions rising 22x globally, proactive measures like JPS's are praised.

• Professors: Enhances efficiency, protects novelty.
• Students: Encourages original thinking.
• International collaborators: Aligns with global standards like APS policies.

Broader Implications for Japanese Higher Education

Japan's top physics departments—UTokyo, Osaka U, Tohoku U—face AI's double-edged sword. Positive: AI accelerates quantum materials modeling. Negative: Erosion of skills if misused. JPS trial sets precedent; expect Chemical Society of Japan or math societies to follow. Universities may integrate similar tools in theses, fostering ethical AI guidelines.

Challenges and Limitations of Current Detection Methods

While 95% accurate, ATIO TEXT may falter on hybrid texts or evolving LLMs. Global stats show detectors improving, but adversarial attacks persist. JPS plans ongoing training data updates. Step-by-step process: (1) Abstract scan, (2) Score generation, (3) Editor alert, (4) Manual verification.

International Comparisons: How Japan Stands Out

Unlike Western journals mandating disclosures (e.g., Nature, Science), JPS quantifies AI via tech. Springer uses Turnitin; Elsevier trials detectors. Japan's offline approach prioritizes security, reflecting cultural emphasis on trust.

Future Outlook: Evolving Standards in Physics Publishing

By 2027, full rollout expected if trial succeeds. Universities like Nagoya U may adopt for internal reviews. Long-term: AI co-authorship norms, watermarking mandates. Actionable insights: Authors—disclose AI use; Reviewers—verify claims; Institutions—train on ethics.

Practical Advice for Japanese Researchers and Students

To thrive amid this shift:

• Use AI as assistant, not author.
• Verify all citations manually.
• Leverage university workshops on ethical AI.
• Explore JPS resources for best practices.

This trial underscores Japan's commitment to scientific integrity, ensuring physics remains a beacon of human ingenuity.

Photo by Annie Spratt on Unsplash