It Asked Better Questions Than Many Humans
Academia prides itself on rigor, yet much of that rigor is hidden behind slow, opaque peer review. I recently published a paper after going through the usual review process. I wasn’t done. I was keen to experiment and try something unconventional: I asked Chatgpt to do something far more threatening to academic comfort:
“Critically review this paper as a journal editor.”
What followed was unsettling.
Not because the AI was brilliant — but because it was relentless. It asked clearer, harder questions than many human reviewers do.
What ChatGPT did better than humans
First, it had no incentive to be polite.
It didn’t soften criticism because it knew me, feared retaliation, or wanted to appear generous. It simply asked:
- What is actually new here?
- Is this theory doing real explanatory work — or just decorating the paper?
- Why do these hypotheses exist at all?
- Would a skeptical editor accept this contribution?
These are the questions that matter.
And too often, they are the ones many reviewers avoid.
Second, it treated journal fit as intellectual, not cosmetic.
Early-career scholars are often told to “reframe” papers for different journals, as if it’s marketing. ChatGPT made something brutally clear: journal fit is conceptual. The same paper that works for one journal can be fundamentally misaligned with another — not because of wording, but because of theory and assumptions.
Still, scholarship remains human work.
If an AI can reliably identify overclaiming, theory misuse, hypothesis inflation, and journal misalignment, then we should ask harder questions.
I’m glad I let AI interrogate my previously published work. It exposed a side of mine that needs improvement. In about 5 years, I expect PhD students to work under Agentic AI advisors!