We all knew AI would even­tu­al­ly gen­er­ate fake cita­tions. That was almost the bor­ing part. The more inter­est­ing ques­tion is why so many of them passed through sys­tems designed to eval­u­ate knowl­edge in the first place.

A recent study audit­ed 111 mil­lion ref­er­ences across 2.5 mil­lion papers and preprints. Its esti­mate: near­ly 147,000 hal­lu­ci­nat­ed cita­tions entered sci­en­tif­ic lit­er­a­ture in 2025 alone, many sur­viv­ing peer review and lat­er appear­ing in pub­lished jour­nal arti­cles. The num­bers are strik­ing. But that was not the part that stayed with me.

The Weak Point

What stayed with me was how lit­tle fric­tion a plau­si­ble-look­ing cita­tion can encounter once a sys­tem is already oper­at­ing near capac­i­ty. Sci­ence has always depend­ed part­ly on trust. Orga­ni­za­tions do, too. Peer review­ers are over­loaded, researchers pub­lish under pres­sure, man­agers skim pre­sen­ta­tions between meet­ings. Very few peo­ple can inde­pen­dent­ly ver­i­fy the assump­tions behind a mar­ket fore­cast, an AI roadmap or a strat­e­gy paper. So cred­i­bil­i­ty often gets assessed indi­rect­ly: insti­tu­tion­al rep­u­ta­tion, inter­nal align­ment, famil­iar lan­guage, confidence.

Large lan­guage mod­els fit remark­ably well into envi­ron­ments like these. They pro­duce com­mu­ni­ca­tion already shaped into plau­si­bil­i­ty. Cor­rect struc­ture. Famil­iar cadence. Con­vinc­ing syn­the­sis. A cita­tion for­mat­ted exact­ly the way a cita­tion should look. Often enough to pass. Not nec­es­sar­i­ly enough to be true.

Plausibility Scales Differently

For years, the dom­i­nant assump­tion was that orga­ni­za­tions main­ly suf­fered from too much infor­ma­tion. Too many emails, reports, dash­boards, PDFs, Slack mes­sages. I’m no longer sure quan­ti­ty was ever the hard­est part. The hard­er part may have been ori­en­ta­tion all along. And sys­tems built around plau­si­bil­i­ty behave dif­fer­ent­ly once plau­si­ble com­mu­ni­ca­tion becomes near­ly free.

I notice this chang­ing the way I read. Flu­en­cy used to be a fair­ly reli­able proxy for com­pe­tence. Now I some­times become more atten­tive when a text feels slight­ly too com­plete. Too fric­tion­less. Too per­fect­ly bal­anced. Not because AI-gen­er­at­ed writ­ing is inher­ent­ly bad. Often it is use­ful. Some­times excel­lent. But increas­ing­ly, I pay atten­tion to oth­er sig­nals: speci­fici­ty, selec­tive empha­sis, traces of con­straint, moments where a text reveals trade-offs or lived famil­iar­i­ty with a subject.

The Familiar Becomes Easier to Reproduce

One detail from the study keeps both­er­ing me. The hal­lu­ci­nat­ed cita­tions dis­pro­por­tion­ate­ly assigned cred­it to already promi­nent schol­ars. That feels less like a bug than a struc­tur­al ten­den­cy. Large lan­guage mod­els com­press prob­a­bil­i­ty dis­tri­b­u­tions from exist­ing sys­tems. The famil­iar becomes eas­i­er to repro­duce. Vis­i­bil­i­ty com­pounds. Canon­i­cal lan­guage becomes even more canonical.

And under­neath all this, a feed­back loop is begin­ning to form. Fake cita­tions enter papers. Papers enter data­bas­es. Data­bas­es enter future mod­el train­ing. Future mod­els gen­er­ate new text from con­t­a­m­i­nat­ed corpora.

Fluency Gains Power

Sci­ence will not col­lapse because of this. But our inter­faces with knowl­edge are chang­ing. More and more peo­ple encounter exper­tise indi­rect­ly now: through sum­maries, gen­er­at­ed syn­the­ses, assis­tants, brief­in­gs, AI-medi­at­ed search. Under those con­di­tions, flu­en­cy gains power.

Which may explain why prox­im­i­ty to peo­ple with proven judg­ment sud­den­ly feels more valu­able again. Experts who still know where a num­ber came from. Peo­ple will­ing to say “I don’t know” before pro­duc­ing anoth­er plau­si­ble syn­the­sis. Com­mu­ni­ties where rep­u­ta­tion depends not only on vis­i­bil­i­ty, but on traceability.

That feels less like nos­tal­gia than adap­ta­tion. Plau­si­bil­i­ty is becom­ing abun­dant. Ground­ed inter­pre­ta­tion may not scale as easily.