I have just fin­ished Caris­sa Véliz’s new book Prophe­cy, and I can­not stop think­ing about it. The philoso­pher from Oxford has writ­ten a witty, sur­pris­ing, and urgent­ly nec­es­sary account of how gen­er­a­tive AI works — not as a truth machine, but as a fortune-teller.

AI Is Not a Truth Machine — It Is a Fortune-Teller

Large lan­guage mod­els do not “know” any­thing; they pre­dict the most prob­a­ble next token, the most plau­si­ble com­bi­na­tion of words they have seen before. They are, as Véliz puts it, built to be for­tune tellers, not truth tellers. They colonise our lives with cor­re­la­tions while ignor­ing every­thing they do not know. And in doing so, they make Big Tech rich­er and the rest of us less safe and less free.

I love the book for exact­ly the rea­son the New York Times review­er Jen­nifer Sza­lai high­lights: it opens the reader’s mind to entire­ly new dimen­sions of what AI actu­al­ly is. Yet for me, as a tech­nol­o­gist who has worked quite a bit with AI, the sin­gle most impor­tant insight is not about arti­fi­cial intel­li­gence at all. It is about what AI reveals — and dra­mat­i­cal­ly accelerates- about the soci­ety we already live in.

We Have Entered Baudrillard’s Simulacrum

We have qui­et­ly slid into what Jean Bau­drillard called the sim­u­lacrum: a stage of real­i­ty in which signs, mod­els, and clas­si­fi­ca­tions no longer rep­re­sent the world; they pre­cede and cre­ate it. Véliz never names Bau­drillard in the pas­sages I found most pow­er­ful, but her analy­sis of how sta­tis­ti­cal cat­e­gories and pre­dic­tive sys­tems work lands in the same territory.

How Classifications Create the World They Claim to Describe

Here is the mech­a­nism she lays bare (and that I have been watch­ing with grow­ing unease for years):

Pre­cise and stan­dard mea­sures are pre­ferred to accu­rate ones. What mat­ters is that mea­sure­ments play the role that we want them to; more than that, they are a truth­ful reflec­tion of reality.

Clas­si­fi­ca­tions have an impact on people’s lives: peo­ple learn to fit the cat­e­go­ry to com­ply with the sys­tem. Cat­e­gories tend to cre­ate the world they pur­port to rep­re­sent. Sta­tis­ti­cal cat­e­gories give rise to indi­vid­ual and col­lec­tive iden­ti­ties. Those who fail to con­form to tax­onomies are stig­ma­tized and exclud­ed, and most peo­ple end up inter­nal­iz­ing the val­ues of bureau­cra­cy. And then the num­bers start work­ing bet­ter — they com­fort­ably inhab­it the world they built after pun­ish­ing or dis­ap­pear­ing who­ev­er or what­ev­er defied their classification.

The Quiet Death of Common Sense

This is the deep­er story. We have stopped treat­ing real­i­ty as some­thing messy, ambigu­ous, and best nav­i­gat­ed by com­mon sense. Instead, we treat it as a more or less fixed set of clas­si­fi­ca­tions and cat­e­gories that serve as guides through an increas­ing­ly com­plex life. We inter­nalise them. They become real­i­ty. Any­one who does not fit is no longer under­stood; they become out­liers, out­siders, prob­lems to be man­aged or ignored.

The cra­zi­est part? Most peo­ple do not even notice the shift. When in doubt about what to do or how to do some­thing, we no longer ask our­selves what com­mon sense or lived expe­ri­ence would sug­gest. We look up the reg­u­la­tion, the guide­line, the risk matrix, and the approved cat­e­go­ry. The clas­si­fi­ca­tion has replaced judg­ment. Bureau­cra­cy has replaced wisdom.

AI: The Ultimate Booster of the Simulacrum

AI is not the cause of this trans­for­ma­tion. It is the ulti­mate boost­er. Where ear­li­er bureau­crat­ic sys­tems were slow and clum­sy, pre­dic­tive algo­rithms are fast, invis­i­ble, and ter­ri­fy­ing­ly effec­tive. They do not mere­ly describe the world; they opti­mise it accord­ing to the cat­e­gories we have already accept­ed. They pun­ish devi­a­tion before it even hap­pens. They make the sim­u­lacrum run smoothly.

The Turkey That Trusted the Pattern

Véliz’s turkey exam­ple (bor­rowed from Bertrand Russell’s chick­en) is per­fect here. The farm ani­mal trusts the pat­tern — food appears every morn­ing — right up until the day it does not. Our soci­ety is doing the same with its clas­si­fi­ca­tions. We have con­vinced our­selves that if we just refine the cat­e­gories enough, stan­dard­ise the mea­sures enough, pre­dict the prob­a­bil­i­ties enough, real­i­ty will final­ly behave. The num­bers will work. The world will fit the model.

It already does — for those who inter­nalise the model. Every­one else dis­ap­pears from the dataset or gets labelled “non-compliant” — outliers.

Why ‘Prophecy’ Is Not Just Another AI Book

This is why Prophe­cy is not just anoth­er AI book. It is a diag­no­sis of a civil­i­sa­tion­al change that most com­men­ta­tors are still miss­ing. The real dan­ger is not that the machines will become con­scious. The real dan­ger is that we have already out­sourced our sense of what is real to the machines, and to the clas­si­fi­ca­tions they supercharge.

Time to Step Outside the Categories

I rec­om­mend Véliz’s book with­out reser­va­tion. Read it for the sharp his­to­ry of pre­dic­tion from ancient ora­cles to insur­ance actu­ar­ies to today’s chat­bots. Read it for the dev­as­tat­ing clar­i­ty on how pre­dic­tion is real­ly about power. But above all, read it for the larg­er story it tells almost in pass­ing: we are liv­ing inside a self-reinforcing sim­u­la­tion of cat­e­gories, and we are learn­ing to love it because it feels safer than the messy, unpre­dictable world it replaced.

The ques­tion is no longer whether AI will change soci­ety. The ques­tion is whether we still remem­ber what soci­ety looked like before the sim­u­lacrum took over, and whether we still dare to step out­side the cat­e­gories it demands we inhabit.

AI Carissa Veliz divination large language models LLM predictions prophets