On the Artificiality of AI

There’s lots of philosophical ink spilled on whether AI conscious or intelligent. (See the Substack of my old TA, B. Scot Rousse’s Without Why, for one of many, many examples.) I wish to focus on a different topic, and for the sake of argument will grant that they are at least intelligent in some sense. The question, then, becomes in what sense. Well, artificially, of course. And what does it mean for something to be artificially intelligent? How is it distinct from natural intelligence? (We’ll set aside my qualms with “artificial” and “natural” as qualifiers for the sake of using terms that readers are already comfortable with. Never let it be said I don’t try to meet my audience where they’re at.)

To answer these questions, I find myself returning, as usual, to Gilbert Simondon’s On the Mode of Existence of Technical Objects. This book remains sorely under-read and appreciated, so consider this blog yet another attempt on my part to promote it and Simondon more generally. And let me say it plainly: I think anyone who thinks of themselves as a serious technologist should read this book.

Ok, so what does Simondon actually have to say about this topic. Well nothing directly about artificial intelligence or neural nets or anything - he wrote the book in the 1950s and those really weren’t things at the time. But he was engaged with cybernetics, which early AI would have fallen under as a science. What he does talk about is information. He’s well aware of Shannon’s theory of information and is skilled at applying the systematic thinking that it requires to all sorts of technologies. (“Technical objects,” to use his term.)

For Simondon, a key distinction between technical objects and living beings like humans is how we process information, and more specifically how memory works for each. When he examines recording devices like film stock, he finds that what is recorded doesn’t have an inherent structure. Clear-cut forms like individual tree leaves are recorded at precisely the same quality as the dirt of a hiking trail; though people may see forms, the recording doesn’t contain them in itself. “The inability of the data-preserving function of the machines is relative to the recording and reproduction of forms.” Moreover, he says “[t]his incapacity is general, it exists at every level.” This fact about technical memory has an important implication: it means that the memory can be wiped and the recording surface reused, and that the iterative recordings are fundamentally different from one another. We know this, he says, because if there is residue left from a previous recording then it “interferes with” the subsequent recording and we consider it damaged.

Living beings, in this case humans, work in the exact opposite way. We recall forms. We have impressions that stay with us and may not be erased between encounters with the form; if such erasure does happen then we consider the person to have experienced something harmful. Instead, according to Simondon, iterative encounters with variations on a form actually improves both our understanding of the thing in question and our ability to better understand it in the future. To quote him at length:

Human memory receives contents that have a formative power in the sense that they overlap with themselves, are grouped, as if acquired experience served as a code for new acquisitions, in order to interpret and fix them: in man and more generally in the living being content becomes coding, whereas in the machine code and content remain separate as condition and conditioned. Content introduced into human memory will superimpose itself on prior content and take form on it: the living is that in which the a posteriori becomes a priori; memory is the function by which a posteriori matters become a priori.

One upshot: don’t ask people to memorize random things unless you have some sort of mnemonic device handy to help them keep it all straight. But recording devices are great for that sort of thing!

This distinction that Simondon makes is useful for a few more reasons than that, though. The first is that it gives the lie to what AI systems are doing. They’re not spotting “patterns” in data. In fact, only people do that. What they’re doing is crunching meaningless numbers and then we’re reading meaning into the results. Numbers don’t speak for themselves; people tell stories with them. That includes LLMs, which really do just use a probabilistic process to generate linguistic tokens or “make tool calls.” We then read that as meaningful text or decisions. (It appears that that is psychologically harmful. Or maybe psychologically-harmed people are the ones who can’t help but become victims to the feint? It’s unclear to me which way that wind blows, though I suspect it’s the former. The famous duck-rabbit image is neither a duck nor a rabbit; it’s a bunch of pixels with some coordinated to look like lines which we can see as one or the other.)

The other reasons is that we can finally think coherently about what machines are good at versus what people are good at. And that you need both kinds of memory, both kinds of processing information, to adequately understand sociotechnical systems and perform resilience effectively! I’ll hold off on elaborating on this point for a future post. But it’s all there if you care to read the book, as I suggest anyone interested in advancing resilience engineering do.