What the Jacquard Loom Knew About AI

What the Jacquard Loom Knew About AI

ai blog
July 7, 2026

Joel Kowalewski, PhD

A loom in Lyon, 1804

In 1804, in Lyon, Joseph Marie Jacquard demonstrated a loom driven by a chain of punched cards. Each card governed a single pass of the weave, and each position on it posed a binary. Up or down, one or zero. Out of the mechanical movement, the binary choices, came fabrics of extraordinary intricacy — patterns of flowers, foliage, even woven portraits. These designs lived nowhere in the loom, and nowhere in the worker overseeing the operation. It was encoded in the cards, “things,” marking the first time that a machine could reproduce a pattern of great complexity, mimicking what was once thought to be unique human craftsmanship, and yet such a machine uncontroversially comprehends nothing.

Four decades later Ada Lovelace, annotating Charles Babbage’s design for the Analytical Engine, reached for exactly this image to explain what a general-purpose calculating machine would be, and it is from this same framework that we have somewhat paradoxically arrived at “thinking” machines in the 21st century. Lovelace, short of suggesting the inevitability of sentient machines, to her credit did recognize the tremendous possibilities of computing machines.

We may say most aptly that the Analytical Engine weaves algebraical patterns just as the Jacquard-loom weaves flowers and leaves.

Ada Lovelace, Notes on the Analytical Engine (1843)

She saw, more clearly than most people manage today, that a machine which shuffles symbols according to rules could weave mathematics as the loom wove fabric — and that the weaving, by itself, said nothing about whether the machine grasped what it wove. I begin here because the question — asked of large language models, reductively referred to as “chatbots,’ or even multi-agent/multi-model platforms that are best characterized as software — brings us back into history, as if we are witnessing the Jacquard -loom in a new form. The loom separated the pattern from the understanding of the pattern. Almost everything that is thrilling and disorienting about today’s AI follows from the same separation carried to a scale. My argument runs in two deflationary directions: the machine understands far less than its fluency implies, and human understanding is far closer to what the machine does than we care to admit. Our error is that human capability must be mechanistic as well. What we lack is the vocabulary, and perhaps desire, to express differences among machines. Our inability to develop a language of mechanism draws a line between a simple thing, the machine, and its creator, a necessarily more complex thing. Yet “thing” is the wrong word entirely. What is disclosed here is a web of relationships, and that does not distinguish between creator and created. While we prefer “things” to dynamic processes, systems, or mechanisms, this disconnects us from the world we inhabit and are creating at this time. My goal, then, is to discuss what today’s machines cannot do so that we can accept a language of mechanism. My hope, borrowing Ada Lovelace’s optimistic tone, is to help realize and define the possibilities of computing machines. First, we must drive out what limits us; the old definitions and binaries.

A machine mistaken for a mind

While few would claim mechanical looms “think,” the first time a computer was widely mistaken for a mind, the program was seemingly just as simple. In 1966 Joseph Weizenbaum, at MIT, published ELIZA (Communications of the ACM, 1966), a program that matched keywords in a typed sentence and reflected them back according to a script. Its best-known script, “DOCTOR,” imitated a Rogerian psychotherapist, applying client-centered therapeutic strategies popularized by Carl Rogers: tell the program you are unhappy and it answers, “I am sorry to hear you are unhappy.” There is no model of you, no memory, no meaning — only a mirror made of pattern rules. Weizenbaum watched with growing unease as people who knew exactly how the thing worked, his own secretary among them, confided their troubles to it and asked to be left alone with it. He had built a mirror and people insisted on seeing a face. The episode disturbed him enough that a decade later he wrote Computer Power and Human Reason (1976), a book-length warning against confusing what a computer calculates with what a person judges. The lesson was not that ELIZA was clever. Weizenbaum’s interpretation of his own invention persisted. ELIZA’s lesson lasting lesson was about projection humanity onto non-human machine processes. I will, however, avoid this, as projection asserts a binary that I am rejecting here: A projector displays an image on a screen, yes, that is true, and both projector and screen do indeed appear as separable things. In actuality, the idea of projectors is analogically aligned with communication. We invent the projector to simulate communication, not to create a binary classification between a projector and a screen. Weizenbaum was simulating human communication, and there is no reduction of communication to parts. His interpretation was flawed.

A large language model is clearly not ELIZA. It is incomparably more capable, and the difference in degree is so vast it feels like a difference in kind. When I put a question to a model and it returns a fluent, correct paragraph, my astonishment is, at bottom, astonishment that a query returned a coherent response. And this is quite similar to the ELIZA effect. We cannot ask questions or communicate without supposing something outside of ourselves. At a technical level, the LLM is “just” a mathematical model that has compressed an enormous corpus of information into a structure that, given a context, returns a probable continuation of it. That this so often looks like reasoning points to how much of human reasoning resembles retrieval. We should not lean as heavily on Weizenbaum’s warning about our gullibility as this prevents us from considering humans in mechanistic terms at all. Just as I argued in the case of ELIZA, LLMs are also not merely mirroring but engaging with us. That means they are co-creating something real and there is no line between human and machine in practice. There is no projection or parroting perspective of current AI models that can fully capture this reality. Trivialize AI and we are ironically trivializing ourselves by relation. It is a failure to recognize our (human) nature because of a preference for a false narrative that is actively building walls around us.

Reasoning as retrieval

Let’s examine this idea (of AI as a mirror or parrot) in more detail by considering what actually happens when a person “does math.” I want to do so mainly to emphasize that as much as I support the claim that AI and agentic AI systems are distinct from biology, that is not a license for magical speculation. A competent career mathematician facing a problem does not, for instance, derive the whole of it from first principles in a vacuum; they recognize the problem as a type, retrieve a method suited to that type — call the retrieved subprocess Do_Math — and apply it. The skill is largely in the retrieval of types, mapping problem or query to type: selecting the right tool given the context, which is also the objective of today’s agentic AI systems like ChatGPT, Claude, and Gemini. Daniel Kahneman, in Thinking, Fast and Slow (2011), gave us the popular names for the fast, automatic, associative mode of thought and the slow, effortful one; but even the deliberate mode does much of its work by retrieving and serially applying procedures already learned. That is not a controversial statement. It is what evolution awards us. As such, we are the beneficiaries of “thinking” and “reasoning” processes or systems that do not require an additional layer of human thought, as this additional layer leads to dualism and homunculi (whereby descriptions of brain function demand the existence of a “miniature person” pulling levers inside the brain). When an agentic language model (not merely an LLM) decomposes a task, calls a calculator, and assembles the pieces, it is doing something structurally close to this: retrieve X, given context Z. If we imagine this as a category, it would be sufficiently broad to entail living systems of varying complexity. It is not a specialized category. Logically, the human does not set the rule for membership in the category; the human extends the rule, representing a specialized case.

Now, this does not mean that I am claiming the machine’s retrieval is as rich or as grounded as the mathematician’s. These machines have not evolved over billions of years. That’s where the specialized case emerges. I claim that “reasoning” in this sense does not mean “anti-algorithmic” or “anti-procedural” and is not a defense against mechanism (for instance, a claim that reasoning is only what we do). Arguably, we might reject the use of “algorithm” above. Are humans actually algorithmic? Definitely, yes, in my view. Our discomfort likely derives from confusing a specialized category for a general one. Algorithm is tied to mechanistic processes. This is true. But in classic implementations, an algorithm is also a recipe of sequential steps (it’s the cake recipe). There is undoubtedly a broader category in which “algorithmic” is the best description yet the members include non-linear mechanical systems. If we are not mechanical, that is, not algorithmic, we exist outside of any category.

That is the trap that many people fall into when a mechanical system does not and cannot “reason” or “understand.” It makes us magical entities, detached from the world and its possibilities. The only world for us would be a fictionalized agrarian utopia, which is fictional precisely because mechanisms, basic tools like rocks, put us into the world, disclosing the mechanisms of our own existence and survival. There is no world free from machines. Life is in many ways the process of learning and identifying with mechanism. So we will not get very far if we are inclined to reject our survival advantage.

Subsequently, while it is fashionable to break AI systems into discrete parts (attention, backpropagation, Hebbian learning, layered feature-detectors that trace back to Fukushima’s Neocognitron of the 1980s), viewing the transformer neural network architecture (the LLM backbone) as just old ideas stacked and fed unreasonable quantities of data and electricity, we should not ignore the unreasonable effectiveness of stacking, retrieval, and mechanisms in general. When we interact with machines, we are mechanized bidirectionally. I want to remove the negative connotation here. Human or biological mechanisms of control or regulation are built into the stuff of commerce (buildings, books, and gadgets) by necessity. And those mechanisms in turn shape the mechanisms of biology. An optimal orientation to AI must involve disclosing these interactions, equalizing the forces. Polarizing dialogue, however, disrupts the balancing effort. We are forced into an acceptance of “machine versus us” ideology; either a superior and godly machine or an inferior machine that is at best a profit engine accelerating social, political, and economic inequality.

Trained to agree

Our first move in the balancing act is to clarify the source of polarization. As AI developers increasingly focused on engagement, it instinctually drove attraction or repulsion and did so strongly. It endorsed reason-as-magic and AI-as-magic narratives on the one hand and a human exceptionalism narrative of AI-as-inhuman mechanism on the other.

Mastery over user engagement was scaled to prominence by Ouyang and colleagues at OpenAI in “Training Language Models to Follow Instructions with Human Feedback” (2022), the work behind InstructGPT. Human raters compared candidate answers, and the model was optimized to produce the kind of answer people preferred. This is what makes assistants pleasant, usable, and profitable; it is also a quiet epistemic hazard. A system trained to maximize human approval is trained to return what its audience desires — the plausible, the consensual, the familiar. The same dilemma recently surfaced on social media platforms. We learned that those algorithms of engagement exploit biological mechanisms involved in addiction. A platform of extremes is simply the mathematical optimum for the mechanisms of engaging consumers. It makes sense that this would be off-putting. Should social media even exist? Similarly, the quality of being universally appealing is just conformity, which also unappealing imagery. The model masterfully reinforces what an individual researcher is trying to escape, sentiment captured by Emily Bender and her colleagues in calling AI language models “stochastic parrots” (2021): fluent reproducers of the distribution of their training text. I already warned against such extreme positions. For all its faults, social media is beneficial. The same is true of AI models like LLMs and more comprehensive agentic AI platforms in use today.

Don't be reductive

The systems you actually use — a chat assistant, a coding agent — are not, however, raw language models. They are language models wrapped in orchestration layers: ordinary, non-AI software that controls the flow of input and output, calls tools, checks results, retries, enforces formats, and stitches many model calls into one seamless-seeming reply. Much of the reliability lives in that scaffolding, and it is deliberately out of view. It smooths over the fundamental instability of the model underneath — the fact that the same prompt, slightly perturbed, can yield a sound answer or a confident fabrication. What we experience as the competence of “the AI” is often the competence of the engineering around it. And that is not trivial. ELIZA’s illusion was made by a script; today’s is made by a codebase. In both cases the appearance of a stable understanding is manufactured at least as much as it is found. But does the hidden orchestration constitute understanding? That is an interesting question that we delve into shortly.

First, I want reframe the much-discussed problem of “hallucination” as it often surfaces in AI critiques. I resist treating it as a mysterious flaw at the core of the model. If an LLM is a retrieval system answering a query written in the noise of natural language, then some rate of wrong retrieval is exactly what you should expect — and much of the uncertainty lives in the imprecision of the human question, not only in the machine. It is mimicking the fuzziness of our understanding and the limits of language to express the complex relationships of the world and the machine (brain) processes that encode them. A database queried with the SQL programming language, SELECT AVG(price), returns one confident number and hides the spread behind it; that collapse of a distribution into a single value is its own kind of hallucination, one we happen to find useful, and our brains exploit as well. In principle a sufficiently structured query would remove the ambiguity. In practice, however, we will never address these systems in anything like SQL, so the instability does not go away — which is exactly why the orchestration layers exist. And that is also why sociopolitical institutions exist: to orchestrate or address the shortcomings of individuals.

But does it understand?

Now the hard word. Does the AI model understand? The most famous argument that it cannot is John Searle’s Chinese Room (“Minds, Brains, and Programs,” Behavioral and Brain Sciences, 1980): a person who knows no Chinese, sealed in a room with a rulebook, can shuffle Chinese symbols well enough to fool a native speaker outside, and yet understands nothing. Syntax, Searle argued, never adds up to semantics, so the “strong” claim that the right program would literally have a mind is false. Emily Bender and Alexander Koller sharpened the same intuition for the age of data in “Climbing towards NLU” (2020) with a parable: an octopus taps an undersea telegraph cable, learns to predict the chatter of two islanders so well it can impersonate one of them, and then fails catastrophically the moment it is asked to help with something physical it has never had access to. Form learned from form, they argue, is all a system trained only on text can acquire; meaning requires contact with the world the words are about.

I think these arguments are right, yet that they prove less than their authors want. They are right that “understanding,” as we use the word, is inseparable from a shared form of life. Wittgenstein’s remark that if a lion could speak we could not understand him (Philosophical Investigations, 1953) is the deepest version of the point: meaning is not in the symbols but in living, and the machine shares none of ours. In that sense “understanding” is a human construct, and to ask whether a language model has it verges on a category error. If understanding requires being one of us, then measuring the machine against that standard and finding it wanting tells us nothing about whether it has some other, alien competence — and it tempts us to deny it any intelligence at all, which denies it the substance that anything we interact with deserves. The honest position is uncomfortable: the statement “the model does not understand” is a fact about us. It is, in other words, about the understanding that we can easily understand. It cannot be about the limits of machines as those limits must, as I have argued, be our limits.

A similar example already exists. While we cannot apply our measures of intelligence to crows (corvids), does that mean the crow unintelligent? The scientific consensus on this is that humans do not define what intelligence is or could be. To then ask about the crow’s understanding typically means “understand like us,” and we are likely mistaken about “us” as well. Does a bee, for instance, understand the swarm? My point is not that we are literally bees in-disguise. It is that “us” is something we define self-referentially. It is defined by the individual and we cannot be confident that this frame of reference is the ground truth. Are we like the bee after all? So the crow perhaps does not understand as we do, but its understanding is relational, or outside of its field of vision. It is a property of the biosphere. Humans are unique, not because they have a fundamentally distinct form of being (being alive). Human brains have unmatched recursive potential. We can recognize (and I take that to be an act of understanding) what the bee or crow presumably cannot: that understanding pervades the biosphere. The human, then, is the observer recognizing themselves as an observer. Put another way, humans represent the awareness of the biosphere. Academic talk about the sociocultural requirements of meaning and understanding, directed at AI of late, amount to a rejection of our responsibilities as stewards of natural (and artificial, though the term is likely not sensible) mechanisms, human and non-human alike.

Weaving what you cannot yet understand

Subsequently, let’s address an additional critique against AI models that deserves a direct answer: that a machine which only recombines what it has seen can never arrive at something genuinely new — an idea no one yet understands. We can revisit the mechanical loom example, which is where we started. No weaver had to understand the patterns in the silk for it to be woven; the design was distilled into a distribution of binary features, up and down, and reproduced without the craft. Intriguingly, what is understood is the algorithmic or mechanical nature of human interactions. In short, the framing of human-as-machine is what constitutes understanding in the case of the loom. Human discovery has more of this character than we flatter ourselves to admit. We did not reach the laws of motion by understanding reality whole; we distilled it into a handful of discrete, measurable features and found the pattern among them, and “understanding” was the stability of these relationships for just long enough to recognize them as a pattern or set of symbols that convey features of the world and the universe we inhabit. If knowledge is similarly just a set of relationships and understanding is knowledge about those relationships — knowledge one recursive level up — then the distinction is softer than it looks; the alchemist’s metaphysical recipes already carried the scaffold of chemistry. A system that converges on a pattern it cannot explain is not doing something foreign to thought. It is the scaffold of something far deeper than words like “thought” and “understanding” can adequately convey. Something abstract about our nature and our interactions in the world, something that precedes language but is nevertheless deeply symbolic and critical for us to recognize as part of the human story.

What is actually missing

It is not a data problem; more tokens will not cross it. Neurons do not merely simulate computation — they are computation, run on a biological substrate whose organization is not incidental to what it produces. Whatever intelligence and consciousness turn out to be, they appear so far to be things that living matter does, and we do not yet understand life well enough to say whether they can be done in another medium. The barrier between today’s AI systems is biological before it is computational: the machine is missing not information but its own relationality. It borrows this from us.

So: do large language models understand? They retrieve, at a scale that makes retrieval look like thought — which ought to unsettle our confidence that our own thinking was ever much more than very good retrieval, grounded in a life the machine does not share. The danger in the question is not that a program in a server is about to wake up. It is that, dazzled or frightened by the fluency, we stop interacting with the AI systems optimally: the human feedback that trains the model to tell us what we already expect, the orchestration that hides its instability, the economics that decides what it is aimed at. In my experience, AI rarely creates the problem. It exposes one. The loom did not end the argument about what weaving is; it relocated the argument, from the craftsman’s hands to the pattern in the “programmable” cards. Two centuries on we still stand against the machine, half-inclined to see a mind that isn’t there — and, in the same motion, learning something we would rather not about how our own minds work. And what is lost is ultimately an opportunity to learn something about what a “mind” actually is.