The quipu: a short warning

An Inca quipu, from the Larco Museum in Lima, Peru.
Source: http://en.wikipedia.org/wiki/Image:Inca_Quipu.jpg

The quipu, or khipu, was a sophisticated Andean record-keeping technology made from knotted cords. The Inka used knots, colours and cord arrangements to encode information, and specialist practitioners made, kept and read them. Important aspects of their numerical structure can still be interpreted, but much of the knowledge needed to recover their fuller meanings was lost following conquest and the disruption of khipu practices.

The warning is this: An information system can leave behind records or outputs while the interpretive knowledge needed to understand how meaning was produced becomes weakened or detached from the people responsible for acting on them.

AI-supported organisations could create a version of this problem even while producing abundant records. Outputs may remain visible and reviewable, yet the person approving them may become less able to reconstruct why certain evidence was emphasised, which alternatives were excluded, what assumptions shaped the analysis or where value judgements entered the process. Formal authority can remain human while practical agency shifts earlier into the AI-supported workflow.

The stick chart: a different relationship with technology

A Micronesian navigational chart from the Marshall Islands, made of wood, sennit fiber and cowrie shells.
Source: https://commons.wikimedia.org/wiki/User:Cullen328

Marshallese stick charts reveal a different relationship between knowledge and technology. Different chart types represented islands, routes and patterns in ocean swells, and many were studied and memorised on land rather than carried to sea. During a voyage, navigation still depended on situated experience, sensory judgement and responsibility for the course taken. The chart supported the navigator’s knowledge; it did not navigate on the navigator’s behalf.

The stick chart did not replace the navigator. It helped form the navigator

The Marshallese stick chart is also where our recent research on value-centric AI begins. We use it to make a specific point: a technology for orientation can help people develop and exercise judgement in complex conditions without removing the need for situated judgement. This leads us to ask how AI-supported sociotechnical systems can keep humans not only “in the loop”, but “in practice”.

That is a more demanding test of human oversight. A system can preserve a formal approval step while gradually shifting the work of framing, interpreting and judging upstream into AI-supported activity. One risk we identify is capability erosion: if people routinely rely on AI to draft, classify, summarise or recommend, they may stop practising the judgement needed to challenge its outputs. The person still signs off, but the consequential framing has already happened elsewhere.

Keeping humans in practice means designing AI-supported work so that people continue to interpret evidence, contest framing, resolve value tensions, make consequential interventions and remain responsible for the course taken.

This does not require people to redo everything the AI has done. The point is to preserve the situated expertise needed to know when an output fits the context, when it does not, what assumptions deserve challenge and when the system should be overridden.

For governance, the question becomes: is the system helping people develop and exercise the judgement needed to interpret, contest, explain and take responsibility for consequential decisions?

What this changes for AI governance

That shift in practical agency is what we describe as delegated agency: humans may retain formal authority while practical agency moves earlier into the workflow. Two further problems can follow. The first is accountability drift, where people remain answerable for outcomes over which their practical control has diminished. The second is a legitimacy gap: even technically strong decisions can be difficult to defend when those responsible cannot explain, contest or revise how they were reached.

Governance designed around preserved human practice therefore has to reach inside the workflow. Provenance, assumptions, confidence boundaries and contextual information need to remain reconstructable. The role assigned to AI should be explicit, whether advisory, assistive or decision-executing, with meaningful override rights, escalation paths and stop conditions. Important organisational values and responsibility for resolving trade-offs need clear owners. And evaluation should ask whether people retain the capacity to interpret, challenge and exercise accountable judgement over time, alongside measures of accuracy, safety and productivity.

For leaders, this is a test of organisational capability, not just procedural compliance. Can the people responsible for an AI-supported decision still reconstruct how it was reached, interrogate the evidence and assumptions, challenge the system’s framing, override it where necessary and explain the outcome? AI adoption should be judged not only by the outputs it produces, but by the independent judgement people remain able to exercise.

A human signature at the end of an AI-supported process tells us who approved the outcome. It tells us little about how that person participated in reaching it: whether they interpreted the evidence, challenged assumptions, shaped the framing or retained the expertise needed to intervene. Like the Marshallese stick chart, the technology should support people in developing and exercising the expertise needed to navigate complex conditions. Human judgement should remain practised, consequential and accountable.