Complementarity-Aware Collaboration

Two people use the same AI system, with the same access, on the same class of problem. One gets measurably better at their job over eighteen months. The other gets measurably worse at the parts of the job the system doesn't do for them. Same tool. Opposite trajectories. The difference is not aptitude with the technology. It's a skill almost nobody names, trains for, or hires for — the ability to know, moment to moment, whether your own judgment or the system's recommendation is more likely to be right.

Call it complementarity-aware collaboration: not humans and AI doing different tasks, but a person who understands, in each specific situation, what each party actually does better — and adjusts accordingly, instead of defaulting to either systematic deference or systematic override.

Why "use AI more" is the wrong training goal

Most corporate AI training optimises for fluency — better prompts, faster workflows, more features used. Fluency is real and worth having. It is also nearly orthogonal to the skill that determines whether AI use compounds someone's judgment or quietly erodes it. A fluent user who defers to the system by default gets faster at producing outputs they can no longer independently evaluate. That's not a hypothetical risk. It's the default trajectory of unstructured AI adoption, because deference is the path of least resistance and the system never signals when it's wrong with the same confidence it signals when it's right.

Training that stops at fluency produces exactly the asymmetry this creates: people who can operate the tool and people who can evaluate what it produces, with a shrinking overlap between the two groups the longer unstructured use continues.

The three components, and why the third is the one that matters

Complementarity-aware collaboration rests on three interdependent competencies. The first two are necessary. The third is the one organisations consistently fail to build.

Domain understanding

Knowledge of the subject matter sufficient to evaluate an AI output on its merits, not merely receive it. Without this, the human step in a workflow is not a check. It's a rubber stamp with a person attached. This is why AI deployed to compensate for a skills gap tends to produce the worst version of this failure: the person asked to supervise the system is the same person who lacked the expertise the system was brought in to cover, which means nobody in the loop can tell good output from confident-sounding output.

The absence shows up quietly. A junior analyst using an AI system to draft financial commentary will accept a plausible-sounding explanation of a variance because they don't yet have the reference points to recognise it's the wrong explanation — not because the tool failed them, but because evaluating its output was never something they could do, tool or no tool. The system didn't create the gap. It just made the gap harder to see, because the output no longer looks unfinished the way a junior analyst's own uncertain first draft would.

Process understanding

A working model of how the system transforms inputs into outputs — what kinds of errors it's prone to, what it optimises for, what categories of input it systematically handles worse. This isn't the same as technical expertise in how the model works internally. It's operational literacy: knowing, for example, that a model trained predominantly on one type of case will produce fluent, confident, and occasionally wrong answers on edge cases it saw rarely — and knowing which of your own recurring problems look like edge cases to the system, even when they don't feel unusual to you.

This is the competency most easily built through deliberate practice rather than accident, because it's really pattern recognition about a specific tool's failure modes, and failure modes are discoverable through structured probing rather than years of passive exposure. Someone who has spent a focused afternoon deliberately testing where a system breaks knows more about its process boundaries than someone who has used it daily for a year without ever pushing past the cases it handles comfortably.

Metacognitive awareness — the rare one

The capacity to know, in the moment, whether your judgment in this specific situation is more or less reliable than the system's recommendation — and to act on that assessment rather than falling back on a fixed rule. This is genuinely rare, because it requires two things most professional training never builds together: enough humility to recognise when the system knows something you don't, and enough confidence in your own reasoning to override it when you're the one who's right. Most people default to one end or the other. Systematic deference feels efficient. Systematic override feels rigorous. Neither is complementarity — both are a fixed rule standing in for judgment that would need to vary by situation.

What it looks like when the third competency is missing

Consider a common pattern: a professional uses an AI system daily, trusts it on the categories of task where it has proven reliable over months of use, and — because that trust was earned honestly on those categories — extends the same trust to a new category the system has never handled well. Nothing about their process changed. What changed is that reliability is not a property of the system in general; it's a property of the system on a specific class of problem, and generalising trust across categories is exactly the failure mode that domain and process understanding, on their own, don't protect against. Only metacognitive awareness — actively asking "is this the kind of problem where I've verified this system is reliable, or does it just feel similar" — catches that shift before it produces a bad decision.

The inverse failure is just as common and gets far less attention: a domain expert who distrusts every AI output on principle, re-derives everything from scratch, and gets none of the genuine leverage the tool offers on the categories where it has, in fact, proven reliable. This isn't rigor. It's the same fixed-rule substitute for judgment as blind deference, just pointed the other way — and it's expensive, because it forfeits real capability the organisation is already paying for.

Why organisations hire and train for the wrong half of this

Job postings for AI-adjacent roles increasingly list "AI fluency" or "prompt engineering" as a required skill. Almost none list the skill this piece is actually about, because it doesn't have a clean name in most job descriptions and it doesn't demo well in an interview. Fluency is observable in thirty minutes — watch someone use the tool, see if they're fast and competent with it. Complementarity-aware collaboration is only observable over time, across enough decisions to see whether someone's calibration between trust and scepticism is actually tracking the system's real reliability, or just their comfort level with it.

That observability gap has a predictable consequence: companies optimise hiring, training, and performance review around what's easy to measure. Fluency becomes the proxy for competence, and the actual determinant of whether AI use compounds or erodes judgment goes unmeasured and unmanaged — right up until the eighteen-month gap between teams becomes visible in output quality, and by then it's a much harder thing to retrain than it would have been to build correctly the first time.

The fix isn't complicated to describe, even if it's harder to implement than a training day: performance conversations that ask not just "are you using the tool" but "walk me through a recent case where you disagreed with it — what happened, and were you right." That single question, asked consistently, does more to surface and develop this competency than any fluency-focused training programme, because it forces the reflection that builds calibration instead of assuming fluency will produce it as a side effect.

How this connects to human-in-the-loop

Complementarity-aware collaboration is the individual competency that makes a properly designed human-in-the-loop system actually work. A loop can have the right structure — the right information, the right authority, the right moment — and still fail if the person occupying it defaults to deference or override rather than situational judgment. Structure creates the conditions for good judgment. It doesn't supply the judgment itself. That has to come from the individual, and it's a specific, learnable skill rather than a fixed trait some people have and others don't.

Building the skill deliberately

Three practices show up consistently in people and teams that develop this well, none of which require special access to model internals.

Track disagreements, not just outcomes. Every time your judgment and the system's recommendation diverge, log which one turned out right and why. Over enough instances, a pattern emerges — the categories where you should trust the system more than instinct suggests, and the categories where your scepticism has been consistently vindicated. Most people carry a vague, unexamined sense of this. Writing it down converts intuition into something you can actually calibrate against.

Interrogate confidence, not just content. A system that states a wrong answer and a system that states a right answer often use identical language patterns to do it — fluent, declarative, unhedged. Train the habit of asking what evidence the output is actually resting on, independent of how it's phrased. This is the single highest-leverage defence against automation bias, because it targets the exact mechanism — confident tone read as reliability — that makes deference feel reasonable in the moment.

Deliberately test the boundary. Occasionally give the system a problem just outside the category where you know it performs well, specifically to observe how it fails. Systems rarely announce "I'm uncertain here" in a way that's easy to notice. Learning their failure signature — where confidence and correctness quietly decouple — is the fastest way to build the process understanding the second competency requires, and it only comes from active probing, not passive use.

Why this is an organisational problem, not just a personal one

Companies that treat AI competency as a training-day topic — a session on prompting, a policy document, done — are building the fluency layer and skipping the one that determines whether fluency helps or hurts over time. Complementarity-aware collaboration doesn't show up in adoption metrics. Usage goes up either way, whether people are getting sharper or getter duller in the process. It shows up eighteen months later, in the gap between teams that got measurably better at their jobs and teams that got measurably faster at producing work they can no longer independently evaluate — the same gap the two people at the start of this piece represent, playing out at organisational scale.

Measuring for it requires tracking something most companies don't currently track: not whether people use the system, but whether their independent judgment on the underlying problem is holding up or eroding. That's a harder metric to build than an adoption dashboard. It's also the one that predicts whether an AI programme compounds an organisation's capability or quietly hollows it out.

Frequently asked

Is complementarity-aware collaboration the same as critical thinking about AI?

Related, but narrower and more specific. Critical thinking is a general disposition to question claims. Complementarity-aware collaboration is the specific, situational skill of knowing whether your judgment or the system's is more reliable on this particular problem, right now — which requires calibrated knowledge of both your own track record and the system's, not just a general habit of scepticism.

Can this skill be taught, or is it just experience?

It can be taught deliberately, faster than passive experience builds it — the three practices above (tracking disagreements, interrogating confidence rather than content, deliberately testing the boundary) are designed to compress the calibration that would otherwise take years of unstructured use into something closer to months, because they force the explicit reflection that unstructured use leaves implicit.

Does more AI experience automatically improve this skill?

No — this is the central mistake behind most AI training strategy. Volume of use builds fluency reliably. It builds complementarity-aware collaboration only if the person is actively tracking where their judgment and the system's diverge and why. Without that active tracking, more experience just as often entrenches whichever default — deference or override — the person started with.

António Martins is an AI-Human Systems Architect and founder of Bitsapiens, working on the architecture that connects strategy, decisions, human capability, operating systems, and technology.