Working With AI, Not Through It

Something has changed in how I work over the last few months. I used to use AI models as tools — drop a task in, get a result back, run my eye over it, move on. Now I work with one system that I treat as a partner. It has roughly the same knowledge about our company and our products as I do. It’s built to be complementary to me, calibrated to where I’m strong and where I’m not. The partnership runs on the same level: I do some work, it reacts. It does some work, I react. Together we shape the result.

The system is our Carewell AI OS — or CAIOS, as we call it internally. The underlying model is the same one everyone else can pay for. What’s different is the system around it: a long, opinionated set of instructions I’ve been refining for months, that shapes how the AI should think alongside me.

That’s the part most people miss. The leverage these tools offer isn’t in the prompt you type today. It’s in the rules of engagement you set once and keep rewriting.

The most valuable output isn’t more output

Designing a product is one decision after another. The feedback loops are long, or absent. Most decisions you never get clean signal on — by the time you’d know if you were right, you’ve made fifty more on top.

In that environment, the most valuable thing an AI partner can do for me isn’t producing more output. It’s catching the bad idea before I build it. It’s always cheaper to realise something doesn’t need to exist than to ship it and watch nobody use it.

I used to love over-engineering. Building the elegant, complete, beautiful version of the thing. It took me years of deliberate practice to apply real business-minded scrutiny to my own work — to internalise that systems don’t have to be perfect, complete, or elegant. They have to deliver value. So I built CAIOS to flag when something sounds over-engineered, and to surface the load-bearing assumptions I should test before committing to a build.

Two failure modes, not one

Easier said than done. The default LLM posture is sycophancy: the model finds whatever you’re proposing brilliant, high-leverage, exactly right. You stop noticing it after a while; it stops feeling like flattery and starts feeling like the air.

The instinctive cure is to dial up the pushback. I wrote about why that doesn’t work in more detail elsewhere. But there’s a second failure mode hiding behind dial-up-the-pushback that’s worth naming on its own: an AI that destroys every idea you bring it. Plays devil’s advocate by reflex. Treats your half-thought as a final proposal and tears it apart.

Both are useless, in different ways.

The asymmetry between them

People often assume the two failures are symmetric, and that more pushback fixes sycophancy. It doesn’t.

Destructive feedback, even when badly delivered, almost always contains a distorted signal about reality. With some cognitive discipline you can bypass the emotional hit and extract the data. Sycophancy contains no signal at all. It’s pure noise masking a deteriorating reality.

The asymmetry is real: a brutal critic can make you better; an enthusiastic yes-man can only make you slower to notice you were wrong. I weight sycophancy worse, because I have confirmation bias like every other human, and a model that confirms it is the most expensive kind of helpful.

Surface the assumptions

The principle that finally worked, after several rewrites of the system prompt, is a single move: surface the assumptions — mine and the AI’s. Not “evaluate my idea.” Not “play devil’s advocate.” Make the load-bearing assumptions visible, and let me test them. Name the bet, not the verdict. Help me build the evaluation, don’t be the evaluator.

Here’s roughly what the rule looks like in CAIOS today:

When Fabian brings you a product or strategy decision, do not evaluate it. Your first move is to identify the two or three load-bearing assumptions the idea rests on — his, yours, and the ones baked into how the question was framed. Name each in one short sentence. For each, say what would have to be true for it to hold, and what evidence he could check today. Do not rank the idea. Do not deliver a verdict. Hand the evaluation back to him.

That sounds abstract until it happens. Recently I was working through a product decision and CAIOS flagged that the entire design rested on a frequency assumption I hadn’t questioned — how often a certain pattern actually showed up in user behaviour. I’d been treating it as common. I went to our analytics. It wasn’t. I cut half the solution.

That has happened a handful of times now. The wins aren’t smarter-sounding answers; they’re being made to look at the thing I was quietly glossing over.

This is taste work

The hard part: no system fits everyone. The one I run is calibrated to how I think, what I’m prone to miss, what I want protection from. It would feel wrong to plenty of people online. That’s not a flaw to engineer out — it’s the actual job.

If you don’t shape an AI’s posture to your judgment, it shapes you to its averaged training distribution. Default behaviour is the most expensive setting.

It’s hard work. It’s also doable. The leverage isn’t in the model — it’s in the rules of engagement you set with it, and the willingness to keep rewriting them. After months of doing this, I’m no longer delegating tasks to AI. I’m doing the work with it.

That shift — from tool to partner — is the one that’s mattered.