We're Measuring AI Adoption. We Should Be Measuring Whether Anything Got Better.

Does adoption matter if nothing gets better?

Right now, in a lot of companies, someone is building a dashboard that tracks what percentage of employees used an AI tool this week. Seats activated. Prompts per person. Logins trending up and to the right. Somewhere a slide is being finished that says "73% adoption," and everyone in the room nods.

I want to gently ruin that adoption slide.

Adoption is not an outcome. It's activity. And after twenty years of building analytics functions, the fastest way I know to spot a team that doesn't yet understand its own goal is to watch what it measures. Struggling teams count what's easy. Logins are easy. Seats are easy. Prompts per week are easy. Whether any of it made the work better is hard, so it quietly doesn't get counted. 

We reach for the number we can get, not the one that's true. That's the oldest trap in my field, wearing a shiny new outfit.

And here's the mechanism, which is almost a law of nature. The moment you make a number the goal, people optimize for the number instead of the thing it was supposed to stand for. Make adoption the KPI and you will get adoption. You'll also get people pasting a paragraph into a chatbot and pasting it right back out so the usage counter ticks up and the mandate is satisfied. The dashboard turns green. Nothing actually improved. You didn't measure value. You measured compliance, and you taught people to perform it.

Now add the part that actually bothers me, which isn't the bad metric. It's what the bad metric does to people.

When "are you using AI enough" becomes something a leader tracks, it stops being a tool and starts being surveillance. People feel it immediately. They start using AI to look productive rather than to be productive. They stop telling you the truth, which is that for a lot of their real work, the tool didn't help, or made it worse, and they quietly worked around it to hit the number you were watching. You wanted a transformation. You built a pressure to pretend.

So here's what I actually ask my team, and it is not "how much did you use it."

I ask whether it gave them time back, and what they did with that time. I ask where it helped and, just as important, where it got in the way, because the second answer is more useful than the first. I make it completely safe to say "I tried it here and it wasn't better." That sentence isn't a failure to manage away. It's the most valuable data point in the whole rollout, and a mandate is specifically designed to make sure you never hear it.

The things worth measuring are harder to put on a slide, but they're the only ones that matter. Did a decision get better, or get made faster. Did the quality of the work go up. Did someone get an hour back and spend it on the part of the job that actually needs a human, the judgment, the relationship, the hard conversation nobody wants to have. That last one is the whole point for me. I don't want AI so my team can produce more slop faster. I want it so the people I lead can spend more of their day on the work that only a person can do.

An adoption dashboard is sizzle. It's activity dressed up as progress, and it looks great right up until someone asks what changed. The steak is quieter and much harder to fake: better decisions, higher-quality work, and people with a little more room to be human at their jobs.

None of this means don't roll out AI. Roll it out. Give people great tools and real training. Just be honest about what you're measuring, and don't confuse motion with progress. The number of people using a thing has never once told you whether the thing was worth using.

So if you're the person about to present the adoption slide, I'd offer one swap. Instead of "what percentage of our people used AI this week," try "what got measurably better, and what didn't, and how do we know." It's a harder slide to build. It's the only one that tells the truth.

Honest question for the leaders here: what are you tracking for AI right now, and does it tell you anything actually got better? I'd love to hear who's found a metric that isn't just activity in disguise.