The on-chain data looked healthy. Wallet addresses were growing, transactions were moving, token distribution was spreading across the chart the way you want it to. By every number on the dashboard, we were acquiring users.
We weren't. We were acquiring addresses.
A single person can hold twenty wallets. A whale can distribute tokens across hundreds. The metric we were tracking wasn't wrong. It just wasn't measuring what we thought it was measuring. We had built a reporting layer on top of the wrong unit, and every goal we were hitting was a goal we had defined incorrectly.
That was the moment I understood something about data that no dashboard had ever shown me: data doesn't tell you what's happening. It tells you what it was built to count. The rest is your job.
"Data doesn't tell you what's happening. It tells you what it was built to count. The rest is your job."
For a while there was no clear fix. The numbers were technically accurate. Nobody had lied. The dashboard was doing exactly what we built it to do. That was the problem.
The question worth sitting with was simpler than the solution: what did we actually want to know? Not what were we tracking, but what were we trying to understand. Real people choosing the product, returning to it, holding value in it. We had never built anything that could tell us that.
The fix, when it came, was less elegant than I wanted. A second identifier at the point of wallet connection. An email. A social login. Anything that could tie an address to a human being. It wasn't airtight, but it gave us something we hadn't had: a bridge between what the chain recorded and who was actually there.
What shifted wasn't just the metric. It was a habit I didn't have before: asking, before anything gets measured, what is this actually counting, and is that the same thing as what we want to know.
Four Questions. Every Build. In Order.
Most PMs encounter data after the fact. The feature ships, the dashboard populates, someone pulls a report two weeks later. By then, the decisions that shaped what gets measured were already made, quietly, in the architecture, in the events that were or weren't instrumented, in the metrics someone chose to surface and the ones nobody thought to track.
Thinking about data as a layer means making those decisions deliberately, before anything ships. Four questions, every time.
These four layers aren't a checklist. They're a sequence of questions that should be live before a single line of code is written. The PM who waits for the data to arrive after launch isn't thinking about data. They're reacting to it.
Nobody Tells You This When You Start Building
Most product decisions I've seen made in bad faith weren't made by bad people. They were made by people who had the wrong number, or the right number measuring the wrong thing, or no number at all and a deadline anyway. The dashboard looked fine. The goal was green. The product was quietly failing in a way nobody had instrumented.
Thinking about data as a layer doesn't make you an analyst. It makes you the person in the room who knows what the numbers actually mean, which ones to trust, and which questions they can't answer. That's not a technical skill. It's a thinking habit.
And like most thinking habits, it starts before you open any tool, before the first line of code, before the first feature ships.
The Rules, Simplified
- Ask what you're counting.
- Ask if that's the same thing as what you want to know.
- Ask who is here, really.
- Ask whether the product is working, not just whether people are using it.
- Ask how people actually move, not how you designed them to move.
- Ask what they're telling you that no metric is capturing.
- Do all of this before the build, not after.
The data doesn't arrive. You design for it, or you go without it.
The rest is just building.