← Writing / № 001 / Growth · Data

Be crazy about data

"A feature can look fine from every metric you're tracking and still be failing some of the users. The only way you find out is by being a bit crazy about it."

We launched autoswap for our Liquidity Pools on EtherHub. It was THE feature we'd been planning, building, testing and retesting for months. In fact, we launched EtherHub with this feature in mind. Which means, one single tap will get you to perform a complex investment (usually 9+ steps) right away.

But, after launching it we saw that the feature was broken.

A huge chunk of users were planning the investment, approving it, but it ended up failing. From the data in the database, it looked like a hit. But from the user's standpoint, it looked like a scam.

We caught it in 45 minutes. Here's how.

— The setup

The first few days after launching something important are when you find out where your product/feature is leaking. So I was watching. Looking at it in different angles.

That's the not-so-glamorous part of growth work. You have to actually care about the data. Not just collect it.

That's what got me to dig in.

— What I did differently

Since at EtherHub we're a team of 2, I need to use some tools to get things going, otherwise I'd be stuck in SQL all day.

Here's what I did: open Claude Desktop, which I have wired to PostHog via their MCP connector, and just ask.

The strategy here is one sentence: stop describing your data to the AI, let it see the data itself. I swear it'll find a different angle that you didn't expect.

Without the connector, you're copy-pasting query results and translating context. With it, Claude runs everything. Queries, reads the sequences, pulls the events, everything on its own.

My first guess was just the tip of the iceberg, and then Claude did a whole investigation. The connection between Claude desktop and PostHog made me catch it in a few minutes and get it resolved in a few hours.

— The angle matters

Just to be clear, I'm not saying that "AI does analysis for you". I drove the investigation and I asked the questions.

Every hypothesis used to take a few SQL queries, or a few more lines in my Jupyter notebook.

So I'm here today to say, go ahead, use Claude desktop (or any other LLM you like), then connect to PostHog (well, if you don't use it yet you should definitely consider it) and ask questions before you jump into SQL.

Remember, don't give AI your hypothesis right off the bat, let it think and give you another angle. That's the beauty of it.

— So what happened?

• Few minutes to root cause
• Same day fix
• Affected users got a little something from us for the trouble
• PostHog Dashboard created so I can keep a closer eye on it

— The Takeaway

Collecting data is the easy part. Everyone has PostHog or Mixpanel or something else.

A feature can look fine from every metric you're tracking and still be failing some of the users (or most of them sometimes). The only way you find out is by being a bit crazy about it. Especially by asking dumb questions and chasing them down.

That's why I love dumb questions, they always lead to something unexpectedly important.