daily

Metrics That Matter: The Art of Taming Numbers in AI Projects

In the world of AI and TPM, metrics can be both a guide and a trap. Join me as I share insights on leading vs. lagging indicators, the importance of storytelling with data, and the pitfalls of vanity metrics, all wrapped in a humorous narrative from my journey in tech.

TPMxAI

12 May 2025 — 4 min read

Metrics That Matter: The Art of Taming Numbers in AI Projects

Metrics: Your Project'S Best Friends

Picture this: It’s a Tuesday morning, and I’m staring at a colorful health dashboard that looks like a rainbow exploded in a spreadsheet. Each KPI is flashing like a siren, yelling at me about our project’s performance. I’m half-tempted to reach for my sunglasses. Welcome to the glamorous world of Technical Program Management, where numbers dance to the tune of our projects’ successes and failures.

As a seasoned TPM, I’ve learned that metrics are like good friends; some are incredibly supportive, while others might just be there for the party. It’s crucial to differentiate between leading and lagging indicators. Leading indicators are your optimistic friends, the ones who call you before your birthday to remind you that there’s cake coming. In our field, leading indicators can include early user engagement metrics or the speed of feature deployment. They help us predict future success by showing what’s happening now.

On the other hand, lagging indicators are those friends who show up late to the party. They’re great for reflecting on what has already happened but don’t help you figure out if you’re on track for that slice of cake! In AI projects, lagging metrics could be sales figures after a new feature launch or user satisfaction scores collected post-implementation. While they provide valuable insights, they often leave us scrambling to play catch-up.

Now, let’s talk about KPI trees. These bad boys are the equivalent of a family tree for your project metrics. If you’ve ever felt overwhelmed by the sheer volume of data, a KPI tree can help you visualize relationships between different metrics. It’s like organizing your sock drawer; once you see the patterns, everything feels a bit more manageable. For example, if you’re tracking an AI model’s performance, your top-level KPI might be accuracy. Beneath that, you could have sub-metrics like precision and recall, showing how each piece contributes to the whole. It’s a beautiful thing when you can clearly see how your efforts feed into the bigger picture.

Of course, not all metrics are created equal. Enter the villain of our story: vanity metrics. These are the metrics that look great on paper but don’t tell you much about the health of your project. Think of them as the friends who only post their highlight reels on social media. Sure, they may have a million likes, but those numbers might not translate to real-world success. In the AI realm, vanity metrics could be the number of downloads of your app without considering user retention or engagement. They can lead you down a rabbit hole of false confidence, and nobody wants to be the TPM who celebrates too early.

But here’s where the real magic happens: storytelling with data. As TPMs, we don’t just throw numbers at stakeholders and hope they get it. No, we craft narratives that connect the dots. For instance, rather than saying, “Our model’s accuracy is 90%,” I might say, “Our model accurately predicts customer behavior 90% of the time, which means we’re not just selling products; we’re anticipating needs. This allows us to improve customer satisfaction and boost sales.” Suddenly, those numbers come alive, and stakeholders can see the tangible impact of our work.

Making trade-offs visible is another vital aspect of our metrics work. Picture yourself in a meeting room, passionately discussing why we should prioritize improving our model’s precision over increasing user engagement. You might have a beautiful pie chart showing how each trade-off impacts overall business goals. When stakeholders can see the potential consequences laid out in front of them, they’re more likely to buy into your vision. It’s not just about numbers; it’s about the story those numbers tell.

And let’s not forget the role of AI in this metrics game. AI systems can analyze vast amounts of data, providing insights that were previously hidden. Imagine having an AI model that predicts the likelihood of a project delay based on historical data. As a TPM, you can take proactive measures instead of playing the blame game later. However, be cautious—relying too heavily on AI without understanding the underlying metrics can lead to blind spots. Remember, AI is a tool, not a crutch.

So, dear junior TPMs, as you navigate the intricate world of metrics, keep these principles in mind: prioritize leading indicators, craft narratives that resonate, and ensure you’re not getting lost in vanity metrics. The dance with numbers can be tricky, but when you learn to tell stories with data, you’ll not only captivate stakeholders but also illuminate the path for your projects.

In conclusion, let’s embrace the quirks of metrics in our TPM journey. With a sprinkle of humor and a dash of humility, we can turn our dashboards from chaotic rainbows into clear, actionable roadmaps.