daily

Lessons from the Trenches: The Realities of AI in Technical Program Management

As a seasoned Technical Program Manager, I reflect on my experiences with AI projects and share lessons on vendor evaluation, ethical data use, and sustainable adoption. Together, we can navigate the complexities of AI and ensure its responsible integration into our workflows.

TPMxAI

06 Apr 2025 — 4 min read

Lessons from the Trenches: The Realities of AI in Technical Program Management

Balancing Excitement And Responsibility

It was a Friday afternoon, the kind that usually draws out the best in my colleagues—laughter, coffee, and the faint hum of optimism that the weekend was just around the corner. Yet, there I was, staring at my screen, feeling a mix of anxiety and determination as I prepared for an AI project kickoff meeting. I could sense the excitement in the air, but I also felt the weight of responsibility. How do you manage something as unpredictable as AI while ensuring that we don’t get swept up in the hype?

Reflecting on my years as a Technical Program Manager (TPM), I realize that my role has shifted dramatically with the advent of AI technologies. The buzzwords are everywhere—machine learning, neural networks, natural language processing—but the real work lies in bridging the gap between innovation and practical application. Let’s dig into how we can manage AI projects effectively, keeping our feet on the ground while our heads explore the clouds.

Evaluating Vendors: Beyond the Hype

When it comes to choosing AI vendors, it’s tempting to be dazzled by flashy presentations and promises of revolutionary capabilities. I remember a project where we were tasked with selecting an AI vendor to enhance our customer service operations. The vendor pitched an impressive demo, showcasing how their chatbot could handle a plethora of queries. Yet, as a TPM, I knew that the glitzy surface didn’t reveal the full story.

My team and I began by seeking out references, digging into case studies, and even reaching out to current clients. It was crucial to understand not just the technology’s capabilities, but also its limitations. What support would we receive post-implementation? How did they handle data security and compliance? These questions helped peel back the layers of vendor promises and reveal the substance beneath.

Guardrails for Data and Ethics

Data is the lifeblood of AI, but it’s also a double-edged sword. Early in my career, I learned the hard way that failing to establish guardrails can lead to catastrophic results. During a project involving predictive analytics, we inadvertently fed our model biased data, which skewed results and eroded trust among stakeholders.

Now, I advocate for an ethics-first approach from the onset. This includes involving legal and compliance teams early in the process and ensuring that our data sources are diverse and representative. We set clear guidelines around data usage, privacy, and bias mitigation strategies to ensure we are not just compliant, but also responsible.

Defining Rollout Phases: The Power of Incremental Adoption

In my experience, one of the most effective strategies for AI deployment is to adopt a phased rollout approach. I recall a project where we launched a new AI-driven feature without sufficient testing. The result? A cascade of issues that led to frustrated users and a demoralized team.

Now, I recommend clearly delineating rollout phases: starting with a pilot, gathering feedback, and making adjustments before a full-scale launch. This iterative approach not only mitigates risks but also empowers teams to learn and adapt. It creates a culture of continuous improvement, which is invaluable in the rapidly evolving AI landscape.

Measuring Latency and Throughput: The Unsung Heroes of AI Performance

As TPMs, we often focus on high-level goals and outcomes, but the devil is in the details—namely, latency and throughput. During a project involving real-time data processing, we encountered performance issues that our initial KPIs didn’t capture.

By incorporating specific metrics around latency and throughput, we were able to identify bottlenecks and address them proactively. This not only improved the user experience but also showcased the value of our AI solution to stakeholders. Remember, in the world of AI, fast and accurate is often better than flashy.

Establishing Feedback Loops: The Heartbeat of Continuous Improvement

Finally, a lesson I’ve learned is the importance of maintaining a feedback loop between product teams and operations. After one project, we launched an AI feature without a clear mechanism for collecting user feedback. The result? We missed critical insights that could have improved functionality and user satisfaction.

Now, I prioritize establishing regular check-ins and surveys to gather feedback from users and operational teams. This practice not only enhances the product but also fosters a sense of ownership among team members. AI is not a set-it-and-forget-it endeavor; it requires ongoing collaboration and communication.

A Thoughtful Reflection

As I look back on my journey as a TPM navigating the complexities of AI, I’m reminded that while the technology holds immense potential, our approach must be grounded in reality. We have the responsibility to cut through the hype, surface risks, and prioritize sustainable adoption. It’s a balancing act, but one that can lead to transformative outcomes.

To the junior PMs out there, remember that your role is not just to manage projects but to be a steward of responsible innovation. Embrace the challenges and learn from them.