How I Debugged a Production Issue Live on a Call

This is the story of when I was pulled into a live production issue with my CEO and the client’s CTO on the line. Oh Lord, what an unforgettable moment.

On a random Tuesday afternoon, I was giving a presentation on how to create a new AWS Service Catalog product when a Slack ping popped up:

“Urgent! Please jump on this call. Thanks.”

I thought it was a quick side question. I was wrong.

No prep. No warning. And when I joined, I realized the call already had our CEO, my Senior Manager and the client’s CTO  (waiting for answers).

The customer-facing application had gone down. Errors everywhere. Executives staring at me to make it right.

Panic could have set in, but instead, I leaned on my training. (Maybe it’s the military background: always be ready, even when you don’t get the luxury of a heads-up.)

What happened?

As I started searching, I saw it: the app was throwing “Access Denied” errors. I was told by the client: “Users couldn’t perform a critical action in production.”

In that moment, you don’t get to Google. You don’t get to delay. You fix or you fail.

First things first, I checked the logs, traced the issue, and found the culprit: a misconfigured S3 bucket policy. The application had lost permission to write logs.

What I did?

Right there, live on the call, I updated the IAM role, adjusted the bucket policy, and ran a quick test. Seconds later, functionality was restored.

The CEO breathed. The CTO nodded. And I exhaled for the first time in what felt like an hour.

Crisis averted.

What did I learn?

That day reinforced something I’ll never forget: in AWS, a small misconfiguration can have massive customer impact. And when the pressure is highest, it’s not just technical skills that save you, it’s mindset, communication, and calm execution.

Here’s the 4-step framework I rely on when the stakes are at their peak:

  • Stay calm, breathe, and listen first
    Don’t rush. Understand the issue before you touch anything.
  • Triage and narrow the Scope
    Focus only on what’s broken. Strip away noise.
  • Communicate clearly
    Executives don’t need jargon, they need confidence you’re in control.
  • Act decisively, then validate
    Fix, test, confirm. Don’t guess.

My Key Takeaway

That call could have gone south fast. Instead, it turned into one of the defining moments of my career.

Pressure doesn’t just test you, it forges you. And in the cloud world, those tests don’t always come at 2 AM like the memes say. They come when you least expect them.

The real question is: when it happens to you, will you be ready?

Thanks for reading. Be sure to check out my other articles, projects, tutorials, and free resources to help you grow as an AWS Cloud engineer. And don’t forget to subscribe to my newsletter to get the latest cloud news, DevOps trends, and updates on all my upcoming projects.

Leave a Reply

Your email address will not be published. Required fields are marked *