Saturday, August 10, 2024

Apologies

From the time we are little, the adults in our lives teach us the basics of civil interaction and discourse by word and example. Say "hello" when you first meet someone, "goodbye" when you leave. Look at someone when you are speaking to them, and do the same when they talk to you. Say "please" when you want something, "thank you" when you get it. Listen quietly to others if they are talking, and if you have to interrupt, start with "excuse me." Ask permission before touching stuff that's not yours. And if you are wrong, accept the blame, apologize, and try to remedy the situation without making it worse.

But what about a company? If we accept Mitt Romney's famous comeback that "Corporations are people," should they not try and follow these same golden rules? Many do, teaching their people to greet customers warmly and listen empathetically, acknowledge problems, and do all they can to remedy issues. You see it with Amazon, with Verizon, with American Express, with almost any major company that wants to have better relations with their clientele. And you saw it with CrowdStrike a few weeks ago. 

That company is a leader in cybersecurity, and its products are used by major players to protect their systems. Big companies use its suite of Falcon products to provide detection, antivirus and firewall capabilities. As they have done countless times before, CrowdStrike rolled out an update to its software, a process that was routine in the past. Except this time it didn't go so smoothly. Instead of seamlessly happening in the background, it caused any computer it touched to go into an endless reboot loop, producing the so called "blue screen of death." Healthcare, airlines, financial services, public transportation, even Starbucks: all were shut down as their computers went from being indispensable tools that ran the minutia of business to bricks that just winked.

The company released a fix, but because the affected computers were effectively lobotomized, it fell to IT staff to fix them manually one at a time, by some estimates 8.5 million machines. Hundreds of engineers worked around the clock to pull each PC from its death spiral. Eventually the problem was remediated, but not before flight delays, rescheduled surgeries and McDonald's in Japan closing some of its stores due to a "cash register malfunction." Estimates are that it cost Fortune 500 companies $5.4 billion in damages with only 10% to 20% covered by insurance.

OK, mistakes happen. This was not cyberterrorism, nor a nefarious plot by some state actor looking to limit the consumption of Big Macs in Osaka. Rather, it was an error in a line of code that looked OK when they tested it, but obviously didn't play nicely in the real world. The only real surprise is that it doesn't happen more often.

Going back to the "people" playbook, the company correctly mea culpa'd. A statement attributed to Founder and CEO George Kurtz read in part, "I want to sincerely apologize directly to all of you for today's outage. All of CrowdStrike understands the gravity and impact of the situation." He went on to reassure users that it was just an error, and that a fix was in place. And he promised to take steps to prevent it from happening in the future. By the book. If only they had left it there. 

Shortly after that a number of professionals at affected companies received an email saying that the company recognizes "the additional work that the July 19 incident has caused. And for that, we send our heartfelt thanks and apologies for the inconvenience. To express our gratitude, your next cup of coffee or late night snack is on us!" Attached was an UberEats credit code for $10 for each to use. You can argue it was a very personal gesture, even if one poster on X said "I literally wanted to drive my car off a bridge this weekend and they bought me coffee. Nice."

Snarky comments aside, it was a humanizing move, one that tried in a small way to lessen the wrong. Unfortunately, it seems it had its own glitch. Either because people started sharing the credit online or for other reasons, the code was quickly discontinued and rendered invalid. The apology was good, the remedy not so much. Or as another poster put it, "ClownStrike. Nuff said."

Nothing is ever as simple as it seems. And it's all made harder as technology has changed everything, especially how we communicate and interact. You can't look a text in the eye, discussions on Zoom are rife with overlaps and awkward pauses, and fixes can backfire if the tech fails further. CrowdStrike did try, and that's worth something. And a blown free cup of coffee hardly rises to the level of the Hippocratic oath's "Do No Harm." But remedies only work if they make things better, not worse. Perhaps they should have left it at "We're really sorry."

-END-

Marc Wollin of Bedford takes the blame when he screws up. His column appears weekly via email and online http://www.glancingaskance.blogspot.com/ and https://marcwollin.substack.com/, as well as via Facebook, LinkedIn and Twitter.


No comments: