If not, why not?
- We got more traffic than we expected.
- The traffic pattern was different from our load test.
- We had a hardware failure, the only single piece that could take out the site failed, what bad luck!
Those are the common excuses we see and none of them are acceptable answers when you consider the financial and customer damage outages can cause.
All too often we see organizations perform a technology load test of their site with little focus on recreating the real life conditions they will see when they open their online doors on Black Friday. This usually is a result of it being an IT driven test that does not take into account the wider business plan. The main reasons we hear for doing it this way: is that it is too hard, too much time, not enough resources, etc.
The math against those excuses is pretty simple. Most traffic based outages that we have seen last at least an hour and normally occur around the peak traffic period. If you are planning on doing $5 million during your peak hour and have a 40% margin you are losing $2,000,000 in that hour. It’s our experience that loss is substantially more than cost of effective planning and testing.
There are some less tangible factors that need to be considered like customer frustration, additional call volumes to call center, and employee fatigue if the outages are extended; a favorite is the Wall Street Journal factor. Think about the hit you would take by ending up in the Journal saying you were down during the holidays.
We have also found that teams that own the solutions are not the first in line to call out their own deficiencies. No one likes to criticize their own kid.
Teams will overlook issues that have been around a long time because “it’s never caused a problem before.” They will explain away minor glitches saying ‘that’s not a real problem’. The key point to remember is that EVERYTHING is different when you are at the holiday peak. The load is concentrated, there are more buyers than browsers, the marketing plan is rev’d up, and promotions are double or triple the normal average. All of these factors should force you to look at everything with a critical eye. This is where outside perspective becomes a real asset.
There are disasters, hurricanes and earthquakes that you can’t avoid, but unless one of those things happens on Black Friday your site being down isn’t one of them. It’s not easy, it’s not fun, but you need to do the work. It’s much better to read about someone else’s issues in the Journal than your own.
Co-authored by Chris Hauca and Justin Finnegan, Acquity Group