Standard Bank blames ‘generic switch’ failure for outage
Standard Bank has blamed the failure of a “generic switch” for the disruption of services experienced on Saturday.
The bank apologised for the massive outage experienced over the weekend, saying the “generic switch”, which is responsible for capturing incoming and outgoing transactions, failed and Standard Bank engineers had to perform “a lot of heroics” within hours to restore services.
Standard Bank suffered an outage on Saturday, which impacted its ATMs, point-of-sale, mobile app, internet and credit card transactions.
The bank’s services became partially inaccessible at first and become wholly inaccessible within hours. The outage lasted for six hours.
Yesterday, Lungisa Fuzile, Standard Bank SA CEO, and Khomotso Molabe, chief engineering officer, provided a glimpse of what led to the outage, while apologising to the public for the disruption.
Fuzile hosted an urgent media briefing yesterday, saying: “We felt it will be important for us to present ourselves to you [media], the South African public, our clients and our employees…and take ownership of what happened over the last weekend, and to express our heartfelt and most sincere apology for the inconvenience caused to everyone by those events.”
He noted the disruption was “huge, profound and pervasive”.
Shedding some light on the technical aspects of what led to the disruption, Molabe said: “At about 7:30am on Saturday, we started experiencing degradation in one of the components (generic switch) that is responsible for switching transactions related to card payments into the bank and that regression that started at 7:30am; and around 9am, it became worse so that the majority of our customers were not able to transact.
“We offer our deep apology for the severity of Saturday’s incident and the impact it had on a lot of clients. Now, as I acknowledge this failure of this particular component, I also want to emphasise its resilience. We have never seen its failure at least in the past four years. It has been a very stead component, which is why when it happened, our engineers responded very quickly to make sure we can restore services.”
He added: “Our engineers had to perform a lot of heroics in this regard; technically building a brand new environment, testing it and putting it to market so that we can actually assist customers. This why we managed to recover all our services early afternoon Saturday so that clients could continue with the things they needed to do.”
Further, Molabe said, as Standard Bank is pushing ahead with its transition from financial services to a platform business, the lender has placed special focus on customer experience and promised to minimise service disruptions.
“We have been embarking on a digitisation and transformation journey, and we have gone through several horizons. We have gone through the first horizon, which we shared publicly, when we were looking at modernising a lot of our backend systems.
“Then over the years, we transitioned to building our client-facing, client experience capabilities, such as the mobile app, where we have millions of people transacting, and now we are transitioning from being a financial services company to becoming a platform services company.
“We invested a lot. We also have partnerships with the likes of Amazon Web Services and Microsoft for our cloud journey. This will go a long way in making sure we can improve the resilience and our posture, and also be able to scale our systems for the growth that is coming.”
In the past three years, Africa’s biggest lender has partnered with cloud providers Microsoft Azure, Amazon Web Services and Salesforce, to develop infrastructure it says will enable it to build partnerships with vendors and service providers to co-create a wide range of customised solutions for its clients.
“We are currently focused on a lot of investments focusing on client experience, but also making sure our systems continue to be resilient so that we can continue meeting our client expectations,” said Molabe.
“I want to conclude by saying that all our engineers and the whole organisation is working around the clock to make sure we can continue to minimise impact in the event we have technical glitches, and this continues to be the number one focus for the organisation going forward.”