The only clue that there’s something out of the ordinary going on inside the Centre for High Performance Computing (CHPC) in Rosebank in Cape Town is a line of chillers feeding icy air into the building. The chillers are on stilts, and were lifted out of the way of the Liesbeek River in flood some years ago. There are also two small buildings, each housing a 1GW generator, should there be any interruption to the power supply.
When we democratise this computational capability, we ensure that the most significant developments emerge from spaces we least expect.
Mervyn Christoffels, CHPC
Mervyn Christoffels, CHPC director, is showing me around the centre, which is on the CSIR campus, and says he’s interested in how the equipment he’s in charge of will shape and advance South Africa’s future of computing. “When we democratise this computational capability, we ensure that the most significant developments emerge from spaces we least expect.” Christoffels qualified as an electrical engineer, before completing his MBA and M.Com degrees. He’s now studying for his PhD in leadership at UCT Graduate School of Business. He has worked as ICT director at UWC and Unisa, before being appointed as CHPC director two years ago.
Trains, drugs, buildings
High-performance computing (HPC) plays a crucial role in advancing scientific discovery, he says, and the centre is a shared resource for academics and educational institutions to conduct research. Use cases for the CHPC’s cluster include high-speed train design, through to drug discovery. It was also used by engineers in enhancing the aerodynamics of the eMendi Building in Gqeberha, whose curved lines are reminiscent of its namesake that was sunk in 1917.
The CHPC is also the failover facility for the South African Weather Service (SAWS), and all SAWS weather modelling is done at the CHPC. The compute is done on the one petaflop Lengau (cheetah, in seTswana) cluster, he says, which can accommodate research programmes from small initiatives with just one participant, to large-scale projects involving 30 to 40 members, consuming millions of computational hours.
The centre is an associate member of CERN’s A Large Ion Collider Experiment (ALICE) project, and contributes a tier-two node to the CERN grid computing infrastructure. The node is used to study the physics of strongly interacting matter at extreme energy densities. “Through collaborations like this, we’re not only exposed to world-class scientific research, but also to world-class infrastructure and cutting-edge technology,” says Christoffels. The value the centre offers to the research community extends beyond computing power, and his team has specialists in bioinformatics and chemistry to physics and engineering, among other disciplines, who assist researchers in executing algorithms or models on the cluster.
He says the “customer” serves as both the user and interpreter of their results. “They generate these results by utilising the available facilities and services. While the CHPC plays a crucial support role in ensuring proper installation and maintenance of application software, and may aid in interpreting results and troubleshooting, the user or researcher ultimately creates their own research and outputs through their access to these HPC services,” he says.
The centre uses OpenStack as its cloud platform and researchers can collaborate by sharing workflows and datasets for local or global science projects. It also has an AI cluster to train, execute, and scale complex AI models, and is in the process of securing a quantum computing access service. Christoffels says the CHPC doesn’t have dedicated AI chips, and is using GPUs for machine learning applications.
Time for an upgrade
As a national facility funded by the Department of Science, Technology and Innovation, the resource costs are budgeted for and carried by the department and where there is a charge, it is at a cost recovery level. “If a researcher were to approach Amazon or Google – companies that operate for profit – the cost per core would be significantly higher than the 45c per hour (excl. VAT) charged to access a single core at the CHPC,” he says. The CHPC’s cluster operates 24 hours a day, seven days a week, and is currently running at 100% capacity. An upgrade is planned for later this year.
Christoffels says the upgrade is long overdue, and its current equipment is now over seven years old. When Lengau was launched in 2017, it was ranked as the 115th fastest supercomputer in the world, but within a few years, it didn’t even feature in the list of the world’s top 500.
It recently purchased a new cluster, a four-petaflop HPC system, which cost the CHPC around R400mn. The Lengau cluster consists of 24 racks, while the new system, which is four times more powerful, has 10, and uses direct water cooling.
Christoffels says HPC services used to be required by research institutions and universities, but he now sees more corporate and entrepreneurial users. Use cases include simulations, or game development. But most of these use cases don’t need complex, high-performance systems capable of handling multiple workloads; instead, they just need to manage a single workload or domain.
The CHPC’s HPC Ecosystem Project is a good example of this. As part of this initiative, it collaborates with supercomputer centres in the US and Europe to secure outdated technology, upgrade obsolete clusters, and redistribute this technology to Square Kilometre Array (SKA) locations across Africa, SADC countries, and historically disadvantaged universities. “I want to stress that our goal is to democratise this technology,” he says. “We’re working to to make it easier to access these services so that more South Africans can make a meaningful and impactful contribution to world-class research.”
Share