Between a rack and a hard place

Read time 10min 40sec

With demand for network, storage and computing capacity on the rise, South African end-user organisations and service providers are investing heavily in revamping their existing data centres, as well as building new facilities.

As they do so, they're starting to put greener, more energy-efficient cooling and power technologies in place and replacing what is left of their legacy hardware and software with commoditised products such as Intel servers.

Even though falling hardware costs, automation solutions and technologies such as virtualisation are helping to ease data centre management, operating managers and CIOs are feeling the squeeze from tight budgets and rapidly-rising energy costs.

“We are caught between a rack and a hard place,” says Craig MacPherson, senior operations manager at Internet Solutions (IS). “We are expected to do more than before with less resources than ever.”

Data centre technology is shrinking, becoming more efficient and more affordable as a result of commoditisation, says Eckart Zollner, business development manager at New Telco SA, a company that provides co-location services to multiple telecom operators and Internet service providers so they can all interconnect at a single facility.

But these trends are offset somewhat by the constant growth in the bandwidth, processing and storage requirements of the average company. What's more, the shift towards online business means that most companies, especially service providers, must invest heavily in creating highly available infrastructure.

As they revamp their data centres, many South African organisations are building them to The Uptime Institute's Tier 3 or even Tier 4 standards for redundancy and resilience. To achieve Tier 3 certification, companies need an infrastructure that offers 99.982% availability.

In our industry, the data centre is critical to our daily operations.

JSE CIO Riaan van Wamelen

The data centre site needs to have full redundancy of its core capacity components such as processors, dual-powered equipment, redundant generators and uninterruptible power supplies, and multiple telecoms uplinks.

This year, New Telco SA, a joint venture between JSE-listed Jasco and Germany's New Telco, completed the construction of a Tier 3 data centre with 250 square metres of floor space. This includes ensuring the site has autonomy from the Eskom grid in the event of a power failure and installing redundant telecoms connections from multiple operators.

Downtime experienced by South African cellular operators in recent months and the interruption of the international BlackBerry service have highlighted the importance of uptime to South African companies, says Zollner. Few companies can afford the loss of revenues and the corporate embarrassment that accompany a prolonged systems outage.

Where it all happens

For the Johannesburg Stock Exchange (JSE), the data centre is such a critical element of its business that it has decided to retain in-house control over its infrastructure in a market where many organisations are outsourcing.

"In our industry, the data centre is critical to our daily operations," says JSE CIO Riaan van Wamelen.

The JSE, like other stock exchanges around the world, is making heavy investments in its data centre. The stock exchange is investing significantly into the renewal of its applications as well as in upgrading its data centre infrastructure to bring it line with Tier 3 standards.

During the course of 2012, the JSE aims to move its equity trading system from the London Stock Exchange to Johannesburg. In addition, it plans to replace back-office systems such as internal surveillance systems, clearing systems and accounting applications.

The relocation of is trading engine will allow the JSE to improve the stability of its trading environment by removing the risks of downtime caused by instability of international connectivity. In addition, the JSE is planning to offer co-location services in future.

The benefit for JSE members and their clients is that they will be better able to use algorithmic trading strategies if they locate their servers near the bourse's matching engine. This is a high-pressure environment where every millisecond counts. The key word is latency. Like stock exchanges around the world, the JSE is looking to reduce latency.

Says Van Wamelen: “One of the key considerations in the day-to-day running of a data centre lies in not only having the required infrastructure environment, but also running and maintaining the data centre operations in line with best practice. The required skills to manage the operations appropriately are scarce on the ground and aren't cheap.”

On the infrastructure side, the major challenge lies in managing infrastructure and server sprawl, he adds. The JSE is using server virtualisation technologies to manage its server environment in an efficient manner. Sophisticated monitoring tools are being implemented as part of the data centre upgrade.

The power struggle

New Telco, too, has invested in automation technologies to keep its overheads down. Environmental monitoring and remote management tools at the New Telco data centre alert technicians when there is a power failure, a sudden rise in the temperature of the data centre or other issues that need to be addressed.

The network is monitored online from the operations centre in Frankfurt, Germany. “We automate as much as possible to keep staff levels down and pricing competitive,” says Zollner.

Electricity, which used to be cheap and reliable in SA, is a major problem for data centre managers who are trying to keep costs down. Zollner says power costs in SA are still reasonably low compared to many other countries, but have crept to nearly 20% of the average data centre's running costs.

New Telco is investigating prospects in other parts of Africa and has found that in some countries, power costs could exceed 35% of operating expenditure.

The fact that SA has ambient data temperatures slightly higher than most European countries and buildings that are not as well insulated contributes to the power bill because the cooling requirements are a little higher, Zollner adds. The cost of power is a major driver for green data centres across the world.

New Telco in Germany is investing in green data centre technologies and is looking at building data centres based on renewable energy sources such as fuel-cell technology. The local joint venture may adopt these technologies once they have proven themselves.

South African companies and regulators are still dealing with the carbon footprint of power-hungry data centres in a reactive manner, says Zollner. Organisations in Europe are being more proactive because subsidies and regulations give them more incentive to invest in green technologies.

Internet Solutions is one South African company that is taking the greening of its data centre seriously. The Dimension Data division last year began the construction of a new Tier 3 data centre in Johannesburg to cater for the growth of its business as well to improve efficiencies in its data centre environment.

Completion of the first phase of the facility was scheduled for mid-November at the time of writing.

The new facility will eventually accommodate more than 700 cabinets, with more than 200 to be provisioned in the first phase.

Greening the data centre

Technology has advanced a great deal in the 18 years or so that have passed since IS was established, says MacPherson. IS realised that it might not be cost-effective to revamp its existing data centres with the new infrastructure technologies that are available today.

We automate as much as possible to keep staff levels down and pricing competitive.

Eckart Zollner, New Telco SA

Up until 15 years ago, servers were configured in towers rather than racks, blanking panels were not yet in wide use, and UPS technologies were not as efficient as the ones available today. With a clean slate, IS has been able to put the latest cooling and management technologies in place in the new data centre.

IS has designed the new data centre around cooling technology from KyotoCooling, which it expects to provide a 93% saving on cooling costs and a 40% saving on the power bill. The KyotoCooling technology should allow IS to achieve an average annual mechanical power usage effectiveness (PUE) of below 1.2. Older cooling systems generate a PUE of around 1.8 or higher.

The cost savings mostly come from the KyotoCooling concept, which delivers 'free cooling' for 75% of the year. The concept of free cooling is to use outside air to remove heat from the data centre whenever possible rather than expensive, power-hungry refrigeration. The Kyoto heat wheel separates the outside air from the inside air to regulate temperature in the data centre.

Building a new data centre or expanding an existing one isn't just about the technology, but also about its physical location and construction. MacPherson says one of the biggest factors in setting up a new data centre is deciding where it should be located to strike a balance between cost and convenience.

A data centre needs to be on an urban node with a reliable electricity supply and good telecoms links from multiple suppliers, and it should preferably have line of sight to high spots for wireless transmitters. In addition, it needs to be in a part of town that employees and customers will feel safe driving to in the dead of night.

There's also a lot of municipal and administrative red tape, which can in some cases take as much time as building the infrastructure, says Zollner. Often, it can take months for a company to secure city council permission to trench the road to lay down some fibre. Whereas European companies will use machines to do the digging, SA still relies on manual labour to do the work, Zollner notes.

The upgrade treadmill

Zollner says the rapid pace of technology change is perhaps the biggest challenge companies face in their data centres. Ideally, companies should be revamping their data centres every three years to keep up with the advance of technology, but most data centres are filled with technology that is seven years old or older.

Many end-user organisations that invested heavily in their own data centre infrastructure a few years back are now looking to defray some of the costs by commercialising them and selling off some of their spare capacity, says Zollner. Others are moving towards outsourcing to get off the upgrade treadmill while still having access to the latest technology.

The JSE is one organisation that is frequently refreshing its technology. Historically, it has replaced its equities trading engine around every five years, for example. As a result, it has little legacy technology in its data centre because it has introduced new technologies such as virtualisation, blade servers and storage area networks over the past few years.

Where possible, it has used industry-standard technologies to power its applications. But Van Wamelen admits the costs of staying up to date with data centre technology are high. With the upgrade of its data centre to cater for co-location services, the JSE is hoping to create a new revenue stream.

The commoditisation of computer hardware is rapidly changing the face of the data centre, with the footprint of the technology shrinking all the time and vendors finding innovative ways to rapidly deploy data centre infrastructure.

Data centres are becoming industrialised to provide more flexibility to end-user organisations. The likes of IBM, for example, are offering modular data centres that feature standardised and repeatable designs so they can be deployed quickly on customer sites.

Telecoms industry veteran Zollner says that when SA's cellular operators were building their data centres in the early 1990s, the trend was towards building large, centralised infrastructures. Now, the trend is towards building a network of smaller, distributed data centres to provide a higher level of redundancy.

One concept that is beginning to gain currency in Europe is the 'containerised' data centre, which crams data centre gear such as server racks and storage into a shipping container for easy portability, says Zollner.

These solutions are seen as perfect for remote sites, for organisations that need mobile infrastructure, and for organisations that want to grow out infrastructure without tearing down are rebuilding their existing data centres.

See also