Performance anxiety? It`s not just about I/O

Storage performance means different things to different users. But most storage managers agree on this: enterprise data centre systems must perform at their best, with the highest possible bandwidth, scalability, reliability and availability.

All of these metrics, and more, come under the umbrella of storage performance. The challenge is many companies don`t have the tools to measure existing performance and don`t know the best way to evaluate vendor solutions (separating hype from facts).

This article will show you how to determine your performance requirements and evaluate vendor solutions against those requirements, while keeping your eye on the bottom line - so you make the right storage investment for your IT organisation.

In my experience, performance actually isn`t a major issue for most storage buyers early in the purchase decision-making process. It usually becomes an issue when a buyer is comparing the performance of one storage vendor to another. Or comparing a vendor`s product performance to in-house baseline performance. But because most customers don`t have the tools to effectively measure existing performance, the vendor is in the driver`s seat, and you, the buyer, are often unable to verify vendor performance claims independently.

A more informed storage buyer is a more effective one. Here`s how to get a better handle on your performance requirements - and make better storage buying decisions.

Before you can determine your IT organisation`s storage performance requirements you must catalogue a range of factors - starting with your applications, servers and networks. Overall performance is important. However, when performance is a concern, considerable attention is focused on throughput in IOPs (Input/Output per second) and MB/s. While these factors are important, response time (usually based on milliseconds/msec) is critical.

For example, what good is 200 000 IOPs or 300MB/s throughput if your response time is 8-20 msec vs the average .5 to 3 msec? Ideally, you want the best combination of throughput and response time possible.

Once you`ve determined your storage performance requirements, you`re in a better position to evaluate vendor offerings - and separate marketing fluff from reality.

Some vendors will give you "artificial" performance results from an engineering lab, calculated with best case scenarios, and unrealistic configurations that are not actually used in real world business applications.

To understand a particular vendor`s performance claims, you should look to one or more sources for validation, including:

* Independent verification and testing reports by a recognized market analyst such as IDC, Veritest, Mercer Group, etc.
* Customer references using the same, or similar, solution as the one you are considering.

And, ideally, bring performance testing in-house. Ask vendors you`re considering to bring their equipment onsite and test. This is not the first choice of most vendors, as it adds significant time and cost to the sales cycle. However, it`s worth asking and will give you the most accurate predictable performance with real-world data.

Still, there are risks involved when evaluating vendors; and there is some truth to the old adage that you get what you pay for. Two vendors may propose solutions that are not, in fact, equal, but are at different price points. Most buyers would choose the less expensive solution; but be certain that you are comparing apples to apples or the lower cost solution could equal a slower performance solution. A common example of this is when a vendor brings a solution that has a smaller footprint and carries a lower cost than a competing solution. But that less expensive solution may deliver lower performance, manageability and scalability.

And, if, say, a vendor`s proposed solution includes a Cisco switch, a comparable solution must include a Cisco switch for a fair benchmark. It`s very important to compare storage solutions with:
* Similar equipment
* Similar disk sizes
* Similar speeds

If, when evaluating competing solutions, you see any difference between supposedly comparable solutions, ask, "Why is there this difference, and what does it mean to my business?"

It comes down to this: The only approach you should NOT take is to believe one vendor`s performance claims over another without first-hand or third-party validation - regardless of their market share. After all, a vendor making performance claims greater than those of another vendor has everything to gain. But you`ve got a lot to lose.

Ultimately, your decision to purchase storage should be based on more than performance and other technical requirements: the FOB factor is one perspective all companies should consider when acquiring any storage product or solution. It means the financial, operational and business benefits.

Quite often, the true business benefits of any storage solution are unclear. So buyers may focus on technical issues that actually have near-zero impact on their businesses. Unfortunately, vendors sometimes promote these non-essential technical issues in a competitive ploy that ends up distracting buyers from their true business objectives. Consider it a red flag if your vendor takes this tact, asking unnecessary questions such as: "What type of memory does your solution utilise?" or "How does disk drive aerial density affect your solution`s performance?" Such questions are akin to asking, "What is your favourite colour?"

At best, these are distractions and should raise questions in your mind about the capabilities of the vendor asking them.

Lastly, remember that a miniscule performance advantage (such as one millisecond difference) will not typically be noticeable to your users. You should evaluate the potential business benefits of any solution, whether your needs are enabling faster time to market, faster response, higher customer satisfaction, or more. And, be sure to look at how easily a given solution can adapt to your changing business requirements. That`s the kind of storage performance that will give you true peace of mind.