ICASA testing 'not up to scratch'
The Independent Communications Authority of SA's (ICASA's) method of checking the service provided by cellphone companies has again been condemned, with questions around its methodology and technical aspects being raised.
This comes after ICASA earlier this week published quarterly quality of service reports, covering KwaZulu-Natal and the Western Cape. All three of the major operators' networks had shortcomings during ICASA's drive-by tests, but MTN fared the worst when it came to connecting calls when the combined results are looked at. Yet, MTN performed the best in ICASA's dropped call measures, while neither Cell C nor Vodacom met targets on an average basis.
ICASA's method of driving and talking to test networks has previously been taken to task by the operators, which took umbrage at the lack of standards. ICASA has been working with the South African Bureau of Standards (SABS) to develop a testing methodology, but this has yet to be wrapped up.
Cell C says it is difficult to determine the accuracy of the results in the "absence of an industry-recognised testing methodology". While it has yet to analyse the reports, it notes it has previously questioned ICASA's results.
In KwaZulu-Natal, Cell C scored 94.54% in network accessibility, but dropped 6.99% of all calls. In the Western Cape, the third-largest operator met ICASA's accessibility targets, but its dropped call rate was 5.36%.
In terms of the end-user and subscriber charter regulations, published in 2009, no more than 3% of calls should drop, while 95% of all attempted calls should go through, based on recommendations made by the International Telecommunication Union.
Robin Maduray, MTN GM of transmission planning and optimisation, agrees there was a problem with the test monitoring system used by ICASA.
The operator, SA's second largest, says this was brought to ICASA's attention before the reports were published. It believes the findings do not reflect the true overall end-user experience its customers are experiencing now in the affected areas, says Maduray. The test environment error has been resolved, MTN adds.
MTN met the target for both provinces for dropped call rates, but failed average accessibility tests, with 91.1% for KwaZulu-Natal and 92.15% in the Western Cape.
Vodacom spokesman Richard Boorman says the methodology has improved and it supports ICASA in refining the testing process in conjunction with the SABS. However, he notes "it's obviously not ideal to see issues highlighted by these trials and we're using the data to pinpoint problem areas and make changes".
Boorman notes that some of the issues experienced with its network were because some test areas were in nature reserves, where base stations cannot be built. He adds in one instance, in Worcester, the dropped call rate was 10 times higher than recorded at the same site a day later, so Vodacom is tracing that fault.
Vodacom passed the accessibility test in both provinces, but failed to meet ICASA's dropped call targets. All three of the operators are investing this year, with Vodacom spending R9 billion, MTN R7.5 billion and Cell C R2.3 billion.
Ovum analyst Richard Hurst says the reports are probably worth their "salt", but notes the situation could be worse, such as in Nigeria, where the regulator imposed a SIM card sale ban because of the lack of quality. However, he adds, ICASA (which did not respond to a request for comment) needs to be better empowered so it has the skills and technical resources that will leave its results unchallenged.