Cloud and big data analytics
The ITWeb Cloud Summit is in its second year in the current format, and brings you more than ever before in experts, content, case studies, and relevant local insight.
We have the ear of the experts in cloud in South Africa and the 2018 event will continue to deliver on its mandate: to dissect the A - Z of cloud for a more agile, cost-efficient and secure business environment.
We spoke to Charlene George, MD, Verve Digital about its take on the cloud and big data analytics.
ITWeb: Why would an organisation choose to move big data analytics to the cloud?
George: There's not necessarily one right answer to whether it's "better" to run data analytics in the company's data centre or the cloud.
The cloud has many benefits, but other factors such as compliance or regulatory requirements may dictate storing and analysing data on-premises in a data centre.
The cloud offers elastic scalability, so it is typically fast and easy to add more storage for incoming data or to get more compute power for advanced analytics. The cloud also makes sense for the growing volume of big data now generated outside a company's firewall. It is expensive to move raw big data back to an on-premises data centre and also then pay for the extensive storage and compute power required for analysis. For these reasons, many companies are moving analytic processing to the cloud, closer to the big data sets and cloud platforms where storage and compute are significantly cheaper than on-premises.
For the near future, most companies will have hybrid architectures, with some data and analytic processing in the cloud (e.g. for large raw datasets) and some data and analytic processes kept on-premises (e.g. for highly regulated data.)
With this being the trend, data movement and integration technologies are critical considerations when choosing a data lake technology or designing hybrid architectures.
ITWeb: What are the pros and cons associated with moving big data to the cloud?
George: The pros would include, rapid provisioning; elastic scalability; higher availability and efficiency; relevant real-time analysis; lower up-front costs; try before you commit to large project; ability to focus on core competency; skills requirements and new technologies.
The cons include costs of data migration and integration; lack of best practices; potentially higher costs, and security.
ITWeb: What questions should we be asking our partners when considering moving to the cloud?
George: Selecting the right partner for your cloud-based analytics deployment is crucial. Organisations should look for an experienced partner that enables them to democratise data across their organisation, putting analytics in the hands of those who need the insight most, regardless of their level of technical expertise.
Additionally, the partners need to be able to provide data movement and integration technologies to ensure that the customer can cost-effectively move data into and out of the data lake in the cloud as required and also be able to integrate to systems both in the cloud and on-premises as required.
The right cloud partner will accelerate business outcomes by leveraging industry, domain, and implementation expertise to apply proven solutions to all of the company's business challenges, enabling users to realise maximum value from their data and improve decision-making throughout the organisation.
ITWeb: What are the most important pointers organisations should remember when contracting with a cloud operator for this purpose?
George: The financial model adopted by cloud providers could mean that while it is easy and relatively cheap to put data into the cloud, it is difficult and extremely expensive to get it out again. The company needs to ensure that they can extract their data in a product-independent format should they wish to move to another cloud provider in the future.
Cloud services and managed service providers must show they are actively providing analytics services on the data already held within their organisation, and provide access to affordable, effective data evaluation processes and options for quick, business-focused activity.
When evaluating cloud analytic options it is important to consider the cost and time required to transfer data to the cloud provider. The company must also assess the cloud provider's support for version control and metadata about data sets, models and analysis results. Those features will be important factors in long-term manageability.
The performance and SLA requirements of a data lake are highly dependent on its role within production data processes and the importance of analytics to a company's success. If the processes are important to the bottom line, it's critical to ensure reliable performance. One problem with data lakes to date has been highly complex software that's difficult to optimise for performance, a problem that can be compounded when installing software on unfamiliar cloud infrastructure. No one wants highly skilled, expensive data scientists and analysts spending time troubleshooting software, or waiting around, when they could be focused on analytics. Therefore, it is critical to know and understand the SLA's that the cloud vendor is offering.
Before entering into any agreement with a cloud vendor, it's critical to know what the costs will be.
It is also important to understand the security solution being offered by the cloud vendor. It is very important to consider data in transit as well as at rest, which is obviously directly related to how data movement to the cloud is architected and managed. Encryption techniques can be used to keep data safe, though it can be challenging to manage end-to-end security. Companies should work closely with their cloud providers' security experts to ensure their security requirements are met and guaranteed.
ITWeb: What do you see as the biggest drivers/motivators for the move to cloud?
George: The following come to mind: standardisation and simplification; seasonality; business continuity and disaster recovery; application upgrades or performance issues; regulatory and compliance; information security can be a significant legal, regulatory and compliance challenges; speed to market and the IOT, which is driving the need for more secure big data analytics.
ITWeb: Why did you agree to present at the cloud summit?
George: It was an honour to be selected for the ITWeb Cloud Summit. I am in the process of building my business, so being able to present not only creates brand awareness for Verve Digital but if attendees see value in what we present, it gives credibility on the subject.
ITWeb: What is it that you bring to the table and what do you want attendees to take away with them after your presentation?
George: We bring cloud solutions in the form of Platform as a service and the associated professional services such as migration, deployment, integration and optimisation. When customers are looking to host ERP or analytics in the cloud, we would like them to think of us.