A common misconception when it comes to big data technologies is that it is only applicable to organizations that manage big sets of information such as financial, public sector and utility industries. In reality, Big Data isn’t just about big sets of information. In today’s networked economy, billions of users are digitally connected across the globe. This digital data is everywhere—in every segment, in every economy and in every organization. Valuable analytics on digital data can substantially improve decision making, minimize risks, and uncover valuable insights that would otherwise remain hidden. When we think of big data, many of us limit our thinking to the “internally” generated data within our organizations and in case of industries that are not highly data intensive or for small/mid sized firms, we are yet to discover the full potential of opportunities that Big Data could unleash by leveraging non-traditional sources of data such as social media web data, RFID tags and GPS.
The use of big data is becoming a key way for leading companies to outperform their peers regardless of size of the organization or the volume of the “internally generated” data.
Let us take a look at some of the big data trends highlighted by a report by McKinsey Global Institute “The next frontier for innovation, competition and productivity”.
In the sectors that were examined by McKinsey, estimated opportunities of hundreds of billion/yearly in savings or new businesses by unleashing the potential of Big Data .
- 40% projected growth in global data generated per year
- 60 % potential increase in retailers’ operating margins possible with big data
- $600 Billion potential annual consumer surplus from using personal location data globally
Some examples of the immediate business opportunities that companies (regardless of the size of the internally generated data) can take advantage of are:
Transparent information environment –Significant value can be captured merely by making data available and transparent and by applying simple analytics. Potential gains increase sharply as stakeholders move to advanced analytics techniques such as applying complex algorithms and segmentation techniques.
Sentiment analysis – data from social media enable organizations to perceive and analyze client sentiment in order to better tailor marketing campaigns, products and services.
Predictive models – based on real-time data streams, Big Data technologies can determine likelihood to churn and take pre-emptive actions for customer retention.
Social technologies – not only understand holistically the client (the 360-degree view), but we can understand the clients network of family, friends and peers in order to build the client 720-degree view.
Location data – better understand behaviour, better offers based on location.
Operational improvement: RFI and sensor networks allows (retailers) to get insights into demand and better manage inventory and supply chains.
It was also noted that some sectors are positioned for better gains from big data.
Let us take a look at what’s driving the latest trends of big data.
We live in a networked economy where value is created and shared by all members of the network rather than by individual companies.
The networked economy creates major challenges for every business.
….First, the flow of information is no longer contained within the physical boundaries of companies. In order to leverage and control information and to create a single version of truth (from systems, social media, machines, web, databases etc.) companies have to pull information from many many sources that are not even “talking“ to each other – this complexity leads to waste.
….Second, the speed of outside marketplace is changing faster than our current ability to respond, by the time we react and take actions the information has already become irrelevant. As a result we are losing time, energy and even customers.
Big data solves this challenge.
Big data isn’t just about big sets of internally generated information. Big data is about creating new growth opportunities by leveraging information from sources that we were not possible to be utilized before. Big Data is also about the extreme volatility and variety of these sources . For every business the type of external sources could be different at any point in time— as the business processes , industries and challenges are always changing. And that’s where Big Data becomes an extremely powerful tool for small /mid size businesses.
However, there are some serious barriers for the Big Data adoption, specially for small to mid size companies.
The report McKinsey Global Institute “The next frontier for innovation, competition and productivity” also highlights one of the biggest barriers for companies to take full advantage of big data.
Let us take a look at what those challenges are.
Lack of IT skills: According to MGI “Companies and other organizations and policy makers need to address considerable challenges if they are to capture the full potential of big data. A shortage of the analytical and managerial talent necessary to make the most of big data is a significant and pressing challenge and one that companies and policy makers can begin to address in the near term. The United States alone faces a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts to analyze big data and make decisions based on their findings.”
Lack of infrastructure: Big Data technologies are capital intensive. The next big challenge is the need to ensure that the right infrastructure is in place and that incentives and competition are in place to encourage continued innovation; that the economic benefits to users, organizations, and the economy are properly understood; and that safeguards are in place to address public concerns about big data.
Big Data technologies deployed in Cloud solve this challenge and logically make sense for companies to quickly start taking advantage of the Big Data opportunities.
Click Here to read my blog about cloud to lean more.
SAP offers a range of technologies that address Big Data use cases and business requirements. The SAP HANA platform makes possible instant analysis of large amounts of multistructured data and the embedding of analytics into operational applications.SAP HANA integrates structured and unstructured data from internal and external sources, and can work on detailed data without aggregations. The SAP HANA appliance enables organizations to analyze their business operations using large volumes of detailed operational data in real time while business is happening. Operational data is captured in memory and made available for instant analysis, eliminating the typical lag time between when data is captured in business applications and analysis of that data from reporting and analysis systems.
Finally, with the latest innovations — using compression the underlying database tables are now utilized more resourcefully. This hardware innovation enables customers to store vast sets of information in a much smaller footprint. Reduction in the size of the database usage also reduces the costs of enabling SAP HANA for customers. This creates a remarkable opportunity for us to enable HANA in a less costly way.
Sodales Solutions can help you with a successful realization of your Big Data projects. Our team members are certified and highly experienced in HANA enablement.
Curious to learn more? Send me your questions at sana.salam@sodalessolutions.com .
