5 Tips for Improving Data Analysis
The challenge of Big Data and Manual Analytical Methodologies continues to grow. With Big Data enterprise adoption rates increasing from year to year, companies face the daunting task of integrating and making sense of all this new information. For companies looking for real business value in analyzing this data, we’ve got a few tips to help them through the process.
Tip 1: Don’t Pre-Determine What Matters – Take ALL Data into Account
Big Data comes from many disparate sources. Aggregating it is challenging. You need to normalize and analyze huge sets of figures. However, pre-determining what’s important creates a “silo effect”– observing only through a narrow lens.
“Correlations and patterns from disparate, linked data sources yield the greatest insights and transformative opportunities” Merv Adrian, Research VP, Gartner. Use a robust ETL system to take any possible data into account and logically determine what is relevant. Avoid misconceptions due to lack of information. What you discover will often be surprising.
Tip 2: Filter Out Noise and Focus Only on What Matters
Big Data greatly increases the volume, velocity, and complexity of data streaming into the organization. Data from numerous disparate sources makes it harder to find correlations between seemingly unrelated phenomena.
Collecting input from 3rd party aggregators, blogs, and public sources, along with your traditional customer, market and operational data, results in significantly greater data noise. Whatever is irrelevant to your analytical goals must be filtered out early on. Analytics based on Domain Expertise, along with Time-Series analysis, will remove the noise while leaving in exceptions that might be important to enable early detection of breaking trends, market changes, performance issues and unrealized opportunities.
Tip 3: Verify Uniqueness
Now that you’ve gotten rid of irrelevant outliers and social media noise, how do you confirm if a trend is problematic or just part of the norm? For example, should an exceptional drop in sales of a given product in a specific region be reported as a separate problem?
The user must ask if the data is unique in comparison with the behavior exhibited by all possible peer groups. In other words: Is this happening everywhere else? (e.g. other products, other stores, other geographies, competing products) Segmenting an opportunity or a business issue is essential for generating actionable data.
Tip 4: Confirm Consistency
Once you’ve verified a trend as a problem or an opportunity, you must dig further to establish why the trend occurred.
Go deeper into the weeds, look at all related data, detect precisely what is unique here, and you will arrive at an informed analysis. The key to this process is Domain Expertise.
Tip 5: Stay Agile and Keep Your Solution Relevant
One of the key reasons for many failed BI projects is the inability to continuously morph the solution as the business and market change. Left unattended these variations create a gap between the business and where the breadth of the solution resides.
Be mindful that analytics are only as good as the data that comes in. If it’s not fresh, the results will be off. The solution must quickly adapt the analytics parameters as needed by the business. They therefore need to be kept constantly up-to-date with the latest internal and external data.
An introduction of a new packaging size for your drug; a promotion by your competitor; a new oncology specialist in your territory; a new payer contract… Any change in the business landscape influences the market. You must take it ALL into account to perform a meaningful analysis that will truly drive the business and affect the bottom line.