| By Rick Kawamura | Article Rating: |
|
| March 8, 2012 06:00 AM EST | Reads: |
7,071 |
I recently presented, "The Moneyball Approach to Big Data - Creating an Unfair Advantage," at the Wall Street Technology Association's Hot Technologies Forum in New York. Everyone is talking about Big Data, but when it comes to taking action, most are taking a "wait-and-see" approach, and that concerns me. Skepticism or "late-adopter" mentality is understandable - if you want to forego a low-risk, high-reward opportunity and let your competition gain the advantage.
My job is to create value for my customers, and I'd hate to see anyone miss out on the opportunity presented by Big Data.
What's the Problem?
The Corporate Executive Board identified three potential barriers to Big Data implementation:
- Information Attainability (the right information is available and easy to find)
- Information Usefulness (information is of good quality and is usable in format)
- Employee Capability (employees analyze information effectively to make good decisions)
While these are definitely three potential barriers to implementation, the main problem I see in the market is pre-implementation where the "wait and see" mentality is caused by either a feeling that Big Data is over-hyped ("maybe it will go away"), or paralysis by analysis (the enormity and complexity of Big Data is too confusing to take action).
What Is Big Data?
The most common definition I've seen for Big Data is summarized by the three Vs:
- Volume: It's big - terabytes and petabytes of data
- Variety: It comes in many forms - internal, external, structured, and unstructured
- Velocity: It is growing and changing rapidly - making real-time capture and action hugely important
This definition is always supported by numbers showing the vastness and enormity of Big Data:
- The New York Stock Exchange creates 1 terabyte of data per day (InformationWeek)
- 10,000 payment-card transactions are made per second worldwide (American Banker)
- 30 billion pieces of content are shared on Facebook every month (McKinsey)
- Twitter feeds generate 8 terabytes of data every day (InformationWeek)
The Internet plays a huge role in the rapid growth of Big Data, giving individuals the ability to post and upload immense amounts of pictures, text, video, and mobile data. It also gives businesses a channel to offer access to customers and partners through web-based applications (think Oracle, salesforce.com, social media, procurement, logistics, publishers, etc.). One way to visualize this explosion of applications and data is the Bessemer Venture Partners Cloudscape. And that's just in the cloud. Don't forget all of the apps and data behind the firewall of every organization, whether commercial, governmental, or charitable. Big Data truly is BIG.

Start Small
Before you go out and buy massive amounts of storage, take a look at what you currently consume and utilize and start from there in easily digestible portions. Forrester estimates that enterprises currently utilize less than 5% of available data. In a survey of global executives, IBM shows that 33% have made decisions with inaccurate data or data they don't trust; half don't have sufficient information from across their organization to do their job; 75% believe more predictive information would drive better decisions; yet 87% have yet to even start taking advantage of opportunities to leverage information to their advantage. You don't need to immediately implement a solution getting you to 100% or even 50% of available data - 6-10% will do for now.
If there are 200 million tweets a day equaling 8 terabytes of data, but only 1,000 of those tweets relate to your product or company, do you need to store and analyze all 8 terabytes every day? Think of it this way: there's a huge difference between "I have terabytes of data - videos, satellite pictures, social media conversations, and research reports" and "I know where Public Enemy #1 is." It comes down to Data vs. Intelligence. Data is useless if you can't extract meaningful intelligence from it. And the quality of the intelligence is much less dependent on the volume of data than it is on the relevance of the data and your ability to access it.
The Magic Words: Relevance and Accessibility
Although Big Data lives up to its name, don't get caught up in all the massive numbers. Focus on what's relevant to your business. Consider this: Sybase published Big Data, Big Opportunity that stated "for the median Fortune 1000 company... a 10% increase in usability of data translates to an increase of $2.01 billion in total revenue per year, [and] a 10% increase in accessibility to data translates to an additional $65.67 million in net income per year." Just because your company currently may have access to only 5 percent of the relevant data that is available, don't despair. You don't have to go from 5 percent to 100 percent. You really only need to go from 5 percent to 5.5 percent to reap great rewards.
The Secret to Taming Big Data
Despite all the hype and discussion around Big Data's massiveness, I've yet to find a single article mentioning the difficulty of accessing data that is spread throughout all of the various source applications. Until recently there were no Big Data integration platforms that could deal with the exploding number of applications and all of the data they contain, as well as the speed at which both are changing. Just a glance at the daily domain statistics on www.domaintools.com gives you an idea of the volume of sites being created, deleted, and transferred every 24 hours. Not every integration solution can manage the intensity of that kind of change to give you access to the relevant data - the business intelligence - your business needs when you need it.
The whole point of gaining access to relevant data is that it must be actionable. Otherwise it's a big waste of time and effort. What's amazingly useful about Big Data, and the web-based nature of so much of it, is that with a Big Data integration platform you can access any data you can see on a website and you can just as easily transform that data, perform an operation on it, and automate a resulting action. Here's an example:
You know that consumers and even your B2B purchasers research prices online and that loyalty to any one vendor has deteriorated as buyers have more pricing knowledge just a search and mouse-click away. But you're smarter than your competitors because you're already doing the extra 10 percent. You set up automated monitoring of your competitor's pricing, and when their price drops below yours, your Big Data integration platform calculates the difference plus 10%, logs into your ecommerce site and adjusts your prices automatically, all in mere moments. The beauty is that this can all be set up in hours, if not a few days, and you don't have to bring in an army of developers or consultants to create custom code to do any of it.
If I told you I could guarantee any application or data you can see in your web browser (customer data, bank transactions, twitter, blogs, supply chain vendors, government data, competitor prices, etc.) could be automatically accessed and loaded into the application, database, or spreadsheet of your choice, how many game-changing Big Data projects could you imagine? Understanding the point-in-time cash position of billions of dollars across 300 banks? No problem. Monitoring competitor pricing on 50,000 SKUs every day? Simple. Automating a 23-step manual invoicing process to get paid millions of dollars two days faster? Done. Real-time, automated access to the relevant data you need is the key to success with Big Data.
Every company can benefit from Big Data in many ways, but most don't realize it. Hundreds of scenarios are possible using real-time application integration platforms that could save your company millions of dollars; grow revenue by double-digit percentages; create more personalized products that delight your customers; automate real-time feedback on your brand, products, and competitor prices; create your own custom research that allows you to see trends before your competitors do; and overall make your company a much more agile business that scales with your new-found vigor and growth. Don't let the size of Big Data paralyze you; get real-time access to the data that is relevant to your company's growth and take action.
Published March 8, 2012 Reads 7,071
Copyright © 2012 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Rick Kawamura
Rick Kawamura, VP of Marketing at Kapow Software, is the driving force behind global marketing at Kapow Software. He and his team are accelerating the company’s growth and expanding its leadership in the application integration market worldwide.
- Cloud Expo New York | Danger Ahead: Why File Sync Is NOT Endpoint Backup
- Session Topics: 12th Cloud Expo / Cloud Expo New York
- Cloud Expo NY: Best Practices for Architecting Your Cloud Infrastructure
- Cloud Expo New York: Aligning Your Cloud Security with the Business
- Overview of the OpenStack Cloud
- Cloud Expo New York: Managing Legal Risks in Cloud Computing
- Cloud Expo NY: Environmental Pressures Drive an Evolution in File Storage
- Cloud Expo NY: Accelerating Cloud Computing with Intel SSD Technology
- Is Cloud Safer Than Your Traditional Datacenter?
- NIST to Sponsor FFRDC Widespread Adoption of Integrated CyberSecurity
- Apple’s Key Rubber-Band Patent Found Invalid Again
- Cloud Expo New York: Anatomy of an Internet Scale Application
- Cloud Expo New York | CEO Insider: Overcoming Cloud Barriers
- Cloud Expo New York | Danger Ahead: Why File Sync Is NOT Endpoint Backup
- Session Topics: 12th Cloud Expo / Cloud Expo New York
- Cloud Expo NY: Best Practices for Architecting Your Cloud Infrastructure
- Cloud Expo New York: Aligning Your Cloud Security with the Business
- Overview of the OpenStack Cloud
- Cloud Expo New York: Managing Legal Risks in Cloud Computing
- Cloud Expo NY: The Promise of an End-to-End SDN Solution - Can It Be Done?
- Guest Post: Typical CIO Conversation
- Cloud Expo NY: Environmental Pressures Drive an Evolution in File Storage
- Technology Benefit Cycle: What Gartner & Geoffrey Moore Aren’t Telling You
- Amazon Makes Virtual Private Clouds Its Default
- Effective Page Authorization In JavaServer Faces
- The Top 250 Players in the Cloud Computing Ecosystem
- Cloud Expo New York Call for Papers Now Open
- SOA Focus - Web Services Security in Java EE
- IBM Security Report Predicts Mobile/Satellite Attacks in 2005
- Industry Experts Discuss the State of Cloud Computing
- The Cloud Computing Kettle Heats Right Up
- The Top 100 Bloggers on Cloud Computing
- The Next Chapter in the Virtualization Story Begins
- Java Application Security in the Corporate World
- ColdFusion Security Best Practices
- Cloud Expo 2011 East To Attract 10,000 Delegates and 200 Exhibitors



























