Cloud Security Authors: Yeshim Deniz, Elizabeth White, Liz McMillan, Dana Gardner, Maria C. Horton

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog

@CloudExpo: Article

Big Data: Taming the Beast for Competitive Advantage

I’d hate to see anyone miss out on the opportunity presented by Big Data

I recently presented, "The Moneyball Approach to Big Data - Creating an Unfair Advantage," at the Wall Street Technology Association's Hot Technologies Forum in New York. Everyone is talking about Big Data, but when it comes to taking action, most are taking a "wait-and-see" approach, and that concerns me. Skepticism or "late-adopter" mentality is understandable - if you want to forego a low-risk, high-reward opportunity and let your competition gain the advantage.

My job is to create value for my customers, and I'd hate to see anyone miss out on the opportunity presented by Big Data.

What's the Problem?
The Corporate Executive Board identified three potential barriers to Big Data implementation:

  1. Information Attainability (the right information is available and easy to find)
  2. Information Usefulness (information is of good quality and is usable in format)
  3. Employee Capability (employees analyze information effectively to make good decisions)

While these are definitely three potential barriers to implementation, the main problem I see in the market is pre-implementation where the "wait and see" mentality is caused by either a feeling that Big Data is over-hyped ("maybe it will go away"), or paralysis by analysis (the enormity and complexity of Big Data is too confusing to take action).

What Is Big Data?
The most common definition I've seen for Big Data is summarized by the three Vs:

  • Volume: It's big - terabytes and petabytes of data
  • Variety: It comes in many forms - internal, external, structured, and unstructured
  • Velocity: It is growing and changing rapidly - making real-time capture and action hugely important

This definition is always supported by numbers showing the vastness and enormity of Big Data:

  • The New York Stock Exchange creates 1 terabyte of data per day (InformationWeek)
  • 10,000 payment-card transactions are made per second worldwide (American Banker)
  • 30 billion pieces of content are shared on Facebook every month (McKinsey)
  • Twitter feeds generate 8 terabytes of data every day (InformationWeek)

The Internet plays a huge role in the rapid growth of Big Data, giving individuals the ability to post and upload immense amounts of pictures, text, video, and mobile data. It also gives businesses a channel to offer access to customers and partners through web-based applications (think Oracle, salesforce.com, social media, procurement, logistics, publishers, etc.). One way to visualize this explosion of applications and data is the Bessemer Venture Partners Cloudscape. And that's just in the cloud. Don't forget all of the apps and data behind the firewall of every organization, whether commercial, governmental, or charitable. Big Data truly is BIG.

Start Small
Before you go out and buy massive amounts of storage, take a look at what you currently consume and utilize and start from there in easily digestible portions. Forrester estimates that enterprises currently utilize less than 5% of available data. In a survey of global executives, IBM shows that 33% have made decisions with inaccurate data or data they don't trust; half don't have sufficient information from across their organization to do their job; 75% believe more predictive information would drive better decisions; yet 87% have yet to even start taking advantage of opportunities to leverage information to their advantage. You don't need to immediately implement a solution getting you to 100% or even 50% of available data - 6-10% will do for now.

If there are 200 million tweets a day equaling 8 terabytes of data, but only 1,000 of those tweets relate to your product or company, do you need to store and analyze all 8 terabytes every day? Think of it this way: there's a huge difference between "I have terabytes of data - videos, satellite pictures, social media conversations, and research reports" and "I know where Public Enemy #1 is." It comes down to Data vs. Intelligence. Data is useless if you can't extract meaningful intelligence from it. And the quality of the intelligence is much less dependent on the volume of data than it is on the relevance of the data and your ability to access it.

The Magic Words: Relevance and Accessibility
Although Big Data lives up to its name, don't get caught up in all the massive numbers. Focus on what's relevant to your business. Consider this: Sybase published Big Data, Big Opportunity that stated "for the median Fortune 1000 company... a 10% increase in usability of data translates to an increase of $2.01 billion in total revenue per year, [and] a 10% increase in accessibility to data translates to an additional $65.67 million in net income per year." Just because your company currently may have access to only 5 percent of the relevant data that is available, don't despair. You don't have to go from 5 percent to 100 percent. You really only need to go from 5 percent to 5.5 percent to reap great rewards.

The Secret to Taming Big Data
Despite all the hype and discussion around Big Data's massiveness, I've yet to find a single article mentioning the difficulty of accessing data that is spread throughout all of the various source applications. Until recently there were no Big Data integration platforms that could deal with the exploding number of applications and all of the data they contain, as well as the speed at which both are changing. Just a glance at the daily domain statistics on www.domaintools.com gives you an idea of the volume of sites being created, deleted, and transferred every 24 hours. Not every integration solution can manage the intensity of that kind of change to give you access to the relevant data - the business intelligence - your business needs when you need it.

The whole point of gaining access to relevant data is that it must be actionable. Otherwise it's a big waste of time and effort. What's amazingly useful about Big Data, and the web-based nature of so much of it, is that with a Big Data integration platform you can access any data you can see on a website and you can just as easily transform that data, perform an operation on it, and automate a resulting action. Here's an example:

You know that consumers and even your B2B purchasers research prices online and that loyalty to any one vendor has deteriorated as buyers have more pricing knowledge just a search and mouse-click away. But you're smarter than your competitors because you're already doing the extra 10 percent. You set up automated monitoring of your competitor's pricing, and when their price drops below yours, your Big Data integration platform calculates the difference plus 10%, logs into your ecommerce site and adjusts your prices automatically, all in mere moments. The beauty is that this can all be set up in hours, if not a few days, and you don't have to bring in an army of developers or consultants to create custom code to do any of it.

If I told you I could guarantee any application or data you can see in your web browser (customer data, bank transactions, twitter, blogs, supply chain vendors, government data, competitor prices, etc.) could be automatically accessed and loaded into the application, database, or spreadsheet of your choice, how many game-changing Big Data projects could you imagine? Understanding the point-in-time cash position of billions of dollars across 300 banks? No problem. Monitoring competitor pricing on 50,000 SKUs every day? Simple. Automating a 23-step manual invoicing process to get paid millions of dollars two days faster? Done. Real-time, automated access to the relevant data you need is the key to success with Big Data.

Every company can benefit from Big Data in many ways, but most don't realize it. Hundreds of scenarios are possible using real-time application integration platforms that could save your company millions of dollars; grow revenue by double-digit percentages; create more personalized products that delight your customers; automate real-time feedback on your brand, products, and competitor prices; create your own custom research that allows you to see trends before your competitors do; and overall make your company a much more agile business that scales with your new-found vigor and growth. Don't let the size of Big Data paralyze you; get real-time access to the data that is relevant to your company's growth and take action.

More Stories By Rick Kawamura

Rick Kawamura, VP of Marketing at Kapow Software, is the driving force behind global marketing at Kapow Software. He and his team are accelerating the company’s growth and expanding its leadership in the application integration market worldwide.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

@ThingsExpo Stories
@ThingsExpo has been named the Top 5 Most Influential Internet of Things Brand by Onalytica in the ‘The Internet of Things Landscape 2015: Top 100 Individuals and Brands.' Onalytica analyzed Twitter conversations around the #IoT debate to uncover the most influential brands and individuals driving the conversation. Onalytica captured data from 56,224 users. The PageRank based methodology they use to extract influencers on a particular topic (tweets mentioning #InternetofThings or #IoT in this ...
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in Embedded and IoT solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 7-9, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and ...
Cloud based infrastructure deployment is becoming more and more appealing to customers, from Fortune 500 companies to SMEs due to its pay-as-you-go model. Enterprise storage vendors are able to reach out to these customers by integrating in cloud based deployments; this needs adaptability and interoperability of the products confirming to cloud standards such as OpenStack, CloudStack, or Azure. As compared to off the shelf commodity storage, enterprise storages by its reliability, high-availabil...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Arch...
The IoT industry is now at a crossroads, between the fast-paced innovation of technologies and the pending mass adoption by global enterprises. The complexity of combining rapidly evolving technologies and the need to establish practices for market acceleration pose a strong challenge to global enterprises as well as IoT vendors. In his session at @ThingsExpo, Clark Smith, senior product manager for Numerex, will discuss how Numerex, as an experienced, established IoT provider, has embraced a ...
November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Penta Security is a leading vendor for data security solutions, including its encryption solution, D’Amo. By using FPE technology, D’Amo allows for the implementation of encryption technology to sensitive data fields without modification to schema in the database environment. With businesses having their data become increasingly more complicated in their mission-critical applications (such as ERP, CRM, HRM), continued ...
SYS-CON Events announced today that Cloudbric, a leading website security provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Cloudbric is an elite full service website protection solution specifically designed for IT novices, entrepreneurs, and small and medium businesses. First launched in 2015, Cloudbric is based on the enterprise level Web Application Firewall by Penta Security Sys...
We are reaching the end of the beginning with WebRTC, and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will w...
When people aren’t talking about VMs and containers, they’re talking about serverless architecture. Serverless is about no maintenance. It means you are not worried about low-level infrastructural and operational details. An event-driven serverless platform is a great use case for IoT. In his session at @ThingsExpo, Animesh Singh, an STSM and Lead for IBM Cloud Platform and Infrastructure, will detail how to build a distributed serverless, polyglot, microservices framework using open source tec...
The Open Connectivity Foundation (OCF), sponsor of the IoTivity open source project, and AllSeen Alliance, which provides the AllJoyn® open source IoT framework, today announced that the two organizations’ boards have approved a merger under the OCF name and bylaws. This merger will advance interoperability between connected devices from both groups, enabling the full operating potential of IoT and representing a significant step towards a connected ecosystem.
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
A completely new computing platform is on the horizon. They’re called Microservers by some, ARM Servers by others, and sometimes even ARM-based Servers. No matter what you call them, Microservers will have a huge impact on the data center and on server computing in general. Although few people are familiar with Microservers today, their impact will be felt very soon. This is a new category of computing platform that is available today and is predicted to have triple-digit growth rates for some ...
SYS-CON Events announced today that Roundee / LinearHub will exhibit at the WebRTC Summit at @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LinearHub provides Roundee Service, a smart platform for enterprise video conferencing with enhanced features such as automatic recording and transcription service. Slack users can integrate Roundee to their team via Slack’s App Directory, and '/roundee' command lets your video conference ...
SYS-CON Events announced today that Enzu will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their online busine...
SYS-CON Events announced today that SoftNet Solutions will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. SoftNet Solutions specializes in Enterprise Solutions for Hadoop and Big Data. It offers customers the most open, robust, and value-conscious portfolio of solutions, services, and tools for the shortest route to success with Big Data. The unique differentiator is the ability to architect and...
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
In past @ThingsExpo presentations, Joseph di Paolantonio has explored how various Internet of Things (IoT) and data management and analytics (DMA) solution spaces will come together as sensor analytics ecosystems. This year, in his session at @ThingsExpo, Joseph di Paolantonio from DataArchon, will be adding the numerous Transportation areas, from autonomous vehicles to “Uber for containers.” While IoT data in any one area of Transportation will have a huge impact in that area, combining senso...
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The IoT has the potential to create a renaissance of manufacturing in the US and elsewhere. In his session at 18th Cloud Expo, Florent Solt, CTO and chief architect of Netvibes, discussed how the expected exponential increase in the amount of data that will be processed, transported, stored, and accessed means there will be a huge demand for smart technologies to deliver it. Florent Solt is the CTO and chief architect of Netvibes. Prior to joining Netvibes in 2007, he co-founded Rift Technologi...
For basic one-to-one voice or video calling solutions, WebRTC has proven to be a very powerful technology. Although WebRTC’s core functionality is to provide secure, real-time p2p media streaming, leveraging native platform features and server-side components brings up new communication capabilities for web and native mobile applications, allowing for advanced multi-user use cases such as video broadcasting, conferencing, and media recording.