Welcome!

Cloud Security Authors: Yeshim Deniz, Pat Romanski, Ed Featherston, Elizabeth White, Liz McMillan

Related Topics: Recurring Revenue, Microsoft Cloud, Containers Expo Blog, @CloudExpo

Recurring Revenue: Article

Big Data on Grids or on Clouds?

Now that we have a new computing paradigm, Cloud Computing, how can Clouds help our data?

Now that we have a new computing paradigm, Cloud Computing, how can Clouds help our data? Replace our internal data vaults as we hoped Grids would? Are Grids dead now that we have Clouds? Despite all the promising developments in the Grid and Cloud computing space, and the avalanche of publications and talks on this subject, many people still seem to be confused about internal data and compute resources, versus Grids versus Clouds, and they are hesitant to take the next step. I think there are a number of issues driving this uncertainty.

Grids didn't keep all their promises
Grids did not evolve (as some of us originally thought) into the next fundamental IT infrastructure for everything and everybody. Because of the diversity of computing and data environments, we had to develop different middleware (department, enterprise, global, compute, data, sensors, scientific instruments, etc.), and had to face different usage models with different benefits. Enterprise Grids were (and are) providing better resource utilization and business flexibility, while global Grids are best suited to complex R&D collaboration with resource sharing. For enterprise usage, setting up and operating Grids was often complicated and did not remove all the (data) bottlenecks. For researchers this characteristic was seen to be a necessary evil. Implementing complex applications on supercomputers has never been easy. So what.

Grid: the way station to the Cloud
After 40 years of dealing with data processing, Grid computing was indeed the next big thing for the grand challenge R&D expert, while for the enterprise CIO, the Grid was a way station on its way to the Cloud model. For the enterprise today, Clouds are providing all the missing pieces: easy to use, economies of scale, business elasticity up and down, and pay-as you go (thus getting rid of some capital expenditure). And in cases where security matters, there is the private Cloud, within the enterprise's firewall. In more complex enterprise environments, with applications running under different policies, private Clouds can easily connect via the Internet to (external) public Clouds -- and vice versa -- forming a hybrid Cloud infrastructure that balances security with efficiency.

Different policies, what does that mean?
No data processing job is alike. Jobs differ by priority, strategic importance, deadline, budget, IP and licenses. In addition, the nature of the code often necessitates a specific computer architecture, operating system, memory, storage, and other resources. These important differentiating factors strongly influence where and when a data processing job is run. For any job, a set of specific requirements decide on the set of specific policies that have to be defined and programmed, such that any of these jobs will run just according to these policies. Ideally, this is guaranteed by a dynamic resource broker that controls submission to Grid or Cloud resources, be they local or global, private or public.

Grids or Clouds?
One important question is still open: how do I find out, and then tell the resource broker, whether my data should run on the Grid or in the Cloud? The answer, among others, depends on the algorithmic structure of the program, which might be intolerant of high latency and low bandwidth. The performance limitations are exhibited mainly by parallel applications with tightly-coupled, data-intensive inter-process communication, running in parallel on hundreds or even thousands of processors or cores.

The good news is, however, that many applications do not require high bandwidth and low latency. Parameter studies often seen in science, engineering, and business intelligence, where a single self-contained application executes with many different parameters, resulting in many independent jobs. The list of examples is extensive -  analyzing the data from a particle physics collider, identifying the solution parameter in optimization, ensemble runs to quantify climate model uncertainties, identifying potential drug targets via screening a database of ligand structures, studying economic model sensitivity to parameters, and analyzing different materials and their resistance in crash tests, to name just a few.

Big Data needs Grids or Clouds, and often both
Obviously, there is no "Grids or Clouds" for the enterprise. There is just "Grids and Clouds", it really depends on the individual scenario. In general, CIOs have to evaluate three different scenarios:

  • (1) the Private Cloud: optimizing and virtualizing the company's internal enterprise IT infrastructure, including the data layer (here is where Momentum can help);
  • (2) the Hybrid Cloud: do (1) and connect to external clouds;
  • or (3) the Public Cloud: do (2) and successively move data (processing) to the external cloud provider.

The choice for the best-suited scenario depends on many aspects: sensitive / competitive data and applications (e.g. medical patient records), individual return on investment, security policies, interoperability between private and public clouds, lose of control when moving data outside the corporation, cloud-enabling data and applications, the current software licensing model, protection of intellectual property, legal issues, and more.

The good news is that CIOs can always start with a hybrid infrastructure in mind: combining private and public cloud resources, balanced according to specific requirements. This provides the best of both worlds, avoiding the worst of each individual world.

More Stories By Pawel Plaszczak

Pawel’s international software engineering experience includes work at CERN, British Telecommunications and Argonne National Laboratory. In 2003, Pawel founded GridwiseTech to lead pioneering work for the early adopters of scalable systems. Under Pawel's leadership the company has won the trust and respect of customers including Turner Broadcasting, Ricoh, and Philips, and led numerous research efforts for international consortia. Pawel is the author of numerous articles and tutorials, the book "Grid Computing: The Savvy Manager's Guide", and a frequent speaker at professional conferences and events.

@ThingsExpo Stories
Announcing Poland #DigitalTransformation Pavilion
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
CloudEXPO | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
The best way to leverage your CloudEXPO | DXWorldEXPO presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering CloudEXPO | DXWorldEXPO will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at CloudEXPO. Product announcements during our show provide your company with the most reach through our targeted audienc...
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution. In his session at @ThingsExpo, Akvelon expert and IoT industry leader Sergey Grebnov provided an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
JETRO showcased Japan Digital Transformation Pavilion at SYS-CON's 21st International Cloud Expo® at the Santa Clara Convention Center in Santa Clara, CA. The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
In past @ThingsExpo presentations, Joseph di Paolantonio has explored how various Internet of Things (IoT) and data management and analytics (DMA) solution spaces will come together as sensor analytics ecosystems. This year, in his session at @ThingsExpo, Joseph di Paolantonio from DataArchon, added the numerous Transportation areas, from autonomous vehicles to “Uber for containers.” While IoT data in any one area of Transportation will have a huge impact in that area, combining sensor analytic...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Michael Maximilien, better known as max or Dr. Max, is a computer scientist with IBM. At IBM Research Triangle Park, he was a principal engineer for the worldwide industry point-of-sale standard: JavaPOS. At IBM Research, some highlights include pioneering research on semantic Web services, mashups, and cloud computing, and platform-as-a-service. He joined the IBM Cloud Labs in 2014 and works closely with Pivotal Inc., to help make the Cloud Found the best PaaS.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
I think DevOps is now a rambunctious teenager - it's starting to get a mind of its own, wanting to get its own things but it still needs some adult supervision," explained Thomas Hooker, VP of marketing at CollabNet, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.