Welcome!

Cloud Security Authors: Elizabeth White, Liz McMillan, Maria C. Horton, Pat Romanski, Yeshim Deniz

Related Topics: @DevOpsSummit, Microservices Expo, Containers Expo Blog

@DevOpsSummit: Article

Test Environment Stability | @DevOpsSummit #APM #CloudNative #Monitoring

Find confidence in the systems you are delivering

How often is an environment unavailable due to factors within your project's control? How often is an environment unavailable due to external factors? Is the software and hardware in the environment up to date with the target production systems? How often do you have to resort to manual workarounds due to an environment?

These are all questions that you should ask yourself if testing environments are consistently unavailable and affected by outages. Here are three key metrics that you can track that can help avoid costly downtime.

Metric: Availability and Uptime Percentage
QA and staging environments seldom require the same level of uptime as production, but tell that to a team of developers working 24/7 on a project that has an aggressive deadline and you will have a very unhappy team. As a test environment manager, you know that when a QA system is unavailable, you get immediate calls from developers and managers.

Understanding why an outage happened is critical for communicating with a development team. If you follow a problem management process for production outages, similar to the process throughout the rest of your IT environment, then you should also follow this process with test environment management. Very often a QA environment will become unavailable due to a factor outside the control of a test environment manager. If one team pushes bad code that interrupts the QA process for all teams you need to be able to identify this clearly.

How to measure availability and uptime?
To truly understand availability and uptime, you must keep track of system availability with a standard monitoring tool such as Zabbix or Nagios. If your systems are visible to the public internet, you can also use hosted platforms like Pingdom to measure system availability.

An uptime of 95% is usually sufficient for a QA or staging environment. If your development is limited to a few time zones, you can also further qualify this by only measuring availability during development hours. While production availability commitment is often closer to 99% or 99.5%, you don't have to treat every QA outage as an emergency. But, be aware that developers may have other opinions-95% uptime still allows for eight hours of downtime a week, so it may be best to aim higher.

How does this metric motivate concrete action?
When you measure system availability and make these numbers public, you encourage test environment managers to make a commitment to uptime. This results in fewer obstacles for QA and development, allowing them to deliver software faster. There's nothing more debilitating to an organization than disruptions in QA and testing. Keeping a close eye on availability and tracking as a metric also encourages movement towards always-available QA systems.

Metric: Mean Time Between Outages
If a system has a 95% availability, then almost 75 minutes of downtime is acceptable every day. If the system fails for ten minutes every hour during an eight-hour work day due to a build or deployment, you'll be creating a QA or staging environment that has a 5% chance of losing developer and QA confidence. To get an accurate picture of system availability you need to couple an availability percentage metric with the mean time between outages (MTBO).

How to Measure MTBO?
If you follow a process that keeps track of outages and strives to understand the root causes of these outages, you'll amass a database of issues that can be used to derive the MTBO. With a monitoring system configured to calculate availability percentages automatically, you can use this same system to record your MTBO.

Example Metric: Goal for MTBO
A goal for MTBO, depends greatly on your availability goal. The lower your availability goal, the higher your MTBO should be. For example, if you have a 95% uptime commitment then your outages need to be spaced over a day or a week. You might have eight hours of downtime each weekend to perform system upgrades or a nightly build and deploy process that takes about an hour, but what you can't have is an MTBO of 45-60 minutes. This will mean that QA and staging systems will be unavailable for a few minutes every hour, which will result in dissatisfied customers.

How does this metric motivate concrete action?
If your MBTO is very short, this suggests that build and deploy activity from a continuous integration environment is frequently interrupting both development and QA. If your MBTO is very high, but your availability is very low (95% or lower) this means that you are experiencing multi-hour downtime at least once a day.

When measuring MBTO, encourage your release engineers and test environment managers to work together to create build and deployment scripts that don't affect availability, and encourage staff to approach QA and staging uptime with care. Without this metric, you run the risk of having teams grow complacent with frequent, low-level unavailability as long as they satisfy overall availability metrics.

Metric: Downtime requirement for a test environment build and deploy
When software is deployed to any system, there is a natural tendency for disruption. If new code is being deployed to an application server, that server often requires a restart so that new code can be loaded. If a web server, such as Apache or Nginx, is being reconfigured this often requires a fast restart measured in seconds.

Some of these build and deploy related disruptions can be avoided through the use of load balancers and clusters of machines. On the largest projects, this is essential in both production as well as staging and QA systems.

An example is a QA system for a large bank's transaction processing system. There are so many teams that depend on this system to be up and running 24/7 that causing any disruption would run the risk of freezing the QA process across the entire company.

Other build and deploy downtimes are unavoidable. A frequent example is when there are changes to a database schema. Certain changes to tables and indexes require systems to be stopped and rebooted to reach a state where database activity isn't competing with DDL statements.

The downtime requirement for a given build and deploy to a test environment is a central measure that is directly related to the availability metrics already discussed.

How to Measure Build/Deploy Downtime?
It's simple: run a build and deployment and keep track of the downtime that falls into the timespan of each build and deploy function. If you have a continuous integration system, such as Jenkins or Bamboo, grab the timestamps of the last few builds and look at your monitoring metrics on QA and staging to see if there is a system impact.

Example Metric: Goal for Build/Deploy Downtime
Your goal for this metric depends on your level of availability. If you are working on a shared service, build and deploy downtime requirements should be as close to zero as possible. If you are working on a less critical application, then build and deploy downtime should be measured in minutes or seconds.

How does this Metric Motivate Concrete Action?
This metric encourages release engineers and test environment managers to drive build and deploy downtime to zero. With the tools available to developers and DevOps professionals, it is possible to achieve zero-downtime deployments to QA and staging systems. Doing this will give internal customers more confidence in the systems you are delivering.


DXWorldEXPO LLC, the producer of the world's most influential technology conferences and trade shows has announced the conference tracks for CloudEXPO | DXWorldEXPO 2018 New York.

DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City.

Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.

A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes.

Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)

Sponsorship Opportunities Here

Speaking Opportunities Here

Sponsorship and Speaking Inquiries: [email protected].

2018 Conference Agenda, Keynotes and 10 Conference Tracks

DXWordEXPO New York 2018 and Cloud Expo New York 2018 agenda present 222 rockstar faculty members, 200 sessions and 22 keynotes and general sessions in 10 distinct conference tracks.

  • Cloud-Native | Serverless
  • DevOpsSummit
  • FinTechEXPO - New York Blockchain Event
  • CloudEXPO - Enterprise Cloud
  • DXWorldEXPO - Digital Transformation (DX)
  • Smart Cities | IoT | IIoT
  • AI | Machine Learning | Cognitive Computing
  • BigData | Analytics
  • The API Enterprise | Mobility | Security
  • Hot Topics | FinTech | WebRTC

Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)

DXWorldEXPO | CloudEXPO 2018 New York cover all of these tools, with the most comprehensive program and with 222 rockstar speakers throughout our industry presenting 22 Keynotes and General Sessions, 200 Breakout Sessions along 10 Tracks, as well as our signature Power Panels. Our Expo Floor brings together the world's leading companies throughout the world of Cloud Computing, DevOps, FinTech, Digital Transformation, and all they entail.

As your enterprise creates a vision and strategy that enables you to create your own unique, long-term success, learning about all the technologies involved is essential. Companies today not only form multi-cloud and hybrid cloud architectures, but create them with built-in cognitive capabilities.

Cloud-Native thinking is now the norm in financial services, manufacturing, telco, healthcare, transportation, energy, media, entertainment, retail and other consumer industries, as well as the public sector.

CloudEXPO is the world's most influential technology event where Cloud Computing was coined over a decade ago and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals.

FinTech Is Now Part of the DXWorldEXPO | CloudEXPO Program!

Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expensive intermediate processes from their businesses.

Accordingly, attendees at the upcoming 22nd CloudEXPO | DXWorldEXPO November 11-13, 2018 in New York City will find fresh new content in two new tracks called:

  • FinTechEXPO
  • New York Blockchain Event

which will incorporate FinTech and Blockchain, as well as machine learning, artificial intelligence and deep learning in these two distinct tracks.

Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)

Sponsorship Opportunities Here

Speaking Opportunities Here

Sponsorship and Speaking Inquiries: [email protected].

FinTech brings efficiency as well as the ability to deliver new services and a much improved customer experience throughout the global financial services industry. FinTech is a natural fit with cloud computing, as new services are quickly developed, deployed, and scaled on public, private, and hybrid clouds.

More than US$20 billion in venture capital is being invested in FinTech this year. DXWorldEXPOCloudEXPO are pleased to bring you the latest FinTech developments as an integral part of our program.

DXWorldEXPO | CloudEXPO are accepting speaking submissions for this new track, so please visit Cloud Computing Expo for the latest information or contact us at [email protected]

Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)

Sponsorship Opportunities Here

Speaking Opportunities Here

Sponsorship and Speaking Inquiries: [email protected].

Download Slide Deck ▸ Here

Only DXWorldEXPO | CloudEXPO bring together all this in a single location:

Attend DXWorldEXPO | CloudEXPO. Build your own custom experience. Learn about the world's latest technologies and chart your course to Digital Transformation.

22nd International DXWorldEXPO | CloudEXPO, taking place November 11-13, 2018, in New York City, will feature technical sessions from a rock star conference faculty and the leading industry players in the world.

Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)

Sponsorship Opportunities Here

Speaking Opportunities Here

Sponsorship and Speaking Inquiries: [email protected].

Download Slide Deck: ▸ Here

Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS - software, platform, and infrastructure as a service.

With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.

Every Global 2000 enterprise in the world is now integrating cloud computing in some form into its IT development and operations. Midsize and small businesses are also migrating to the cloud in increasing numbers.

Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)

Sponsorship Opportunities Here

Speaking Opportunities Here

Sponsorship and Speaking Inquiries: [email protected].

Download Slide Deck: ▸ Here

Companies are each developing their unique mix of cloud technologies and services, forming multi-cloud and hybrid cloud architectures and deployments across all major industries. Cloud-driven thinking has become the norm in financial services, manufacturing, telco, healthcare, transportation, energy, media, entertainment, retail and other consumer industries, and the public sector.

Sponsorship Opportunities

DXWorldEXPO | CloudEXPO are the single show where technology buyers and vendors can meet to experience and discus cloud computing and all that it entails. Sponsors of DXWorldEXPO | CloudEXPO will benefit from unmatched branding, profile building and lead generation opportunities through:

  • Featured on-site presentation and ongoing on-demand webcast exposure to a captive audience of industry decision-makers.
  • Showcase exhibition during our new extended dedicated expo hours
  • Breakout Session Priority scheduling for Sponsors that have been guaranteed a 35-minute technical session
  • Online advertising on 4,5 million article pages in SYS-CON's i-Technology Publications
  • Capitalize on our Comprehensive Marketing efforts leading up to the show with print mailings, e-newsletters and extensive online media coverage.
  • Unprecedented PR Coverage: Unmatched editorial coverage on Cloud Computing Journal.
  • Tweetup to over 100,000 plus Twitter followers
  • Press releases sent on major wire services to over 500 industry analysts.

Secrets of Our Most Popular Sponsors and Exhibitors ▸ Here

For more information on sponsorship, exhibit, and keynote opportunities, contact [email protected].

Sponsorship Opportunities Here

Download Slide Deck:Here

Speaking Opportunities

The upcoming 22nd International DXWorldEXPO | CloudEXPO November 11-13, 2018 in New York City, NY announces that its Call For Papers for speaking opportunities is now open.

Secrets of Our Most Popular Faculty Members ▸ Here

Submit your speaking proposal Here or by email [email protected].

Download Slide Deck: ▸ Here

About DXWorldEXPO LLC

DXWorldEXPO LLC is a Lighthouse Point, Florida-based trade show company and the creator of DXWorldEXPODigital Transformation Conference & Expo. The company produces and presents CloudEXPO, DevOpsSummitFinTechEXPO Blockchain Event, the world's most influential conferences and trade shows.

More Stories By Dalibor Siroky

Dalibor Siroky, CEO and Co-Founder, Plutora: Dalibor has close to 15 years of leadership, consulting, enterprise product, and operations experience in Australia, Asia, and Europe. Before co-founding Plutora, Dalibor was the founder and managing director of Finotaur, a leading provider of independent management consulting services to Wealth Management, Investment Management, Private Banking, and Payment institutions within the Asia Pacific region. Earlier in his career, Dalibor served as the CIO of financial advisory software at Macquarie Bank, head of solution architecture at Commonwealth Bank of Australia, and as a management consultant at PricewaterhouseCoopers. Dalibor holds an MS in Software Engineering with distinction from the University of Oxford and an MBA with honors from the University of Chicago. Dalibor is also a graduate of the Royal Military College of Australia and served as a Captain in the Army.

@ThingsExpo Stories
Organizations planning enterprise data center consolidation and modernization projects are faced with a challenging, costly reality. Requirements to deploy modern, cloud-native applications simultaneously with traditional client/server applications are almost impossible to achieve with hardware-centric enterprise infrastructure. Compute and network infrastructure are fast moving down a software-defined path, but storage has been a laggard. Until now.
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
IoT is at the core or many Digital Transformation initiatives with the goal of re-inventing a company's business model. We all agree that collecting relevant IoT data will result in massive amounts of data needing to be stored. However, with the rapid development of IoT devices and ongoing business model transformation, we are not able to predict the volume and growth of IoT data. And with the lack of IoT history, traditional methods of IT and infrastructure planning based on the past do not app...
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
DXWorldEXPO LLC, the producer of the world's most influential technology conferences and trade shows has announced the 22nd International CloudEXPO | DXWorldEXPO "Early Bird Registration" is now open. Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
Here are the Top 20 Twitter Influencers of the month as determined by the Kcore algorithm, in a range of current topics of interest from #IoT to #DeepLearning. To run a real-time search of a given term in our website and see the current top influencers, click on the topic name. Among the top 20 IoT influencers, ThingsEXPO ranked #14 and CloudEXPO ranked #17.
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant tha...
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
Major trends and emerging technologies – from virtual reality and IoT, to Big Data and algorithms – are helping organizations innovate in the digital era. However, to create real business value, IT must think beyond the ‘what’ of digital transformation to the ‘how’ to harness emerging trends, innovation and disruption. Architecture is the key that underpins and ties all these efforts together. In the digital age, it’s important to invest in architecture, extend the enterprise footprint to the cl...
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
"We are a well-established player in the application life cycle management market and we also have a very strong version control product," stated Flint Brenton, CEO of CollabNet,, in this SYS-CON.tv interview at 18th Cloud Expo at the Javits Center in New York City, NY.
In his session at @ThingsExpo, Arvind Radhakrishnen discussed how IoT offers new business models in banking and financial services organizations with the capability to revolutionize products, payments, channels, business processes and asset management built on strong architectural foundation. The following topics were covered: How IoT stands to impact various business parameters including customer experience, cost and risk management within BFS organizations.
While the focus and objectives of IoT initiatives are many and diverse, they all share a few common attributes, and one of those is the network. Commonly, that network includes the Internet, over which there isn't any real control for performance and availability. Or is there? The current state of the art for Big Data analytics, as applied to network telemetry, offers new opportunities for improving and assuring operational integrity. In his session at @ThingsExpo, Jim Frey, Vice President of S...
With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, discussed some of the security challenges of the IoT infrastructure and related how these aspects impact Smart Living. The material was delivered interac...