|By Steve Weisfeldt||
|January 21, 2013 08:00 AM EST||
Developing with Performance Testing in Mind
A tester friend of mine recently came to me with a complaint that I think is fairly common in the testing community. He said, "Every time there is a new release of the software for us to test, we have to rework our testing scripts." I've heard this complaint throughout my career, not only in performance testing but in functional testing with automation tools as well.
This state of affairs arises from three fairly straightforward observations:
- Change is inevitable. Everything changes, and in no industry is this more apparent than software development. It makes no sense for testers to ask developers to stop changing the code, but it does make sense to encourage wise changes.
- Developers and testers don't always communicate well. The proverbial wall between developers and testers is still quite formidable. When developers throw a new version over the wall to be tested, too often they've given little thought to how it will be tested.
- All testing tools are not created equal. Some make it easier to identify and handle changes than others. If your testing tool is designed to handle change well, then your entire team is better positioned to embrace change rather than fear it.
Thinking Like a Tester
Most development organizations make a real effort to improve communication between developers and testers, but it's not always enough. Beyond encouraging developers to talk with testers, I ask them to take it a step further and think like testers.
I find that it's a good idea for developers to sit through some of the training that the test engineers complete. In my experience, the developers who do are more careful and avoid making arbitrary changes with little or no justification. They don't, for example, change the name of a field in a form simply because they don't agree with the name the initial developer gave it. When developers are aware of the kinds of changes that make a tester's job harder and what kinds of changes make it easier, then from an organizational standpoint the entire process is more productive.
An Analogy from the Early Days of Functional Testing
Some of the earliest automated functional testing tools for GUIs would simply record the location of the mouse pointer on the screen during tests, and then play back those mouse clicks to execute the test script. If a developer moved the location of a button on the screen, the script would break. Other tools would record the label on the button, so the button could be moved around the UI without breaking the script but changing the button text from "Submit" to "OK" would break the script. More advanced tools used the button's ID to identify it in the script so that the developer could change both the position and the label of the button without making the tester's job more difficult.
One key lesson here is that the choice of testing tool makes a big difference in the productivity of the testing team when the software under test changes, even in relatively trivial ways.
The other key lesson is that developer awareness of testing tools and procedures goes a long way in facilitating a smooth testing operation. I saw this firsthand during a training session I gave years back. While describing how button label changes affected testing, a developer who happened to be sitting in on the training sat upright when he finally understood why his colleagues in testing were so frustrated by many of his changes. He never knew why they objected so much to his changing a button label from "Clear" to "Reset". Going forward, that knowledge didn't stop the developer from making necessary changes. It did, however, make him pause when he made such changes to consider whether they were really necessary.
Performance Testing Tools That Make It Easy to Handle Change
In performance testing, we are not concerned with the location of buttons, but we're not immune to seemingly trivial changes.
For example, when a web form is submitted to the server, the form fields will be a series of name-value pairs. Changing the name of a form field, adding a field, or deleting a field can cause problems during performance testing. With a less capable testing tool, these problems can be hard to identify and diagnose, especially if there is poor communication between developers and testers.
File difference viewers (diff viewers) that enable the tester to compare multiple recordings against each other are particularly helpful in pinpointing the changed fields. When it's time to modify the script, an effective tool will enable you to add, delete, and update fields without programming. Just right-click and choose add, or simply drag-and-drop to update your load testing script.
Form fields are relatively easy to handle for load testers. Parameters that are session specific are more difficult. (These parameters change from session to session but stay the same for the duration of each user session). By default, the hard-coded session values are captured by a load testing tool in each script, and a test engineer needs to parameterize them to make the script usable for load testing. Double-clicking on a hard-coded value to make it a variable is easier than diving into the script code. Here again, tools that help automate the process can reduce test creation time from many hours to a few minutes.
When a new script is needed or maintenance is required on an existing script, tools that are easier to use can make the task orders of magnitude faster.
Overcoming the Fear of Change
I know of development teams that gradually became more and more afraid to change their software because of the difficulties that the changes introduced in testing and elsewhere in the process. Needless to say, this had a negative effect on their ability to deliver new features and fixes. A root of the problem, it turned out, was the testing tool that they were using, which made changes arduous and error-prone. Once they switched to a modern tool, the required script changes were easier to make. Performance testing times shrank from a week to less than a day and development was once again free to make long-needed changes. Agile development shops in particular depend on this ability to rapidly implement changes in testing scripts, and get the tests going in minutes or hours instead of days and weeks.
If your organization is starting to fear change, encourage your developers to think like testers and encourage your testers to use tools that make inevitable change easier to handle.
Hadoop as a Service (as offered by handful of niche vendors now) is a cloud computing solution that makes medium and large-scale data processing accessible, easy, fast and inexpensive. In his session at Big Data Expo, Kumar Ramamurthy, Vice President and Chief Technologist, EIM & Big Data, at Virtusa, will discuss how this is achieved by eliminating the operational challenges of running Hadoop, so one can focus on business growth. The fragmented Hadoop distribution world and various PaaS solutions that provide a Hadoop flavor either make choices for customers very flexible in the name of opti...
May. 24, 2015 12:30 AM EDT Reads: 3,461
In the consumer IoT, everything is new, and the IT world of bits and bytes holds sway. But industrial and commercial realms encompass operational technology (OT) that has been around for 25 or 50 years. This grittier, pre-IP, more hands-on world has much to gain from Industrial IoT (IIoT) applications and principles. But adding sensors and wireless connectivity won’t work in environments that demand unwavering reliability and performance. In his session at @ThingsExpo, Ron Sege, CEO of Echelon, will discuss how as enterprise IT embraces other IoT-related technology trends, enterprises with i...
May. 24, 2015 12:00 AM EDT Reads: 4,057
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal an...
May. 23, 2015 11:00 PM EDT Reads: 2,320
Wearable devices have come of age. The primary applications of wearables so far have been "the Quantified Self" or the tracking of one's fitness and health status. We propose the evolution of wearables into social and emotional communication devices. Our BE(tm) sensor uses light to visualize the skin conductance response. Our sensors are very inexpensive and can be massively distributed to audiences or groups of any size, in order to gauge reactions to performances, video, or any kind of presentation. In her session at @ThingsExpo, Jocelyn Scheirer, CEO & Founder of Bionolux, will discuss ho...
May. 23, 2015 09:00 PM EDT Reads: 4,946
The true value of the Internet of Things (IoT) lies not just in the data, but through the services that protect the data, perform the analysis and present findings in a usable way. With many IoT elements rooted in traditional IT components, Big Data and IoT isn’t just a play for enterprise. In fact, the IoT presents SMBs with the prospect of launching entirely new activities and exploring innovative areas. CompTIA research identifies several areas where IoT is expected to have the greatest impact.
May. 23, 2015 09:00 PM EDT Reads: 4,647
Every day we read jaw-dropping stats on the explosion of data. We allocate significant resources to harness and better understand it. We build businesses around it. But we’ve only just begun. For big payoffs in Big Data, CIOs are turning to cognitive computing. Cognitive computing’s ability to securely extract insights, understand natural language, and get smarter each time it’s used is the next, logical step for Big Data.
May. 23, 2015 08:00 PM EDT Reads: 1,772
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
May. 23, 2015 07:00 PM EDT Reads: 1,307
We are reaching the end of the beginning with WebRTC, and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will want to use their existing identities, but these will have credentials already that are (hopefully) i...
May. 23, 2015 07:00 PM EDT Reads: 4,011
17th Cloud Expo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS – software, platform, and infrastructure as a service.
May. 23, 2015 05:00 PM EDT Reads: 2,103
The Industrial Internet revolution is now underway, enabled by connected machines and billions of devices that communicate and collaborate. The massive amounts of Big Data requiring real-time analysis is flooding legacy IT systems and giving way to cloud environments that can handle the unpredictable workloads. Yet many barriers remain until we can fully realize the opportunities and benefits from the convergence of machines and devices with Big Data and the cloud, including interoperability, data security and privacy.
May. 23, 2015 04:00 PM EDT Reads: 4,668
The 17th International Cloud Expo has announced that its Call for Papers is open. 17th International Cloud Expo, to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, APM, APIs, Microservices, Security, Big Data, Internet of Things, DevOps and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal today!
May. 23, 2015 02:00 PM EDT Reads: 4,018
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series data. By focusing on enterprise applications and the data center, he will use OpenTSDB as an example t...
May. 23, 2015 02:00 PM EDT Reads: 6,229
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using the URL as a basic building block, we open this up and get the same resilience that the web enjoys.
May. 23, 2015 01:00 PM EDT Reads: 6,745
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
May. 23, 2015 01:00 PM EDT Reads: 931
Container frameworks, such as Docker, provide a variety of benefits, including density of deployment across infrastructure, convenience for application developers to push updates with low operational hand-holding, and a fairly well-defined deployment workflow that can be orchestrated. Container frameworks also enable a DevOps approach to application development by cleanly separating concerns between operations and development teams. But running multi-container, multi-server apps with containers is very hard. You have to learn five new and different technologies and best practices (libswarm, sy...
May. 23, 2015 12:00 PM EDT Reads: 1,706
SYS-CON Events announced today that DragonGlass, an enterprise search platform, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. After eleven years of designing and building custom applications, OpenCrowd has launched DragonGlass, a cloud-based platform that enables the development of search-based applications. These are a new breed of applications that utilize a search index as their backbone for data retrieval. They can easily adapt to new data sets and provide access to both structured and unstruc...
May. 23, 2015 12:00 PM EDT Reads: 1,658
Converging digital disruptions is creating a major sea change - Cisco calls this the Internet of Everything (IoE). IoE is the network connection of People, Process, Data and Things, fueled by Cloud, Mobile, Social, Analytics and Security, and it represents a $19Trillion value-at-stake over the next 10 years. In her keynote at @ThingsExpo, Manjula Talreja, VP of Cisco Consulting Services, will discuss IoE and the enormous opportunities it provides to public and private firms alike. She will share what businesses must do to thrive in the IoE economy, citing examples from several industry sector...
May. 23, 2015 12:00 PM EDT Reads: 1,792
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, June 9-11, 2015, at the Javits Center in New York City. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be
May. 23, 2015 11:15 AM EDT Reads: 1,625
An entirely new security model is needed for the Internet of Things, or is it? Can we save some old and tested controls for this new and different environment? In his session at @ThingsExpo, New York's at the Javits Center, Davi Ottenheimer, EMC Senior Director of Trust, reviewed hands-on lessons with IoT devices and reveal a new risk balance you might not expect. Davi Ottenheimer, EMC Senior Director of Trust, has more than nineteen years' experience managing global security operations and assessments, including a decade of leading incident response and digital forensics. He is co-author of t...
May. 23, 2015 11:00 AM EDT Reads: 5,445
The Internet of Things is a misnomer. That implies that everything is on the Internet, and that simply should not be - especially for things that are blurring the line between medical devices that stimulate like a pacemaker and quantified self-sensors like a pedometer or pulse tracker. The mesh of things that we manage must be segmented into zones of trust for sensing data, transmitting data, receiving command and control administrative changes, and peer-to-peer mesh messaging. In his session at @ThingsExpo, Ryan Bagnulo, Solution Architect / Software Engineer at SOA Software, focused on desi...
May. 23, 2015 11:00 AM EDT Reads: 3,782