|By Steve Weisfeldt||
|January 21, 2013 08:00 AM EST||
Developing with Performance Testing in Mind
A tester friend of mine recently came to me with a complaint that I think is fairly common in the testing community. He said, "Every time there is a new release of the software for us to test, we have to rework our testing scripts." I've heard this complaint throughout my career, not only in performance testing but in functional testing with automation tools as well.
This state of affairs arises from three fairly straightforward observations:
- Change is inevitable. Everything changes, and in no industry is this more apparent than software development. It makes no sense for testers to ask developers to stop changing the code, but it does make sense to encourage wise changes.
- Developers and testers don't always communicate well. The proverbial wall between developers and testers is still quite formidable. When developers throw a new version over the wall to be tested, too often they've given little thought to how it will be tested.
- All testing tools are not created equal. Some make it easier to identify and handle changes than others. If your testing tool is designed to handle change well, then your entire team is better positioned to embrace change rather than fear it.
Thinking Like a Tester
Most development organizations make a real effort to improve communication between developers and testers, but it's not always enough. Beyond encouraging developers to talk with testers, I ask them to take it a step further and think like testers.
I find that it's a good idea for developers to sit through some of the training that the test engineers complete. In my experience, the developers who do are more careful and avoid making arbitrary changes with little or no justification. They don't, for example, change the name of a field in a form simply because they don't agree with the name the initial developer gave it. When developers are aware of the kinds of changes that make a tester's job harder and what kinds of changes make it easier, then from an organizational standpoint the entire process is more productive.
An Analogy from the Early Days of Functional Testing
Some of the earliest automated functional testing tools for GUIs would simply record the location of the mouse pointer on the screen during tests, and then play back those mouse clicks to execute the test script. If a developer moved the location of a button on the screen, the script would break. Other tools would record the label on the button, so the button could be moved around the UI without breaking the script but changing the button text from "Submit" to "OK" would break the script. More advanced tools used the button's ID to identify it in the script so that the developer could change both the position and the label of the button without making the tester's job more difficult.
One key lesson here is that the choice of testing tool makes a big difference in the productivity of the testing team when the software under test changes, even in relatively trivial ways.
The other key lesson is that developer awareness of testing tools and procedures goes a long way in facilitating a smooth testing operation. I saw this firsthand during a training session I gave years back. While describing how button label changes affected testing, a developer who happened to be sitting in on the training sat upright when he finally understood why his colleagues in testing were so frustrated by many of his changes. He never knew why they objected so much to his changing a button label from "Clear" to "Reset". Going forward, that knowledge didn't stop the developer from making necessary changes. It did, however, make him pause when he made such changes to consider whether they were really necessary.
Performance Testing Tools That Make It Easy to Handle Change
In performance testing, we are not concerned with the location of buttons, but we're not immune to seemingly trivial changes.
For example, when a web form is submitted to the server, the form fields will be a series of name-value pairs. Changing the name of a form field, adding a field, or deleting a field can cause problems during performance testing. With a less capable testing tool, these problems can be hard to identify and diagnose, especially if there is poor communication between developers and testers.
File difference viewers (diff viewers) that enable the tester to compare multiple recordings against each other are particularly helpful in pinpointing the changed fields. When it's time to modify the script, an effective tool will enable you to add, delete, and update fields without programming. Just right-click and choose add, or simply drag-and-drop to update your load testing script.
Form fields are relatively easy to handle for load testers. Parameters that are session specific are more difficult. (These parameters change from session to session but stay the same for the duration of each user session). By default, the hard-coded session values are captured by a load testing tool in each script, and a test engineer needs to parameterize them to make the script usable for load testing. Double-clicking on a hard-coded value to make it a variable is easier than diving into the script code. Here again, tools that help automate the process can reduce test creation time from many hours to a few minutes.
When a new script is needed or maintenance is required on an existing script, tools that are easier to use can make the task orders of magnitude faster.
Overcoming the Fear of Change
I know of development teams that gradually became more and more afraid to change their software because of the difficulties that the changes introduced in testing and elsewhere in the process. Needless to say, this had a negative effect on their ability to deliver new features and fixes. A root of the problem, it turned out, was the testing tool that they were using, which made changes arduous and error-prone. Once they switched to a modern tool, the required script changes were easier to make. Performance testing times shrank from a week to less than a day and development was once again free to make long-needed changes. Agile development shops in particular depend on this ability to rapidly implement changes in testing scripts, and get the tests going in minutes or hours instead of days and weeks.
If your organization is starting to fear change, encourage your developers to think like testers and encourage your testers to use tools that make inevitable change easier to handle.
SYS-CON Events announced today that Cloudian, Inc., the leading provider of hybrid cloud storage solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Cloudian, Inc., is a Foster City, California - based software company specializing in cloud storage software. The main product is Cloudian, an Amazon S3-compliant cloud object storage platform, the bedrock of cloud computing systems, that enables cloud service providers and enterprises to build reliable, affordable and scalable cloud storage solu...
Apr. 27, 2015 01:00 AM EDT Reads: 2,756
Temasys has announced senior management additions to its team. Joining are David Holloway as Vice President of Commercial and Nadine Yap as Vice President of Product. Over the past 12 months Temasys has doubled in size as it adds new customers and expands the development of its Skylink platform. Skylink leads the charge to move WebRTC, traditionally seen as a desktop, browser based technology, to become a ubiquitous web communications technology on web and mobile, as well as Internet of Things compatible devices.
Apr. 27, 2015 12:45 AM EDT Reads: 2,334
SYS-CON Events announced today that Gridstore™, the leader in hyper-converged infrastructure purpose-built to optimize Microsoft workloads, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Gridstore™ is the leader in hyper-converged infrastructure purpose-built for Microsoft workloads and designed to accelerate applications in virtualized environments. Gridstore’s hyper-converged infrastructure is the industry’s first all flash version of HyperConverged Appliances that include both compute and storag...
Apr. 27, 2015 12:45 AM EDT Reads: 4,665
“With easy-to-use SDKs for Atmel’s platforms, IoT developers can now reap the benefits of realtime communication, and bypass the security pitfalls and configuration complexities that put IoT deployments at risk,” said Todd Greene, founder & CEO of PubNub. PubNub will team with Atmel at CES 2015 to launch full SDK support for Atmel’s MCU, MPU, and Wireless SoC platforms. Atmel developers now have access to PubNub’s secure Publish/Subscribe messaging with guaranteed ¼ second latencies across PubNub’s 14 global points-of-presence. PubNub delivers secure communication through firewalls, proxy ser...
Apr. 27, 2015 12:00 AM EDT Reads: 3,883
SYS-CON Events announced today that IDenticard will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. IDenticard™ is the security division of Brady Corp (NYSE: BRC), a $1.5 billion manufacturer of identification products. We have small-company values with the strength and stability of a major corporation. IDenticard offers local sales, support and service to our customers across the United States and Canada. Our partner network encompasses some 300 of the world's leading systems integrators and security s...
Apr. 27, 2015 12:00 AM EDT Reads: 5,277
SYS-CON Events announced today that On the Avenue Marketing Group, a sales and marketing firm that utilizes events to market and sell products to consumers, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. On the Avenue Marketing Group (OTA) is a sales and marketing firm that utilizes events to market and sell products to consumers. On behalf of our clients, we attend thousands of fairs, festivals, expos, concerts, conferences, and sporting events annually, helping them reach millions of individuals ...
Apr. 26, 2015 07:30 PM EDT Reads: 3,507
Containers and microservices have become topics of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 16th Cloud Expo at the Javits Center in New York June 9-11 will find fresh new content in a new track called PaaS | Containers & Microservices Containers are not being considered for the first time by the cloud community, but a current era of re-consideration has pushed them to the top of the cloud agenda. With the launch of Docker's initial release in March of 2013, interest was revved up several notches. Then late last...
Apr. 26, 2015 07:00 PM EDT Reads: 2,892
“In the past year we've seen a lot of stabilization of WebRTC. You can now use it in production with a far greater degree of certainty. A lot of the real developments in the past year have been in things like the data channel, which will enable a whole new type of application," explained Peter Dunkley, Technical Director at Acision, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Apr. 26, 2015 05:00 PM EDT Reads: 4,492
SYS-CON Events announced today that Ciqada will exhibit at SYS-CON's @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Ciqada™ makes it easy to connect your products to the Internet. By integrating key components - hardware, servers, dashboards, and mobile apps - into an easy-to-use, configurable system, your products can quickly and securely join the internet of things. With remote monitoring, control, and alert messaging capability, you will meet your customers' needs of tomorrow - today! Ciqada. Let your products take flight. For more inform...
Apr. 26, 2015 03:00 PM EDT Reads: 1,934
Health care systems across the globe are under enormous strain, as facilities reach capacity and costs continue to rise. M2M and the Internet of Things have the potential to transform the industry through connected health solutions that can make care more efficient while reducing costs. In fact, Vodafone's annual M2M Barometer Report forecasts M2M applications rising to 57 percent in health care and life sciences by 2016. Lively is one of Vodafone's health care partners, whose solutions enable older adults to live independent lives while staying connected to loved ones. M2M will continue to gr...
Apr. 26, 2015 03:00 PM EDT Reads: 1,535
SYS-CON Media announced today that @WebRTCSummit Blog, the largest WebRTC resource in the world, has been launched. @WebRTCSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. @WebRTCSummit Blog can be bookmarked ▸ Here @WebRTCSummit conference site can be bookmarked ▸ Here
Apr. 26, 2015 03:00 PM EDT Reads: 2,515
SYS-CON Events announced today that GENBAND, a leading developer of real time communications software solutions, has been named “Silver Sponsor” of SYS-CON's WebRTC Summit, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. The GENBAND team will be on hand to demonstrate their newest product, Kandy. Kandy is a communications Platform-as-a-Service (PaaS) that enables companies to seamlessly integrate more human communications into their Web and mobile applications - creating more engaging experiences for their customers and boosting collaboration and productiv...
Apr. 26, 2015 02:00 PM EDT Reads: 2,752
Dave will share his insights on how Internet of Things for Enterprises are transforming and making more productive and efficient operations and maintenance (O&M) procedures in the cleantech industry and beyond. Speaker Bio: Dave Landa is chief operating officer of Cybozu Corp (kintone US). Based in the San Francisco Bay Area, Dave has been on the forefront of the Cloud revolution driving strategic business development on the executive teams of multiple leading Software as a Services (SaaS) application providers dating back to 2004. Cybozu's kintone.com is a leading global BYOA (Build Your O...
Apr. 26, 2015 02:00 PM EDT Reads: 1,579
The best mobile applications are augmented by dedicated servers, the Internet and Cloud services. Mobile developers should focus on one thing: writing the next socially disruptive viral app. Thanks to the cloud, they can focus on the overall solution, not the underlying plumbing. From iOS to Android and Windows, developers can leverage cloud services to create a common cross-platform backend to persist user settings, app data, broadcast notifications, run jobs, etc. This session provides a high level technical overview of many cloud services available to mobile app developers, includi...
Apr. 26, 2015 02:00 PM EDT Reads: 1,429
SYS-CON Events announced today that BroadSoft, the leading global provider of Unified Communications and Collaboration (UCC) services to operators worldwide, has been named “Gold Sponsor” of SYS-CON's WebRTC Summit, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. BroadSoft is the leading provider of software and services that enable mobile, fixed-line and cable service providers to offer Unified Communications over their Internet Protocol networks. The Company’s core communications platform enables the delivery of a range of enterprise and consumer calling...
Apr. 26, 2015 01:30 PM EDT Reads: 2,562
While not quite mainstream yet, WebRTC is starting to gain ground with Carriers, Enterprises and Independent Software Vendors (ISV’s) alike. WebRTC makes it easy for developers to add audio and video communications into their applications by using Web browsers as their platform. But like any market, every customer engagement has unique requirements, as well as constraints. And of course, one size does not fit all. In her session at WebRTC Summit, Dr. Natasha Tamaskar, Vice President, Head of Cloud and Mobile Strategy at GENBAND, will explore what is needed to take a real time communications ...
Apr. 26, 2015 01:00 PM EDT Reads: 1,801
What exactly is a cognitive application? In her session at 16th Cloud Expo, Ashley Hathaway, Product Manager at IBM Watson, will look at the services being offered by the IBM Watson Developer Cloud and what that means for developers and Big Data. She'll explore how IBM Watson and its partnerships will continue to grow and help define what it means to be a cognitive service, as well as take a look at the offerings on Bluemix. She will also check out how Watson and the Alchemy API team up to offer disruptive APIs to developers.
Apr. 26, 2015 12:00 PM EDT Reads: 1,869
The IoT Bootcamp is coming to Cloud Expo | @ThingsExpo on June 9-10 at the Javits Center in New York. Instructor. Registration is now available at http://iotbootcamp.sys-con.com/ Instructor Janakiram MSV previously taught the famously successful Multi-Cloud Bootcamp at Cloud Expo | @ThingsExpo in November in Santa Clara. Now he is expanding the focus to Janakiram is the founder and CTO of Get Cloud Ready Consulting, a niche Cloud Migration and Cloud Operations firm that recently got acquired by Aditi Technologies. He is a Microsoft Regional Director for Hyderabad, India, and one of the f...
Apr. 26, 2015 12:00 PM EDT Reads: 1,678
The 17th International Cloud Expo has announced that its Call for Papers is open. 17th International Cloud Expo, to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, APM, APIs, Microservices, Security, Big Data, Internet of Things, DevOps and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal today!
Apr. 26, 2015 12:00 PM EDT Reads: 2,329
WebRTC is an up-and-coming standard that enables real-time voice and video to be directly embedded into browsers making the browser a primary user interface for communications and collaboration. WebRTC runs in a number of browsers today and is currently supported in over a billion installed browsers globally, across a range of platform OS and devices. Today, organizations that choose to deploy WebRTC applications and use a host machine that supports audio through USB or Bluetooth can use Plantronics products to connect and transit or receive the audio associated with the WebRTC session.
Apr. 26, 2015 12:00 PM EDT Reads: 1,913