Welcome!

Cloud Security Authors: Liz McMillan, Pat Romanski, Elizabeth White, Yeshim Deniz, Terry Ray

Related Topics: @CloudExpo, Agile Computing, Cloud Security

@CloudExpo: Blog Post

Screen Scraping Bots By @DISTIL | @CloudExpo [#Cloud]

Thieves and unauthorized content aggregators use screen scraping bots and cheap labor to steal your content

Four Ways Screen Scraping Bots Degrade Your Website's Success

Like DDoS attacks, aggressive screen scraping can create huge spikes in traffic, cause brownouts, and even take you down for extended periods of time. Web scraping doesn't make the headlines and most people aren't aware of the damage it can cause to your website's success.

Thieves and unauthorized content aggregators use screen scraping bots and cheap labor to steal your content, repackage it for sale, or place it on unauthorized sites - all of which hurt your SEO, negatively impact your brand, and reduce your ability to monetize your intellectual property. Below I've outlined just a few of the many ways that screen scraping can impact a company and its website.

Screen scraping bots can:

  1. Steal your intellectual property: Your pricing data, product information, web page copy, and more is all vulnerable to web scraping. Once screen scrapers get their hands on it, they either use it to their competitive advantage, post it to another site, or sell it off. In any case, it means your proprietary data falls into the hands of others - for free. This malicious activity can be illustrated when, for example, an ecommerce company runs promotions on popular products and then, minutes later, sees their competitors undercutting their own pricing to get ahead.
  2. Bog down your web infrastructure: Screen scraping consumes bandwidth and compute resources, forcing IT teams to over-provision infrastructure to account for random spikes in bot traffic. Massive screen scraping requests can slow down your website or take you down for extended periods of time. This can happen when bots aggressively screen scrape long tail listings from a directory website, for example. These types of listings do not usually get traffic, so they are not cached in the company's CDN (Content Delivery Network). When bots aggressively scrape these listings, it causes a brown out and user complaints.
  3. Harm your reputation: Brownouts, outages, and duplicate content can harm your reputation. Screen scrapers can repost your content on less reputable sites as well as introduce errors into your content. Sometimes, screen scraping bots register on websites in order to scrape content behind a registration form. Fake registrations can pollute your site and back-end systems and have negative cascading effects. Social media sites often have a lot of fake registrations, which translate into fake social media profiles. When such sites send out monthly newsletters, for example, to fake emails resulting from fake registrations, they incur substantial bounce back rates. This hurts their email reputation, impacting their overall email delivery rates, which leads to lost opportunities to engage prospects and customers, and ultimately lost revenue.
  4. Put your SEO performance at risk: If your stolen content shows up elsewhere on the web or page loading times slow down significantly, it can have a huge impact on your site's search engine rankings. In some cases, it can get you blacklisted altogether, essentially rendering your brand invisible on the web. When Google changed its algorithm to penalize duplicate content, many publishers lost great portions of their traffic, some up to 70%, resulting in near bankruptcy and severe financial turmoil.

Screen scraping is a serious threat that can affect your company's site, customers, and sales in an instant. Websites depend on unique content, fast response times, and security to maintain brand integrity and a competitive edge. Want to protect yourself and your business? Recognize the threats that screen scraping can impose and establish an action plan to outsmart malicious intruders.

More Stories By Rami Essaid

Rami Essaid is the Co-founder and CEO of Distil Networks, the global leader in bot detection and mitigation. He began his career as the founder and CEO of Chit Chat Communications. After a successful exit, he consulted in mobile development.

With over 11 years in communications, network security, and infrastructure management, Rami advised enterprise companies to help improve scalability and reliability while maintaining a high level of security. He attended North Carolina State University where he studied computer engineering.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time t...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
Disruption, Innovation, Artificial Intelligence and Machine Learning, Leadership and Management hear these words all day every day... lofty goals but how do we make it real? Add to that, that simply put, people don't like change. But what if we could implement and utilize these enterprise tools in a fast and "Non-Disruptive" way, enabling us to glean insights about our business, identify and reduce exposure, risk and liability, and secure business continuity?
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.