Welcome!

Cloud Security Authors: Mehdi Daoudi, John Walsh, Liz McMillan, Xenia von Wedel, Elizabeth White

Related Topics: @DXWorldExpo, @CloudExpo, Cloud Security

@DXWorldExpo: Blog Feed Post

Big Data and the Personalization Challenge: The Murky Middle By @Schmarzo | @BigDataExpo #BigData

Personalized recommendations are a real dilemma in the world of big data

What if you knew, through legal means, something about someone where you could intervene to deliver advice to help them perform better or stay out of danger? Should you act on that?

I was recently delivering a lecture at a major university in Texas and one of the participants posed a couple of very thought-provoking scenarios (stay with me as these are sort of long, abstract scenarios, but hey, what better way to provoke an interesting conversation):

What if by virtue of monitoring a student’s social media habits (with the student opting to share their social media activities), the university could 1) flag behaviors that predict student classroom performance problems and 2) could deliver recommendations to the student, faculty and advisors for improving the student’s social, classroom and study habits and increase their probability of graduating on time and with higher grades?

What if by monitoring the student’s study habits and extra-curricular activities (e.g., how often they went to the library and/or the lab and for how long, class attendance, attendance at campus events, extracurricular activities, etc.), the university could generate recommendations to the student, faculty and advisors for improving the student’s social, classroom and study habits and increase their probability of graduating on time and with higher grades?

These scenarios raise all sorts of ethical questions, such as:

  • Would not this be in the best interests of the students to perform this analysis in order to help the students achieve their best outcomes?
  • Would students want this level of analysis as students come to college to earn a degree and achieve a level of classroom performance that will help them get better jobs and earn more money?
  • Is one type of monitoring more intrusive and a potential violation of the student’s privacy (even if they have opted in to share their social media activities)?
  • Where does one draw the line between what is helping the student to increase the probability of their college and post-college success, and what is being a “creeper”?

One of the workshop participants called this the “murky middle” where noble and forthright efforts to help students to be more successful may cross the personal privacy line.

Primary Research (My Kids as Guinea Pigs)
I’ve got to be honest. I don’t have an answer. And I’m really torn by the dilemma of wanting the best performance for students (and my kids), but doing it in such in a way that doesn’t violate their personal privacies or come across as being a “creepy.”

So I asked my children what they thought of these scenarios. Each of my children has a unique perspective:

  • Alec has already graduated from college and has been in the workforce for a couple of years now.
  • Max is currently in grad school and will be living these two scenarios over the next few years.
  • Amelia is evaluating colleges and has to determine if she is comfortable with these levels of monitoring.

Here are their answers:

Scenario #1: What if by virtue of monitoring a student’s social media habits (with the student opting to share their social media activities), the university could 1) flag behaviors that predict student classroom performance problems and 2) could deliver recommendations to the student, faculty and advisors for improving the student’s social, classroom and study habits and increase their probability of graduating on time and with higher grades?

Amelia (evaluating colleges): If I gave my social media information to my future college, I would expect them to skim over my social media accounts. I think in the beginning it could be a great benefit, but for an entire year I do not think it would be beneficial. I believe this because everyone wants to be successful in school and life, but a constant watch over the accounts may get stressful and not necessary. It can often create frustration and lead to hidden activities that could result in something more detrimental.

Max (still in college): If there were a human component to it, I would feel like the college is investing in me. However, if it were electronic I would feel violated. For example, if my college had access to all of this information and a counselor gave me a call, I would feel like someone cares. But if I got an email or text message on my phone offering me suggestions I would feel violated and “big brothered.”

Alec (out of college): I don’t like the idea of anyone – especially a school or other authority type institution – examining my social media to identify trends in terms of how I “tick.” There is something decidedly unsettling about any entity, for whatever purpose, altruistic or not, analyzing my social relations and providing advice on how to improve myself. Ditch certain friends? Don’t go to certain parties? These types of recommendations would only lead to backlash to the system as a whole.

Scenario #2: What if by monitoring the student’s study habits and extra-curricular activities (e.g., how often they went to the library and/or the lab and for how long, class attendance, attendance at campus events, extracurricular activities, etc.) to generate the recommendations to the student, faculty and advisors for improving the student’s social, classroom and study habits that will increase their probability of graduating on time and with higher grades?

Amelia (evaluating colleges): Similar to the first scenario, this scenario would only be helpful in the beginning. The main purpose to going to college is to receive an education on my own. Constantly being tracked and told where to go would help you in college, but would not set you up for the real world. In the real world, you are not going to have someone tell you to go study more or go to classes more. If you miss a class or don’t study, you must face the consequence. Overall, growing up and making mistakes is a part of life and to have that stripped at such a young age may lead to a hard reality check.

Max (still in college): At no point in time would I be okay with my college knowing my location. I feel that crosses the line of caring. I wouldn’t give my college access to something I wouldn’t give my parents access to. To be honest dad, I wouldn’t feel comfortable if you knew how many classes I missed and whether or not I was at the library. I am sure the government knows, but sometimes with electronics, ignorance is bliss.

Alec (out of college): This option is more feasible in my opinion. I would be much more comfortable with an educational institution analyzing my educational decisions. Analysis of my study habits, correlations in terms of grades and time spent in the library or class attendance would prove invaluable in making a logical case to present to the student to convince them that their grades are affected by these factors.

So none of my kids would be happy with this level of monitoring and analysis, with some specific observations:

  • While Amelia (evaluating colleges) felt that some level of monitoring and analysis might be useful at first, it would eventually impede her personal development. Students are going to college to learn to live on their own; monitoring their behaviors and “holding their hands” doesn’t allow them to grow.
  • Alec (already graduated) seems comfortable with the school using some of the classroom data to make performance improvement recommendations, but neither Max (still in school) nor Amelia (evaluating schools) feel comfortable with this.

However, if colleges were to do this level of monitoring and analysis, then the advice or recommendations must have a human component. By having a counselor get involved, you give the data a human face.

I think this last point is critical – give the data a human face. As I’ve discussed many times, the real impact of big data (and data science) is helping the humans in the process by providing recommendations to the humans (teachers, physicians, parole officers, technicians, counselors, coaches, therapists) to make them more effective at the point of customer engagement.

Summary
Let me give you one more scenario to chew on: What if by virtue of analyzing the student’s social media and classroom data, the college is able to make the following probabilistic determinations:

  • There is 65% chance of the student dropping out or flunking out
  • There is a 33% chance that the student is having substance abuse problems
  • There is a 5% chance that the student is going to attempt suicide

What should the college do?

Personalized recommendations are a real dilemma in the world of big data – how much monitoring and analysis is “okay” before crossing over that fine line. And that’s the heart of the challenge: it isn’t a fine line; it’s the murky middle.

Big Data and the Personalization Challenge: The Murky Middle

Read the original blog entry...

More Stories By William Schmarzo

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business”, is responsible for setting the strategy and defining the Big Data service line offerings and capabilities for the EMC Global Services organization. As part of Bill’s CTO charter, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He’s written several white papers, avid blogger and is a frequent speaker on the use of Big Data and advanced analytics to power organization’s key business initiatives. He also teaches the “Big Data MBA” at the University of San Francisco School of Management.

Bill has nearly three decades of experience in data warehousing, BI and analytics. Bill authored EMC’s Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements, and co-authored with Ralph Kimball a series of articles on analytic applications. Bill has served on The Data Warehouse Institute’s faculty as the head of the analytic applications curriculum.

Previously, Bill was the Vice President of Advertiser Analytics at Yahoo and the Vice President of Analytic Applications at Business Objects.

@ThingsExpo Stories
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
An increasing number of companies are creating products that combine data with analytical capabilities. Running interactive queries on Big Data requires complex architectures to store and query data effectively, typically involving data streams, an choosing efficient file format/database and multiple independent systems that are tied together through custom-engineered pipelines. In his session at @BigDataExpo at @ThingsExpo, Tomer Levi, a senior software engineer at Intel’s Advanced Analytics gr...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things’). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing? IoT is not about the devices, it’s about the data consumed and generated. The devices are tools, mechanisms, conduits. In his session at Internet of Things at Cloud Expo | DXWor...
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution. In his session at @ThingsExpo, Akvelon expert and IoT industry leader Sergey Grebnov provided an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.