Organizations synthesize search, social and sensor data streams into insights that guide smarter actions.
What is Collective Intelligence?
Collective intelligence involves analyzing the collective actions and feedback of people, finding patterns and trends, and sharing it back to aid understanding and guide action. Organizations, artists and changemakers are using collective intelligence to analyze opinions and behaviors, identify patterns and trends, and recommend actions or inspire change.
The rise of collective intelligence can be attributed to three broad trends. First, people are sharing immense amounts of location-based, personalized data online, both implicitly by searching, clicking or buying and explicitly by creating profiles, posting status updates, and checking in to locations and events. Second, people are beginning to use sensor-based devices to track and share real world data about our bodies (quantified self) and our devices, houses, and environments (internet of things). Third, web platforms like Google, Facebook, Twitter and LinkedIn are anonymizing and aggregating this data, mining collective intelligence from it themselves, and also making it available for third-party applications via robust APIs.
Web platforms are using data to create reviews of the most important trends and events in the previous year (Google Zeitgeist(video), 2012 Year on Twitter (video), Facebook Year in Review); add new perspective to important political, sports and entertainment events (Amazon Election Heat Map (screenshot), Twitter Political Index, Facebook America Votes 2012 (video),Twitter Oscars Index); and even predict potential career paths (LinkedIn Career Explorer (video)), the spread of communicable diseases (Google Flu Trends (video)) and traffic conditions (Google Maps Traffic (video)).
News and entertainment media organizations are partnering with internet platforms or using their APIs to use search and social data to analyze public opinions, predict the outcome of important events (USA Today/ Twitter Election Meter, Facebook/ CNN Election Insights, E! Heat Gauge (video)) or showcase upcoming artists (MTV Music Meter (video)).
Several web platforms are finding patterns in user profiles, networks and behaviors to make better product, movie, book, music and restaurant recommendations (Amazon, Netflix, Random House’s Bookscout, Goodreads, Pandora, Bundle).
Entrepreneurs and changemakers are creating niche platforms to mine social and search data to improve traffic conditions (Waze (video)), optimize energy consumption (Opower (video)), and aggregate health data to predict outbreak of diseases (Sickweather (video), Flu Near You (video), HealthMap (video)) and even explore effective cures (Patients Like Me (video), NextBio (video)).
Some collective intelligence initiatives have achieved significant impact and scale. For instance, Waze’s community of 36 million drivers shared 90 million user reports on real time traffic, accidents, hazards, police, gas prices and map issues, and Opower has used data from 80 utility companies to help reduce energy consumption by 2 billion kilowatt hours and save $234 million on electricity bills.
The success of such collective intelligence platforms shows that it’s possible to synthesize search, social, sensor and self-reported data from millions of people into meaningful real-time insights that can guide actions and change behaviors at scale.
How does Collective Intelligence work?
Collective intelligence platforms can be classified across three dimensions: the type of data, the method of data analysis, and the possibilities for participation.
Most collective intelligence platforms use a combination of search, social, sensor and self-reported data. Recommendation engines (Amazon) primarily use on-site browsing, buying and rating data, but are beginning to add social data. Navigation apps (Waze) primarily use automatically updated location data from smartphones, with some self-reported data. Many behavior change applications (Opower) use sensor or transaction data from their own or partner devices, but sometimes add in social data. Many platforms from media and entertainment organizations (MTV Music Meter) use social data sourced from social network APIs. Platforms that use search, social or sensor data typically use the public APIs or take a one-time permission from the user. Platforms that use self-reported data from specialized communities often build their own community platforms and add gamification features to encourage people to share data regularly (Patients Like Me).
Different collective intelligence platforms synthesize data in different ways. Some platforms use algorithms to cluster users and products based on viewing, buying, or rating behaviors and show their recommendations in terms of “users who liked these products also bought these other products” (Amazon) or “users who have similar characteristics also behaved in this way” (Opower). Many platforms plot search, social, sensor and self-reported data on maps, based on keywords or metadata, to find shifts in geographical patterns over time (Sickweather). Other platforms find patterns in social conversations through text and link analysis and connect them back to source or profile data (Facebook/ CNN Election Insights). Some platforms allow users to filter through the data based on time, location, popularity or sentiment to get to more nuanced insights.
Many collective intelligence platforms have overlaps with co-creation communities, social curation platforms, and behavior change games, and offer similar possibilities for participation. Crowdsourcing-driven platforms ask users to create profiles, share answers or ideas, and engage with other users’ content (Patients Like Me). Curation-driven platforms ask users to engage with other users’ content and tag their own content so that it might be included (Sickweather). Behavior change driven platforms compare the users’ behaviors with similar others and incentivize them to change their behavior through gamification features (OPower).
Collective Intelligence for Brands
Many organizations and brands are experimenting with collective intelligence in meaningful ways.
A number of organizations have created ideation platforms to crowdsource insights from employees, partners and customers, and some have even used these insights to create new product and service offerings (Dell Ideastorm, MyStarbucksIdea). Many other organizations have created long-term public or private insights communities to get a more nuanced understanding of consumer behavior, and some have even shared these insights back with the community. For instance, Nestle launched the Happily Healthy Project (video) quiz to help Australians measure their Happily Healthy Quotient and compare it to nation and state averages, filtered by a number of demographic variables like age, gender and income. Other organizations have partnered with independent community platforms to get insights about specialized high-value communities. For instance, several pharmaceutical companies have partnered with Patients Like Me to understand patient needs and drug performance.
Other organizations have taken the social curation route to synthesize and share insights from social conversations around important events. For instance, KPMG built WEF Live to curate the conversations around World Economic Forum and highlight the most important themes from WEF delegates and WEF watchers from around the world. During the 2012 London Olympics, GE tied up with NBC to track Twitter conversations around the games. Almost all major brands are trying to use big data, including search and social data, to understand and engage with consumers. For instance, Vicks combined aggregated search data from Google Flu Trends with demographic data to target moms in high flu zones with ads for their premium Flu Thermometer. @WalmartLabs has analyzed vast amounts of social data (“fast data”) and combined it with public web data and proprietary data to create the social genome, a living database of entities (people, events, topics, products, locations, organizations) and their relationships. It is now building a series of collective intelligence social applications using the social genome, starting with the social gift recommendation app Shopycat (video).
Finally, some organizations are building platforms and products to synthesize and share insights from sensor data, both in the quantified self-space and the internet of things space. For instance, the Nike+ (video) and Adidas MiCoach (video) range of wearable sensor-enabled products enable people to track their workouts, compare themselves with friends and similar others, and even compete with others. Audi partnered with MIT to create a Road Frustration Index (video) based on traffic and weather conditions, reported accidents and driver sentiment from social data.
Collective Intelligence case studies
Throughout the year, we have tracked the conversations around a number of collective intelligence programs and branded programs in our weekly insights reports and quarterly magazines; here are a few highlights.
Collective Intelligence program: MTV Music Metermtvmusicmeter.com
MTV Music Meter is a platform that ranks artists by social popularity and helps people discover new music.
Mashable’s Brenna Ehlrich explains how it works:
“MTV worked with music intelligence company The Echo Nest — which recently partnered with Island Def Jam — to develop an algorithm that combs through blogs, social media, video and more traditional metrics (like radio plays and sales) to determine which bands are getting the most attention each day.”
The algorithm also segregates artists by type, by analyzing where people are talking about them. As Billboard contributor Glenn Peoples noted:
“Where an artist is being talked about influences the Music Meter list where that artist appears. For example, indie rock artist Bon Iver showed up on Music Meter’s mainstream list after winning a Grammy for best new artist.”
Then, MTV Music Meter provides curated content about the artists with 30-second song previews from music partner Rhapsody; articles, bios and photos from Rolling Stone, Pitchfork and the MTV archive; and tour dates from Songkick.
Collective Intelligence program: CNN I’m Votingfacebook.com
CNN partnered with Facebook to create the I’m Voting app to encourage people to discuss political issues and pledge to vote, and to share insights from these conversations in their coverage of the 2012 presidential elections.
In a press release, CNN shared:
“The app will enable people who use Facebook to commit to voting and endorse specific candidates and issues. Commitments to vote will be displayed on people’s Facebook timeline, news feed, and real-time ticker…”
Govind, member of the MSLGROUP Insights Network commented:
“I love the fact that this initiative gets media to partner people in recognizing and thinking of real issues, and lets people see that they are not alone. Also, as this movement grows, political parties get to see that they need to deliver.”
Meghan McCain, daughter of 2008 presidential candidate John McCain, blogged:
“In my opinion, It will be really interesting to see how this Facebook integration influences conversations surrounding the election among young voters, and if it will become a platform for bipartisanship.”
Analysts acknowledge the potential of the I’m Voting app to use metrics gathered from surveys and insights gleaned from conversations, both to predict trends and to better understand the views of the masses. Online radio host Tim Berge noted:
“Currently, about 25-hundred Facebook users have pledged to vote in November. Of the participating users, 53 percent said they are Democrat, while 25 percent are Republican, and 22 percent said they are Independent.
“And, despite what the candidates may be saying recently in their campaign attacks… most Facebook users are listing the economy as the most critical issue.”
Several people have criticized the data collected from the app, pointing out that it does not truly represent the view of Americans but of Facebook and CNN users, the majority of whom are democratic.
Branded program: Vicks Mobile Ad Campaign
In early 2012, Vicks combined three layers of data to reach moms in high flu zones with mobile ads for their premium Behind Ear Thermometer. Moms only received ads if they were within three miles of a retailer selling the thermometer. On clicking the ads, moms were shown a video on the benefits of the thermometer and directed to the nearest retailer selling the thermometer.
First, Vicks used Google Flu trends to find out which areas were experiencing high incidences of flu. Dr. Robert Brecht, a specialist in healthcare marketing, explained how the raw data was validated and made accessible:
“Google’s [Flu] Trends is based on a formula to estimated flu activity based solely on searches. Google was able to do that by correlating flu-related Web searches with actual data from the Center for Disease Control (DCD) (sic). By combining the search keywords with the IP address of searchers which provides searchers’ locations, Google is able to estimate regional flu activity within a day of outbreaks compared to a week or two lag with CDC reports.”
Second, Vicks reached moms and expecting moms through mobile apps such as Pandora, which collect user data including age, gender, marital status and whether they are parents.
Andrew Adam Newman, a journalist at New York Times, noted:
“A mobile campaign by Blue Chip Marketing Worldwide, which is based in Chicago, places the ads for the thermometer within popular apps like Pandora that collect basic details about users, including their sex and whether they are parents, and can pinpoint specific demographics to receive ads.”
Third, Vicks used real-time data from location based mobile advertising network Where to target moms when they were within 3 miles of a closest retail store that stocks the Behind Ear Thermometers.
Michael Johnsen, who covers medical marketing news, wrote:
“The ad targets users who arguably have a higher need for the product — a factor that would presumably increase the purchase intent with that branded call to action.”
Branded program: Nike FuelBandnike.com
In 2012, Nike introduced the Nike FuelBand – a wearable product that measures people’s daily activities and workouts in a virtual metric called NikeFuel. People can view their performance data on their smart phones and the Nike+ website and can compare results and NikeFuel earned with friends and members of the 7 million strong Nike+ community.
Nike targets the “everyday athlete” with the FuelBand. As journalist Jessica Stanley observed
“Just Do It’ is one of the best positioning statements in the world, but customers started to change. Don’t just say it, help us.”
The FuelBand does this by re-positioning everyday activities and chores as a sport, measuring people’s daily activities on web and mobile dashboard, and rewarding them for doing more. The concept of instant feedback immediately appealed to self-trackers, like Jenna Wortham, who commented
“From the moment I wrapped the band around my wrist, I was enamored with the idea of a device that could help me collect data about my habits and behavior, so that I could try to improve them.”
Ever present on the wrists of the owner, the FuelBand displays the amount of NikeFuel earned for the day, and motivates people to meet their daily goal.
MSLGROUP’s Gaurav Mishra talks about how the NikeFuel band has helped him become more active:
“I am a big believer in breaking down a large challenge into small challenges and ticking them off in public. I remember that the year I first bought a Nike+ shoe was the year I ran most regularly. The instant feedback and the sense of progress were almost addictive. Then, I lost the sensor, and lost my stride. I bought a NIkeFuel band a few weeks back and I have seen my activity levels go up significantly since then. Instead of taking a taxi, I walk 3+ km to work, both ways, and I am planning to buy a bike for the weekends. I even created a goal on Nike Plus to finish 2012 active.”
As Alyson Shontell reflected:
“Realizing how inactive I was during certain hours has made me more active in my spare time.”
The Nike FuelBand is the latest addition to Nike’s suite of fitness tracking products, all of which incorporate some elements of games, networks and data to help people achieve their fitness goals.
Future of Collective Intelligence
In the near future, we expect more social platforms like Google, Facebook, Twitter and LinkedIn to synthesize user data to share insights that help users get a new understanding of their own behaviors (how families interact on Facebook). We also expect social platforms to create more data-driven applications that help users make meaningful decisions and change their behaviors (LinkedIn Career Explorer, Google Flu Trends).
We also expect the social data space to explode with new, specialized players. Gnip, Topsy and DataSift (video) aggregate data from multiple social platforms, provide applications to recombine and analyze them, and APIs for third party developers to build applications on them. Other data players are focusing on building social data applications for a specific industry: Dataminr for financial services, BlueFinLabs (video) and Second Sync (video) for Television, Next Big Sound video and The Echo Nest for music, and ReviewPro for hotels. We also expect other data startups to focus on sensor data (SensorCloud (video)) and transaction data (Swipely (video)).
We expect that big corporations will acquire many of these social data startups. For instance, Twitter acquired TV social data startup BlueFin, Intuit acquired personal finance startup Mint, eBay acquired personal recommendation startup Hunch, and Walmart acquired social commerce startup Kosmix (now @WalmartLabs). Other organizations will partner with platforms like Kaggle or DataKind to run crowdsourced data challenges.
We also expect that organizations will shift their focus from collecting and analyzing data to creating applications that use the data to help their users get better understanding and make better decisions. As a result, social curation tools like MassRelevance, insight community tools like CommuniSpace and crowdsourcing tools like BrightIdea will all strengthen their features around visualizing and showcasing data back to the users to guide action.
Finally, we expect that “fast data” will be the next big thing after “big data”, as organizations seek to analyze data streams from social conversations, search queries, sensors, and transactions, find patterns and actionable insights, and share it back with users to help them make better decisions, all in real time.
This is the eighth report from our upcoming People’s Insights Annual Report titled “Now & Next: Future of Engagement,” to be published as an interactive iPad app and a Kindle eBook. The report will highlight the ten most important frontiers that will define the future of engagement for marketers, entrepreneurs and changemakers: Crowdfunding, Behavior Change Games, Collaborative Social Innovation, Grassroots Change Movements, Co-creation Communities, Social Curation, Transmedia Storytelling, Collective Intelligence, Social Live Experiences and Collaborative Consumption.
In each of these reports, we start by describing why they are important, how they work, and how brands might benefit from them; we then examine web platforms and brand programs that point to the future (that is already here); then finish by identifying some of the most important features of that future, with our recommendations on how to benefit from them.
Do subscribe to our email newsletter to receive each report and also an invite to download a free copy of the interactive iPad app.