A Controversy Measurement System for Facebook
Duration
2020.10 - 2020.12
Team
Wonil Choi, Max Hu
Tools
Figma, Sketch, ProtoPie
My Role
UX/UI Design, UX Research
about the project
What is a controversy measurement system for Facebook?
In a society that’s becoming increasingly divided in the age of social media, a Controversy Measurement System for Facebook was designed to alert users of content that is highly polarized, show how polarized a post might be on a scale of 1 to 5, and allow users to filter posts above a certain level of controversy if they wish to do so.
My Contributions
As a UX/UI Designer, I collaborated with my teammate to brainstorm and design the features and UIs for our core experiences. As a User Researcher and Design Researcher, I investigated related academic research that confirmed our assumptions and provided the working principles behind our design. I also conducted user interviews and consolidated research insights.
Project background
The filter bubble effect
Our society is becoming increasingly divided in the age of social media. Highly personalized content feeds create filter bubbles that trap users in a "self-confirming" feedback loop that may radicalize their opinions over time.
The Filter Bubble Effect (Image from Internet)
user research
Understanding the right user group
To find the relevant information, we needed to talk to the right social media users. We sent out a screener survey to identify those who are mainly exposed to new information through videos and articles on social media and are somewhat critical of personalized social media feeds.
Insights from our screener survey
52/55
watch videos on social media
highest of all activities
35/55
read articles on social media
2nd highest of all activities
54/55
are aware that social media is tailored to their preferences
16/55
rated their experience below average
due to various reasons
We selected and interviewed six survey participants who best fit our criteria.
Core Insights
#1
Users often passively consume content casually presented to them.
#2
It often requires energy and stress for users to consciously pay attention when they are mindful of "filter bubbles".
Defining Our Users
Our target users are active on social media, aware of the filter bubble effects, and actively trying to be more critical of what they see.
See the Persona
See the Journey Map
Defining Our Goals
Our users are aware of the filter bubble effect but still struggle to stay mindful of potential biases,
How might we create a media consumption experience that promotes the awareness of the context of information and lessen the filtering burden?
Further Secondary Research
But how can we even measure controversy?
Before we create any designs that address controversies, we must learn how to understand controversies on social media. I researched related academic papers and discovered a study on how to quantify social media controversies.
Quantifying Controversy on Social Media, Kiran Garimella, Gianmarco de Francisci Morales, Aristides Gionis, Michael Mathioudakis, 2017
Quantifying Controversy on Social Media is a research study from 2017 that analyzed data on user interactions with selected controversial topics (listed in the illustration above). It used actions such as "like", "comment", or "repost" as forms of approval of the original message and thus establishing a connection between the two posts.
If you represent posts as dots, then link and pull two dots closer when you establish a connection, you can generate a graph that clearly shows if a topic is controversial. Like the example graphs below, each graph represents a controversial topic. However, the left two graphs show more polarizing discussions as there are clearly two distinctive sides. This means people on either side of the discussion are mostly communicating between themselves and reaffirming each other's opinions instead of listening to those on the other side.
This study also provided methods to quantify how polarized a certain post might be with Random Walk Scores. Random Walk Scores capture how many connections need to be made on the graph to connect a post to another post on the other side of the discussion.
Conceptualizing
Most of our interviewees use Facebook regularly, so we decided to design for Facebook. Since Facebook also has the most diverse media types, it's easier to apply our designs to other social media platforms in the future.
After our brainstorming sessions, we decided to develop "controversy flags" and "controversy filter" as our core features.
Initial Designs
Controversy Flag
In Post Notification
VS.
VS.
Information Screen Designs
VS.
VS.
Controversy Filter
Entry Point
Setting the Target Score
User Testing
Validating our initial designs
Social media controversy is a delicate topic to tackle. As our society becomes increasingly divided, it's getting increasingly difficult for people to see things from different perspectives. It can be challenging to convey to users that the contents they are viewing are one-sided. That's why we decided to conduct two rounds of user testing to validate our assumptions and test the understanding of our designs.
Thinking Out Loud
Screen Sharing
The goals of our testing sessions are:
determining if users find the language understandable
determining if users can distinguish the difference between Neutrality Score for a topic and Neutrality Score for a post
checking general usability
We gave our test participants two tasks:
to identify the post that Facebook has flagged in their feed
to find and set up their controversy filter
Testing Insights from Round 1
#1
The language we considered "neutral" was too vague for users. We need to update the copies to be more direct and clear
#2
Showing a Neutrality score can overcomplicate the concept and the message we wish to convey. We need to instead focus on the visualization of "how controversial a post is"
For the second round of user testing, we asked our test participants to fill out the SUS (System Usability Scale) that evaluates the readiness of our design.
Testing Insights from Round 2
Our improved design successfully resolved the major issues we identified in the first round of user testing, however:
#1
We need to change the "dashboard visual" of our alert info page into a linear gauge
#2
We need to make "custom filter setup" a requirement upon users' first visit to controversy filter settings
Results from our SUS (System Usability Scale)
This is some text inside of a The average SUS score from our tests is 73. In reference, a SUS score above 68 would be considered above average.
Final Design
Controversy Flag
Care-free browsing, we keep you aware
With our design, Facebook now flags posts that might be controversial based on collected response data. Users can dismiss the alerts or click on them to read more about how Facebook flags posts.
Controversy Filter
Never stress about hidden biases
The controversy filter feature allows Facebook users to filter posts that might be controversial based on their needs. The system helps users choose their desired feed controversy scale by asking users to identify content that might be controversial for them. First-time users are required to go through this process to set up their controversy filter.
next steps
More user testing & Further research
Although we have conducted two rounds of user testing, there is still much room for improvement. An appropriate next step for us would be to continue user testing and improve our SUS score. We also wish to implement our design to the web version of Facebook and other social media platforms.
Many of the design decisions we made were based on insights gathered from people who are already aware of the negative effects of filter bubbles. In the future, we're curious to interview people who are unaware of this issue and whose social media content is biased.
Many of the design decisions we made were based on insights gathered from people who are already aware of the negative effects of filter bubbles. In the future, we're curious to interview people who are unaware of this issue and whose social media content is biased.
Takeaways
The importance of user testing & Working with academic research
This project reflected the importance of desktop and academic research. The research paper on quantifying social media controversies helped to define our concepts and made our features possible. Although this project was my first time basing my designs on academic research, I'm sure it will not be my last.
This project also greatly benefited from user testing. User testing helped us iterate our visual designs and improve our UX writing. It also gave me experience working with SUS scores which helped us evaluate our usability and determine the readiness of our product. I wish to apply this method to my future projects and user testing sessions.
This project also greatly benefited from user testing. User testing helped us iterate our visual designs and improve our UX writing. It also gave me experience working with SUS scores which helped us evaluate our usability and determine the readiness of our product. I wish to apply this method to my future projects and user testing sessions.
This project was also inspired by the Netflix documentary The Social Dilemma. If you wish to learn more about the filter bubble effects of social media, this documentary is a great way to start.
Thank you for reading!
