ControCurator

By Benjamin Timmermans @8w

My research is about

Using crowdsourcing to extract human interpretation

I'm trying to achieve

Understand controversies through human and machine data

My research question is

How to enable the discovery and understanding of controversial issues and events by combining human-machine workflows.

Content

  • The ControCurator Project
  • Measuring Controversy
  • ControCurator Methodology
  • Results

The ControCurator Project

COMMIT: SEALINCMEDIA

Who's Involved


Lora Aroyo: Crowdsourcing (non-expert) annotations from CrowdTruth

Evangelos Kanoulas: Combining active learning with expert and crowd annotations

Johan Oomen: Analysis and mining of broadcast news

Edwin Kuipers: Analysis and mining of user-generated data

Goal of the project

How to enable the discovery and understanding
of controversial issues and events by combining
human-machine active learning workflows.

Vaccination

Pro
  • Vaccines save lives
  • Ingredients are safe
  • Eradicated smallpox and polio
  • Vaccines prevent medical cost
Against
  • Vaccines have side effects
  • Ingredients are harmful
  • Freedom of choice
  • It is unnatural

Climate Change

Pro
  • Humans are responsible
  • Greenhouse effect
  • CO2 rising rate unnatural
  • No change in Sun heat
Against
  • Humans are not responsible
  • Natural fluctuation
  • Not link with CO2 rise
  • Caused by Sun activity

Abortion

Pro
  • Fundamental right
  • Person after birth
  • Incapable of feeling pain
  • Empowers women
Against
  • It is murder
  • Person from conception
  • Fetuses feel pain
  • Killing human being

Wicked problems

Problems that are (near) impossible to solve

Solutions are not right or wrong, but good and bad.

as defined by Rittel and Webber in 1973

Why this is relevant

  • Unbiased source for journalists, government
  • Preventing filter bubbles
  • Identify fake news
  • Understand how issues develop

Measuring Controversy

controversy

noun | con·tro·ver·sy | \ˈkän-trə-ˌvər-sē

An argument that involves many people who strongly disagree about something

Merriam-Webster

Prolonged public disagreement or heated discussion

Oxford English Dictionary

When people often disagree and debate with opposing viewpoints

Mejova, Y., Zhang, A. X., Diakopoulos, N., & Castillo, C. (2014).

The Five Aspects of Controversy


  • Actors
  • Polarity
  • Openness
  • Time-Persistence
  • Emotion

Actors

Many people are involved in discussions on this topic

Abortion
Vaccination
Climate Change

Polarity

There are opposing views on this topic

Openness

There are public discussions on this topic

It is relevant and open to society

Time-Persistence

The discussions on this topic last for a long time

Emotion

The topic of this article triggers strong emotional responses

Your values and beliefs are questioned

Hypothesis

The controversy of an issue can be measured through how it is discussed.

Methodology

Dataset

Discussions on a certain issue

  • Issues: 10k Guardian articles
  • Discussion: 700k related comments and social media

Training and evaluation

Human annotated data

  • Issues: 6k Guardian articles
  • Discussion: 5 representitive comments per article

Crowdsourcing setup

Controversy Classifier

Discussions on a certain issue

  • Overlap of words
  • Sentiment strength
  • Antonym frequency
  • Contradiction probability
  • Duration of discussion
  • Amount of comments

Implementation

Results

Crowdsourcing correlation

Not Controversial
Controversial

Evaluation of the Aspects

Evaluation of the Algorithm


Conclusions

  • Measuring controversy through discussions
  • Five key aspects
  • More precise and real-time

ControCurator

controcurator.org/browse