A COLLECTION OF RECIPES

FOR THOSE WHO LOVE TO COOK

WITH DIGITAL METHODS

COMPILED BY
Liliana Bounegru
Jonathan Gray
Tommaso Venturini
Michele Mauri

1
A FIELD GUIDE TO FAKE NEWS
Compiled by Liliana Bounegru, Jonathan Gray, Tommaso Venturini
and Michele Mauri

This guide explores the use of digital methods to trace the

production, circulation and reception of fake news online. It is a
project of the Public Data Lab with support from First Draft.
The Public Data Lab (publicdatalab.org) is an interdisciplinary
network seeking to facilitate research, democratic engagement and
public debate around the future of the data society.

First Draft (firstdraftnews.com) is dedicated to improving skills and

standards in the reporting and sharing of information that emerges
online.

This is a SAMPLE of A Field Guide

to Fake News released to coincide
with the International Journalism
Festival in Perugia in April 2017.
If youd like to be notified when
the full version is released, please
visit: fakenews.publicdatalab.org
For further information you can
email contact@publicdatalab.org

2017 Public Data Lab.

The guide is released under the Creative Commons Attribution
License (creativecommons.org/licenses/by/3.0/). This means you
can freely share, adapt and draw on this work as long as you give
credit, as per the terms of the license.

If you reproduce or draw on material from this guide, wed be

grateful if you could credit and link back as per the following
statement: A COLLECTION OF RECIPES
This article draws on A Field Guide to Fake News, a
collaboration of the Public Data Lab and First Draft. For further
FOR THOSE WHO LOVE TO COOK
details see: http://fakenews.publicdatalab.org WITH DIGITAL METHODS
The copyright for the research that this guide draws upon remains
with its respective contributors.

This project would not have been possible without the contributions
of researchers from the following institutions:
Centre for Journalism Studies, University of Ghent
Citizen Data Lab, Amsterdam University of Applied Sciences
Density Design, Politecnico di Milano
Digital Methods Initiative, University of Amsterdam
Institut National de Recherche en Informatique et en
Automatique (INRIA)
Institute for Policy Research, University of Bath
Laboratoire dtude des Sciences et des Techniques (STS-Lab),
Universit de Lausanne
Laboratoire Interdisciplinaire Sciences Innovations Socits
(LISIS), Universit Paris-Est Marne-la-Valle
Mdialab, Sciences Po, Paris
Techno-Anthropology Lab, Aalborg University Copenhagen
University of Siegen

Published by Public Data Lab.

Sample released in April 2017.
 CONTENTS

Contributors
Co-investigators
Liliana Bounegru, University of Ghent, University of Groningen 7 Introduction
Jonathan Gray, University of Bath
Tommaso Venturini, Univ Lyon, Inria, ENS de Lyon, CNRS, UCB
Michele Mauri, Density Design, Politecnico di Milano

DensityDesign Researchers CHAPTER 1

Agata Brilli, Politecnico di Milano
ngeles Briones, Politecnico di Milano
Carlo De Gaetano, Politecnico di Milano
15 MAPPING FAKE NEWS HOTSPOTS ON
Gabriele Colombo, Politecnico di Milano
Mariasilvia Poltronieri, Politecnico di Milano
FACEBOOK
Michele Invernizzi, Politecnico di Milano
Michele Mauri, Politecnico di Milano
18 1. What publics does fake news animate on facebook?
Paolo Ciuccarelli, Politecnico di Milano
Tommaso Elli, Politecnico di Milano
38 2. How may the trajectory of a fake news story be
Facilitators
traced on facebook?
Alex Gekker, University of Amsterdam
Anders Munk, Aalborg University
50 3. Do fact-checking initiatives reach the publics of
Bilel Benbouzid, University of Paris-Est Marne-la-Valle
Erik Borra, University of Amsterdam
fake news on facebook?
Esther Weltevrede, University of Amsterdam
Jonathan Gray, University of Bath
Jorn Preuss, University of Siegen
Liliana Bounegru, University of Ghent, University of Groningen
Marc Tuters, University of Amsterdam CHAPTER 2
Mathieu Jacomy, Sciences Po
Natalia Snchez, University of Amsterdam 59 TRACING THE CIRCULATION OF FAKE NEWS
Nicolas Baya-Laffite, University of Lausanne
Richard Rogers, University of Amsterdam ON THE WEB
Sabine Niederer, Hogeschool van Amsterdam
Tommaso Venturini, Univ Lyon, Inria, ENS de Lyon, CNRS, UCB 62 1. Where do fake news originate? By what sites are
Analysts they first retransmitted?
Agata Troost, University of Utrecht
Anders Grundtvig, Aalborg University 76 2. Which are the most visible sources related to a fake
Anna Keuchenius, University of Amsterdam
Antonio Martella, DataActive story? When and by whom are they mentioned?
Asbjrn Fleinert Mathiasen, Aalborg University
Asger Gehrt Olesen, Aalborg University
Carlo Santagiustina, University of Amsterdam
Charlotte Leclercq, University of Amsterdam
Daniel Bach, Aalborg University CHAPTER 3
Daniela Demarchi, University of Amsterdam
Ecesu Erol, University of Amsterdam 93 USING TRACKER SIGNATURES TO MAP THE
Emil Grosby Jrgensen, Aalborg University
Emillie Victoria de Keulenaar, University of Amsterdam TECHNO-COMMERCIAL UNDERPINNINGS OF
Joep Voorn, University of Amsterdam
Jorn Preuss, University of Siegen FAKE NEWS SITES
Kaspar Beelen, University of Amsterdam
Katerina Gladkova, University of Amsterdam
This is a sample of
95 1. Do fake news sites use different kinds of trackers
Lieke Kersten, University of Amsterdam
Lisanne Blomberg, University of Amsterdam A Field Guide to from mainstream media sites?
Manon van Hoek, University of Amsterdam Fake News, which
Maria Hayat, University of Amsterdam
will contain further
101 2. How can fake news and mainstream media sites be
Marlene Scherf, University of Amsterdam
Michel Blonk, University of Amsterdam chapters on how to profiled based on their tracker usage?
Mintsje de Witte, University of Amsterdam map and investigate
Mischa Benjamin Szpirt, Aalborg University
fake news, online
107 3. How do tracker ecologies on fake news sites change
Pieter Vliegenthart, University of Amsterdam
Rina Tsubaki, European Journalism Centre misinformation and over time?
Ronja Ingeborg Lofstad, Aalborg University other associated
Sal Hagen, University of Amsterdam
phenomena
113 4. Which other websites share the same tracker IDs as
Stefani Mans, University of Amsterdam
Stefanie Voortman, University of Amsterdam including recipes on fake news sites?
Talia Castellanos Usigli, University of Amsterdam memes, bots, trolls,
Zo Versteegen, University of Amsterdam
propaganda and fact-
119 5. Do trackers associated with hyper-partisan and
Research and Editorial Assistant checking initiatives. misinformation sites vary across language spheres?
Daniela Demarchi, University of Amsterdam
A FIELD GUIDE TO
FAKE NEWS
[1] See, for example Robert
Darnton, The True History
of Fake News, The New
York Review of Books,
February 2017, available at:
http://www.nybooks.com/
daily/2017/02/13/the-true-
history-of-fake-news/
[2] See, for example, Sky
Views: Facebook's fake
news threatens democ-
racy, Sky News: http://
news.sky.com/story/sky-
views-democracy-burns-
as-facebook-lets-fake-
news-thrive-10652711
INTRODUCTION
[3] See, John Naughton, offending content;[3] and a host of new startups and technical
What is fake news? And what can be done about it? Facebook and Twitter
fixes from authenticated content to automated fact-
could pay the price
Depending on who you ask, fake news is said to represent for hate speech, The checking projects.[4]
a step-change in information warfare; an emerging form of Guardian: https://www.
theguardian.com/com-
cynical profiteering; an engine for energising alt-right and mentisfree/2017/mar/19/ Across these different kinds of responses, observers
other digitally mediated grassroots political mobilisations john-naughton-germany-
agree that fake news cannot be straightforwardly defined.
fine-social-media-sites-
around the world; a partisan battle cry for a new liberal facebook-twitter-hate- Fake news comes in many different shades. This need
ministry of truth; an unwanted byproduct of the online speech
not be taken as proof of the futility of investigating this
platforms which organise our digital societies; or a canary [4] See, for example, Full Fact, phenomenon. On the contrary: accepting that there is no
call signalling a collapse of consensus around established The State of Automated
easy way to demarcate between fake from non-fake
Factchecking, https://
institutions and processes of knowledge production, fullfact.org/automated across all cases opens interesting research opportunities. It
heralding a new post-truth era in politics and public life. and http://www.512tech.
is precisely because its forms and contents are designed to
com/technology/austin-
startup-wants-end-fake- mimic those of mainstream media and precisely because
According to some commentators fake news is just old wine news-and-fake-every-
it travels through similar circuits that fake news offers us
thing-else-the-internet/
in new bottles and similar misinformation phenomena EcchWFgrl4PQmjPvm- the occasion to study not just the strategies and formats of
have existed for at least as long as the printing press and kzycJ/
fakeness, but the politics and composition of the media and
other communication technologies through which they information environments of the digital age more generally.
circulate.[1] Others suggest that new online platforms
accelerate and supercharge their circulation in a way This guide aims to enrich public debate and catalyse
which introduces hitherto unprecedented challenges and collective inquiry around this rapidly evolving and highly
dynamics.[2] Proposed responses include new media literacy, contested issue by suggesting different ways in which it
educational and fact-checking initiatives; new laws, policies can be empirically studied, mapped and investigated online.
and fines for technology companies who fail to remove
INTRODUCTION 7
 Ultimately our hope is not just to provide better accounts of
the issue of fake news and phenomena associated with it,
but also to contribute to more substantive forms of public
engagement around it. We hope this guide will contribute
to facilitating broader public debate and involvement
around processes of reshaping platforms and policies, laws
and infrastructures, technologies and standards that are
implicated in the circulation of fake news phenomenon.
This includes remaining attentive to possible unintended
consequences of these different responses, as well as other
interests and concerns.
The guide explores the notion that fake news is not just
another type of content that circulates online, but that it is
precisely the character of this online circulation and reception
that makes something into fake news. In this sense fake
news may be considered not just in terms of the form or
content of the message, but also in terms of the mediating
infrastructures, platforms as and participatory cultures which
facilitate its circulation. In this sense, the significance of fake
news cannot be fully understood apart from its circulation online.
It is the register of this circulation that also enables us to
trace how material that starts its life as niche satire can be
repackaged as hyper-partisan clickbait to generate advertising
money and then continue life as an illustration of dangerous
political misinformation.
As a consequence this field guide ecourages a shift from
focusing on the formal content of fake news in isolation to
understanding the contexts in which it circulates online.
This shift points to the limits of a deficit model approach
which might imply that fake news thrives only because
of a deficit of factual information. In the guide we suggest
new ways of mapping and responding to fake news beyond
identifying and fact-checking suspect claims including
thicker accounts of circulation as a way to develop a richer
understanding of how fake news moves and mobilises
people, more nuanced accounts of fakeness and responses
which are better attuned to the phenomenon.
8 A FIELD GUIDE TO FAKE NEWS
[5] For a recent overview see
Shannon Matterns Cloud
and Field, Places Journal:
https://placesjournal.org/
article/cloud-and-field/
watercolour portraits, anatomical illustrations, cartographic
charts, satellite imagery or infrastructural diagrams of the
Instead we illustrate a range of methods and procedures
which readers may use in order to explore fake news
INTRODUCTION
While online and platform metrics often serve to take
measure of engagement by means of what Theodore Porter
calls thin descriptions i.e. aggregated quantities such
as total likes, shares, posts we suggest different ways of
exploring how different publics engage with and ascribe
meaning to fake news and how this moves and mobilises
different actors in the process. In doing so while we start
our inquiry with fake news, we end up surfacing a wide
range of grassroots political, media and participatory
cultures online and the social and political issues around
which they assemble. Some of these may challenge and
prompt a rethinking of our ideas of the forms and formats of
grassroots political action online.
We have adopted the metaphor of the field guide in the
tradition of a number of recent guides which transpose the
language and imagery of mapping places, flora and fauna
onto the cloud, digital infrastructures and life online.[5]
However this metaphor stands in need of some qualification.
Many classical natural historical field guides aspire to
provide systematic taxonomies of natural phenomena by
taking them out of their contexts in order to abstract and
compare their features. By contrast with our guide we aim
to do precisely the opposite not to decontextualise, but to
recontextualise fake news phenomena by suggesting ways
to follow them in the wild: as they travel across the web,
search engines, digital platforms, fact-checking initiatives
and news websites.
We do not set out to provide a definitive single set of
phenomenon in question or even lists of characteristic
features which may be used for the purposes of identification.
phenomena online for themselves. As part of this process we
wish to extend the repertoire of mapping practices which are
publicly available to make sense of fake news online and in
9

[6] See Philip E. Agre, To-
ward a Critical Technical
Practice: Lessons Learned
in Trying to Reform AI, in
Geof Bowker, Les Gasser,
Leigh Star, and Bill Turner
(eds), Bridging the Great
Divide: Social Science,
Technical Systems, and Co-
operative Work, Mahwah,
NJ: Erlbaum, 1997. Avail-
able at: http://polaris.gseis.
ucla.edu/pagre/critical.
html
[7] Shannon Mattern, Cloud
and Field, Places Journal:
https://placesjournal.org/
article/cloud-and-field/
[8] See, for example, the work
of Pamela Smith and her
colleagues on the Making
and Knowing project
at Columbia University:
http://recipes.hypotheses.
org/7430 and http://www.
makingandknowing.org/
10
this sense the graphics that we provide can be understood as
temporary placeholders to encourage further exploration.
We also draw attention to different ways of examining how
things are categorised and labelled as fake news and the politics
of these practices of classification. In this sense we hope to
cultivate what has been called critical technical practice[6]
which in this case would include reflection on the use of
digital methods and digital data and how these not only serve
to designate phenomena which can be straightforwardly and
independently picked out, but how these very methods may
also be involved in the process of articulating what fake news
is. As Shannon Mattern puts it, in undertaking to investigate
fake news online we should be aware of the shadows cast
by our presence as explorers in the field.[7] And rather than
producing maps for the sake of producing maps, we should
consider what maps do, who and what they are for and
the effects that they produce as social, cultural and political
devices.
Insofar as we focus on providing procedures for inquiry
rather than pictures of the phenomena, this guide may
also be considered a kind of recipe book. Recent research
suggests that there is an interesting relation between the
documentation of recipes and the emergence of procedural
knowledge in the early modern period such that practices
of writing down processes for cooking and craft are
entangled with the history of the emergence of scientific
method.[8] Over the past few decades the metaphors of the
recipe and the cookbook have also become popular in
relation to software programming. In our guide, we illustrate
different approaches to mapping and investigating fake news
online through a series of methodological recipes. As with
many cookery books, our aim is not just to support readers in
following the specific recipes that we present, but rather to
use these recipes to illustrate a certain approach to cooking
with the hope that readers are inspired to adapt, modify and
venture beyond them. We also include a number of serving
suggestions about how they may be put to work
A FIELD GUIDE TO FAKE NEWS
We hope that the recipes in this guide will enrich
investigations of fake news in a way which has affinities with
a common narrative approach in mystery fiction namely the
scenario that in pursuit of solving an apparently simple crime,
the plot thickens, the cast grows, the questions multiply and
there are unexpected twists or changes of perspective. By
following the production, circulation and responses to fake
news online we may end up being drawn into things that
we do not set out to investigate: whether the media strategies
of fake news publishers, propagandists, trolls or bots; the
commercial and technical architectures of online content;
the politics and dynamics of viral content; and how social
life adapts, evolves and innovates in response to some of the
worlds biggest online platforms and websites. In this sense,
it will be clear that fake news involves more than a few rogue
producers or state conspiracies and raises important and
difficult questions about the role of digital technologies in
society and how we mutually shape and are shaped by them.
In Edgar Allan Poes classic mystery story The Purloined
Letter, the prefect of police G and his colleagues
search for a letter said to contain scandalous information
behind wallpaper, under carpets, in the legs of furniture
and in cushions, only to eventually find the letter hiding
in plain sight. In a similar vein, we may consider the
[9] See, Noortje Marres, The
Redistribution of Methods:
algorithmically mediated circulation of fake news on digital
On Intervention in Digital platforms in terms of what Noortje Marres and Carolin
Social Research, Broadly
Conceived, The Socio-
Gerlitz characterise as distributed accomplishment or what
logical Review, June 2012; Mike Ananny and Kate Crawford describe as relational
and Mike Ananny and Kate
Crawford, Seeing Without
achievement.[9] This entails a shift from seeing in systems
Knowing: Limitations of the as a kind of looking under the hood, to seeing across
Transparency Ideal and its
Application to Algorithmic
a diverse range elements which are implicated in the
Accountability, New Media patterning of collective life online.
and Society, December
2016.
Many of the researchers who have contributed to the guide
[10] M. S. Abildgaard, A.
Birkbak, L. Bounegru, J.
share a background in a field called Science and Technology
Gray, M. Jacomy, A. K. Studies (STS). Some of the lines of inquiry pursued in the
Madsen, A. K. Munk (forth-
coming) Fake News: Seven
guide are informed by a forthcoming paper exploring what
Lines of Inquiry. STS can bring to the study of fake news.[10] The recipes are
INTRODUCTION 11
 also informed by a digital methods research approach that issues and controversies as they unfold on digital media, as
has developed through an engagement with this field and well as of the mediating capacities of platforms, algorithms
which many of us have contributed to through our teaching and infrastructures which shape life online.
[11] See, for example, Richard and research.[11] We also draw upon our fields interest
Rogers, Digital Methods,
Cambridge, MA: MIT Press,
in public engagement and participation around digital The data sprint format has also helped us to catalyse new
2013. technologies and data infrastructures.[12] As such our focus is experimentation and empirical work in a comparatively
[12] See, for example, Noortje
less on advancing particular legal or technical fixes, than on short period of time a distinct advantage given the
Marres, Material Partic- facilitating processes of public engagement and democratic pace of developments around fake news. For this we
ipation: Technology, the
Environment and Everyday
deliberation including provoking curiosity about different are immensely grateful to researchers, graduates and
Publics, London: Palgrave ways of seeing the issue and imagination about the different students at Density Design (Politecnico di Milano, Italy),
Macmillan, 2012.
ways in which we might respond. the Digital Methods Initiative (University of Amsterdam,
Netherlands), the European Journalism Centre, the
The material in this guide has been produced through Laboratoire Interdisciplinaire Sciences Innovations Socits
[13] See Tommaso Venturini,
Anders Munk and Axel
a series of data sprints and research workshops in (Universit Paris-Est, France), the mdialab (Sciences Po,
Meunier, Data-Sprint: A Amsterdam, Copenhagen and Milan, hosted by members Paris, France), the Media of Cooperation research group
Public Approach to Digital
Research in C. Lury, P.
of the Public Data Lab. The data sprint is a short form (University of Siegen, Germany), the STS-Lab (University
Clough, M. Michael, R. working format that has emerged at the intersection of Lausanne, Switzerland), the Techno-Anthropology Lab
Fensham, S. Lammes, A.
Last, & E. Uprichard (Eds.)
between Science and Technology Studies and New Media (Aalborg University Copenhagen, Denmark) and the Utrecht
Interdisciplinary Research Studies, drawing on approaches associated with open-source University without whose energy, creativity and dedication
Methods, London: Rout-
ledge, 2017.
software development, open data and civic hacking in order this project would not have been possible.
to convene a range of actors to collaborate around the co-
[14] Jonathan Gray, Liliana
Bounegru and Lucy Cham-
production of data and research projects including between
bers (Eds.) The Data Jour- fields of practice with different outlooks.[13]
nalism Handbook, Sebas-
topol, CA: OReilly Media,
Jonathan Gray, Liliana Bounegru, Tommaso Venturini
2012, available at: http:// Two of us have a background in data journalism, having London, March 2017
datajournalismhandbook.
org/
co-edited The Data Journalism Handbook and undertaken
various initiatives in this field.[14] This guide builds on a
[15] Upon being invited to
become Director of the
long-standing interest in supporting productive encounters
Argentine National Li- between data journalists and digital researchers. While fake
brary at a moment which
coincided with the dete-
news seems like a remarkably ripe area for experimentation
rioration of his eyesight, between these two fields, just as the writer Jorge Luis Borges
Borges famously wrote: No
one should read self-pity
lamented being granted books and night at one touch,[15]
or reproach / into this it is not without a sense of irony that we note that as public
statement of the majesty
/ of God; who with such
attention around this issue grows, fake news websites are
splendid irony / Granted beginning to vanish leading to proposals for a fake news
me books and night at one
touch. See J. L. Borges,
archive amongst our contributing researchers. Happily the
Poem of the Gifts in approaches and analytical techniques in this guide may be
Selected Poems: Volume
2, London: Penguin Books,
used to inform collaborations between data journalists and
2000, p. 95. digital researchers around the study of other contentious
12 A FIELD GUIDE TO FAKE NEWS INTRODUCTION 13
 Chapter #1

MAPPING
FAKE NEWS HOTSPOTS
ON FACEBOOK

What publics does fake news

animate on Facebook?

How may the trajectory of a

fake news story be traced on
Facebook?

Do fact-checking initiatives
reach the publics of fake news
on Facebook?

14 A FIELD GUIDE TO FAKE NEWS

Introduction - This section provides a set generating engagement . This
[1] See, John Herrman, In-
side Facebooks (Totally
[1]

of recipes for tracing the circulation of fake fake news dissemination machine
Insane, Unintentionally
Gigantic, Hyperpartisan)
Political-Media Machine,

news on Facebook. The focus is on circulation and responses to it, is what the
Agust 2016, The New
York Times: https://www.
nytimes.com/2016/08/28/
because false and misleading knowledge claims recipes in this section enable to
magazine/inside-face-
books-totally-insane-un-
intentionally-gigantic-hy-
are not born fake news. To become fake explore. The first two recipes
perpartisan-political-me-
dia-machine.html

news they need to mobilise a large number of
publics including witnesses, allies, likes and
shares, as well as opponents to contest, flag
and debunk them. Facebooks architecture
poses challenges to the study of circulation
of content due to the nature of its access and
permissions system. Hence we focus on tracing
the publics of fake news through its most
publicly accessible entities: pages and groups,
which may be considered to constitute already
assembled publics.
Around the 2016 US presidential elections
commentators have noted the emergence of
a Facebook-native, hyper-partisan political
media machine that was highly effective in
gathering large numbers of followers and
16 A FIELD GUIDE TO FAKE NEWS
ON FACEBOOK 17
focus on mapping the publics
that are energised by fake news on Facebook, as
well as the trajectories through which fake news
stories travel on Facebook. The third recipe
provides an approach to address the effectiveness
of fact-checking initiatives in reaching the
publics of fake news on Facebook. Through these
recipes we aim to gesture towards different ways
of providing thicker accounts of circulation and
engagement around fake news on social media
beyond the thin descriptions of aggregated
counts and metrics.
CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS
 START
CHAPTER 1 RECIPE 1
list of 22 URLs of political fake input URLs in
news stories
Source: BuzzFeed News CrowdTangle

WHAT PUBLICS
DOES FAKE NEWS
output data

> Fake news story URLs

> Facebook pages and groups

ANIMATE ON FACEBOOK?
that share the URLs
> Number of interactions
per each page or group
> Date of sharing
of the story

identify time intervals with

highest frequency of publication
of fake news stories

a
BEFORE STARTING identify key related WHICH MEDIA AND POLITICAL
events with
This recipe takes as a starting point a list of EVENTS ARE SUCCESSFUL IN

visualise
Google News
fake news stories. There are different ways of SETTING THE FAKE NEWS
obtaining these lists including starting with AGENDA?
existing lists as well as creating your own. To
illustrate this recipe we use an already existing
list of 22 fake news stories about various political b
issues pertaining to the 2016 presidential
WHICH FACEBOOK PAGES

import data in
elections in the US that generated most

visualise
engagement on Facebook. These were identified RAWGraphs AND GROUPS SHARE FAKE
by BuzzFeed News. NEWS?

The recipe comprises of four steps. We start by

identifying the themes that are exploited in our
c
set of stories as well as the key political events

import data in
which they editorialise (a ). Next we identify DO FACEBOOK PUBLICS HAVE

visualise
Gephi
the most prominent Facebook pages and groups PREFERRED STORY THEMES?
that share these stories (b ). We also explore
whether certain publics have preferred story
themes (c) and profile the publics that are d
energised by fake news stories about the US WHAT KINDS OF PUBLICS ARE

visualise
manually categorise
elections (d )? facebook pages ENERGISED BY FAKE NEWS?

18 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 19
START
list of 22 URLs of political fake input URLs in
news stories
Source: BuzzFeed News CrowdTangle

output data

> Fake news story URLs

CHAPTER 1 RECIPE 1
> Facebook pages and groups
that share the URLs a. EXAMINE THE THEMES EXPLOITED
> Number of interactions
per each page or group IN FAKE NEWS STORIES AND
> Date of sharing
of the story IDENTIFY THE EVENTS WHICH THEY
EDITORIALISE
identify time intervals with
highest frequency of publication This analysis may be done by qualitatively
of fake news stories
examining the content of each article and
identifying key political or media events
a related to the issues exploited in the articles,
which occurred around the publication date
identify key related WHICH MEDIA AND POLITICAL
events with of each story. This is done to enable a better
EVENTS ARE SUCCESSFUL IN
visualise

Google News understanding of the issues that animate the

SETTING THE FAKE NEWS
publics that circulate fake news.
AGENDA?
If the content of the fake news article is no
longer available on its original URL you
may use the Internet Archives WayBack
Machine to check whether an archived
version of the URL is available.
To identify key events occurring around
the dates of publication of the stories which
are related to the themes exploited in the
stories you may use a news aggregator such
as Google News Search as well as news
article archives.
An annotated timeline of stories and
relevant events occurring around the same
dates might provide a starting point for
reflection about the relationship between
political and media events and fake news
stories.

20 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 21
 3 November

WHICH MEDIA AND

EMPIREHERALD.COM
African Billionaire Will Give $1 Million To
Anyone Who Wants To Leave America if
POLITICAL EVENTS ARE Donald Trump is Elected President
DISCRIMINATION AGAINST MINORITIES

SUCCESSFUL IN SETTING
3 October
ABCNEWS.COM.CO 5 November
THELASTLINEOFDEFENSE.ORG
THE FAKE NEWS AGENDA?
Actor Bill Murray Announces 2016
Presidential Run Van Full Of Illegals Shows Up To Vote
PACIFISM 11 October Clinton At SIX Polling Places, Still
WORLDNEWSDAILYREPORT.COM
Think Voter Fraud Is A Myth? 11 December
Timeline of best performing fake VOTER FRAUD, ANTI-IMMIGRATION SENTIMENT
ABCNEWS.COM.CO
ISIS Leader Calls for American
news stories about the US elections Muslim Voters to Support Hillary 5 November
Obama Signs Executive Order
Clinton Banning The Pledge Of Allegiance In
on Facebook in 2016 and events they ANTI-MUSLIM SENTIMENT THELASTLINEOFDEFENSE.ORG Schools Nationwide
editorialise. Successful fake news 14 October
WHOA! Hillary Caught On Hot Mic NATIONALIST SENTIMENT
Trashing Beyonce With RACIAL SLURS!
stories appear to exploit populist 7 September
WORLDNEWSDAILYREPORT.COM ANTI-CLINTON
12 December
BURRARDSTREETJOURNAL.COM 5 November ABCNEWS.COM.CO
themes such as anti-establishment RUPAUL CLAIMS TRUMP TOUCHED
Obama Signs Executive Order Declaring
President Obama Confirms He Will HIM INAPPROPRIATELY IN THE DENVER GUARDIAN
sentiment, nationalist and anti- Refuse To Leave Office If Trump Is 1990S FBI Agent Suspected in Hillary Email Investigation Into Election Results;
Elected Revote Planned For Dec. 19th
immigration sentiment as well as
MISOGYNY
Leaks Found Dead in Apparent
PRESIDENTIAL RACE CORRUPTION OF POLITICAL ESTABLISHMENT
18 September 15 October Murder-Suicide
perceived or projected weaknesses of

Fake news stories

USANEWSFLASH.COM CORRUPTION OF THE POLITICAL ESTABLISHMENT
POLITICOPS.COM
political candidates such as misogyny Mike Pence: Sarah Palin Is My Role FBI Agent Suspected in Hillary Email
11 November
Model For Beautiful, Smart American Leaks Found Dead in Apparent
and corruption. A number of events at Women - Newslo Murder-Suicide TMZHIPHOP.COM
the end of July, mid-October and early 6 July GENDER DISCRIMINATION CORRUPTION OF THE POLITICAL ESTABLISHMENT Trump Offering Free One-Way
CHRISTIAN TIMES 26 September Tickets to Africa & Mexico for Those
November are successful in setting the ENDING THE FED 17 October Who Wanna Leave America
BREAKING Romanian Hacker With
fake news agenda. Access To Clinton Emails Found Pope Francis Shocks World, CONSERVATIVESTATE.COM ANTI-IMMIGRANT SENTIMENT

Dead In Jail Cell Endorses Donald Trump for Hillary Clinton In 2013: I Would Like To See
22 November
ANTI-CLINTON President, Releases Statement People Like Donald Trump Run For Office;
23 July CORRUPTION OF THE POLITICAL ESTABLISHMENT Theyre Honest And Cant Be Bought ABCNEWS.COM.CO
BIZSTANDARDNEWS.COM PRO-ENTREPRENEURIALIST SENTIMENT Donald Trump Protester Speaks Out: I
8 March
Graham Says Christians Must Was Paid $3,500 To Protest Trumps Rally
REACT365.COM ANTI-CLINTON
Support Trump or Face Death Camps
Obama passed law for grandparents ANTI-DEMOCRAT 25 July
to get all their grandchildren every KYPO6.COM
weekend
N/A
Pope Francis Shocks World,
11 March Endorses Hillary Clinton for
President, Releases Statement
HEAVIERMETAL.NET ANTI-TRUMP 29 July
RAGE AGAINST THE MACHINE To BURRARDSTREETJOURNAL.COM
Reunite And Release Anti Donald Trump Claims America Should Never
Trump Album Have Given Canada Its Independence
ANTI-TRUMP
NATIONALIST SENTIMENT

January February March April May June July August September October November December

23 Jul Trump attacks "corrupt establishment",

Donald Trump accepts nomination Beyonce, Jay Z & Clinton share a stage 5 Nov
states Clinton should be "locked up" 13 Oct
and media is in "war against him" NY Times
Wikileaks reveals Democratic Party 25 Jul Vox
has a bias against Bernie Sanders No charge to Hillary Cinton and no further investigation over email scandal
The Most Explosive WikiLeaks 6 Nov
Clinton Revelations (So Far) FBI Director Comey
28 Jul
Hillary Clinton accepts nomination Breitbart Arizona Anti-Immigrant Sheriff to Deploy Deputies at Polling Places
The Guardian

Events
Trump Says Sexual Assault Allegations
14 Oct
are "Made-Up Stories"
IB Times Donald Trump continues to accuse Democrats of voter fraud.
On Saturday 5th November, Trump claimed the late opening of 7 Nov
Michelle Obama denounces a voting site in a Latino neighorhood of Las Vegas due
Trump's comment on women to long lines pointed to a "rigged system."
NY Times Democracy Now

Ninth Woman Says Trump Arizona Anti-Immigrant Sheriff

15 Oct 11 Nov
Inappropriately Groped or Kissed Her to Deploy Deputies at Polling Places
The Guardian The Guardian

22 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 23
 CHAPTER 1 RECIPE 1

START b. IDENTIFY THE FACEBOOK PAGES AND

list of 22 URLs of political fake input URLs in GROUPS THAT SHARE THESE STORIES
news stories
Source: BuzzFeed News CrowdTangle This may be done with a social media
monitoring tool such as the browser extension
of CrowdTangle. The number of followers
output data per page or group as well as the number of
> Fake news story URLs
> Facebook pages and groups
interactions per posts should be recorded in a
that share the URLs
> Number of interactions
spreadsheet alongside the names of pages and
per each page or group groups that share fake news stories.
> Date of sharing
of the story
Please note that a fake news story may be
reposted on a number of different websites.
For this reason a methodological decision
needs to be taken from the outset as to
whether only the pages and groups that
share the original URL of the story will be
recorded or whether all pages and groups
that share all versions of the fake news
story will be collected.
You may want to take note of the pages or
groups which shared the highest number
of fake news stories as well as the total
number of interactions generated by each
b
group or page.
WHICH FACEBOOK PAGES
import data in

If you use CrowdTangle please note

visualise

RAWGraphs AND GROUPS SHARE FAKE

that for Facebook the tool returns the top
NEWS?
500 most important posts from verified
pages as well as from pages with more than
750.000 followers.[1] [1]
See, CrowdTan-
You may use a circle packing visualisation gles Frequently
Asked Ques-
to represent the pages and groups that tions, available
share fake news items as well as the at: https://apps.
crowdtangle.
number of stories which they share and com/chrome-ex-
the number of interactions which they tension/faq

generate. You may use RAWGraphs for

this operation.

24 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 25
WHICH FACEBOOK PAGES
AND GROUPS PROMOTED Women SCOUTS for TRUMP (c)

THE HIGHEST NUMBER OF We Endorse

Donald J. Trump

FAKE NEWS? WHICH ONES THE TRUMPERS!!!

Washington State for
Donald Trump 2016

CREATED THE HIGHEST THE TRUMPS STORM GROUP

ENGAGEMENT? WOMEN4TRUMP

NO Hillary for 2016

Facebook pages and groups that TRUMP - SPEAK OUT
AGAINST ISLAMIZATION
share fake news items, sized OF AMERICA

according to the number of items NEVER Sicilians For

HILLARY Donald Trump
they share and coloured according TRUMP FORCE ONE
CLINTON

to their number of followers. Each Veterans for Donald Trump

TRUMP OF CHAMPIONS Hillary Clinton NEVER HILLARY
page can share the same item more vote blue
than once. The pages and groups that
Hillary Clinton in 2020
share the highest number of stories Newslo
are primarily pro-Trump supporters
and anti-Hillary groups. The page DONALD TRUMP
Hillary Clinton Revolution VOTE TRUMP ONLY -
The Australian Tea Party COMMANDER IN CHIEF !!!!
that generates the highest number of Donald Trump for President! THE AMERICAN PARTY RISING

interactions with fake news stories is F@CK YOU

HILLARY CLINTONDemocratic D.L. Hughley
the fan page dedicated to republican The Burrard Street Journal WE HATE YOU! News
Donald Trump for
Rock Feed
and Events
TV commentator, Jeanine Pirro. President Connecticut
Anonymous The VETERANS FOR PRESIDENT
World is Ours Donald Trump DONALD J TRUMP 2016
Indies 4 Trump Criminalize VS. Hillary
The Church All Things Conservatism Clinton
of PUNK #trumpvictory Resist Donald J. Trumpf
#neverhillary Donald Trump
The Political Movement
Deplorables for TRUMP 1 Million Strong For
Hillary Clinton in 2016 Conservative
Veterans Donald
Chonda Pierce for America Trump
The Empire Herald
FU Trump President

One Nation Under God

Females Republican
AMOUNT OF INTERACTIONS for TRUMP Citizens Patriotic
Being Liberal Make America Party
For Trump
Means Being Great Again
1 30 Judge Jeanine Donald Trump For President
a Hypocrite 2 Colorado UNITE TO FIGHT
Pirro has Fans
Don P for Donald BRING AMERICA
0 73,732 Trump 2016 BACK !!! TRUMP 2016
Donald Trump
The Original Wake MY VOTE'S FOR
PATRIOT Freedom Don't Donald J. Trump Donald Trump Commander
FAKE NEWS POSTED Up America
AMERICANS Come Free for PRESIDENT America's in Chief 2020 HILLARY IN TRUMP Reclaim
2016 & 2020 The Deplorables America
FOR DONALD TRUMP & Devildoc Facebook Group President (c)
1 30 Friends Who
The Resistance: Like Donald J. Trump
The Last Line
Of Defense
Patriots for a TrumpNation
Free Republic -
REPUBLICANS - CONSERVATIVES -
Trump Squad
TEA PARTY PATRIOTS-
RIGHT WING AMERICANS
Politicked

Trump Trump USA's CEO

Trump America
The Trump Tomorrow's Train
American Party News Today.

26 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 27
START
list of 22 URLs of political fake input URLs in
news stories
Source: BuzzFeed News CrowdTangle

output data

> Fake news story URLs

CHAPTER 1 RECIPE 1
> Facebook pages and groups
that share the URLs c. IDENTIFY WHETHER FACEBOOK
> Number of interactions
per each page or group PUBLICS HAVE PREFERRED STORY
> Date of sharing
of the story THEMES
To explore whether particular story themes
assemble publics and to qualitatively profile
those publics based on the stories that animate
them you may conduct a network analysis of
Facebook pages and groups connected by the
stories which they share.
Starting from the dataset extracted with
CrowdTangles browser extension, you
may create a network file where each time
a Facebook group or page posts a fake news
story a link is established between that page
or group and that story.
You may use Table2Net to convert your
CSV (comma-separated values) file into a
network file and Gephi to explore the [2]
See, Mathieu
network. A force-directed layout algorithm Jacomy, Tommaso
such as ForceAtlas2[2] can help you Venturini, Sebas-
tien Heymann and
visualise the outcomes. Mathieu Bastian,
c ForceAtlas2, a
Identify which stories are most successful
import data in

Continuous Graph
DO FACEBOOK PUBLICS HAVE in energising publics as well as whether
visualise

Layout Algorithm
Gephi
PREFERRED STORY THEMES? publics have preferred story themes.
for Handy Network
Visualization De-
signed for the Ge-
phi Software, June
2014, PLoS ONE:
http://journals.
plos.org/plosone/
article?id=10.1371/
journal.
pone.0098679

28 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 29
DO FACEBOOK PUBLICS
HAVE PREFERRED
STORY THEMES?
Network of Facebook pages and groups
connected by the fake news stories
which they share. Notable is the core of
the network which consists of a series
of pages and groups associated with
Trump supporters which are animated
by anti-Hillary stories.
FAKE AMOUNT OF
NEWS URL INTERACTIONS
K
T
H N Fake News Headlines
A FBI Agent Suspected in Hillary Email
Leaks Found Dead in Apparent
Murder-Suicide
B Hillary Clinton In 2013: I Would Like
L To See People Like Donald Trump Run
D.L. Hughley
For Office; Theyre Honest And Cant
Be Bought
C ISIS Leader Calls for American Muslim
Voters to Support Hillary Clinton
P D Donald Trump Protester Speaks Out:
O "I Was Paid $3,500 To Protest
Trump's Rally"
E Obama Signs Executive Order
Declaring Investigation Into Election
Results; Revote Planned For Dec. 19th
The Burrard R
Street Journal F WHOA! Hillary Caught On Hot Mic
Trashing Beyonce With RACIAL
SLURS!
G Van Full Of Illegals Shows Up To Vote
Clinton At SIX Polling Places, Still
Hillary Clinton Think Voter Fraud Is A Myth?
Revolution H RAGE AGAINST THE MACHINE To
Reunite And Release Anti Donald
Colorado for Donald Trump Album
M Trump 2016 S
J Reclaim America The Resistance: The Last I Obama Signs Executive Order Banning
Line Of Defense The Pledge Of Allegiance In Schools
Nationwide
One Nation J BREAKING Romanian Hacker With
Under God Access To Clinton Emails Found Dead
America's Veterans C In Jail Cell
are Loved
E K Actor Bill Murray Announces 2016
Presidential Run
Donald Trump
Commander
U L Pope Francis Shocks World, Endorses
in Chief 2020 Hillary Clinton for President, Releases
I
Q F Statement
G M Pope Francis Shocks World, Endorses
F@CK YOU HILLARY
Stop the Mosque Donald Trump for President, Releases
in Bendigo Statement
CLINTON WE HATE YOU!
N Trump Claims America Should Never

0 98,032 D Donald Trump

Have Given Canada Its Independence
For President O Mike Pence: Sarah Palin Is My Role
NO Hillary TrumpNation Model For Beautiful, Smart American
for 2016 Women - Newslo
N OF PAGES SHARING
A NEWS ITEM P RUPAUL CLAIMS TRUMP TOUCHED
HIM INAPPROPRIATELY IN THE
B 1990S
Q President Obama Confirms He Will
Refuse To Leave Office If Trump Is
1 125 Elected
The Trump American Party TRUMP R Graham Says Christians Must Support
FORCE ONE Trump or Face Death Camps
A
VOTE TRUMP ONLY Make America S African Billionaire Will Give $1 Million
THEAMERICAN Great Again To Anyone Who Wants To Leave
PARTY RISING Donald Trump America if Donald Trump is Elected
FACEBOOK AMOUNT OF for President President
PAGE OR INTERACTIONS
TRUMP - SPEAK OUT AGAINST T Trump Offering Free One-Way Tickets
GROUP ISLAMIZATION OF AMERICA to Africa & Mexico for Those
Who Wanna Leave America
U Obama passed law for grandparents to
0 65,649
get all their grandchildren every
weekend

30 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 31
START
list of 22 URLs of political fake input URLs in
news stories
Source: BuzzFeed News CrowdTangle

output data

> Fake news story URLs

CHAPTER 1 RECIPE 1
> Facebook pages and groups
that share the URLs d. PROFILE THE PUBLICS ANIMATED BY
> Number of interactions
per each page or group FAKE NEWS
> Date of sharing
of the story
This may be done by conducting a qualitative
analysis of all Facebook pages that share fake
news items based on self-descriptions available
on their About pages.
You may take an emergent coding
approach to identify the themes that
emerge from the description of pages.
You may take note of a more generic
category (e.g. grass-roots activism) as
well as a more specific one (e.g. anti-
establishment).
Sum up the amount of followers across all
pages belonging to the same category.
A treemap visualisation may be used to
represent the weight and hierarchy of each
category. You may use RAWGraphs for
this operation.

d
WHAT KINDS OF PUBLICS ARE
visualise

manually categorise
facebook pages ENERGISED BY FAKE NEWS?

32 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 33
WHAT KINDS OF WORKERS NEWS GRASSROOTS ACTIVISTS/ CITIZEN VIGILANTES PUBLIC FIGURE

PUBLICS ARE ENERGISED Truckers Small

Bussiness
Steelworkers News Trending Liberal
News News
Convervative
News
For Free
Society
Againt Pro
Ted Cruz Bernie
For
Disaster
Against
Islam
Against Pro Latino
Financial
Artist Porn
Actress
Preparness Terrorism

BY FAKE NEWS?
Actor Republican
OTHER Politician
Types of Facebook publics animated Italian Climate Page Critical Liberal News / Republican
For Liberty
and Property
White For
Lives Information
Against Trump Against
Republicans
News news News News
by fake news, according to a manual Left Skeptics Dedicated
Aggregators
Matter Freedom
to the
classification of pages that share fake Philipines
Againt
Gun
news items. Notable are grassroots Owners
Journalist
activists for a variety of issues, SOCIAL CHANGE Against ENTERTAINERS
News News Alternative
Pro Latino Human Progessives Aggregator News Mainstream Anti-conservatism US Patriots
political candidate loyalists as well as rights Media
entertainers. Against Gun
Control
Against Gun Advocates Anonymous
Violence
Against
Corruption
Non-PC News

EARLY CAMPAIGNERS Pro-Trump

Early 2020 Early 2020 Early 2020

Bernie Trump Hillary
Campaigners Campaigners Campaigners
LOYALISTS
Hillary Loyalists Trump Loyalists

Bernie Loyalists

DISCUSSION SPACE
Political Anti
Commentary establishment

Politcal
Commentary

Anti-Democrat

BUSINESS MUSIC/ENTERTAIMENT
Punk Tattoo Fashion COMMUNITY
Spritual Democrat Community Pro Military
HIERARCY
Community
Bussiness Motivational
Speaker Progessive
CATEGORY
Community
Dating app
Sub-category Survivalist
Republican Community

FACEBOOK PAGES
Pro-Black
PER CATEGORY
Religious
OCCUPY MOVEMENT

1
2
3 CLICKBAIT
WOMEN-FOCUSED

34 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 35
 3 November
EMPIREHERALD.COM
African Billionaire Will Give $1 Million To
Anyone Who Wants To Leave America if
Donald Trump is Elected President Women SCOUTS for TRUMP (c)
DISCRIMINATION AGAINST MINORITIES
3 October
ABCNEWS.COM.CO 5 November We Endorse
THELASTLINEOFDEFENSE.ORG Donald J. Trump
Actor Bill Murray Announces 2016 Washington State for
Presidential Run Van Full Of Illegals Shows Up To Vote THE TRUMPERS!!! Donald Trump 2016
PACIFISM 11 October Clinton At SIX Polling Places, Still
WORLDNEWSDAILYREPORT.COM
Think Voter Fraud Is A Myth? 11 December THE TRUMPS STORM GROUP
VOTER FRAUD, ANTI-IMMIGRATION SENTIMENT
ABCNEWS.COM.CO
ISIS Leader Calls for American
Muslim Voters to Support Hillary Obama Signs Executive Order
5 November

CHAPTER 1 RECIPE 1
Clinton Banning The Pledge Of Allegiance In WOMEN4TRUMP
ANTI-MUSLIM SENTIMENT THELASTLINEOFDEFENSE.ORG Schools Nationwide
WHOA! Hillary Caught On Hot Mic NATIONALIST SENTIMENT NO Hillary for 2016
TRUMP - SPEAK OUT
14 October Trashing Beyonce With RACIAL SLURS!
7 September 12 December AGAINST ISLAMIZATION
WORLDNEWSDAILYREPORT.COM ANTI-CLINTON OF AMERICA
BURRARDSTREETJOURNAL.COM 5 November ABCNEWS.COM.CO
RUPAUL CLAIMS TRUMP TOUCHED NEVER
President Obama Confirms He Will Obama Signs Executive Order Declaring Sicilians For
HIM INAPPROPRIATELY IN THE DENVER GUARDIAN HILLARY Donald Trump
Refuse To Leave Office If Trump Is 1990S FBI Agent Suspected in Hillary Email Investigation Into Election Results; CLINTON
Elected MISOGYNY Revote Planned For TRUMP FORCE ONE
Dec. 19th
Leaks Found Dead in Apparent
PRESIDENTIAL RACE
15 October
CORRUPTION OF POLITICAL ESTABLISHMENT Veterans for Donald Trump
18 September Murder-Suicide TRUMP OF CHAMPIONS

Fake news stories

CORRUPTION OF THE POLITICAL ESTABLISHMENT
Hillary Clinton NEVER HILLARY
POLITICOPS.COM USANEWSFLASH.COM vote blue
Mike Pence: Sarah Palin Is My Role FBI Agent Suspected in Hillary Email
K
11 November
Model For Beautiful, Smart American Leaks Found Dead in Apparent Hillary Clinton in 2020
Women - Newslo Murder-Suicide TMZHIPHOP.COM Newslo
6 July GENDER DISCRIMINATION CORRUPTION OF THE POLITICAL ESTABLISHMENT Trump Offering Free One-Way
CHRISTIAN TIMES 26 September Tickets to Africa & Mexico for Those
17 October

SERVING SUGGESTIONS
BREAKING Romanian Hacker With ENDING THE FED Who Wanna Leave America DONALD TRUMP
Hillary Clinton Revolution
CONSERVATIVESTATE.COM ANTI-IMMIGRANT COMMANDER IN CHIEF !!!! VOTE TRUMP ONLY -
Access To Clinton Emails Found Pope Francis Shocks World, The Australian Tea Party SENTIMENT
Donald Trump for President! THE AMERICAN PARTY RISING
Dead In Jail Cell Endorses Donald Trump for Hillary Clinton In 2013: I Would Like To See
ANTI-CLINTON
T 23 July
President, Releases Statement
CORRUPTION OF THE POLITICAL ESTABLISHMENT
People Like Donald Trump Run For Office;
Theyre Honest And Cant Be Bought
22 November
F@CK YOU
HILLARY CLINTONDemocratic
ABCNEWS.COM.CO
D.L. Hughley

PRO-ENTREPRENEURIALIST SENTIMENTThe Burrard Street Journal

WE HATE YOU! News Rock Feed
BIZSTANDARDNEWS.COM Donald Trump Protester Speaks Out: I Donald Trump for
and Events
8 March
Graham Says Christians Must Was Paid $3,500 To Protest Trumps Rally President Connecticut
REACT365.COM Support Trump or Face Death Camps ANTI-CLINTON Anonymous The VETERANS FOR PRESIDENT
Donald Trump DONALD J TRUMP 2016

Fake News Headlines

Obama passed law for grandparents ANTI-DEMOCRAT World is Ours
25 July Indies 4 Trump
N Criminalize VS. Hillary

This recipe may be used to better

H to get all their grandchildren every
weekend
KYPO6.COM
Pope Francis Shocks World,
The Church
of PUNK
All Things
#trumpvictory
Conservatism Clinton
Resist Donald J. Trumpf
A FBI Agent Suspected in Hillary Email
N/A #neverhillary Donald Trump
11 March Endorses Hillary Clinton for The Political Movement
Deplorables for TRUMP
HEAVIERMETAL.NET
President, Releases Statement
ANTI-TRUMP 29 July
1 Million Strong For
Hillary Clinton in 2016
Leaks Found Dead in Apparent
Conservative
RAGE AGAINST THE MACHINE To BURRARDSTREETJOURNAL.COM
Chonda Pierce
Murder-Suicide
VeteransDonald

understand the publics that are

Reunite And Release Anti Donald for America Trump
Trump Claims America Should Never The Empire Herald
Trump Album Have Given Canada Its Independence
FU Trump
B Hillary Clinton In 2013: I Would Like
President

L
ANTI-TRUMP
NATIONALIST SENTIMENT
One Nation Under God To See People Like Donald Trump Run
Republican
AMOUNT OF INTERACTIONS
Females
Citizens
For Office; Theyre Honest And Cant
Patriotic
D.L. Hughley for TRUMP
For Trump Being Liberal
Be Bought Make America Party
January February March April May June Means Being Great Again
1 30 July August September October November December

animated by fake news and the meaning

Judge Jeanine Donald Trump For President
a Hypocrite 2 Colorado
Pirro has Fans
Don P for Donald C ISIS Leader Calls for American Muslim
UNITE TO FIGHT
BRING AMERICA
0 73,732 Trump 2016 Voters to Support Hillary Clinton
BACK !!! TRUMP 2016

P
23 Jul Trump attacks "corrupt establishment",
Donald Trump accepts nomination Beyonce, Jay Z & Clinton share a stage 5 Nov Donald Trump
states Clinton should be "locked up" 13 Oct
and media is in "war against him"
The Original Wake
NY Times PATRIOT Freedom Don't Donald J. Trump Donald Trump Commander D Donald Trump Protester
MY VOTE'S FOR Speaks Out:
FAKE NEWS POSTED
O
Up America in Chief 2020 TRUMP Reclaim
Wikileaks reveals Democratic Party 25 Jul Vox AMERICANS Come Free for PRESIDENT America's "I2016
Was Paid $3,500 To America
HILLARY IN
Protest

making activities that they engage in

has a bias against Bernie Sanders FOR DONALD TRUMP & Devildoc Facebook Group President (c) & 2020 The Deplorables

1
The Most Explosive WikiLeaks
Clinton Revelations30
(So Far)
No charge to Hillary Cinton and no further investigation over email scandal
FBI Director Comey
6 Nov
Friends Who
Trump's Rally"
28 Jul
Hillary Clinton accepts nomination Breitbart Like Donald J. Trump
Arizona Anti-Immigrant Sheriff to Deploy Deputies at Polling Places
The Guardian
The Resistance:
The Last Line
E Obama Signs Executive Order

Events
Trump Says Sexual Assault Allegations
14 Oct
Of Defense
Patriots for a Declaring Investigation Into Election
TrumpNation
are "Made-Up Stories"
Free Republic - Results; Revote Planned For Dec. 19th
R
IB Times Donald Trump continues to accuse Democrats of voter fraud. REPUBLICANS - CONSERVATIVES -

around fake news, i.e. how they enroll

On Saturday 5th November, Trump claimed the late opening of 7 Nov Trump Squad
Michelle Obama denounces The Burrard a voting site in a Latino neighorhood of Las Vegas due
TEA PARTY PATRIOTS-
Trump's comment on women Street Journal to long lines pointed to a "rigged system." Politicked F WHOA! Hillary Caught On Hot Mic
RIGHT WING AMERICANS
WORKERS NEWS GRASSROOTS ACTIVISTS/ CITIZEN VIGILANTES PUBLIC FIGURE
NY Times Democracy Now
Trashing
Trump
Beyonce
Trump USA's CEOWith RACIAL
Trump America
Arizona Anti-Immigrant Sheriff
TruckersThe Small
Trump Steelworkers News
Tomorrow's Trending Liberal Convervative ForSLURS!
Free
Train Againt Pro For Against Against Pro Latino Artist Porn
Ninth Woman Says Trump American Party
Inappropriately Groped or Kissed Her
15 Oct
to Deploy Deputies at Polling Places
11 Nov Bussiness News Today. News News News Society Ted Cruz Bernie Disaster Islam Financial Actress
G Van Full Of IllegalsPreparness
Shows Up ToTerrorism
Vote

fake news in the service of their own

The Guardian The Guardian

Clinton At SIX Polling Places, Still

Hillary Clinton OTHER Think Voter Fraud Is A Myth? Actor Republican
Politician
Revolution Italian Climate Page Critical Liberal News / Republican
For Liberty
HandRAGE
Property AGAINST
White For
Lives Information
Against Trump Against
THE MACHINE ToRepublicans
News news News News
Left Skeptics Dedicated
Reunite And Release Anti Donald
Matter Freedom

issue work. This approach may inform

to the Aggregators
Colorado for Donald Trump Album
Againt
S
Philipines
M Trump 2016 Gun

J Reclaim America The Resistance: The Last

SOCIAL CHANGE
I Owners
Obama Signs Executive Order Banning
ENTERTAINERS Journalist
Line Of Defense News News Alternative The Pledge Of Allegiance In Schools
Against
Pro Latino Human Progessives Mainstream
Aggregator News
NationwideAnti-conservatism US Patriots

a thicker description of the impact of

rights Media

One Nation JAgainst

BREAKING
Gun Romanian Hacker With
Control
Under God Against Gun Access To Anonymous
Advocates Clinton Emails Found Dead
America's Veterans C Violence In Jail Cell
Against
Corruption
are Loved
E
fake news that moves away from its
Non-PC News K Actor Bill Murray Announces 2016
EARLY CAMPAIGNERS
Presidential Run
Pro-Trump
Donald Trump
Commander Early 2020 Early 2020 Early 2020
U L Pope Francis Shocks World, Endorses
in Chief 2020 Hillary Clinton for President, Releases
I
Bernie Trump Hillary
Campaigners Campaigners Campaigners
Q
viral character (the single engagement
Statement
LOYALISTS
F
OF G Hillary Loyalists
M Pope Francis Shocks World, Endorses
Trump Loyalists

TIONS F@CK YOU HILLARY

Stop the Mosque Bernie Loyalists
Donald Trump for President, Releases
in Bendigo Statement
CLINTON WE HATE YOU!

number or metric) to understanding who

DISCUSSION SPACE N Trump Claims America Should Never

98,032 D Donald Trump

Political
Commentary
Anti
establishment
Have Given Canada Its Independence
For President O Mike Pence: Sarah Palin Is My Role
NO Hillary TrumpNation Politcal Model For Beautiful, Smart American

it mobilises and how.

for 2016 Commentary
Women - Newslo
Anti-Democrat P RUPAUL CLAIMS TRUMP TOUCHED
HIM INAPPROPRIATELY IN THE
B 1990S
BUSINESS MUSIC/ENTERTAIMENT
Q President Obama Confirms He Will
Punk Tattoo Fashion COMMUNITY
Refuse To Leave Office If Trump Is
HIERARCY Spritual Democrat Community Elected Pro Military
Community
Bussiness Motivational
The Trump American Party TRUMP Speaker R Graham Says Christians Must Support
CATEGORY Progessive
FORCE ONE Trump or Face Death Camps
A Community

VOTE TRUMP ONLY Sub-category Make America Dating app

Survivalist
Republican Community
S African Billionaire Will Give $1 Million
THEAMERICAN Great Again To Anyone Who Wants To Leave
PARTY RISING Donald Trump America if Donald Trump is Elected
OF for President
FACEBOOK PAGES President
TIONS PER CATEGORY Pro-Black
TRUMP - SPEAK OUT AGAINST Religious
T Trump Offering Free One-Way Tickets
ISLAMIZATION OF AMERICA OCCUPY MOVEMENT to Africa & Mexico for Those
Who Wanna Leave America
U Obama passed law for grandparents to
65,649 1
2 get all their grandchildren every
3 weekend CLICKBAIT
WOMEN-FOCUSED

36 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 37
 START
select a fake news story
you want to trace
CHAPTER 1 RECIPE 2

design queries to identify

HOW MAY THE TRAJECTORY URLs where story is published

Google Search

OF A FAKE NEWS STORY

BE TRACED ON FACEBOOK? compile list of URLs
where story is published

input URL in

input each URL in

CrowdTangle
CrowdTangle

output data
output data for each URL
BEFORE STARTING > Fake news story URLs
> Facebook pages and > Fake news story URLs
groups > Facebook pages and
For this recipe it is recommended that a fake that share the URLs groups
> Number of followers that share the URLs
news story is taken as a starting point and the per page or group > Number of followers
> Date of sharing per page or group
URL or URLs or on which it is published are of the story > Date of sharing
of the story
identified. To illustrate this recipe we have input data to
selected as case studies two prominent stories input data to
RAWGraphs
about the 2016 US presidential elections, namely RAWGraphs
"Trump Offering Free One-Way Tickets to
Africa & Mexico for Those Who Wanna Leave
America," a story that exploits anti-immigrant visualise visualise

sentiment and "Rage Against the Machine

to Reunite and Release Anti Donald Trump
Album," which exploits anti-Trump sentiment. a b
This recipe comes in two flavours. In step one HOW DOES THE STORY HOW DOES THE STORY
you will learn to trace Facebook pages and "RAGE AGAINST THE "TRUMP OFFERING FREE
groups in which the original story URL is posted MACHINE TO REUNITE ONE-WAY TICKETS TO AFRICA
and plot them on a timeline (a). In step two AND RELEASE ANTI & MEXICO FOR THOSE WHO
this analysis will be extended to all URLs on DONALD TRUMP ALBUM" WANNA LEAVE AMERICA" AND
which a story has been republished (b). TRAVEL ON FACEBOOK? ITS DEBUNKED VERSIONS
TRAVEL ON FACEBOOK?
38 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS
ON FACEBOOK 39
 START CHAPTER 1 RECIPE 2
select a fake news story
you want to trace a. IDENTIFY FACEBOOK PAGES AND
GROUPS THAT SHARE A FAKE NEWS
STORY VIA THE ORIGINAL URL
Facebook pages and groups that share a fake
news story may be detected through a social
media monitoring tool such as CrowdTangles
browser extension.
The names of these pages and groups, their
followers count, the interactions that they
generate as well as the date of sharing of the
story may be recorded in a spreadsheet per
input URL in
story URL.
CrowdTangle To explore the temporal dynamics of
the circulation of the fake news story
on Facebook, you may plot its trajectory
across pages and groups on a timeline.
output data
> Fake news story URLs RAWGraphs can be used to create the
> Facebook pages and
groups
base layer of the visualisation. Note which
that share the URLs
> Number of followers
publics engage with the story as well as
per page or group
> Date of sharing
whether the moment of debunking a story
of the story affects its circulation.
input data to To take the analysis one step further, a
RAWGraphs qualitative analysis of how fake news is
enrolled by each of these pages to support
issue work may be undertaken. This may
visualise
be done by examining the context in which
the stories are shared, i.e. whether they
are shared uncritically or called out as fake
as well as how they are framed in relation
a
to the issues represented by the pages that
HOW DOES THE STORY share them. It is to be noted that such
"RAGE AGAINST THE analysis might at times be difficult due to the
MACHINE TO REUNITE fact that for the most prominent fake news
AND RELEASE ANTI stories Facebook posts that share them are
DONALD TRUMP ALBUM" often no longer available via the Facebook
TRAVEL ON FACEBOOK? interface and API.

40 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 41
HOW DOES THE STORY Pubished the 11th of March 2016 on heaviermetal.net
RAGE AGAINST THE MACHINE
"RAGE AGAINST THE To Reunite And Release Anti Donald Trump Album

MACHINE TO REUNITE The Church of PUNK

AND RELEASE ANTI Justice Through Music

Rock Feed

DONALD TRUMP Rock.It Boy Entertainment

Indie DaaDee
ALBUM" TRAVEL ON Magic 89.3

FACEBOOK?
98 Rock

Alabama Citizens for Bernie Sanders 2020

Anonymous The World is Ours

Trajectory of Rage Against the
This page is very offensive
Machine to Reunite and Release Bernie Sanders for President 2020
Anti Donald Trump Album" story on Occupy Orange County CA
Facebook pages and groups. The story Wunderkynd
circulates best between March and June Ballin' On A Budget
2016 as satire amongst English language No Music No Life.
music and entertainment groups. It RAGE AGAINST THE MACHINE CHRISTMAS NO.1 2009

is revived in November after the US Circo Voador

elections, when it is also picked up by FM 949

Italian music and political pages. Bernie Sanders - Support from abroad

Boogat

FM 949

107.9 The Mix

Occupy London

Occupy movement

United Steelworkers Activists for Political Revolution

United People for Latinos in Film TV and Theater

Missoula for our Green Party Revolution

Our Revolution Continues- Activist Resources

STORIES SHARED ON FB Dj Mike Style!!! The story reappears on Facebook
Post Page where is shared Heavier Metal at the 11th of November, shared by
the page Apartment Khunpa
METAL ARMADA
PAKISTAN RISING
Rock Feed
Interactions Heavier Metal

Rock Feed No sharing of the story URL on Facebook from late May to early November
Music and/or Radio
Apartment Khunpa
Activism The Guitar Mag

Politics Banda Bassotti

Others Marcello Belotti - Sinistra Italiana EU

ROCK Mundial
POSTS INTERACTIONS Rock Feed is sharing the story
both at the earliest as well as The Church of PUNK
at the end of the first wave of posting. Radioactivo 98.5
0 1588
Los Angeles Punk Museum

Veterans Against Rep. Ignorance

POSTS ON A SAME PAGE FU Trump

Occupy Orange County CA

11/03/2016 APRIL MAY JUNE JULY AUGUST SEPTEMBER OCTOBER NOVEMBER DECEMBER

42 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 43
 START
select a fake news story
you want to trace

design queries to identify

URLs where story is published

Google Search
CHAPTER 1 RECIPE 2

b. IDENTIFY FACEBOOK PAGES AND

compile list of URLs
where story is published GROUPS THAT SHARE ALL INSTANCES
OF A FAKE NEWS STORY
As fake news stories may be republished by a
number of sources, the previous analysis may
be enriched by tracing the circulation not only
input each URL in
of the original URL on which the chosen story
CrowdTangle is posted but all instances of story republication
across a number of different sites.
To identify the websites which republish
output data for each URL
> Fake news story URLs a story as well as those which debunk it,
> Facebook pages and
groups you may query the title of the fake news in
that share the URLs
> Number of followers
a search engine of choice (e.g. Google
per page or group
> Date of sharing
Web Search) using a research browser [1] [1]
See instructions
of the story and extract the URLs corresponding to on how to set up
input data to instances of republication and debunking a research brows-
er in this video
RAWGraphs of the story from the returned list of tutorial: https://
results. www.youtube.com/
watch?v=bj65X-
Query the resulting URLs in a social r9GkJM
visualise
monitoring tool (such as CrowdTangle)
to get the list of Facebook groups and pages
that share the URLs corresponding to both
b the fake story and its debunked versions.
HOW DOES THE STORY You may plot these pages on a timeline to
"TRUMP OFFERING FREE see whether different fake news sources
ONE-WAY TICKETS TO AFRICA spawn different story trajectories on
& MEXICO FOR THOSE WHO Facebook and whether debunked versions
WANNA LEAVE AMERICA" AND are being acknowledged.
ITS DEBUNKED VERSIONS
TRAVEL ON FACEBOOK?
44 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS
ON FACEBOOK 45
HOW DOES THE Published the 11th of November 2016 on viralmugshot.com
Trump Offering Free One-Way Tickets to Africa & Mexico
STORY "TRUMP for Those Who Wanna Leave America

OFFERING FREE ONE- VIRALMUGSHOT.COM

The first instance of the story appears o
n viralmugshot.com in
early November, but
C rowdTangle only ind
Prince Sam
icates instances of Facebo TheCelebtricity
WAY TICKETS TO
ok sharing in late February
The first repost has been Don P

AFRICA & MEXICO FOR

published by myhiphop.org on
MYHIPHOP.ORG the 11th of November NYMAG.COM/SELECTALL
The first debunk has been

THOSE WHO WANNA Daily Media Buzz New York Magazine

Select All
published by nymag.com
on the 30th of December

LEAVE AMERICA" YEPSEE.COM The story is published

AND ITS DEBUNKED on Yepsee.com and has VANITYFAIR.COM

Amazing and Crazy videos
Crazy Accidents 18+
Damn
VERSIONS TRAVEL ON Embarrassing Party Photos
EpicFail.com
I post disturbing things.
FACEBOOK? New York Meta
The Craziest fight videos
Timeline of "Trump Offering THUGIFY.COM
Free One-Way Tickets to Africa
ThugLifeClips
& Mexico for Those Who Wanna
Leave America story and its sites
been shared on eight Pakistan Rising
different Facebook vanityfairmagazine
FAKE STORY INSTANCES
pages/groups during the VFHive
same day
Earliest version (original)
THEVERGE.COM Reposted Version
TechKnowledgeIt
verge Debunked version
NEWSER.COM POSTS ON FACEBOOK
newser Post Page that shares it

of publication on the web and DAILYNEWSPOSTS.INFO

PAKISTAN RISING
Facebook. The story is republished DONALD TRUMP
BUZZFEED.COM
COMMANDER IN CHIEF !!!! Interactions
without critical context on multiple NO Hillary for 2016
BuzzFeed News
BuzzFeed Politics
clickbait sites in the week following TRUMP - SPEAK OUT AGAINST ISLAMIZATION
OF AMERICA - FIM DA COLONIZAO ISL
BuzzFeed UK News
BuzzFeed World Fake Story
its original publication. This gives the Trump USA's CEO Isaac Saul
StyleHaul Debunk
story multiple lives on Facebook. Its TrumpNation The European Journalism Observatory | EJO
FAKE STORY INSTANCES BuzzFeed
sharing on a fake news site animates VOTE TRUMP ONLY - THE AMERICAN PARTY RISING Coloradans For Bernie POSTS INTERACTIONS
Ethical Journalism Network
politicalEarliest
publics versionwhile
(original)its sharing on Hillary Clinton The People's Choice

clickbaitReposted
sites Version
sees the story being 0 1588
CIVILIZED.LIFE
recycled as clickbait by viral pages. ATRENDING.NET
Debunked version civilized.life
The publics sparked into being by the Exposing The Truth POSTS ON A SAME PAGE
fake news story and the debunked
POSTS ON FACEBOOK
version thereof do not overlap. QATARDAY.COM
INQUISITR.COM
Post Page that shares it theinquisitr
Qatar Day

PAKISTAN RISING

DIYHILFE.COM POLITICO.COM
Interactions
JimHeathTV
Mobbing Hilfe LionPublishers INEWS.CO.UK
mnsucollegedemocrats
Fake Story modernliberals theipaper
PolPreposterous
Debunk
HOPEFORNIGERIAONLINE.COM politico TRUEAMERICANS.ME

POSTS INTERACTIONS Hope for Nigeria pg/politicomag

President Donald J. Trump
TRUMP - SPEAK OUT AGAINST [...]
politicomediadesk
0 1588
PENamerican
EDUBLOG.SCHOLASTIC.COM
LUXURYANDGLAMOR.COM DavidWilliamBateman
EmmaClarkLibrary
POSTS ON A SAME PAGE Daily Video NOWTORONTO.COM
No sharing on Facebook from mid November to late December nowmagazine
2017
NOV 13 NOV 20 NOV 27 DEC 04 DEC 11 DEC 18 DEC 25 JAN 01 JAN 08 JAN 15 JAN 22 JAN 29 FEB 05 FEB 12 FEB 19 FEB 26 MAR 05 MAR 12

46 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 47
 The story reappears on Facebook on
the 11th of November, shared by the
page Apartment Khunpa

CHAPTER 1 RECIPE 2
Published the 11th of November 2016 on viralmugshot.com
Trump Offering Free One-Way Tickets to Africa & Mexico
for Those Who Wanna Leave America

The first instance of the story appears o

VIRALMUGSHOT.COM n viralmugshot.com in
early November, but
C Prince Sam
rowdTangle only ind TheCelebtricity

Apartment Khunpa
The first repost has been
published by myhiphop.org on
MYHIPHOP.ORG the 11th of November
icates instances of Facebo
ok sharing in late February
Don P
NYMAG.COM/SELECTALL
The first debunk has been

SERVING SUGGESTIONS
Daily Media Buzz New York Magazine published by nymag.com
Select All
on the 30th of December

The Guitar Mag

YEPSEE.COM
Amazing and Crazy videos
Crazy Accidents 18+
Damn
Embarrassing Party Photos
EpicFail.com
The story is published
on Yepsee.com and has VANITYFAIR.COM
been shared on eight Pakistan Rising
different Facebook vanityfairmagazine
pages/groups during the
VFHive

This recipe may be used to

I post disturbing things. same day
New York Meta

Banda Bassotti
The Craziest fight videos
THEVERGE.COM
TechKnowledgeIt
THUGIFY.COM verge

understand the trajectory of a

ThugLifeClips
NEWSER.COM

Marcello Belotti - Sinistra Italiana EU

newser

DAILYNEWSPOSTS.INFO
BUZZFEED.COM
DONALD TRUMP
COMMANDER IN CHIEF !!!!

fake news story on Facebook, the

BuzzFeed News
NO Hillary for 2016 BuzzFeed Politics
TRUMP - SPEAK OUT AGAINST ISLAMIZATION BuzzFeed UK News
OF AMERICA - FIM DA COLONIZAO ISL BuzzFeed World

ROCK Mundial
Trump USA's CEO Isaac Saul
StyleHaul
TrumpNation The European Journalism Observatory | EJO
FAKE STORY INSTANCES BuzzFeed
VOTE TRUMP ONLY - THE AMERICAN PARTY RISING Coloradans For Bernie

different phases of its life cycle

Ethical Journalism Network
Earliest version (original) Hillary Clinton The People's Choice

Reposted Version

The Church of PUNK

CIVILIZED.LIFE
ATRENDING.NET
Debunked version civilized.life
Exposing The Truth

as well as key moments and

POSTS ON FACEBOOK
INQUISITR.COM
QATARDAY.COM
Post Page that shares it theinquisitr
Qatar Day

Radioactivo 98.5
PAKISTAN RISING
POLITICO.COM

intermediaries associated with its

Interactions DIYHILFE.COM
JimHeathTV
Mobbing Hilfe LionPublishers INEWS.CO.UK
mnsucollegedemocrats
Fake Story modernliberals theipaper
PolPreposterous
Debunk
HOPEFORNIGERIAONLINE.COM politico TRUEAMERICANS.ME

Los Angeles Punk Museum

dissemination.
POSTS INTERACTIONS Hope for Nigeria pg/politicomag
President Donald J. Trump
TRUMP - SPEAK OUT AGAINST [...]
politicomediadesk
0 1588
PENamerican
EDUBLOG.SCHOLASTIC.COM
LUXURYANDGLAMOR.COM DavidWilliamBateman
EmmaClarkLibrary
POSTS ON A SAME PAGE Daily Video NOWTORONTO.COM

Veterans Against Rep. Ignorance

No sharing on Facebook from mid November to late December nowmagazine
2017
NOV 13 NOV 20 NOV 27 DEC 04 DEC 11 DEC 18 DEC 25 JAN 01 JAN 08 JAN 15 JAN 22 JAN 29 FEB 05 FEB 12 FEB 19 FEB 26 MAR 05 MAR 12

FU Trump

Occupy Orange County CA

TOBER NOVEMBER DECEMBER

48 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 49
 START
CHAPTER 1 RECIPE 3
List of 22 URLs of political
fake news stories
Source: BuzzFeed News

DO FACT-CHECKING
INITIATIVES REACH THE
query
Fake News Title 1 + Fake
Fake News Title 2 + Fake
. . . in

PUBLICS OF FAKE NEWS Google Web Search

ON FACEBOOK?
Retain the top ranked URL
of a debunk per fake news story

...

merge all URLs in a single list

BEFORE STARTING
This recipe takes as a starting point a list of input URLs in
fake news stories. There are different ways CrowdTangle
of obtaining these lists including starting
with existing lists as well as creating your
own. To illustrate this recipe we use an
output data
already existing list of 22 fake news stories > URLs of fake news story
or debunked version thereof
about various political issues pertaining to > Facebook pages and groups
the 2016 presidential elections in the US that that share either the fake
news story or its correction
generated most engagement on Facebook. > number of followers
per page or group
These were identified by BuzzFeed News.
input data to
There are two steps to this recipe. The first
is to identify URLs that circulate corrections RAWGraphs
or debunking web pages for each fake
news story. The second is to explore how
Facebook groups engage with both fake news visualise ARE DEBUNKED VERSIONS
stories and their corresponding debunking OF FAKE NEWS STORIES
web pages (b). ACKNOWLEDGED BY THE
PUBLICS OF FAKE NEWS?
50 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS
ON FACEBOOK 51
 START

List of 22 URLs of political

fake news stories
Source: BuzzFeed News

query
Fake News Title 1 + Fake CHAPTER 1 RECIPE 3
Fake News Title 2 + Fake
. . . in
a. IDENTIFY WEB PAGES WHICH AIM TO
Google Web Search DEBUNK FAKE NEWS STORIES
To identify prominent debunking web pages
for a given fake news story you may use the
Retain the top ranked URL
of a debunk per fake news story Google Web Search engine. Alternatively,
you may also query fact-checking sites for
...
keywords describing a fake news story.
In order to find corrections of fake news
articles queries need to be designed for
each fake news item in your list. One
strategy would be to use the title of the
story in quotation marks followed by the
word fake (e.g. Trump Offering Free
One-Way Tickets to Africa & Mexico for
Those Who Wanna Leave America, fake).
You may use the search engine ranking
as an indication of salience of correction
and select the highest ranked URLs
corresponding to a corrected version of the
fake news story in question.
The result of this step is a list of URLs
containing the most highly ranked
debunking web pages per fake news story.

52 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS

ON FACEBOOK 53
 START

List of 22 URLs of political

fake news stories
Source: BuzzFeed News

query
Fake News Title 1 + Fake CHAPTER 1 RECIPE 3
Fake News Title 2 + Fake
. . . in
b. MAP THE OVERLAP BETWEEN THE
Google Web Search PUBLICS OF FAKE NEWS STORIES AND
WEB PAGES WHICH AIM TO DEBUNK
THEM
Retain the top ranked URL
of a debunk per fake news story
Facebook pages and groups that share both fake
news stories as well as web pages which aim to
... debunk them may be detected through a social
media monitoring tool such as CrowdTangles
browser extension.
merge all URLs in a single list
To explore whether the debunking web
+
pages are acknowledged by the publics
which share the fake news stories, identify
input URLs in
whether there is an overlap between the
Facebook pages and groups that share fake
CrowdTangle
news stories and those debunking web
pages issued in responses.
This may be illustrated by means of a
output data
> URLs of fake news story circle packing visualisation. You may use
or debunked version thereof
> Facebook pages and groups RawGraph for this operation.
that share either the fake
news story or its correction
> number of followers
per page or group

input data to

RAWGraphs

visualise ARE DEBUNKED VERSIONS

OF FAKE NEWS STORIES
ACKNOWLEDGED BY THE
PUBLICS OF FAKE NEWS?
54 A FIELD GUIDE TO FAKE NEWS CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS
ON FACEBOOK 55
ARE DEBUNKING
WEB PAGES
ACKNOWLEDGED
BY THE PUBLICS OF
FAKE NEWS?
Fake news pages and debunking
web pages have different publics
on Facebook. Only six of the pages
that share fake news stories have
acknowledged web pages which aim
to debunk them. While Google looks
to prioritise debunking web pages, on
Facebook it is fake news stories that
circulate better. While both progressive
and conservative pages share fake
news stories it is primarily progressive
Facebook pages and those pertaining
to journalists and fact-checking
initiatives that share web pages which
aim to debunk fake news stories.
Page or group which shares both
fake story and its correction
Fake
Debunk
POST INTERACTIONS
100 41271
56
ON FACEBOOK
Rage Against the Machine To Reunite
And Release Anti Donald Trump Album
Pubished the 11th of March 2016
on heaviermetal.net
Rupaul claims Trump touched
him inappropriately in the 1990s
Pubished the 14th of October 2016
on worldnewsdailyreport.com
D.L. Hughley
Hillary Clinton In 2013: I Would Like
To See People Like Donald Trump Run
For Office; Theyre Honest And Cant
Be Bought
Pubished the 17th of October 2016
on conservativestate.com
Obama Signs Executive Order
Declaring Investigation Into Election
Results; Revote Planned For Dec. 19th
Pubished the 12th of December 2016
on www.abcnews.com.co
Hillary Clinton
for President in 2016
Connecticut
Progressives
Actor Bill Murray Announces
2016 Presidential Run
Pubished the 3th of October 2016
on www.abcnews.com.co
African Billionaire Will Give $1
Million To Anyone Who Wants
To Leave America if Donald Trump
is Elected President
Pubished the 3th of November 2016
on www.empireherald.com
Graham Says Christians
Must Support Trump or Face
Death Camps
Pubished the 23th of July 2016
on www.bizstandardnews.com
Pope Francis Shocks World,
Endorses Hillary Clinton for
President, Releases Statement
Pubished the 25th of July 2016
on www.kypo6.com
Read the damn news
A FIELD GUIDE TO FAKE NEWS
57
Mike Pence: Sarah Palin Is
My Role Model For Beautiful,
Smart American Women
Pubished the 18th of September 2016
on www.politicops.com
BREAKING Romanian Hacker
With Access To Clinton Emails
Found Dead In Jail Cell
Pubished the 6th of July 2016
on Christian Times
Bestwebtools
Anonymous Portugal
Internacional
Trump Claims America
Should Never Have Given
Canada Its Independence
Pubished the 29th of July 2016
on www.burrardstreetjournal.com
Pope Francis Shocks World,
Endorses Donald Trump for
President, Releases Statement
Pubished the 26th of September 2016
on Ending The Fed
CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS
President Obama Confirms
He Will Refuse To Leave Office
If Trump Is Elected
Pubished the 7th of September 2016
on www.burrardstreetjournal.com
ISIS Leader Calls for American
Muslim Voters to Support
Hillary Clinton
Pubished the 11th of October 2016
on worldnewsdailyreport.com
Van Full Of Illegals Shows Up To
Vote Clinton At SIX Polling Places,
Still Think Voter Fraud Is A Myth?
Pubished the 5th of November 2016
on www.thelastlineofdefense.org
Obama Signs Executive Order
Banning The Pledge Of Allegiance
In Schools Nationwide
Pubished the 11th of December 2016
on www.abcnews.com.co
Trump Offering Free One-Way
Tickets to Africa; Mexico for Those
Who Wanna Leave America
Pubished the 11th of November 2016
on www.tmzhiphop.com
Obama passed law for grandparents
to get all their grandchildren
every weekend
Pubished the 8th of March 2016
on www.react365.com
WHOA! Hillary Caught On
Hot Mic Trashing Beyonce
With RACIAL SLURS!
Pubished the 5th of November 2016
on www.thelastlineofdefense.org
FBI Agent Suspected in Hillary
Email Leaks Found Dead
in Apparent Murder-Suicide
Pubished the 15th of October 2016
on www.usanewsflash.com
Donald Trump Protester
Speaks Out: I Was Paid $3,500
To Protest Trump's Rally
Pubished the 22th of November 2016
on www.abcnews.com.co
Criminalize
Conservatism
PolitiFact
 Chapter #2

1.3.A
Rage Against the Machine To Reunite
And Release Anti Donald Trump Album
Pubished the 11th of March 2016
on heaviermetal.net
Actor Bill Murray Announces
2016 Presidential Run
Pubished the 3th of October 2016
Mike Pence: Sarah Palin Is
My Role Model For Beautiful,
Smart American Women
on www.abcnews.com.co Pubished the 18th of September 2016
on www.politicops.com
President Obama Confirms
He Will Refuse To Leave Office
If Trump Is Elected
Pubished the 7th of September 2016
on www.burrardstreetjournal.com
TRACING THE
Trump Offering Free One-Way
Tickets to Africa; Mexico for Those
Who Wanna Leave America
CIRCULATION OF FAKE
NEWS ON THE WEB
Pubished the 11th of November 2016
on www.tmzhiphop.com

CHAPTER 1 RECIPE 3
Obama passed law for grandparents
to get all their grandchildren
every weekend
Pubished the 8th of March 2016
on www.react365.com

Rupaul claims Trump touched
him inappropriately in the 1990s
Pubished the 14th of October 2016
on worldnewsdailyreport.com
D.L. Hughley
African Billionaire Will Give $1 BREAKING Romanian Hacker ISIS Leader Calls for American WHOA! Hillary Caught On
Million To Anyone Who Wants With Access To Clinton Emails Muslim Voters to Support Hot Mic Trashing Beyonce
To Leave America if Donald Trump Found Dead In Jail Cell Hillary Clinton With RACIAL SLURS!
is Elected President Pubished the 6th of July 2016 Pubished the 11th of October 2016 Pubished the 5th of November 2016
Pubished the 3th of November 2016 on Christian Times on worldnewsdailyreport.com on www.thelastlineofdefense.org
on www.empireherald.com
Bestwebtools
Anonymous Portugal
Internacional

Hillary Clinton In 2013: I Would Like

To See People Like Donald Trump Run
For Office; Theyre Honest And Cant
Be Bought
Pubished the 17th of October 2016
on conservativestate.com
Graham Says Christians
Must Support Trump or Face
Death Camps
Trump Claims America
Should Never Have Given
Canada Its Independence
Van Full Of Illegals Shows Up To
Vote Clinton At SIX Polling Places,
Still Think Voter Fraud Is A Myth?
FBI Agent Suspected in Hillary
Email Leaks Found Dead
in Apparent Murder-Suicide
Where do fake news stories originate?
By which sites are they first disseminated?
Pubished the 23th of July 2016 Pubished the 29th of July 2016 Pubished the 5th of November 2016 Pubished the 15th of October 2016
on www.bizstandardnews.com on www.burrardstreetjournal.com on www.thelastlineofdefense.org on www.usanewsflash.com

Page or group which shares both

fake story and its correction

Which are the most visible sources

Obama Signs Executive Order
Fake Declaring Investigation Into Election
Results; Revote Planned For Dec. 19th
Debunk Pubished the 12th of December 2016
on www.abcnews.com.co
Pope Francis Shocks World, Pope Francis Shocks World, Obama Signs Executive Order Donald Trump Protester
Endorses Hillary Clinton for Endorses Donald Trump for Banning The Pledge Of Allegiance Speaks Out: I Was Paid $3,500
President, Releases Statement President, Releases Statement In Schools Nationwide To Protest Trump's Rally
Pubished the 25th of July 2016 Pubished the 26th of September 2016 Pubished the 11th of December 2016 Pubished the 22th of November 2016
on www.kypo6.com on Ending The Fed on www.abcnews.com.co on www.abcnews.com.co

related to a fake story? When and by

Hillary Clinton
POST INTERACTIONS Criminalize
for President in 2016
Conservatism

whom are they mentioned?

100 41271 PolitiFact
Connecticut
Progressives

Read the damn news

SERVING SUGGESTIONS
This recipe may be used as
one way to assess the impact
of attempts to debunk fake
news by examining whether
debunking responses to fake
news are acknowledged on
the platform that generates
most engagement with fake
news, Facebook, and by the
particular publics which share
and engage with fake news.

58 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 59
Introduction - Fake news are not just false
news. They are interesting not so much
because their content or form are different
from that of authentic news, but because they
travel as much as (and sometimes more than)
mainstream news. If a blog claims that Pope
Francis endorses Donald Trump, it's just a lie. If
the story is picked up by dozens of other blogs,
retransmitted by hundreds of websites, cross-
posted over thousands of social media accounts
and read by hundreds of thousands, then it
becomes fake news.
The following recipes investigate the
circulation of fake news for two reasons. Firstly,
from a political point of view many have
expressed disappointment that techniques and
tactics commonly used to tackle fake news have
not lived up to expectations. Fact-checking and
debunking, in particular, often do not succeed
in preventing the circulation of hoaxes and
rumors. On the contrary: they can inadvertently
60 A FIELD GUIDE TO FAKE NEWS
contribute to making them even more visible
on the web. A better understanding of how fake
news travels online can help to inform responses
that are more attuned to the phenomena. From
a methodological point of view, as there is
no ontological difference between fake and
authentic news, studying fake news circulation
can help us understand more about how other
kinds of news travels.
This recipe comes in two flavours. Firstly,
we propose a manual variant which can be
executed without the need for any particular tool
or technical knowledge. This variant is easy to
execute but also time consuming. It is based on
a search for web pages referring to fake news
stories and on the manual identification of the
dates of their publication and the sources that
they cite. Secondly, we propose a semi-automated
version which allows this approach to be scaled
to more pages, but demands more technical skills
and may require more manual verification.
CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 61
 START
open Chrome browser
in incognito mode
CHAPTER 2 RECIPE 1

query

WHERE DO FAKE [ Pope AND endors* AND (Trump OR Clinton)

AND NOT (hoax OR fake news OR lie OR debunk) ]

Google Web Search

NEWS ORIGINATE?
a
BY WHAT SITES ARE THEY compile a list with
the top resulting URLs
input URLs in a spreadsheet

FIRST RETRANSMITTED?
> The page advocating
or debunking the fake
news item
> Rank in the search
engine results
> the date of publication
of each of the page

record all sources cited in the appearances

of your story, noting down:
+ > if they are cited as evidence
+ or counter-evidence
> if they are cited through
+ hyperlink, textual reference
BEFORE STARTING or copying/pasting of its
content
For this recipe you will need to choose a fake
news story whose circulation youd like to
Make sure you visit both the results of
trace. The more distinctive your story is, the your initial search and all the sources
cited by those results
easier it will be to follow its circulation. In b
the example, we focus on the Pope Endorses
HOW DO THE
Trump story that was widely circulated extract network of instances
OCCURRENCES OF THE
around the US Elections. and references with import data in
POPE ENDORSES

visualise
Table2Net Gephi
TRUMP STORY CITES
EACH OTHER?

Plot URLs on a timeline with

c
Graph Recipes
WHAT IS THE LIFE OF
THE POPE ENDORSES
TRUMP STORY

visualise
ACCORDING TO PAGES
IN SEARCH ENGINE
RESULTS?
62 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 63
START
open Chrome browser
in incognito mode
CHAPTER 2 RECIPE 1

a IDENTIFY AND ANALYSE THE

OCCURRENCES OF YOUR STORY IN SEARCH
query
[ Pope AND endors* AND (Trump OR Clinton) ENGINE RESULTS
AND NOT (hoax OR fake news OR lie OR debunk) ]
Successful fake news stories always appear on
Google Web Search
several web pages. In the first step of this recipe,
you will identify and collect information about these
a
occurrences.
compile a list with input URLs in a spreadsheet
the top resulting URLs > The page advocating Identify the occurrences of your story by
or debunking the fake
news item querying one or more search engines (we used
> Rank in the search
engine results Google Web Search). Since fake news stories
> the date of publication
of each of the page
evolve while circulating, consider keywords that
may capture different variants of the story.
Rely on search engines to rank results by
relevance and concentrate on the first results
(under the working assumption that they are the
ones that circulated the most).
To avoid "filter bubbles" and personalised
results, consider using a dedicated research
browser.[1] [1]
See instructions
Be aware that search engines may give more on how to set up a
research browser
visibility to debunkers than to original sources in this video tuto-
and be careful not to overestimate the rial: https://www.
youtube.com/
circulation of debunkers based on this. Also watch?v=bj65X-
bear in mind that with this approach you see r9GkJM

the phenomena through the eyes of the search

engine that you selected which will become a
part of your story or research.
Record all relevant metadata for each relevant
search result. You can collect as many variables
as you like, but make sure to characterise how
the page refers to the story (e.g. as a reliable
news source, or as a problematic claim to be
debunked), as well as some indicator of its
visibility (e.g. the rank in the search results).

64 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 65
START
open Chrome browser
in incognito mode

query
[ Pope AND endors* AND (Trump OR Clinton)
AND NOT (hoax OR fake news OR lie OR debunk) ]

Google Web Search

CHAPTER 2 RECIPE 1

b. EXTRACT THE NETWORK OF

compile a list with
the top resulting URLs
record all sources cited in the appearances
of your story, noting down:
+ > if they are cited as evidence
+ or counter-evidence
+ > if they are cited through
hyperlink, textual reference
or copying/pasting of its
content
Make sure you visit both the results of
your initial search and all the sources
cited by those results
extract network of instances
and references with import data in
visualise
a
REFERENCES AROUND YOUR STORIES
input URLs in a spreadsheet
> The page advocating Fake news is supported or opposed through
or debunking the fake
news item a network of references: websites that share
> Rank in the search
engine results rumours cite other pages to support their
> the date of publication
of each of the page claims, while debunking initiatives flag toxic
websites or refer to sources denying them.
In this step, we will trace the network of
references of a specific fake news story.
Record all sources cited in the occurrences
of your story. For each, note down if the
source is cited as evidence or counter-
evidence and if it is cited through a
hyperlink (e.g. http://snopes.com), a
textual reference (e.g. the website WTOE
5 News) or copying and pasting its
b
content.
HOW DO THE
Make sure you visit not only all the results
OCCURRENCES OF THE
of your initial search, but also all the
POPE ENDORSES

Table2Net Gephi sources cited by those results.

TRUMP STORY CITES
EACH OTHER? Extract the networks of occurrences and
references (you can use Table2Net).
Visualise the network (using Gephi, for
instance), applying a force-directed layout;
sizing the nodes according to the number
of citations they receive; and colouring the
nodes according to how they report the
story (advocating or debunking).

66 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 67
HOW DO THE europe.newsweek.com/fake-...

OCCURRENCES OF rednationrising
complex

THE POPE ENDORSES uk.complex.com/life/2016/...

TRUMP STORY CITES trend.network/2016/09/28/...

reddit.com/r/thedonald/co...

EACH OTHER? mercurynews.com/2016/12/1...

endingthefed
Network of cross-references between commonwealmagazine.org/bl...
the pages mentioning the Pope southendnewsnetwork.com/n...
Endorses Trump story. In the image
the nodes represent the different pages thebassbarn.com/forum/12-...
godlikeproductions.com/fo...
on which the fake story appears. The cnbc.com/2016/12/30/read-...
comparison of the colour of the nodes msn
heavy.com/news/2016/11/fa...
(which indicates whether the page cites uk.businessinsider.com/fa...
nymag.com/selectall/2016/... groups.google.com/forum/#...
or debunks the story) and their size quora
theguardian.com/commentis... buzzfeed_partisan
(which indicates the number of citations
catholicnews
received by the page) reveals the greater enh latimes
visibility of the debunking and neutral nytimes.com/2016/11/13/op...
raptureforums.com/forums/... foxnews.com/world/2016/12...
pages compared to websites that spread snopes
linkedin.com/pulse/pope-f...
the fake story as authentic earnthenecklace.com/fact-...
nytimes.com/2016/11/13/op...
gvnews.com/opinion/factch...
niemanlab.org/2016/11/the...
wtoe5 dinarvets.com/forums/inde... guardian
morningnewsusa.com/pope-f... erumors
theglobeandmail.com/news/...
justplainpolitics.com/sho...
newsbreakshere.com/pope-f...
buzzfeed_strangest vox.com/new-money/2016/11...
sconews.co.uk/news/52118/...
newstatesman.com/world/20...
independent.co.uk/news/pe...
thejournal.ie/fake-news-f... hoaxslayer truthorfiction.com/pope-f... buzzfeed_macedonia
hackinghumanity.com/uncat...
wp
buzzfeed dailydot.com/unclick/face...
NUMBER OF MENTIONS thatsfake.com/pope-franci...

1 18
reuters.com/article/us-us...

nbcnews.com/news/us-news/...
slate.com/blogs/future_te...
wtoe5_about
cathola.com/2017/02/19/po...
factcheck.org/2016/10/did... guardian_pope
apnews.com/86073aedf0c245...
wgal.com/article/did-the-...
Fake News
cbsnews.com/news/pope-fra...
buzzfeed_viralfake
wgal
Debunk
msn.com/en-us/news/crime/... thedailybeast.com/article...
bigstory.ap.org/article/5...
Neutral
kypo6
youtube.com/watch?v=8pzfw...

68 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 69
START
open Chrome browser
in incognito mode

query
[ Pope AND endors* AND (Trump OR Clinton)
AND NOT (hoax OR fake news OR lie OR debunk) ]

Google Web Search

CHAPTER 2 RECIPE 1

c. VISUALIZE THE FAKE NEWS INSTANCES

a
OVER TIME
compile a list with input URLs in a spreadsheet
the top resulting URLs > The page advocating The network extracted in the previous step
or debunking the fake
news item can help you understanding not only who cited
> Rank in the search
engine results whom, but also how and in which direction your
> the date of publication
of each of the page fake news travelled. To reveal the circulation use
the dates that you collected in the first step of this
recipe.
record all sources cited in the appearances
of your story, noting down: Arrange the network of instances extracted
+ > if they are cited as evidence in the previous step chronologically. You can
+ or counter-evidence
+ > if they are cited through use different visual styles to represent the
hyperlink, textual reference
or copying/pasting of its different kind of citations (to do so, we used
content
a custom script of Graph Recipes tools.

Make sure you visit both the results of

your initial search and all the sources
cited by those results
b
HOW DO THE
extract network of instances
and references with import data in
OCCURRENCES OF THE
POPE ENDORSES
visualise

Table2Net Gephi
TRUMP STORY CITES
EACH OTHER?

Plot URLs on a timeline with

c
Graph Recipes
WHAT IS THE LIFE OF
THE POPE ENDORSES
TRUMP STORY
visualise

ACCORDING TO PAGES
IN SEARCH ENGINE
RESULTS?
70 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 71
WHAT IS THE LIFE OF
THE POPE ENDORSES
TRUMP STORY
Buzzfeed Thejournal
Theguardian Vox

ACCORDING TO PAGES
Truthorfiction Dinarvets

Godlikeproductions
Washingtonpost Truthorfiction

IN SEARCH ENGINE Thedailybeast

Catholic-news
Linkedin
Groups

RESULTS?
Commonwealmagazine Earnthenecklace
Wtoe5news Justplainpolitics
Newsbreakshere
Thebassbarn
Snopes
Chronological network of the cross- Nymag
Niemanlab
references between the pages Nytimes
Nytimes
mentioning the Pope Endorses Uk
Morningnewsusa
Trump story. In this image it is not Theguardian

the nodes, but their connections that Hoax-slayer Buzzfeed Hackinghumanity

Thatsfake
are highlighted: their colour indicates Gvnews Independent
Newstatesman
whether a source is cited as evidence Sconews Cnbc
Everynewshere Raptureforums
or counter-evidence of the story and Kypo6 Msn
Youtube
the style the type of citation. The Trend Reddit
high presence of orange dotted lines Twitter Europe

(counter-evidence textual citations) Endingthefed

shows how debunking initiatives tend to Quora

Latimes
mention original sources but not link to
Wgal
them. This technique is used to flag fake
Buzzfeed Heavy
websites without increasing their online
visibility by explicitly linking to them. Msn

Factcheck

Slate

source in original list

Buzzfeed Dailydot
source identified through
the analysis
Complex Uk
CITED
SOURCE
SOURCE Buzzfeed Bigstory
No sharing of the story URL from March to July
Nbcnews
TYPE OF CITATION
Apnews
counter-evidence Foxnews
cited as hyperlink
Theglobeandmail
counter-evidence Cbsnews
cited as text
Southendnewsnetwork
evidence cited
as hyperlink
Mercurynews
evidence cited
as text Reuters
evidence cited
as copy-paste Theguardian Cathola

GOOGLE RANK
WITHOUT DATE JANUARY FEBRUARY MARCH APRIL MAY JUNE JULY AUGUST SEPTEMBER OCTOBER NOVEMBER DECEMBER JANUARY FEBRUARY
1 100
2016 2017

72 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 73
 mercurynews.com/2016/12/1...

endingthefed
commonwealmagazine.org/bl...

southendnewsnetwork.com/n...

/12-...
godlikeproductions.com/fo...
nbc.com/2016/12/30/read-...
heavy.com/news/2016/11/fa...
msn
insider.com/fa...
groups.google.com/forum/#...
quora
buzzfeed_partisan
catholicnews CHAPTER 2 RECIPE 1
latimes

foxnews.com/world/2016/12...
Buzzfeed
linkedin.com/pulse/pope-f... Thejournal
SERVING SUGGESTIONS
Theguardian Vox

nthenecklace.com/fact-...
This recipe may be used to repurpose data
Truthorfiction Dinarvets

Godlikeproductions
Washingtonpost Truthorfiction

gvnews.com/opinion/factch... obtained through search engine results in

Linkedin
Thedailybeast
Catholic-news Groups
Commonwealmagazine
wtoe5
Earnthenecklace

dinarvets.com/forums/inde...
Justplainpolitics
guardian
order to identify and follow the different
Wtoe5news Newsbreakshere
Thebassbarn

/pope-f... erumors
Snopes
Nymag

occurrences of a given fake news story

Niemanlab

stplainpolitics.com/sho...
Nytimes
Nytimes

newsbreakshere.com/pope-f... Uk

as it is cited and referenced by different

Morningnewsusa
Theguardian

est vox.com/new-money/2016/11...
Hoax-slayer Buzzfeed Hackinghumanity

online sources, as well as to retrace its

Thatsfake

sconews.co.uk/news/52118/... Gvnews Independent

Newstatesman
Sconews Cnbc
newstatesman.com/world/20... Everynewshere

circulation over time. You will also be able

Raptureforums
independent.co.uk/news/pe... Kypo6 Msn
Youtube

hoaxslayer
Trend
truthorfiction.com/pope-f...
Reddit

Endingthefed
buzzfeed_macedonia
Twitter Europe
to see when the debunking activities took
place, who promoted them and what effect
Quora

wp
Latimes

dailydot.com/unclick/face...
Wgal

thatsfake.com/pope-franci...
Buzzfeed

Msn
Heavy
this had on the circulation of fake news
Factcheck stories and web pages.
Slate

nbcnews.com/news/us-news/...
Buzzfeed Dailydot

Complex Uk

Buzzfeed Bigstory
No sharing of the story URL from March to July
Nbcnews
cathola.com/2017/02/19/po... Apnews

guardian_pope
Foxnews
Theglobeandmail
Cbsnews

Southendnewsnetwork

com/article/did-the-... Mercurynews

Reuters

buzzfeed_viralfake
wgal
Theguardian Cathola

/en-us/news/crime/... thedailybeast.com/article...
bigstory.ap.org/article/5...
WITHOUT DATE JANUARY FEBRUARY MARCH APRIL MAY JUNE JULY AUGUST SEPTEMBER OCTOBER NOVEMBER DECEMBER JANUARY FEBRUARY

o6 2016 2017

youtube.com/watch?v=8pzfw...

74 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 75
 START

CHAPTER 2 RECIPE 2

find a query identifying

WHICH ARE THE MOST

your fake news story

VISIBLE SOURCES identify the occurrences

of your story in
compile a list of potential sources

websites collected

RELATED TO A FAKE STORY?

and curated
Radarly by the project
Dcodex by Le Monde

WHEN AND BY WHOM export all data

ARE THEY MENTIONED?

> Full text of each crawl the websites with
occurrence
> Date of publication Hyphe
> Metadata

search mentions export a graph file

of potential sources
in the instances of your story it represents
the general
Csv Rinse Repeat connectivity
among potential
BEFORE STARTING sources

This recipe enables a scaling up of the approach

presented in the previous recipe, but requires a export a graph file
create a basemap
network with
bit more technical knowledge, as well as some
bigger datasets. In particular, you will need to Gephi
have access to:
A web archive (we used Radarly by
Project the instances
Linkfluence). of your story
visualise

b on the basemap
A list of all the possible web sources in
which your chosen fake news story may WHICH ARE THE visualise
a
have appeared (we used the list curated by SOURCES CITED IN THE
OCCURRENCES OF THE WHAT ARE THE
Le Monde Dcodex). produce time-sliced
FAKE NEWS STORY? networks using MAIN SPHERES
Graph Recipes IN THE FRENCH
c
MEDIA SYSTEM?
HOW MANY OCCURRENCES
OF THE FAKE NEWS ARE visualise

PUBLISHED IN EACH PERIOD

OF TIME AND WHAT
SOURCES DO THEY CITE?
76 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 77
 START

compile a list of potential sources CHAPTER 2 RECIPE 2

websites collected
and curated a DEFINE A BASE MAP OF NEWS
by the project
Dcodex by Le Monde
PROVIDERS
Identify (or compile) a list of all the possible
Web sources in which you think your fake
crawl the websites with news item might have appeared (try to be as
Hyphe exhaustive as possible). You can use one of the
many lists of fake news websites maintained by
debunking initiatives and combine it with a list
export a graph file of mainstream media outlets.
it represents
the general Identify how the sources in your list are
connectivity
among potential associated with each other through web
sources
crawling and hyperlink analysis. We used
Hyphe for this.
create a basemap Visualise the resulting network and apply a
network with
force-directed layout algorithm to identify
Gephi clusters of sources. You can use Gephi
for this task.
Manually highlight and name the clusters.
visualise

a
WHAT ARE THE
MAIN SPHERES
IN THE FRENCH
MEDIA SYSTEM?

78 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 79
WHAT ARE THE MAIN
SPHERES IN THE FRENCH
MEDIA SYSTEM?
Network analysis of the media
sources active in French public SOCIAL MEDIA AND OTHERS
debate. The image shows the news GROUPE CENTRE-FRANCE
La Rpublique du Centre
sources listed by the Dcodex project L'Yonne rpublicaine
SATIRICAL WEBSITES
by Le Monde and the hyperlinks Snapchat

connecting them. A force-directed

layout has been applied to reveal the
GOSSIP MEDIA
Wikihow INTERNATIONAL MEDIA
main clusters of websites and their
Instagram
respective associations and positions. US MEDIA
Facebook
Tumblr
Vice
Mashable
Reddit
Version Femina
LOCAL MEDIA Canal+ The Washington Post
Twitter
EAST FRANCE MEDIA Bloomberg
L'Est rpublicain
The Guardian The New York Times
Wanted Pedo
YouTube
Al Kanz BuzzFeed
Le Progrs
Paris Match The Independent
L'Alsace-Le Pays
Dailymotion UK MEDIA

Change.org The Daily Telegraph

France Inter Russia Today RUSSIAN MEDIA

Ouest-France Europe 1 Les Echos Conscience du peuple
RTL
Wikipdia
Mes Opinions Debunkers de hoax Les Chroniques de Rorschach
AFP

Le Figaro Slate
BFMTV L'Express Lablogalupus
Le Monde alalumieredunouveaumonde

Blog du"Plan C", par Etienne Chouard Stop Mensonges

Franceinfo
Libration
Le6emetiroir
Huffington Post
EXTREME-RIGHT MEDIAen France
Le Parisien-Aujourdhui Reopen911 Rvolution Vibratoire
Le Point Les Crises
Arrt sur images
Novopress Les Moutons enrags
Marianne
Cercle des Volontaires IlFattoQuotidiano.fr
Contre-Info L'Obs
Les Brins d'Herbes
Link between
providers Agoravox

Egalit et Rconciliation

Provider

CONSPIRACY THEORISTS
NUMBER OF CONNECTIONS MAINSTREAM FRENCH MEDIA

80 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 81
 CHAPTER 2 RECIPE 2
START
b. HIGHLIGHT THE OCCURRENCES OF YOUR
STORY ON THE BASE MAP
In this step we will explore how fake news stories
find a query identifying
your fake news story are associated with different sources. This is
interesting, as while a fake news story might
for example start out its life as a piece of satire,
identify the occurrences compile a list of potential sources
of your story in
websites collected
as it travels it can become more prominently
Radarly
and curated associated with non-satirical sources. Here we
by the project
Dcodex by Le Monde will identify which of the sources in the base map
of the French media system are mentioned in the
export all data
pages in which your fake news story occurs.
> Full text of each crawl the websites with
occurrence Create a query that identifies the fake news
> Date of publication Hyphe story that you want to trace. Use keywords
> Metadata
specifically associated with your story and
the stop-words to exclude "false positives".
search mentions export a graph file
of potential sources
in the instances of your story it represents Identify the occurrences of your story,
the general
running your query on the archive that
Csv Rinse Repeat connectivity
among potential
sources
you have chosen to use. For each of the
results, collect the full text and the date of
publication.
export a graph file
create a basemap Detect mentions of the sources of your base
network with
map, searching for the URLs as well as for
Gephi
the names of your sources (e.g. sputniknews.
com, Sputnik). In our example we used a
custom script for CSV Rinse Repeat.
Project the instances
of your story Project the occurrences of your story onto
b on the basemap
your base map, by merging them with the
WHICH ARE THE visualise source network and connecting each of them
SOURCES CITED IN THE to the sources that they mention. While
OCCURRENCES OF THE keeping the source-nodes fixed, apply a force
FAKE NEWS STORY? directed spatialisation algorithm (you can
do this using Gephi) to move the nodes
representing the fake story occurrences
closer to clusters of the base map that they
cite the most.

82 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 83
WHICH ARE THE
SOURCES CITED IN THE
OCCURRENCES OF THE
FAKE NEWS STORY?
Projection of the fake news
occurrences on the network of media La Rpublique du Centre
L'Yonne rpublicaine
sources. In this image, the occurrences Snapchat
of the fake news story are positioned on
the base map displayed by the previous
network according to the sources they Wikihow

cite. The tendency to refer to social Instagram

media is visible as well as the relevance Facebook

Tumblr
Vice
of Russia Today and Sputnik Mashable
Reddit
International in this particular story. Version Femina

Canal+ The Washington Post

Twitter
Bloomberg
L'Est rpublicain
The Guardian The New York Times
Wanted Pedo
YouTube
Al Kanz BuzzFeed
Le Progrs
Paris Match The Independent
L'Alsace-Le Pays
Dailymotion

Change.org The Daily Telegraph

France Inter Russia Today

Ouest-France Europe 1 Les Echos Conscience du peuple
RTL
Wikipdia
Mes Opinions Debunkers de hoax Les Chroniques de Rorschach
AFP

Le Figaro Slate
BFMTV L'Express Lablogalupus
Le Monde alalumieredunouveaumonde

Blog du"Plan C", par Etienne Chouard Stop Mensonges

Franceinfo
Libration
Le6emetiroir
Huffington Post
TRACKER Link between provider
POPULARITY Le Parisien-Aujourdhui en France Reopen911 Rvolution Vibratoire
and fake news article Le Point Les Crises
Arrt sur images
Novopress Les Moutons enrags
Link between Marianne
providers Cercle des Volontaires IlFattoQuotidiano.fr
Contre-Info L'Obs
Les Brins d'Herbes
Agoravox

Provider
Egalit et Rconciliation

Fake News article

NUMBER OF CONNECTIONS

84 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 85
 START

find a query identifying

your fake news story

identify the occurrences compile a list of potential sources

CHAPTER 2 RECIPE 2
of your story in
websites collected
and curated c. VISUALISE THE SPREAD OF YOUR FAKE
Radarly by the project
Dcodex by Le Monde STORY ON THE BASE MAP
In this step, you will reveal how the reference
export all data patterns identified in the previous step evolve
crawl the websites with
> Full text of each
occurrence
over time.
> Date of publication Hyphe
> Metadata Slice your network of occurrences and
references by month, by week or by day,
search mentions
according to the speed of circulation of
export a graph file
of potential sources
it represents
your story. In this example we grouped
in the instances of your story
the general news by month and then zoomed in on
Csv Rinse Repeat connectivity
among potential a four day window to explore the most
sources
important period of circulation.
While keeping the source base map stable,
export a graph file
create a basemap visualise the different temporal slices of
network with
fake news story occurrences.
Gephi
In order to make the changes and patterns
more legible, you can represent the fake
story occurrences not as single nodes, but
Project the instances
of your story through a density heatmap (the example
on the basemap
has been produced using a Graph
Recipes).

produce time-sliced
networks using

Graph Recipes
c
HOW MANY OCCURRENCES
OF THE FAKE NEWS ARE visualise

PUBLISHED IN EACH PERIOD

OF TIME AND WHAT
SOURCES DO THEY CITE?
86 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 87
HOW MANY
OCCURRENCES OF THE
FAKE NEWS STORY
ARE PUBLISHED IN
EACH PERIOD AND
WHAT SOURCES DO
14 MAR 2016 - 11 APR 2016 28 MAR 2016 - 25 APR 2016 11 APR 2016 - 9 MAY 2016 25 APR 2016 - 23 MAY 2016 9 MAY 2016 - 6 JUN 2016 23 MAY 2016 - 20 JUN 2016

THEY CITE?
Temporal evolution of the fake news
story in the whole observed period.
In this image, the occurrences of the
fake news story are divided in slices of
4 weeks (with an overlap of two weeks)
and represented as a density heat map
rather than as individual points. Though
mentions of the story have been present
for more than one year, its circulation
appears to spike up in February 2017, 6 JUN 2016 - 4 JUL 2016 20 JUN 2016 - 18 JUL 2016 4 JUL 2016 - 1 AUG 2016 18 JUL 2016 - 15 AUG 2016 1 AUG 2016 - 29 JUL 2016 15 AUG 2017 - 12 SEP 2016

when a new strand of the fake story

is published by the Russian website
Sputnik International.

BASEMAP
Social Media
and Others
29 AUG 2016- 26 SEP 2016 12 SEP 2016 - 10 OCT 2016 26 SEP 2016 - 24 OCT 2016 10 OCT 2016 - 7 NOV 2017 24 OCT 2017 - 21 NOV 2017 7 NOV 2016 - 5 MAY 2016
Local Media International
Media

Extreme
Right Media Conspiracy
Theorists
Mainstream
French Media

Fake news density

21 NOV 2017 - 19 DEC 2016 5 DEC 2016 - 2 JAN 2017 19 DEC 2016 - 16 JAN 2017 2 JAN 2017 - 30 JAN 2017 16 JAN 2017 - 13 FEB 2017 30 JAN 2017 - 27 FEB 2017

88 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 89
HOW MANY
OCCURRENCES OF THE
FAKE NEWS STORY
ARE PUBLISHED IN
FEBRUARY 2017 AND
WHAT SOURCES DO 14 MAR 2016 - 11 APR 2016 28 MAR 2016 - 25 APR 2016 11 APR 2016 - 9 MAY 2016 25 APR 2016 - 23 MAY 2016 9 MAY 2016 - 6 JUN 2016 23 MAY 2016 - 20 JUN 2016

THEY CITE? 1 FEB 2017 - 5 FEB 2017 3 FEB 2017 - 7 FEB 2017 5 FEB 2017 - 9 FEB 2017 7 FEB 2017 - 11 FEB 2017 9 FEB 2017 - 13 FEB 2017
Temporal evolution of the fake
news story in the February 2017.
This image represents a temporal
zoom of the previous one. Here the
occurrences of the fake news story
are break up in slices of 4 days (with
an overlap of two days).

6 JUN 2016 - 4 JUL 2016 20 JUN 2016 - 18 JUL 2016 4 JUL 2016 - 1 AUG 2016 18 JUL 2016 - 15 AUG 2016 1 AUG 2016 - 29 JUL 2016 15 AUG 2017 - 12 SEP 2016

11 FEB 2017 - 15 FEB 2017 13 FEB 2017 - 17 FEB 2017 15 FEB 2017 -19 FEB 2017 17 FEB 2017 - 21 FEB 2017 19 FEB 2017 - 23 FEB 2017

BASEMAP
Social Media
and Others
29 AUG 2016- 26 SEP 2016 12 SEP 2016 - 10 OCT 2016 26 SEP 2016 - 24 OCT 2016 10 OCT 2016 - 7 NOV 2017 24 OCT 2017 - 21 NOV 2017 7 NOV 2016 - 5 MAY 2016
Local Media International
Media

Extreme
Right Media Conspiracy
Theorists
Mainstream
French Media

21 FEB 2017 - 25 FEB 2017 23 FEB 2017 - 27 FEB 2017 25 FEB 2017 - 1 MAR 2017 27 FEB 2017 - 3 MAR 2017
Fake news density

21 NOV 2017 - 19 DEC 2016 5 DEC 2016 - 2 JAN 2017 19 DEC 2016 - 16 JAN 2017 2 JAN 2017 - 30 JAN 2017 16 JAN 2017 - 13 FEB 2017 30 JAN 2017 - 27 FEB 2017

90 A FIELD GUIDE TO FAKE NEWS CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB 91
 Chapter #3

SOCIAL MEDIA AND OTHERS USING TRACKER SIGNATURES

TO MAP THE TECHNO-
La Rpublique du Centre
GROUPE CENTRE-FRANCE
L'Yonne
La rpublicaine
Rpublique du Centre
L'Yonne rpublicaine Snapchat
SATIRICAL WEBSITES
Snapchat

COMMERCIAL UNDERPINNINGS
CHAPTER 2 RECIPE 2
Wikihow

GOSSIP MEDIA
Wikihow
Instagram
INTERNATIONAL MEDIA

20 JUN 2016 - 18 JUL 2016 4 JUL 2016 - 1 AUG 2016

Version Femina
Facebook
Facebook Instagram

18 JUL 2016 - 15 AUG 2016

Vice
Tumblr
Vice
Mashable
Tumblr
1 AUG 2016 - 29 JUL 2016
Reddit
US MEDIA
OF FAKE NEWS SITES
15 AUG 2017 - 12 SEP 2016
Mashable
Reddit
Version Femina Canal+ The Washington Post
LOCAL MEDIA Canal+
Twitter The Washington Post
Twitter Bloomberg
NCE MEDIA
Do fake news sites use
L'Est rpublicain
TheBloomberg
Guardian The New York Times
Wanted Pedo
L'Est rpublicain YouTube
The Guardian The New York Times

different kinds of trackers from

Wanted Pedo Al Kanz BuzzFeed
Le Progrs YouTube
Al Kanz Paris Match BuzzFeed The Independent
Le Progrs L'Alsace-Le Pays

mainstream media sites?

Paris Match Dailymotion The Independent
L'Alsace-Le Pays
Dailymotion UK MEDIA
1 FEB 2017 - 5 FEB 2017 3 FEB 2017 - 7 FEB 2017
Change.org
5 FEB 2017 - 9 FEB 2017 7 FEB 2017 - 11 FEB 2017 9 FEB 2017 - 13 FEB 2017
The Daily Telegraph

Change.org The Daily Telegraph

Russia Today
France Inter
Ouest-France Europe 1 Les Echos Conscience du peuple

How can fake news and

France Inter RTL Russia Today RUSSIAN MEDIA
Ouest-France Europe 1 Les Echos Conscience du peuple
RTL Wikipdia
Mes Opinions Debunkers de hoax Les Chroniques de Rorschach

mainstream media sites be profiled

AFP11 FEB 2017 - 15 FEB 2017 Wikipdia
13 FEB 2017 - 17 FEB 2017 15 FEB 2017 -19 FEB 2017 17 FEB 2017 - 21 FEB 2017 19 FEB 2017 - 23 FEB 2017

Mes Opinions Debunkers de hoax Les Chroniques de Rorschach

AFP
Le Figaro Slate
BFMTV Lablogalupus

based on their tracker usage?

L'Express alalumieredunouveaumonde
Le Figaro Slate Le Monde
BFMTV L'Express Lablogalupus
Le Monde alalumieredunouveaumonde
Blog du"Plan C", par Etienne Chouard Stop Mensonges
Franceinfo
BlogLibration
du"Plan C", par Etienne Chouard
21 FEB 2017 - 25 FEB 2017 23 FEB 2017 - 27 FEB 2017 Stop Mensonges
25 FEB 2017 - 1 MAR 2017 27 FEB 2017 - 3 MAR 2017 LEGENDA

Franceinfo Le6emetiroir
Libration
12 SEP 2016 - 10 OCT 2016 26 SEP 2016 - 24 OCT 2016 10 OCT 2016 - 7 NOV 2017
Huffington Post 24 OCT 2017 - 21 NOV 2017 7 NOV 2016 - 5 MAY 2016
How do tracker ecologies on fake
Le Parisien-Aujourdhui en France Le6emetiroir
Reopen911 Rvolution Vibratoire
Le Point
Huffington Post Les Crises
EXTREME-RIGHT MEDIAen France
Le Parisien-Aujourdhui Arrt sur images
Reopen911 Rvolution Vibratoire
Le Point
Novopress

news sites change over time?

Les Moutons enrags
SERVING SUGGESTIONS
Arrt sur images
Marianne Les Crises
Novopress Cercle des Volontaires
Les Moutons enrags IlFattoQuotidiano.fr
MarianneContre-Info L'Obs
Cercle des Volontaires Les Brins d'Herbes
IlFattoQuotidiano.fr
This recipe may be used to
Contre-Info L'Obs
Agoravox
Agoravox
Les Brins d'Herbes

MAINSTREAM FRENCH MEDIA
5 DEC 2016 - 2 JAN 2017
identifyEgalit
which websites appear
et Rconciliation
Egalit et Rconciliation
Which other websites share
most often when a fake news
story is mentioned in different
the same tracker ids as fake
settings. These are not necessarily CONSPIRACY THEORISTS news sites?
the original sources of the fake
news, but are often the most Do trackers associated with
influential media outlets that
hyper-partisan and misinformation
contribute to its circulation
19 DEC (whether
2016 - 16 JANas2017 a rumour 2orJAN
as2017
a - 30 JAN 2017 16 JAN 2017 - 13 FEB 2017 30 JAN 2017 - 27sites
FEB 2017vary across language

debunked story). spheres?

92 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 93

Introduction - Over the past few decades
many responses to misinformation have
focused on mapping and debunking claims
made or repeated by politicians, journalists or
other public figures. What are the prospects of
mapping fake news online not just by looking at
the circulation of claims, but by examining the
technical infrastructures of the websites through
which these claims are published?
Many websites use trackers small
bits of embedded code in order to monitor
engagement, including visitor numbers, visitor
behaviour and the effectiveness of ads. In this
section we look at how data about web trackers
can be repurposed in order to investigate the
technical and commercial underpinnings of
websites associated with fake news and other
misinformation phenomena.
94 A FIELD GUIDE TO FAKE NEWS
CHAPTER 3 RECIPE 1
DO FAKE NEWS SITES
USE DIFFERENT KINDS OF
TRACKERS FROM MAINSTREAM
MEDIA SITES?
BEFORE STARTING
For this recipe you will need two lists of
URLs: one list of fake news URLs and one list
of mainstream media URLs. How these lists
are obtained is a crucial part of the research
process. You can either draw on existing
lists, or create your own (e.g. by compiling a
selection, triangulating from other sources,
or obtaining from different platforms or
media sources). The starting point that you
choose will affect how to read and what
you can do with the results. To illustrate
this recipe, we start with a selection of fake
news pages obtained from a list created by
BuzzFeed News (ordered by most engaged
with content according to the BuzzSumo
tool), as well as a list of mainstream media
web pages obtained by triangulating lists from
BuzzFeed News and Alexa.
CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 95
 START
obtain seed lists CHAPTER 3 RECIPE 1
of fake news URLs
CALCULATE TRACKER USAGE PER
SITE TYPE
From the source code of web pages it is often
possible to see which third-party tracking
input URLs in services are used.
DMI Tracker Tracker Collect data about trackers associated with
the web pages on each list. You may use the
DMI Tracker Tracker tool to collect this
output data to
information.
> Extract list of trackers
associated with web pages
on each list Count the usage of each tracker in fake
> Calculate tracker usage
per site type
news websites and in mainstream news
websites.
You may use a scatter plot to visualise
import data in
the resulting data. Each circle represents
RAWGraphs
one tracker coloured by category. On the
horizontal axis, you can show, for example,
the distribution of trackers usage by
visualise
mainstream media and fake news websites.
On the vertical axis, you can indicate the
overall usage of the tracker. We used
DO MAINSTREAM the RAWGraphs tool to generate this
MEDIA AND FAKE NEWS visualisation.
WEBSITES SHARE THE SAME
TRACKER ECOLOGY?

96 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 97
DO MAINSTREAM MEDIA
AND FAKE NEWS WEBSITES
SHARE THE SAME TRACKER
ECOLOGIES?
Scatterplot representing tracker
usage on a series of fake news and
mainstream media sites. While fake
news sites and mainstream media
sites share popular tracker services DoubleClick

such as Google Adsense, DoubleClick

and Google Analytics, mainstream
ScoreCard Research Beacon
media sites appears more mature and 300
sophisticated in its use of trackers in
terms of the number and diversity of
Google Analytics
trackers that it uses. Google Adsense

250

Facebook Connect
Tracker Popularity

200
Google Publisher Tags Google Syndication
Aggregate Knowledge BlueKai

AppNexus

DoubleClick
LiveRamp Quantcast Ad Exchange-Seller
150
Rubicon
ChartBeat TradeDesk
eXelate Twitter Button

Datalogix

100 Facebook Social Plugins

TRACKER POPULARITY
Moat Facebook Custom Audience
Gravatar
Facebook Social Graph
Omniture (Adobe Analytics)
Wordpress Stats
1 307 RevContent
Outbrain
Content.ad
50 PulsePoint

OwnerIQ ShareThis
TRACKER CATEGORY
Disqus
Ad Acxiom Zypmedia

Analytics
20 40 60 80
MAINSTREAM

Tracker Tracker Fakeness Index (%)

FAKE NEWS
Widget
100%

100%
98 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 99
 Google Analytics
Google Adsense

250

CHAPTER 3 RECIPE 2
Facebook Connect

Tracker Popularity
200

HOW CAN FAKE NEWS

Google Publisher Tags Google Syndication
Aggregate Knowledge BlueKai

AppNexus

150
CHAPTER 3 RECIPE 1
LiveRamp Quantcast AND MAINSTREAM MEDIA SITES DoubleClick
Ad Exchange-Seller

BE PROFILED BASED ON
Rubicon
ChartBeat TradeDesk
eXelate Twitter Button

THEIR TRACKER USAGE?

Datalogix

100 Facebook Social Plugins

ITY
Moat Facebook Custom Audience
Gravatar
Facebook Social Graph
Omniture (Adobe Analytics)
Wordpress Stats
Outbrain RevContent
Content.ad
50 PulsePoint

OwnerIQ ShareThis
RY
Disqus

Acxiom Zypmedia

20 40 60
BEFORE STARTING
80
MAINSTREAM

SERVING SUGGESTIONS Tracker Fakeness Index (%)

FAKE NEWS
For this recipe you will need two lists of
This recipe can be used to profile URLs: one list of fake news URLs and one list
100%

100%
the tracking practices associated of mainstream media URLs. How these lists
with different kinds of websites are obtained is a crucial part of the research
including which trackers are either process. You can either draw on existing lists, or
mainly or exclusively associated create your own (e.g. by compiling a selection,
with fake news websites and what triangulating from other sources, or obtaining
these trackers do as well as from different platforms or media sources). The
identifying most commonly used starting point that you choose will affect how to
trackers. It can also be used for read and what you can do with the results. To
exploring the long tail of smaller illustrate this recipe, we start with a selection
and more specialised trackers. of fake news pages obtained from a list created
by BuzzFeed News (ordered by most engaged
with content according to the BuzzSumo tool),
as well as a list of mainstream media web pages
obtained by triangulating lists from BuzzFeed
News and Alexa.

100 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 101
 CHAPTER 3 RECIPE 2

START
GRAPH RELATIONS BETWEEN PAGES
obtain seed lists
of fake news URLs AND TRACKERS
In order to explore how different URLs share
the same patterns of tracker usage we can
create a network graph to highlight associations
between web pages to their corresponding
input URLs in
trackers.
DMI Tracker Tracker
Extract lists of trackers associated with the
initial lists of fake news and mainstream
output data to
media pages. You may use the DMI
> Extract list of trackers Tracker Tracker tool to collect this
associated with web pages
on each list information.
Create a network in order to show the
tracker usage patterns of the different
import data in
web pages. We used Gephi in order to
visually explore the network using a force
Gephi
directed network layout to help read the
data.
visualise You can annotate the network graph in
order to highlight the cluster of URLs
(e.g. fake news clusters, or mainstream
HOW DO FAKE NEWS SITES media clusters).
AND MAINSTREAM MEDIA
CLUSTER ACCORDING THEIR
TRACKER USAGE?

102 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 103
HOW DO FAKE NEWS
SITES AND MAINSTREAM
MEDIA CLUSTER
ACCORDING THEIR empirenews.net +
news4ktla.com +
straightstoned.com

TRACKER USAGE?
Bipartite network of trackers and
websites that use them. Shared tracker
abcnews.com.co
signatures may be used to explore newsexaminer.net
tracker practices or strategies amongst
a set of websites or to detect fake news
media groups. nationalreport.net

newslo.com
newsbiscuit.com
en.mediamass.net

thevalleyreport.com
thenewsnerd.com newshub.info
satiratribune.com

celebtricity.com
empireherald.com
ncscooper.com burradstreetjournal.com

notallowedto.com
baltimoregazzette.com

dailyfinesser.com
huzlers.com

realnewsrightnow.com

wm21news.com

stuppid.com

kata33. om

NUMBER OF CONNECTIONS

Fake news sites

Mainstream media sites Landrypost.com

Tracker

104 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 105
 news4ktla.com +
straightstoned.com

CHAPTER 3 RECIPE 3
abcnews.com.co
newsexaminer.net

nationalreport.net
HOW DO TRACKER ECOLOGIES
CHAPTER 3 RECIPE 2
newslo.com ON FAKE NEWS SITES
CHANGE OVER TIME??
newsbiscuit.com
en.mediamass.net

thevalleyreport.com
thenewsnerd.com newshub.info
satiratribune.com

celebtricity.com
empireherald.com
ncscooper.com burradstreetjournal.com

notallowedto.com
baltimoregazzette.com

dailyfinesser.com
huzlers.com

SERVING SUGGESTIONS BEFORE STARTING

realnewsrightnow.com

This recipe can be used to explore how a wm21news.com

For this recipe you will need the source code
set of web pages can be grouped based of the same web page (or set of web pages) at
stuppid.com
on their tracker signatures. This provides a two different moments in time. You can obtain
complementary picture to lists or metrics kata33. om saved copies of the same page over time (e.g.
(e.g. of most and least used trackers across through manually or automatically saving the
the pages) by facilitating exploration of source code yourself) or you can use public web
relations between trackers and websites. archiving projects such as the Internet Archives
For example it could be used as a starting The Wayback Machine.
point to identify potential fake news
media groups for further investigation,
or to explore the different web tracking
practices, styles and footprints of fake
news web pages including comparisons
Landrypost.com
between pages associated with different
regions, issues or sources.

106 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 107
 START
obtain seed lists
of fake news URLs
input URLs in
CHAPTER 3 RECIPE 3
The Wayback Machine
GRAPH RELATIONS BETWEEN PAGES
AND TRACKERS
obtain past versions This recipe can be used to identify which
of the pages
trackers were being used by a given web page
merge all URLs in a single list
at different moments in time. It might be useful
+
to chart changes in tracking practices for
example by examining the impact and responses
to events like Google and Facebooks bans of
fake news providers from their ads programs in
input URLs in
November 2016.
DMI Tracker Tracker
Obtain archived copies of a webpage. You
may use The Wayback Machine to see
how a given page changed over time.
output data to
> Extract list of trackers Identify associated trackers with the
associated with web pages
on each list current and previous version of the page.
You may use the DMI Tracker Tracker
tool to collect such information.
Identify the trackers which are only present
import data in
on the first date, the ones that which are
Gephi only present on the second date and the
ones that are shared across both dates.
You can group trackers into three lists,
visualise
colouring them accordingly.

HOW DO FAKE NEWS SITES

ADAPT THEIR TRACKER USAGE
IN RESPONSE TO BLACKLISTING
FROM MAJOR AD NETWORKS?

108 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 109
HOW DO FAKE NEWS SITES Doubleclick
ADAPT THEIR TRACKER Google Analytics
USAGE IN RESPONSE TO Google Adsense
Doubleclick Ad Exchange-seller
BLACKLISTING FROM Google Syndication
MAJOR AD NETWORKS? Scorecard Research Beacon
Gravatar
Tracker ecologies on fake news sites Wordpress Stats
before and after blacklisting from Facebook Connect
major ad networks. While ad networks Exelate
from which fake news sites have been Disqus
Brightroll
blacklisted remain in the source code Adobe Audience Manager
of these sites and hence are present Bluekai
Amazon Associates
in the graphic even after the moment Appnexus
of blacklisting, the visualisation also Bidswitch
Criteo
illustrates new ad networks that fake Mediamath
news sites have moved to. A manual Openx
Pubmatic
review of ad services used to serve ads Tradedesk
on the website interface may help to Facebook Social Plugins
Taboola
further refine this analysis and identify Twitter Button
false positives (i.e. tracker services that Adtech
Advertising.com
are no longer in use but whose code Index Exchange (Formerly Casale Media)
remains embedded in these sites). Pulsepoint
Quantcast
Rubicon
Spoutable
Stickyads
Tapad
Adform
Teads Infectious Media
Turn Inc. Yahoo Ad Manager Plus
Yahoo Ad Exchange Adap.tv
Tubemogul Adroll
Krux Digital Bidswitch
Liveramp Crimtan
Datalogix
Acloudimages
Dataxu
Acuity Ads
Digilant
TRACKER POPULARITY Adscale
Dstillery
Eyeview Getintent
TRACKER Revcontent
Smart Adserver
Improve Digital
Infolinks
Sovrn (Formerly Lijit Networks) Internet Billboard
TRACKER Twitter Advertising Smaato
Zypmedia Smartclip
TRACKER At Internet Spotxchange
Twitter Analytics Switch Concepts
Aggregate Knowledge Yieldlab
TRACKER CATEGORY Lotame Kxcdn
Admarvel Owneriq Beeswax
Clicksor Bidtheatre
Rocket Fuel
Ad Drawbridge
Videology
Chango
Facebook Custom Audience Dotomi
Addthis
Google Publisher Tags Kixer
Gumgum
Facebook Social Graph Mythings
Analytics
Media.net Lockerz Share Netmining
Sekindo Pinterest Pagefair
Doubleverify Sharethis Radiumone
Tracker
Visible Measures Twitter Badge Sumome
Omniture (Adobe Analytics) Typekit By Adobe Tumblr Dashboard
Widget
TRACKERS PRESENT TRACKERS TRACKERS PRESENT
ONLY BEFORE DECEMBER ALWAYS PRESENT ONLY AFTER JANUARY
110 A FIELD GUIDE TO FAKE NEWS CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES 111
 CHAPTER 3 RECIPE 4

WHICH OTHER
Doubleclick
CHAPTER 3 RECIPE 3
WEBSITES SHARE
Google Analytics THE SAME TRACKER IDS
Google Adsense AS FAKE NEWS SITES?
Doubleclick Ad Exchange-seller
Google Syndication
Scorecard Research Beacon
Gravatar
Wordpress Stats
SERVING SUGGESTIONS BEFORE STARTING
Facebook
Given debates Connect
and proposals Before you start you will need to compile
Exelate
about stopping the ad revenue or identify seed lists of fake news or other
of fake news, this recipe may be misinformation websites. We illustrate this
Disqus
used to understand how fake recipe by examining which websites use the
Brightroll
news websites are adapting to same Google Analytics IDs as a list of websites
Adobe Audience
the measures takenManager
by trackers from the EU Disinformation Review.
Bluekai
services, technology companies
and advertisers
Amazon as well as how
Associates
effective these measures are.
Appnexus
For example, it can show which
Bidswitch
trackers have been dropped, which
Criteo
remain and which are added at
Mediamath
different moments in time.
Openx
Pubmatic
Tradedesk
Facebook Social Plugins
Taboola
112
Twitter Button A FIELD GUIDE TO FAKE NEWS WHICH OTHER WEBSITES SHARE THE SAME TRACKER IDS AS FAKE NEWS PAGES? 113

Adtech
 START
obtain seed lists
of fake news URLs
obtain the Google Analytics ID
associated to each page
input IDs in
Spyonweb
get the list of associated pages
obtain screenshots
of associated pages
categorise websites groups
visualise
WHAT MEDIA GROUP STRATEGIES
CAN BE DETECTED THROUGH
SHARED GOOGLE ANALYTICS IDS?
114
CHAPTER 3 RECIPE 4
IDENTIFY WEBSITES WHICH SHARE
TRACKER IDS WITH A SEED LIST OF
PAGES OR SITES
This recipe can be used to identify which other
websites share the same tracker IDs as web
pages on a given list.
Extract the Google Analytics ID for
each URL in your starting list. You can
do this manually (e.g. by looking in the
source code for a string in the form UA-
xxxxxxx) or automatically through web
scraping or other tools (in this example we
wrote a custom script in order to extract
this information from the metadata of the
website).
Obtain a list of pages associated with the
same ID. We used the API of
Spyonweb.com to get this information.
Take a screenshot of each web page. We
used a script to automate the process of
obtaining screenshots, in order to visually
compare the different websites to identify
different kinds of media groups.
Place together screenshots of pages with
the same ID to spot differences and
similarities between websites across and
within groups.
A FIELD GUIDE TO FAKE NEWS WHICH OTHER WEBSITES SHARE THE SAME TRACKER IDS AS FAKE NEWS PAGES? 115
WHAT MEDIA GROUP rt.com
19 disinformation stories
xryshaygh.com
1 disinformation story
defensenews.com
1 disinformation story
newcoldwar.org
7 disinformation stories
almanach.cz
3 disinformation stories

STRATEGIES CAN BE UA-5773642 UA-4839940 UA-841082 UA-15942468 UA-3004323

DETECTED THROUGH
SHARED GOOGLE
ANALYTICS IDS?
A selection of websites which share
Media group Lone Themed network Canadian socialism Themed network
the same Google Analytics IDs, based
(Russia Today) Webmaster (Navy, Airforce) and unions (Mysticism, Liberland)
on seed list from EU Disinformation
Review. This illustrates the diversity of
online settings where claims labelled
as Russian disinformation are shared
from large media groups such as
Russia Today, to themed clusters (e.g.
military or mysticism), and geographical
clusters (e.g. Canadian). One can also assange.rt.com asco.gr armedforcesjournal.com oakvillendp.ca grand-mystical-lodge.com
identify distinctive visual styles and
possible shared CMS features amongst
different websites in these clusters,
which may be used as the basis for
further investigations into the media,
publication and communication
strategies of websites associated with doc.rt.com royalparadise.gr sightlinemediagroup.com socialiststudies.com malachim.cz
online misinformation.

learnrussian.rt.com discoverthassos.com militarytimes.com ndpsocialists.ca liberlandpress.com

russian.rt.com thassosinn.gr airforcetimes.com ccu-csc.ca illuminati-journal.com

catalog.rt.com marinecorpstimes.com oldm.cz

116 A FIELD GUIDE TO FAKE NEWS WHICH OTHER WEBSITES SHARE THE SAME TRACKER IDS AS FAKE NEWS PAGES? 117
 rt.com xryshaygh.com defensenews.com newcoldwar.org almanach.cz
19 disinformation stories 1 disinformation story 1 disinformation story 7 disinformation stories 3 disinformation stories
UA-5773642 UA-4839940 UA-841082 UA-15942468 UA-3004323

CHAPTER 3 RECIPE 5

DO TRACKERS
CHAPTER Lone
3 RECIPE 4

ASSOCIATED WITH HYPER-

Media group Themed network Canadian socialism Themed network
(Russia Today) Webmaster (Navy, Airforce) and unions (Mysticism, Liberland)

PARTISAN AND MISINFORMATION

SITES VARY ACROSS
LANGUAGE SPHERES?
assange.rt.com asco.gr armedforcesjournal.com oakvillendp.ca grand-mystical-lodge.com

doc.rt.com royalparadise.gr sightlinemediagroup.com socialiststudies.com malachim.cz

SERVING SUGGESTIONS BEFORE STARTING

This recipe may be used in the For this recipe, you will need lists of fake news,
learnrussian.rt.com discoverthassos.com militarytimes.com ndpsocialists.ca liberlandpress.com
service of expanding a group of hyper-partisan or misinformation sites in
fake news web pages in order different language spheres in order to compare
to derive lists of other websites their trackers and tracking practices. We
which share the same tracker IDs. illustrate this recipe with reference to hyper-
It may also be used to provide partisan, fake news and misinformation sites in
russian.rt.com
context to the digital strategies
thassosinn.gr airforcetimes.com ccu-csc.ca illuminati-journal.com
Dutch, English and German language spheres.
and media groupings of fake
news providers

catalog.rt.com marinecorpstimes.com oldm.cz

118 A FIELD GUIDE TO FAKE NEWS WHICH OTHER WEBSITES SHARE THE SAME TRACKER IDS AS FAKE NEWS PAGES? 119
 CHAPTER 3 RECIPE 5

START IDENTIFY TRACKERS PER LANGUAGE

obtain seed lists
of fake news URLs SPHERE
Extract trackers associated with lists of the
web pages for each language sphere. We did
this using the DMI Tracker Tracker tool.
Identify the trackers which are shared across
input URLs in
and which are unique to different languages
DMI Tracker Tracker
spheres within the dataset. We did this using
the DMI Triangulation tool.
You can illustrate the results using the visual
output data to
> Extract list of trackers metaphor of magnets. Each of the three
associated with web pages
on each list languages are represented on the corner of a
triangle. The trackers are distributed in the
triangle according to their usage: if a tracker
is used by all three languages it will appear in
compare trackers lists with the middle, if it is used by two languages the
DMI Triangulation Tool tracker will be placed on the edge between
the two and so on.

visualise

DO MISINFORMATION AND
HYPER-PARTISAN WEBSITES
IN DIFFERENT LANGUAGE
SPHERES HAVE DISTINCT
TRACKER ECOLOGIES?

120 A FIELD GUIDE TO FAKE NEWS WHICH OTHER WEBSITES SHARE THE SAME TRACKER IDS AS FAKE NEWS PAGES? 121
DO MISINFORMATION veeseo
Mark & Mini
AND HYPER-PARTISAN Ligatus
Atlas

WEBSITES IN DIFFERENT DUTCH

AdLantic
LANGUAGE SPHERES
HAVE DISTINCT TRACKER Platform161
SkimLinks
LinkShare
ECOLOGIES?
Visualisation of tracker ecologies
associated with hyper-partisan or
misinformation sites across three
language spheres. While popular
ad and widget services such as
DoubleClick, Google Adsense and
Facebook Connect are shared across
language spheres, unique services per
language sphere may also be detected.
For example, trackers associated with
the Russian-language focused Mail. Facebook SumoMe ENGLISH
Connect Google
ru Group are only found in the set of Media
Innovation Tag Manager
websites associated with the German Group First Impression
DoubleClick ChartBeat Content ad
language sphere. Ad Exchange-Seller
Gravatar Spoutable
DoubleClick
Wordpress Distroscale
Stats Google Adsense
Google
Twitter Button Syndication Zedo
Google
Analytics

TRACKER POPULARITY

1 78

TRACKER CATEGORY
INFOnline
Plista
Ad Piwik
Tynt
adNET de
Analytics
VKontakte Widgets
AMP Platform
Tracker M P NewMedia
GERMAN
Widget

122 A FIELD GUIDE TO FAKE NEWS WHICH OTHER WEBSITES SHARE THE SAME TRACKER IDS AS FAKE NEWS PAGES? 123
 CHAPTER 3 RECIPE 5

Facebook SumoMe ENGLISH

Connect Google
Media
Innovation Tag Manager
Group First Impression
DoubleClick ChartBeat Content ad
Ad Exchange-Seller
Gravatar Spoutable
DoubleClick
Wordpress Distroscale
Stats Google Adsense
Google
Twitter Button Syndication Zedo
Google
Analytics

SERVING SUGGESTIONS
This recipe can be used to
identify trackers for further
investigation including
language sphere specific and
cross-language trackers. It
may help to provide lines
ts
of inquiry for looking into
what is distinctive about the
commercial and technical
underpinnings of fake news in
different language spheres.

124 A FIELD GUIDE TO FAKE NEWS WHICH OTHER WEBSITES SHARE THE SAME TRACKER IDS AS FAKE NEWS PAGES? 125
126 A FIELD GUIDE TO FAKE NEWS