From Publisher To Platform: How The Guardian Embraced The Internet Using Content, Search, and Open Source

From publisher to platform: How the Guardian embraced the internet using content, search, and Open Source
Stephen Dunn, Guardian News and Media stephen.dunn@guardian.co.uk, 25th May, 2011 Twitter: @cuica, @openplatform
Thursday, 26 May 2011
From publisher to platform

How the Guardian embraced the Internet using content, search, and Open Source
Stephen Dunn, Guardian News and Media
2
The publishing era
We started a long time ago:
Keyword page Live blogs Apps Mobile site
Twitter updates
Swine flu
Comment
Content partnerships
Newspapers Audio Video Open platform API
To secure the financial and editorial independence To secure the financial and editorial independence of the Guardian in perpetuity. To promote freedom in thein perpetuity of the Guardian press and liberal journalism globally. To become the world's leading liberal voice.
journalism globally To promote freedom in the press and liberal
Open Web Principles
2009
1. Permanent
http://www.ickr.com/photos/fstorr/
A cool URI is one that does not change 1.5 million resources redirected to new scheme
Tim Berners-Lee 1998
2. Addressable
Resources are about something - ready for the social web. We live in the age of point-at-things (Coates 2005)
10
3. Discoverable
Multiple routes to content
Tagging drives discovery
11
4. Open
12
Example: The Hackable Guardian

http:// www.guardian.co.uk/....
/technology/internet /rss /technology/all /rss /environment/climatechange +business/globaleconomy/rss
Results...
14
Site trafc growth

30,000,000 26,250,000 22,500,000
Final Release
Unique Users
First release
Unique Users
18,750,000 15,000,000 11,250,000 7,500,000 3,750,000
Pre - project
40M
Sep 2005
Oct 2006
Nov 2007
Dec 2008
15
However...
16
1 Billion+ Internet Users!
17
18
19
20
...How I stopped worrying about my website and learned to love the whole internet. Matt McAlister
21
The Open Strategy

OPEN IN Bring in data and apps from the Internet
OPEN OUT
Enable partners to build applications using Guardian content and services for other platforms
22
23
"Our most interesting experiments lie in combining what we know with the experience, opinions and expertise of the people who want to participate rather than passively receive.
24
25
26
27
28
29
30
31
32
33
Jack Shenker
The Guardian alongside Al Jazeera was the one news source that everybody on the streets in Tahrir - not just in Cairo but in surrounding cities and major centers of revolutionary activity that people were talking about.
34
The Open Strategy

OPEN OUT
35
22
The Open Platform
36
The suite of services enabling partners to build applications with the Guardian
37
OPEN OUT
38
22
CONTENT API
A service for selecting and collecting content from the Guardian for re-use
DATA STORE
A directory of useful data curated by Guardian editors
POLITICS API
Open database of candidates, voting records, constituencies, election results, live data on election day
Mutualised news!
40
Mutualised news!
41
Mutualised news!
42
43
44
45
46
DATA STORE
A directory of useful data curated by Guardian editors
POLITICS API
POLITICS API
49
<OBLIGATORY DOGFOOD SLIDE >
50
51
Open for Business
56
3 Tiers of access 3 Revenue models

Keyless: Take our headlines. You keep associated
revenues.
Approved: Take our full article content, but with an

advert. Guardian keeps ad revenue, you keep rest-ofpage revenue.
Bespoke: Take, reformat, augment our content

Revenue model to be negotiated. Combination of Media, Fees, Downloads.
57
58
What this means

Open Out: Developers can now access full content APIs on demand with keys post-approved Platform is positioned as a place to do business So rapid scalability, reliability and performance are now core requirements
59
OPEN IN Bring in data and apps from the internet
OPEN OUT Allow partners to build applications using Guardian content and services for other platforms
MICROAPPS
A framework for integrating 3rd party applications into guardian.co.uk
Simple REST/HTTP framework allows lightweight development Applications proxied for performance Apps generally hosted in the cloud, allows hot deployment into production
61
MICROAPPS A framework for integrating 3rd party applications into guardian.co.uk
62
What could I cook?
Bringing it together
64
65
App showcase
66
From publisher to platform

Seeking massive growth, but no longer only broadcasting content on the website User/partner engagement & contribution on Journalism data software applications revenue and ads Support developers and partners with data and APIs, need scalability, reliability, speed
67
Evolving the architecture
68
Web server
Web server
Web server
App server
App server
App server
Memcached (added later)
Oracle
CMS
Web server
Web server
Web server
Why RDBMS?
App server App server App server
5 years ago, fewer alternatives

Memcached Understand operations procedures
Can easily recruit DBAs / devs Developer/ops tools Business critical system: a safe choice
CMS Oracle
Scaling trafc
30,000,000 26,250,000 22,500,000
Unique Users
Unique Users
18,750,000 15,000,000 11,250,000 7,500,000 3,750,000
Sep 2005
Sep 2006
Sep 2007
Sep 2008
71
72
73
74
75
76
77
We chose Solr/Lucene
Can perform complex queries, including full-text search We can change the schema with no downtime Most queries are of similar cost Scales very well horizontally Just worked in the cloud No strange control processes/engines Developers just loved working with it!
78
79
Web servers App server Memcached
Api Solr Solr Solr
RDBMS
Solr
Solr Solr Cloud, EC2

80
CMS
What about Open In?

OPEN OUT
81
22
Apps
Web servers
Proxy
App App App App App App

external hosting app engine etc
App server Memcached
RDBMS
CMS
82
Core In Web servers App App App App App App

external hosting app engine etc
Out
Solr App server Solr Memcached Solr
CMS
Proxy
Solr
Solr Solr Cloud, EC2

83
rdbms
84
85

From Publisher To Platform: How The Guardian Embraced The Internet Using Content, Search, and Open Source

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

From Publisher To Platform: How The Guardian Embraced The Internet Using Content, Search, and Open Source

Transféré par

Droits d'auteur :

Formats disponibles

From publisher to platform: How the Guardian embraced the internet using content, search, and Open Source

Thursday, 26 May 2011