PostgreSQL at myYearbook.

com

Gavin M. Roy
Chief Technology Officer
myYearbook.com

pgEast 2010
About myYearbook.com

2007 - 100M Page views per Month

2010 - 1.5B Page views per Month

Top 5 Social Network in the United States as measured by

Hitwise

In the Top 30 most trafficked sites in the United States as

measured by ComScore

99% Uptime
 2007: Midnight Calls
2008: Sizing hardware
2009: Complex
2010: Connections

Growth is a double-edged sword.

 SAN - Fiber Channel
RamSAN - Fiber Channel
iSCSI? Too slow

Started on DAS
FusionIO?

“The best laid schemes o’ mice an’ men

Gang aft agley”

- Robert Burns, To a Mouse

 Concurrency:
Too many backends
Pool with pgBouncer

Things we’ve learned about

Plan for Growth concurrency & data growth
Too Much Traffic
Too Many Backends
Too Much Lock Contention
Slow PostgreSQL Queries

pgBouncer to the Rescue

Table Partitioning

• Supported in PostgreSQL as of 8.1

• Excellent method for maintaining data

• Allows for removal of aged data without bloat

• Focused SELECTS while allowing ad-hoc SELECT across all partitions

Vertically Partitioning
Data

• Isolate application data in

different database servers

• Replicate common data

needed for joins
Horizontally Partitioning Spread table data across
multiple database servers
Data
 Daily Backups and
Offsite (worst)

Warm Standby

PostgreSQL 9.0 Hot

+Stream

Ancient Mountaineering Adage:

“Anything that can possibly go wrong, does.”

- Jack Sack (1952)
How to kill PostgreSQL performance.
Inadequate Hardware
Not Enough Ram
Not Enough Disk
Controllers
Slow
Communication
CPU Oversaturation
 Long Running Exclusive
High Share Locks
Lock Contention == Death Reduce by partitioning
concurrent reindex, etc
 MVCC
HOT (Tuples)
Bloat == Slower Death Index Bloat
“Knowledge is Power”

- Sir Francis Bacon

Reacting can be Proactive

• check_postgres.pl by Greg Mullane

• Nagios plugin

• Bloat

• Management activity such as last analyze and vacuum

• wal file count, txid wrap around, sequence exhaustion

• many other items it checks

Be Trendy

• Know your database behavior

over time

• Predict future issues and

behavior

• Identify issues as they occur

• Review impact of maintenance

• Know if your heap use exceeds

your index use

• Look for daily change

Trending and Analysis at myYearbook.com

• Cacti - http://www.cacti.net

• Posuta - http://code.google.com/p/posuta/

• pgFouine - http://pgfouine.projects.postgresql.org/

• Staplr - http://github.com/myYearbook/staplr
Any Questions? http://twitter.com/Crad