Académique Documents
Professionnel Documents
Culture Documents
Condential
Konstantin Gredeskoul
CTO, Wanelo.com
12-Step Program for Scaling Web
Applications on PostgreSQL
@kig
@kigster
Proprietary and
What does it mean, to scale on top of
PostgreSQL?
Proprietary and
And why should you care?
Proprietary and
Scaling means supporting more
work load concurrently, where work
is ofen interchanged with users
But why on PostgreSQL?
Because NoNoSQL is hawt! (again)
Proprietary and
Relational databases are great at
supporting constant change in
sofware
They are not as great in auto
scaling, like RIAK or Cassandra
So the choice critically depends on
what you are trying to build
Proprietary and
Huge majority of applications are
represented well by the relational
model
So if I need to build a new product
or a service, my default choice
would be PostgreSQL for critical
data, + whatever else as needed
Proprietary and
This presentation is a walk-through
lled with practical solutions
Its based on a story of scaling
wanelo.com to sustain 10s of thousand
concurrent users, 3k req/sec
But lets explore the application
to learn a bit about wanelo
for our scalability journey
Proprietary and
Founded in 2010, Wanelo (wah-nee-loh, from
Want, Need, Love) is a community and a social
network for all of the world's shopping.
Wanelo is a home to 12M products, millions of
users, 200K+ stores, and products on Wanelo
have been saved into collections
over 2B times
P i
move fast with product development
scale as needed, stay ahead of the curve
keep overall costs low
but spend where it matters
automate everything
avoid reinventing the wheel
learn as we go
remain in control of our infrastructure
Early on we wanted to:
Heroku or Not?
Proprietary and
Assuming we want full control of our
application layer, places like Heroku arent a
great t
But Heroku can be a great place to start. It
all depends on the size and complexity of the
app we are building.
Ours would have been cost prohibitive.
Foundations of web apps
Proprietary and
Are there adequate indexes for the query? Is the database using
appropriate index? Has the table been recently analyzed?
Follows table
Proprietary and
Stories table
Proprietary and
So our index is partial, only on state = active
So this query is a full table scan
But there state isnt used in the query, a bug?
Lets add state = active
It was meant to be there anyway
Proprietary and
Proprietary and
Step 3:
Upgrade Hardware and RAM
Hardware + RAM
Proprietary and
JOYENT
CPU: E5-2690
2.9GHz
AWS
8 SSD drives
SSD Make: ?
CPU: E5-2670
2.6Ghz
Perhaps you get what you pay for afer all.
Proprietary and
Step 4:
Scale Reads by Replication
Scale Reads by Replication
Proprietary and
https://github.com/wanelo-chef/postgres
It installs PG in eg /opt/local/postgresql-9.3.2
Problem: products.saves_count is
incremented every time someone saves a
product (by 1)
UniqueJob extension