Architectural

Design Patterns in
Cloud Computing
Jinesh Varia Technology Evangelist jvaria@amazon.com
They sent me here to talk
But I am here to listen
Please Send Feedback
jvaria@amazon.com
Twitter: @jinman
Cloud Best Practices Whitepaper
Prescriptive guidance to Cloud Architects
Just Google for Cloud Best
Practices to find the link
Abstract
Resources
Focus on your needs, not on hardware specs. As your
needs change, so should your resources.
On-Demand
Provisioning
Ask for what you need, exactly when you need it. Get rid
of it when you dont need
Scalability
Scale out or in depending on usage needs.
No Up-Front
Costs
No contracts or long-term commitments.
Pay only for what you use.
Efficiency of
Experts
Utilize the skills, knowledge and resources of experts.
Cloud Computing Attributes
What makes the Cloud so attractive
The Living and Evolving Cloud
The Cloud
AWS services and features
Most Applications Need:
1. Compute
2. Storage
3. Messaging
4. Payment
5. Distribution
6. Scale
7. Analytics
Amazon RDS
High-Memory Instances
Lower EC2 Pricing
AWS Multi-Factor Authentication
Virtual Private Cloud
Lower Reserved Instance Pricing
AWS Security Center
Reserved Instances in EU Region
Elastic MapReduce
SQS in EU Region
New SimpleDB Features
FPS General Availability
Lower pricing tiers for
Amazon CloudFront
AWS Management Console
Amazon EC2 with Windows
Amazon EC2 in EU Region
AWS Toolkit for Eclipse
Amazon EC2 Reserved
Instances
AWS Import/Export
New CloudFront Feature
Monitoring, Auto Scaling &
Elastic Load Balancing
Amazon Elastic MapReduce
in Europe
EBS Shared Snapshots
SimpleDB in EU Region
Monitoring, Auto Scaling &
Elastic Load Balancing in EU
Amazon CloudFront
Private Content
SAS70 Type II Audit
AWS SDK for .NET
Amazon EC2 with Windows Server
2008,
Spot Instances,
Boot from Amazon EBS
Amazon CloudFront Streaming
Amazon VPC enters Unlimited Beta
AWS Region in Northern California
International Support for AWS
Import/Export
Amazon EC2 Reserved Instances
with Windows, Extra Large High
Memory Instances
Amazon S3 Versioning Feature
Consolidated Billing for AWS
Lower pricing for Outbound Data
Transfer
The Living and Evolving Cloud At Amazon, Every Day is a Launch Day
New Features and Services
Scalability
Characteristics of Truly Scalable Service
Build Scalable Architecture on AWS
A scalable architecture is critical to take advantage of a scalable
infrastructure
Increasing resources results in a proportional increase in
performance
A scalable service is capable of handling heterogeneity
A scalable service is operationally efficient
A scalable service is resilient
A scalable service becomes more cost effective when it
grows
Cloud Architecture Lessons
1. Design for failure and nothing fails
2. Loose coupling sets you free
3. Implement Elasticity
4. Build Security in every layer
5. Don't fear constraints
6. Think Parallel
7. Leverage different storage options
using Amazon Web Services
1. Design for Failure
"Everything fails, all the time"
Werner Vogels, CTO Amazon.com
and nothing will really fail
Avoid single points of failure
Assume everything fails, and design backwards
Goal: Applications should continue to function even if the
underlying physical hardware fails or is removed or replaced.
Design for Failure with AWS
Tools to make your life easier
Use Elastic IP addresses for consistent and re-mappable routes
Use multiple Amazon EC2 Availability Zones (AZs)
Create multiple database slaves across AZs
Use real-time monitoring (Amazon CloudWatch)
Use Amazon Elastic Block Store (EBS) for persistent file systems
EC2 Instance A EC2 Instance B
YourWebTwoDotZeroName.com
LOG
Volume
DATA
Volume
EC2 Instance A
YourWebTwoDotZeroName.com
LOG
Volume
DATA
Volume
EC2 Instance B
Amazon S3
A
v
a
i
l
a
b
i
l
i
t
y

Z
o
n
e

1
A
v
a
i
l
a
b
i
l
i
t
y

Z
o
n
e

2
2. Build Loosely Coupled Systems
The looser they're coupled, the bigger they scale
Independent components
Design everything as a Black Box
De-coupling for Hybrid models
Load-balance clusters
Controller A Controller B Controller C
Controller A Controller B Controller C
Q Q Q
Tight Coupling
Loose Coupling
using Queues
Use Amazon SQS as Buffers
MySQL
Master
Web Server
MyWebSite.com
MySQL
(Slave)
App Server App Server
LB
Web Server
LB
App server
Tapes
Data Tier
Database Server machines with
master and local running
separately, Network storage for
Static objects
App Server Tier
Fleet of machines handling
Application specific workloads
Caching server machines can
be implemented at this layer
App Load Balancer
Hardware or Software solution to
spread traffic over app servers
Web Tier
Fleet of machines handling
HTTP requests.
Web Load Balancer
Hardware or Software solution
to distribute traffic over web
servers
Exterior Firewall Hardware
or Software Solution to open
standard Ports (80, 443)
Backend Firewall Limits
access to application tier from
web tier
Backups on
Tapes Periodic
backups stored on
Tapes usually
managed by 3
rd
party at their site
Availability Zone #n
Availability Zone #1
Auto-scaling group : App Tier
Availability Zone 2
Auto-scaling group : Web Tier Auto-scaling group : Web Tier
RDS
Master
Web Server
MyWebSite.com
ELB: Web Tier
App Server
Cloud
Front
LB
Web Server
SLB
Tomcat
App Server
Web Server Web Server
RDS
Slave
Auto-scaling group : App Tier
App Server
SLB
Tomcat
App Server
RDS
Slave
DNS
Amazon
S3
DB Tier
MySQL RDS DB Instances
(master, local slave, x-AZ slave
for failover) , Automated
backups to S3 all managed by
AWS
Auto-scaling App Tier
Group of EC2 instances running
the actual app. Instances
belong to Auto-scaling group.
Caching servers instances can
be implemented at this layer
App Server Load Balancer
Software LB (e.g. HAProxy) on
EC2 instance to spread traffic
over app server cluster
Auto-scaling Web Tier
Group of EC2 instances
handling HTTP requests.
Elastic Load Balancer
ELB to spread traffic to Web
Server Auto-scaling groups
Edge Caching
High Volume
Static
Content is edge
cached using
CloudFront
Backups
Amazon S3 used
for storing Static
Objects and
Backups
Exterior Firewall no longer
needed because EC2 instances
are controlled with Security
Groups
Backend Firewall no longer
needed
3. Implement Elasticity
Dont assume health or fixed location of components
Use designs that are resilient to reboot and re-launch
Bootstrap your instances: Instances on boot will ask a
question Who am I & what is my role?
Enable dynamic configuration
Elasticity is fundamental property of the Cloud
Use Auto-scaling (Free)
Use Elastic Load Balancing on multiple layers
Use configurations in SimpleDB to bootstrap instance
Managed Development
Environment
Managed
Development
Environment
AWS Cloud
Enterprise IT ISV Startup
3. Implement Elasticity
3 Usecases
Automated
Deployment
Environment
AWS Cloud
SaaS
Paid
AMI
Cloud-powered
Software Lifecycle
management
AWS Cloud
Web 2.0 Product Dev/Test
Apps
Prod
OS
Framework
Your Code
Libraries
Packages
DB Caching
MVC
App Server
Web Server
Linux
JEE
Your Code
Log4J
Spring
Hibernate
Struts
Tomcat
Apache
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
ASP.NET
IIS
Centos
Ruby Runtime
Your Code
logger
RubyGems
memcached
Rails
Mongrel
Apache
Java Stack .NET Stack RoR stack
Standardized Technology Stacks
3. Implement Elasticity
Standardized Application Stacks
3 Approaches to design MDE
Inventory of fully baked AMIs
(Frozen/Ready made)
Golden AMIs with fetch on boot
(Take N Bake)
AMIs with JeOS and Chef Agent
(Made to Order)
More Control
Easier to maintain
Easier to Setup
3. Implement Elasticity
3 approaches to designing your AMIs
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
ASP.NET
IIS
.NET Stack
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
IIS
IIS
.NET AMI
Amazon EC2
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
IIS
IIS
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
IIS
IIS
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
IIS
IIS
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
IIS
IIS
3 Approaches to design MDE
3. Implement Elasticity
1. Frozen Pizza Model
Source Control
Amazon S3
Windows
.NET
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
IIS
IIS
.NET Stack
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
Windows
.NET
IIS
IIS
.NET AMI
Amazon EC2
Golden AMIs with fetch on boot
Windo
ws
.NET
IIS
IIS
Windo
ws
.NET
IIS
IIS
Windo
ws
.NET
IIS
IIS
Windo
ws
.NET
IIS
IIS
Fetch on boot time
3 Approaches to design MDE
3. Implement Elasticity
2. Papa Murphy Pizza Model
Source Control
Amazon S3
RoR Stack
Your Code
Log4Net
Spring.NET
nHibernate
ASP.NET MVC
.NET IIS
IIS
AMI (JeOS)
Amazon EC2
Chef Server
Windows
CHEF Agent
Windows
CHEF
Agent
Centos
Ruby Runtime
Your Code
logger
RubyGems
memcached
Rails
Mongrel
Apache
Cookbooks
Recipes
3 Approaches to design MDE
3. Implement Elasticity
3. Made to Order Pizza Model
3 Approaches to design MDE
Inventory of fully baked AMIs
(Frozen/Ready made)
Golden AMIs with fetch on boot
(Take N Bake)
AMIs with JeOS and Chef Agent
(Made to Order)
More Control
Easier to maintain
Easier to Setup
3. Implement Elasticity
3 approaches to designing your AMIs
4. Build Security in every layer
With cloud, you lose a
little bit of physical
control but not your
ownership
Design with Security in mind
Create distinct Security Groups for each Amazon EC2 cluster
Use group-based rules for controlling access between layers
Restrict external access to specific IP ranges
Encrypt data at-rest in Amazon S3
Encrypt data in-transit (SSL)
Consider encrypted file systems in EC2 for sensitive data
Rotate your AWS Credentials, Pass in as arguments encrypted
Use MultiFactor Authentication
5. Don't fear constraints
More RAM? Distribute load across machines
Shared distributed cache
Re-think architectural constraints
Better IOPS on my database?
Multiple read-only / sharding / DB
clustering
Your hardware failed or messed up config?
simply throw it away and switch to new
hardware with no additional cost
Performance
Caching at different levels (Page, Render, DB)
Hardware Config
does not match?
Implement Elasticity
6. Think Parallel
Experiment different architectures in parallel
Multi-treading and Concurrent requests to cloud services
Run parallel MapReduce Jobs
Decompose a Job into its simplest form
Serial and Sequential is now history
6. Leverage many storage options
Amazon S3: large static objects
Amazon Cloudfront: content distribution
Amazon SimpleDB: simple data indexing/querying
Amazon EC2 local disc drive : transient data
Amazon EBS: persistent storage for any RDBMS + Snapshots on S3
Amazon RDS: RDBMS service - Automated and Managed MySQL
One size DOES NOT fit all
6. Leverage many storage options
Which storage option to use when?
Amazon S3 +
CF
Amazon EC2
Ephemeral
Store
Amazon EBS Amazon
SimpleDB
Amazon RDS
Ideal for Storing Large
write-once,
read-many
types of
objects, Static
Content
Distribution
Storing non-
persistent
transient
updates
Off-instance
persistent
storage for any
kind of data,
Querying light-
weight attribute
data
Storing and
querying
structured
Relational and
referential
Data
Ideal examples Media files,
audio, video,
images,
Backups,
archives,
versioning
Config Data,
scratch files,
TempDB
Clusters, boot
data, Log or
data of
commercial
RDBMS like
Oracle, DB2
Querying,
Mapping,
tagging, click-
stream logs,
metadata,
shared-state
management,
indexing
Complex
transactional
systems,
inventory
management
and order
fulfillment
systems
Not
recommended
for
Querying,
Searching
Storing
Database logs
or backups,
customer data
Relational (joins)
query
Not
recommended
examples
Database, File
Systems
Sensitive data Content
Distribution
OLTP, DW cube
rollups
Simple
lookups
Cloud Architecture Lessons
Best Practices
1. Design for failure and nothing fails
2. Loose coupling sets you free
3. Design for dynamism
4. Build Security in every layer
5. Don't fear constraints
6. Think Parallel
7. Leverage many storage options
AWS community and Ecosystem
AWS Ecosystem
AWS Community
Find help, guidance, assistance when you need it
Phot o: La Pedrera - Casa Mi l , Barcel ona - Ant oni o Gaudi
Migrating
a Web Application
to AWS
Migrating your Web Application
A typical Web App needs:
Step by Step towards AWS
Compute Power
Storage capacity
Content Distribution
Database storage
Messaging
Load balancing
Monitoring
Web Server /
Presentation Layer
Application Server /
Business Logic
Database
Client Browser
Migrating your Web Application - 1/8
Typical Web App Architecture
Store persistent files in Amazon S3 for
lower costs, higher reliability
Client Browser
Migrating your Web Application - 2/8
Amazon S3 for Storage
Use Amazon CloudFront
Amazon CloudFront is a content delivery network
that caches data stored in Amazon S3 across a
network of 14 edge locations around the world
Client Browser
Migrating your Web Application - 3/8
Amazon CloudFront for distribution
Configure Amazon EC2 running your
choice of web server to handle all
incoming web requests.
Client Browser
Migrating your Web Application - 4/8
Amazon EC2 for your choice of web servers
Configure multiple Amazon EC2
instances running your choice of
application server to process requests.
Use Availability Zones and Elastic IPs
for greater reliability and resiliency.
Utilize Auto-scaling and Elastic LB
service
Client Browser
Migrating your Web Application - 4/8
Scale out App servers on Amazon EC2
Use Amazon EBS for Database
Configure an Amazon EBS device to host
your existing relational database.
Snapshots can be automatically backed up
to Amazon S3.
Client Browser
Migrating your Web Application - 5/8
EBS for Persistent Storage and S3 for Snapshots
Use Amazon SQS
Amazon SQS makes it easy to coordinate
between the web server and application
servers.
Client Browser
SQS
Migrating your Web Application - 6/8
Amazon SQS for queuing requests
Use Amazon SimpleDB
Amazon SimpleDB can be used to store
metadata, logfiles, and other information
for your site.
SimpleDB
Client Browser
SQS
Migrating your Web Application - 7/8
Amazon SimpleDB for log files, metadata
Use Amazon SimpleDB
Amazon CloudWatch to monitoring your
Amazon EC2 instances
SimpleDB
Client Browser
SQS
Migrating your Web Application - 8/8
Monitor your Amazon EC2 instances using CloudWatch
Migrating your Web Application
A typical Web App needs:
Step by Step towards AWS
Compute Power
Storage capacity
Content Distribution
Database storage
Messaging
Load balancing
Monitoring
With AWS:
Amazon EC2
Amazon S3
Amazon CloudFront
Amazon EBS
Amazon SQS
Amazon EC2
Amazon CloudWatch
Phot o: Gr and Cany on Hopi Poi nt SunSet
Conclusions
Most Important Lesson From Our Customers:
Start small with a well-defined proof of concept
Build support and awareness in your organization
Once one application is launched others will follow
The day is not too far when applications
will cease to be aware of physical
hardware. Much like plugging in a
microwave in order to power it doesnt
require any knowledge of electricity, one
should be able to plug in an application to
the cloud in order to receive the power it
needs to run, just like a utility. As an
architect, you will manage abstract
compute, storage and network resources
instead of physical servers. Applications
will continue to function even if the
underlying physical hardware fails or is
removed or replaced. Applications will
adapt themselves to fluctuating demand
patterns by deploying resources
instantaneously and automatically, thereby
achieving highest utilization levels at all
times. Scalability, Security, High availability,
Fault-tolerance, Testability and Elasticity
will be configurable properties of the
application architecture and will be an
automated and intrinsic part of the platform
on which they are built.
The day is not too far.
Scalability, Security, High availability, Fault-tolerance, Testability and
Elasticity will be configurable properties of the application
architecture and will be an automated and intrinsic part of the
platform on which they are built.
jvaria@amazon.com Twitter:@jinman
Presentation idea and template from @simon
Thank you!
http://aws.amazon.com