APP-BCA2053

Virtualizing MS Exchange 2010 at General Mills Architecture Details and Lessons Learned

Aaron Kopel, General Mills Curtis Witt, VMware, Inc.

#vmworldapps

Disclaimer

This session may contain product features that are

currently under development.

This session/overview of the new technology represents

no commitment from VMware to deliver these features in any generally available product.

Features are subject to change, and must not be included in

contracts, purchase orders, or sales agreements of any kind.

Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features
discussed or presented have not been determined.

Session Objectives & Agenda

General Mills Background Exchange 2007 Architecture Exchange 2010 Architecture Business Drivers and Benefits VMware Environment and Configuration Key Takeaways Q&A

General Mills Background

Global Company
Headquartered in
Minneapolis, MN

35,000 employees
across 80+ countries

$16.6B annual revenue

Virtualization Highlights
Virtualizing since 2007 94% virtual at HQ,
74% virtual globally

VMware in 80+ sites

across 20+ countries

vSphere ESXi 4.1 and 5.0 2200+ server VMs on 250+ hosts 1200+ View 5.1 users Lab Manager

General Mills Exchange Landscape

32,000+ Mailboxes Highly mobile user base 10,000+ Mobile devices Utilize Microsoft Unified Messaging Regional centralization Exchange infrastructure in 8 sites
UK: 3,030 China: 824

US: 25,735

Hong Kong: 838 Australia: 593

Argentina: 958 S. Africa: 103

5

India: 561

Exchange 2007 Architecture

Started physical and migrated to virtual Virtualized everything except the Cluster Continuous Replication
(CCR) clusters

Server roles in Minneapolis

4 Client Access (CAS) servers 2 Hub Transport (HUB) servers 2 Unified Messaging (UM) servers 3 pairs of CCR clusters (Physical Servers with DAS)

Server roles at remote sites

9 all-in-one servers

Exchange 2010 Architecture

Started 100% virtual never ran physical Server roles in Minneapolis

6 CAS servers 4 Hub servers 3 UM servers One 7-node Database Availability Group (DAG)

Server roles at remote sites

7 all-in-one servers

Performance and sizing

Microsoft Exchange 2010 Mailbox Server Role Requirements Calculator VMware Exchange 2010 on VMware Best Practices Guide

Exchange 2010 DAG Architecture

Single 6-node DAG (split across 2 sites), each hosting 8 active and
8 passive databases

Approximately 4,200 mailboxes per DAG node (25,000+ Total) 2GB Mailbox Quota All mail groomed with 1-year max retention Each nodes active databases replicate 2nd copy to other DAG node VMs, never on the same host or in the same datacenter backup processing

7th DAG node houses 3rd copy of all 48 passive databases for

Exchange 2010 DAG Database Layout

Virtualization Business Drivers and Benefits

Agility
Right size VMs initially, and hot-add vRAM and vCPU as needed Easily expand shared storage to dynamically grow Ability to quickly add additional servers

Operational Simplicity and Efficiency

No need to create one-off architecture Core Compute, Network & Storage is the same as the rest of the environment Easier planned maintenance

10

VMware Environment

Started on ESXi 4.1, now on ESXi 5.0 8 ESXi blade hosts dedicated for Exchange Blade enclosures are shared with other general purpose ESXi hosts No Exchange-purposed blade hosts share the same enclosure No two DAG nodes share the same blade host Everything located at 2 sites, interconnected with private fiber

11

Compute / Memory Config

12

HA / DRS / vMotion

HA Admission Control enabled with 25% CPU / Memory reserved

for failover capacity (2 hosts)

DRS Set to Fully Automated priority 1 & 2 recommendations DAG Nodes set to Manual (no DRS) Using DRS Groups / Rules to pin VMs to Primary or DR Datacenter
and to separate like Exchange server roles.

Succeeding with vMotion and DAG nodes

SameSubnetDelay increased from 1000ms to 2000ms SameSubnetThreshold left at default of 5 vSphere 5 improved vMotion performance for DAG nodes Work very closely with Exchange team to test / qualify this!!

13

Compute / Memory Lessons Learned

VMs created with hot-add CPU and Memory enabled Reserved all memory for DAG nodes Not using CPU reservations Most server roles required increases from original CPU/memory configurations DAG: 46 vCPUs and 24GB48GB CAS: 816GB UM: 24 vCPUs and added another server

Make sure to size CPU / Memory for failover / maintenance capacity

14

Datacenter Network Config

15

Host Network Config

16

Networking Results and Lessons Learned

Using VMXNET3 (default settings) Mailbox VMs have 2 vNICs connected to Exchange and Backup
Networks

Overall daytime bandwidth averages ~2Gb/s Separated vMotion / Backup network to relieve production network
bandwidth

17

Datacenter FC SAN Config

18

Disk / LUN / VMFS Layout

19

DAG Virtual Machine Storage Config

20

Storage Results and Lessons Learned

You can succeed with shared storage! Make sure your SAN has the required capacity and throughput.
(Run Jetstress!)

Plan for Background Database Maintenance

(runs 24x7 @ 5MB/s/Database)

Total Exchange IOPS (all roles) is around 8000IOPS + 3000 for

7th DAG node

Needed to dedicate 4 FA ports per site for Exchange on Tier-1 SAN

Each FA port could handle about 4000IOPS

Increase VMFS heap size to 256 MB

Allows for increase from 20 TB to 64 TB of open VMDK capacity per ESXi host Needed for the gigantic 7th DAG node Details at: http://kb.vmware.com/kb/1004424

21

Uber Diagram

22

Key Takeaways

1. Be on the good side of your Exchange and Storage Teams! 2. Enable hot-add vCPU and vRAM for all VMs easily adjust
sizing for all server roles.

3. Design storage so it is quick and easy to grow! 4. Compared with Exchange 2007, disk I/O is definitely reduced,
but at the expense of CPU and Memory

5. PowerCLI is handy for configuration of complex VMs

(lots of controllers and VMDKs).

6. Review the VMware E2K10 Best Practices Guide(s)

http://www.vmware.com/files/pdf/Exchange_2010_on_VMware_-_Best_Practices_Guide.pdf http://www.vmware.com/files/pdf/using-vmware-HA-DRS-and-vmotion-with-exchange-2010-dags.pdf

23

After The Conference

APP-BCA1684 - Virtualizing Exchange Best Practices By Alex Fontana, VMware

(Monday, 1:00 PM 2:00 PM and Tuesday, 3:30 PM 4:30 PM)

Review content for this session and Alexs session after the conference, at http://www.vmworld.com

24

Questions? THANK YOU

25

FILL OUT A SURVEY

EVERY COMPLETE SURVEY IS ENTERED INTO DRAWING FOR A $25 VMWARE COMPANY STORE GIFT CERTIFICATE

APP-BCA2053

Virtualizing MS Exchange 2010 at General Mills Architecture Details and Lessons Learned

Aaron Kopel, General Mills Curtis Witt, VMware, Inc.

#vmworldapps