ZFS+

by Tegile Systems

Dedup Issues (opensolaris)

Performance
DDT (Dedup Table) needs to be in RAM
Or in L2ARC (SSD), what about reboot/failover
Solution Meta Device

System Hangs
during large deduped file/dataset destroy
very difficult, if not impossible, to get out of this state

Meta Data Device

4KB aligned
No SYNCHRONIZE CACHE
Only keeps 1 copy of metadata (total 2)
SSD aware allocator

Results:
Sustainable Dedup performance
Faster metadata retrieval faster FS
operations
Dedicate L2ARC for caching data

L2ARC
L2ARC feed marker
efficiently, aggressively finding arc buf to feed
L2ARC

L2ARC compression (LZ4)

compress data to cache more data in L2ARC

L2ARC persistence
After importing pool, the content of L2ARC can be
brought back without accessing HDDs

Other Improvement
Dedup Write throttle
Enhance Resilver/Scrub performance, while
reducing the performance impact on pool
LZ4 compression support in dataset
Asynchronous Destroy of dataset (port from
illumos)
Metaslab weighting & selection

Under construction

Resumable send/receive
Targeted Scrub
Fast pool export
And more performance improvements

Performance Monitoring Tools (1)

(from support@s1.tegile.com:tools/zfs/ )

SAN (FC/iSCSI) IO
zvio [-x] [-d zvol] [interval [count]] (see header)

NFS IO
nfsio [-x] [-d share] [interval [count]]
#zvio
Time: 2013 Jul 11 13:44:03
RS
WS
KR
KW
916
723 109803 41819
868
696 83224 39123
1150
382 99512 21860
4
0
512
0
4
0
512
0
4
0
512
0
4
0
512
0
0
0
0
0
0
0
0
0
1143
701 46044 28359

RAV
122629
98086
88608
131072
131072
131072
131072
0
0
41210

WAV
59171
57504
58598
0
0
0
0
0
0
41385

RSV
WSV
WQ
15430
8219
0
7895
6718 14
9032 59326 97
13816
0 145
9060
0 151
429
0 153
1208
0 154
0
0 162
0
0 162
1418 1649796
2

Performance Monitoring Tools (2)

# ./zvio -d ReplicaPool/Local/DRVMWareUser/ZebiDRUser
RS

WS

KR

KW

74

37

9509

310

744

191

RAV

2447 123876

WAV WAL

WF

RSV

WSV

WQ

63764

43

94

17950

21668

91

91

91

37059

45431 122295

62468

51

91

19302 655372

259

23044

15788 123544

62420

53

93

15988 146310 108

426

697

50979

42929 122423

63008

51

92

18673 260558

21

170

247

20273

14748 122000

61083

47

88

17549 110410

65

74

9080

0 125647

428

823

48235

50702 115292

63023

50

92

26117 179435

33

126

33

2342

19017

64977

48

93

342 242114

32

17

2048

0 123361

187

388

17218

94194

60851

50

167

52

19500

2757 119568

54291

51

2096
23079

WAL: write alignment percentage

WF:

full block write percentage

53641

24504

0 108

0 116

55

87

14396 259252

78

16350

64496

96

Performance Monitoring Tools (3)

Tracing pool syncing activities
spasync.d pool (notice outside of )
55157660034455 ---> txg:11745086, start-int:5000 ms, sleep-int:4521 ms.
2028
37166 us dsl_1: txg:11745086, spent
35 ms
toupdate:439
synced:284
zio:0
db_read:2292
throughput:54758
limit:27379
38397
5326 us ddt_1: txg:11745086, spent
4 ms in scan
44980
26592 us dsl_2: txg:11745086, spent
24 ms
...
386701
us
dsl_pool_sync_done
0 ms
548200
us
spa_update_dspace
0 ms
548238
us
spa_handle_ignored_writes
0 ms
548245
us
spa_async_dispatch
0 ms
dsl
84
ms
ddt
4
ms
bplist
4
ms
vdev_sync
8
ms
metaslab_sync
7
ms
vdev_config_sync
281
ms
dsl_sync_done
0
ms
vdev_sync_done
161
ms
spa
548
ms
55158208287010 443256 us<--- txg:11745086 complted in 548 ms.

Other common toosls

/zebi/opt/DTraceToolkit-0.99/hotkernel
/zebi/opt/DTraceToolkit-0.99/iopattern
mpstat
kstat -m zfs
lockstat
mdb
::arc
::kmastat