DSEE Import Rate Improvements through ZFS ARC

Hello all,

My friend Wajih Ahmed blogged today about how the ZFS filesystem Adaptive Replacement Cache can be used to improve DSEE import speed. I observed similar behavior during our 100M entry benchmark at the Sun Benchmark Center.

With the ZFS primarycache=all on the ZFS filesystem containing the ldif import file, it only took approximately 5 hours to import 100M entries. I attempted an import with primarycache set to none and let it run for a long time but was unwilling to wait for it to complete. I estimate that it easily would have taken 18-24 hours to complete. Instead, for fun I set the primarycache to all while the import was running. Below is the excerpt from the import output.

[18/Dec/2009:05:36:58 -0800] - import telco: Processed 18689616 entries -- average rate 1414.1/sec, recent rate 1417.3/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:37:18 -0800] - import telco: Processed 18718370 entries -- average rate 1414.1/sec, recent rate 1425.6/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:37:38 -0800] - import telco: Processed 18753933 entries -- average rate 1414.6/sec, recent rate 1607.9/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:37:58 -0800] - import telco: Processed 18879468 entries -- average rate 1422.0/sec, recent rate 4027.4/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:38:18 -0800] - import telco: Processed 18999088 entries -- average rate 1428.8/sec, recent rate 6128.9/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:38:38 -0800] - import telco: Processed 19115830 entries -- average rate 1435.4/sec, recent rate 5909.0/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:38:58 -0800] - import telco: Processed 19232989 entries -- average rate 1442.1/sec, recent rate 5847.6/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:39:18 -0800] - import telco: Processed 19348836 entries -- average rate 1448.6/sec, recent rate 5825.1/sec, hit ratio 100% -- written to database.
....
[18/Dec/2009:05:43:58 -0800] - import telco: Processed 20999677 entries -- average rate 1539.9/sec, recent rate 5883.3/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:44:18 -0800] - import telco: Processed 21117566 entries -- average rate 1546.3/sec, recent rate 5932.6/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:44:38 -0800] - import telco: Processed 21234080 entries -- average rate 1552.5/sec, recent rate 5860.1/sec, hit ratio 100% -- written to database.
[18/Dec/2009:05:44:58 -0800] - import telco: Processed 21352390 entries -- average rate 1558.9/sec, recent rate 5870.6/sec, hit ratio 100% -- written to database.

Notice that the recent and average import rates remained around 1400/sec up until I enabled the primarycache. Once enabled, the recent import rate jumped from 1400/sec up above 6000/sec in less than 60 seconds. The average import rate then remained around 5800/sec for the remainder of the import. That is approximately a 4.28 times boost in performance!

In Wajih's case, the improvement wasn't as great. I suspect that is due to difference in servers. As you can see from the following specs, my server was faster in every way. Further the storage containing the LDIF import file was dramatically different as well.

Here were the specs of Wajih's server:

Server: Sun X4150

DRAM: 32GB of 667MHz DDR2

CPU Cores: 8 2.33GHz

Zpool Containing LDIF: 146GB from 1 15k SAS disk

Entry Count: 3M

LDIF File Size: 14GB

Here were the specs of my server:

Server: Sun X4270

DRAM: 72GB of 1066MHz DDR3

CPU Cores: 8 HT enabled 2.93GHz (Total threads = 16)

Zpool Containing LDIF: 2TB from 20 virtual disks supplied by Sun StorageTek 9980.

Entry Count: 100M

LDIF File Size: 556GB

Just for fun, I looked at the import rate differential for OpenDS using the same import file. The OpenDS import rate went from around 1,200/sec to 12,000/sec. That is a 10 times boost. Wow!!!

The net of both of our findings is that it is important that you put the LDIF file on a ZFS filesystem that has the primarycache enabled for optimal DSEE import rate.

Have a super day!

Brad

If this information has helped you, please consider helping me through investing in your health and in the health of those you love through purchasing Mannatech wellness products. Ambrotose is the key ingredient of all Mannatech Wellness products.  Place your order at my Mannatech Web Store today.