Solaris 10 10/09 ZFS cache improvements

Larry Wake in an interview by Chhandomay Mandal provides a good overview of the new features of Solaris 10 10/09 (a.k.a. Update 8).


With respect to my work in filesystem caching strategies, this new Solaris release introduces three excellent new features.

First is the introduction of L2ARC cache support. This means that you can now employ readzilla and writezilla SSD devices into any Sun servers.

Second is the introduction of ZFS ARC cache controls through the primarycache (e.g. L1 ARC cache) and secondarycache (e.g. L2 ARC cache) filesystem properties. These new cache controls provide the ability to define what is in and more importantly not in the ZFS ARC cache. You may recall from my blog post on filesystem caching strategies that controlling the contents of the ZFS ARC cache can produce much better and more consistent performance results for data centric services such as directory server and others.

For example, lets say that I am deploying directory server (e.g. DSEE) with the following layout:
* DSEE Bits: ZFS filesystem zpool/bits -primary and secondary caches are disabled
* DS info and txn logs: ZFS filesystem zpool/logs - primary and secondary caches are disabled
* DS DB and ChangeLog: ZFS filesystem zpool/data -primary and secondary caches are enabled

By using the primary and secondary cache controls, I guarantee for the zpool ZFS pool that the
only data stored in the ARC cache is the DS data and changelog.

Here is how to disable both primary and secondary cache for the zpool/logs filesystem at the time of filesystem creation:
# zfs create -o primarycache=none -o secondarycache=none zpool/logs

Here is how to disable both primary and secondary cache for the zpool/logs filesystem at after the filesystem has already been created.
# zfs set primarycache=none zpool/logs
# zfs set secondarycache=none zpool/logs

Note that if you wanted to create and associate pre-tuned ZFS filesystems to a zone at the same time you are creating the zone, you can do this through The Zone Manager with the -r or -w flags. This is possible with the latest release
through the extension that allows you to pass ZFS options like "primarycache=none;secondarycache=none;compression=gzip" to the -r or -w flags. Click here to see full usage help.

The third new feature is the breakout of ZFS ARC cache accounting in the ::memstat kernel metrics. Although the Solaris documentation doesn't make mention of this feature, I presume that it is present in support of the new ARC cache controls. You can see for yourself by running the following command:

# nice -10 echo "::memstat"|mdb -k

Note that you should not run this command on a production server as it may significantly reduce performance of the system while it scans through all physical memory. Note also that the time to complete running is proportional to the amount of physical memory installed in the server.

Here is a sample output of ::memstat metrics prior to Solaris 10 10/09:

Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                      94731               370    9%
Anon                        35113               137    3%
Exec and libs                4544                17    0%
Page cache                 150191               586   14%
Free (cachelist)           394526              1541   38%
Free (freelist)            367163              1434   35%

Here is a sample of what I hope that you will see with Solaris 10 10/09:

Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                     428086              3344    3%
ZFS File Data               25006               195    0%
Anon                     13992767            109318   85%
Exec and libs                 652                 5    0%
Page cache                  24979               195    0%
Free (cachelist)             1809                14    0%
Free (freelist)           1979424             15464   12%
Total                    16452723            128536

Have a super day!

Brad

PS: As soon as I get the chance to download and install Solaris 10 10/09, I will check the memstat data, I will confirm or deny the presence of the new memstat data.