My RSS FeedWhen conducting directory services benchmark analysis, one of the areas that you should carefully monitor is network throughput. There have been several benchmarks that I have conducted where the throughput seemed very low relative to what I expected. When I looked at the network throughput I realized that the throughput was constrained due to the limitations of the network. e.g. 100Mbit network interface card (NIC) delivers <= 12.8MBytes/sec, 1Gbit NIC delivers <= 128MBytes/sec, and 10Gbit NIC delivers <= 1280MBytes/sec.
The most effective Solaris command that I have found to succinctly measure network throughput nicstat (Thanks Brendan). You can install CSWnicstat from blastwave or compile it yourself. To compile nicstat, you will need to download and install your preferred compiler and compile nicstat on a development server. Then copy the binary to the respective production server to monitor the network bandwidth.
Here is the gcc invocation that I used to compile nicstat:
# gcc nicstat.c -o nicstat -lkstat -lgen -lsocket -lrt
One issue that I encountered was that there was no 10GigE nic defined in the code. I just added nxge to the list of supported interfaces on line 105 and was able to compile it with no problems.
Line 105 before the change:
static char *g_network[] = { "be", "bge", "ce", "ci", "dmfe", "e1000g", "el",
Line 105 after the change:
static char *g_network[] = { "nxge", "be", "bge", "ce", "ci", "dmfe", "e1000g", "el",
Here is a sample invocation:
$ nicstat -zsi nxge0 10
Time Int rKb/s wKb/s
04:38:16 nxge0 432.278 339.883
04:38:26 nxge0 461.772 1026.978
04:38:36 nxge0 465.267 1035.693
04:38:46 nxge0 468.381 1045.200
04:38:56 nxge0 459.422 1020.522
04:39:06 nxge0 453.317 1009.152
04:39:16 nxge0 466.867 1043.582
04:39:26 nxge0 466.785 1041.348
04:39:36 nxge0 466.367 1040.200
Enjoy!
Brad
If this information has helped you, please consider helping me through investing in your health and in the health of those you love through purchasing Mannatech wellness products. Ambrotose is the key ingredient of all Mannatech Wellness products. Place your order at my Mannatech Web Store today.
4 comments:
Well
# dladm show-dev -s -i 10 nxge0
Why not use this command, works for me (Solaris 10 10/09 s10s_u8wos_08a SPARC)
Lars, that is a great point. I forgot that with more recent releases of Solaris 10 that you could see network traffic in bytes via dladm show-dev. Clearly, nicstat would be useful for earlier versions of Solaris 10, 9 and 8 where dladm show-dev did not offer this feature. For everyone's benefit, here is an output comparison of the two commands.
# dladm show-dev -s -i 5 e1000g1
ipackets rbytes ierrors opackets obytes oerrors
e1000g1 161315 163167112 0 58835 5443469 0
ipackets rbytes ierrors opackets obytes oerrors
e1000g1 3162 4670910 0 1700 112518 0
ipackets rbytes ierrors opackets obytes oerrors
e1000g1 3083 4673042 0 1642 108242 0
ipackets rbytes ierrors opackets obytes oerrors
e1000g1 3084 4672800 0 1643 108512 0
# nicstat -i e1000g1 5
Time Int rKb/s wKb/s rPk/s wPk/s rAvs wAvs %Util Sat
22:01:46 e1000g1 0.50 0.03 0.75 0.23 677.44 120.40 0.00 0.00
22:01:51 e1000g1 0.10 0.06 1.40 0.40 72.57 158.00 0.00 0.00
22:01:56 e1000g1 249.80 3.00 174.20 38.20 1468.40 80.45 2.07 0.00
22:02:01 e1000g1 838.43 7.70 574.00 115.20 1495.74 68.41 6.93 0.00
22:02:06 e1000g1 838.19 10.63 593.40 161.40 1446.42 67.42 6.95 0.00
22:02:11 e1000g1 760.91 12.07 526.80 185.80 1479.06 66.51 6.33 0.00
Looking at the output differences, it appears that there are two significant differences. First, the values of rbytes (dladm) and rKb (nicstat) are not in agreement. Not completely sure what the source of the difference is but the nicstat data seemed to line up with performance behaviors that I observed. e.g. When the nicstat throughput approached the maximum theoretical limit, performance of what I was benchmark tended to level off. I will do some research to see if I can ascertain the difference in output. The second observation is that nicstat offers better control of what you see without having to parse the output. Lastly, nicstat offers an additional metric (%busy) that could come in handy as well.
Brad
Thanks to Jim Fiori for educating me on the difference between the command outputs. dladm reports bytes per interval. nicstat reports Kilo bytes per second. Thus, if you divide the output provided by dladm by the interval and by 1024, the number should be much closer to the output provided by nicstat.
Thanks Jim!
Brad
Thanks for the information Brad and Lars, it was very helpful. I'm running OpenSolaris 10, and it said the show-dev sub command was not found. I used show-link instead:
dladm show-link ntxn5 -si 300
Where 300=300 seconds. This command gives me, among other stats, the RBYTES average over 5 minutes. So I just divide the RBYTES number by 300 to get bytes per second.
Thanks, I had trouble finding a way to monitor NIC throughput until I found this entry.
Post a Comment