Occasionally I have a need to measure disk I/O in our virtual environments. This is usually because I have somebody telling me that we don’t have enough I/O to do what it is they want to do… often some sql that could be re-crafted to be easier on I/O. Usually they are wrong… but occasionally (like today) they are right on. Anyway, when I have this need, I turn to a few handy tools that have worked well for me int he past.
##iostat iostat is one of my favorite tools. I usually throw the -x flag at it and I get back something like this:
[root@host ~]# iostat -x Linux 2.6.32-431.17.1.el6.x86_64 (host) 07/07/2014 _x86_64_ (2 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 0.84 0.00 0.84 1.88 0.00 96.44 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 4.85 10.38 7.98 1.53 510.72 95.22 63.74 0.07 7.38 4.01 3.81 sdb 0.00 0.00 0.37 0.00 2.93 0.00 8.00 0.00 0.49 0.49 0.02 dm-0 0.00 0.00 10.99 11.90 473.82 95.18 24.86 0.38 15.77 1.60 3.66 dm-1 0.00 0.00 0.40 0.00 3.21 0.00 8.00 0.00 5.79 5.69 0.23 [root@host ~]#
iostat -x n will continually run the iostat command every n seconds. I usually use one for five second intervals depending on what I am trying to get a feel for.
The key metric in iostat is %iowait - this is the amount of time spend waiting on I/O
Another tool that I use from time to time is
sar. It provides some alternate statistics alongside iowait. But again, %iowait is the key metric - the amount of time spent waiting on I/O
##Note: %iowait is the time that the kernel is doing nothing else, but waiting on I/O. Think about that for a minute. In other words a 100% CPU bound machine is going to show zero for %iowait, even if the system is also I/O bound.
sar are both part of the
sysstat meta-package in the CentOS yum repos, and also available for Debian based systems via apt-get.
UPDATE - 2016-06-22: I have been using this a lot recently: `iostat -tx 60 2>&1 | tee iostat.log` It allows me to capture the time `-t` and output it to a log file for later perusal.
UPDATE - 2016-08-03: Also `vmstat -t 6`