Friday, 7 July 2017

RAM, CPU and I/O Throttling with Windows Server Containers / Docker

CPU

Docker provides several options for limiting CPU with containers - one of the more common switches is '--cpus' - which allows you to limit the number of CPU's a container can use - for example if I wanted to ensure a container only used 1 and a half processors I could issue:

docker run -it --name windowscorecapped  --cpus 1.5 microsoft/windowsservercore cmd.exe

There is also a switch called '--cpu-shares' which allows you to delegate which containers would get priority (weighted) access to CPU cycles during (and only during) contention - the default value is 1024 - which can be set to either lower or higher:

docker run -it --name windowscorecapped  --cpus 1.5 --cpu-shares 1500 microsoft/windowsservercore cmd.exe

RAM

Docker on Windows uses a light-weight HyperV VM to create the containers and as a result has the default 2GB memory limit - we can increase this using the -m switch e.g.:

docker run -it --name windowscorecapped  --memory 4G microsoft/windowsservercore cmd.exe

Disk I/O

Docker provides a few options to control I/O throughput - one of the more useful options is the '--blkio-weight' switch which can be found under the 'docker run' command.

--blkio-weight takes an integer between 10 and 1000 - although by default is set to 0 - which disables it.

Unfortunately at this time it appears like the Windows implementation of Docker does not support this feature:

docker: Error response from daemon: invalid option: Windows does not support BlkioWeight.

So instead for Windows we have 'io-maxiops' and 'io-maxbandwidth'.

Neither are as good as the 'blkio-weight' - the 'io-maxiops' is not great because what size of data are you basing the iops calculation on - for example a read or write block size of 8KB might get a very different result then that of a 64KB block.

There are numerous tools that can help simulate disk i/o for different situations - for example databases read / writes which will typically be small but many. One of these tools can be obtained from Microsoft - called diskspd.

Let's firstly create a normal (unthrottled) container and perform some benchmarks:

docker run -it --name windowscore  microsoft/windowsservercore cmd.exe

If you are on Windows Server Core you can download it using PowerShell e.g.:

Invoke-WebRequest -Uri "http://url.com/disk.zip" -OutFile "C:\Disk.zip"

and since Microsoft didn't bother to include a command line unzip utility (not even in Windows 10!) we have to use PowerShell to unzip it:

powershell.exe -nologo -noprofile -command "& { Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]::ExtractToDirectory('C:\Disk.zip', 'disk'); }"

We can then run it like follows (note: This config is aimed at a SQL server - you should change the block size etc. dependant on the use of the server / targeted application):

diskspd –b8K –d30 –o4 –t8 –h –r –w25 –L –Z1G –c5G C:\iotest.dat

Read IO
thread |       bytes     |     I/Os     |     MB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |        55533568 |         6779 |       1.77 |     225.96 |    8.442 |    35.928 | C:\iotest.dat (5120MB)
     1 |        64618496 |         7888 |       2.05 |     262.93 |    7.412 |    34.584 | C:\iotest.dat (5120MB)
     2 |        53665792 |         6551 |       1.71 |     218.36 |    7.792 |    31.946 | C:\iotest.dat (5120MB)
     3 |        63733760 |         7780 |       2.03 |     259.33 |    6.919 |    30.835 | C:\iotest.dat (5120MB)
     4 |        32374784 |         3952 |       1.03 |     131.73 |   14.018 |    44.543 | C:\iotest.dat (5120MB)
     5 |        62357504 |         7612 |       1.98 |     253.73 |    7.571 |    33.976 | C:\iotest.dat (5120MB)
     6 |        53354496 |         6513 |       1.70 |     217.10 |    8.352 |    35.517 | C:\iotest.dat (5120MB)
     7 |        65290240 |         7970 |       2.08 |     265.66 |    7.016 |    33.599 | C:\iotest.dat (5120MB)
-----------------------------------------------------------------------------------------------------
total:         450928640 |        55045 |      14.33 |    1834.80 |    8.064 |    34.699

Write IO
thread |       bytes     |     I/Os     |     MB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |        18046976 |         2203 |       0.57 |      73.43 |   28.272 |    56.258 | C:\iotest.dat (5120MB)
     1 |        21463040 |         2620 |       0.68 |      87.33 |   23.235 |    49.989 | C:\iotest.dat (5120MB)
     2 |        17686528 |         2159 |       0.56 |      71.97 |   31.750 |    64.613 | C:\iotest.dat (5120MB)
     3 |        20946944 |         2557 |       0.67 |      85.23 |   25.756 |    56.673 | C:\iotest.dat (5120MB)
     4 |        11157504 |         1362 |       0.35 |      45.40 |   47.233 |    72.614 | C:\iotest.dat (5120MB)
     5 |        21438464 |         2617 |       0.68 |      87.23 |   23.498 |    50.224 | C:\iotest.dat (5120MB)
     6 |        17784832 |         2171 |       0.57 |      72.37 |   30.173 |    59.930 | C:\iotest.dat (5120MB)
     7 |        21282816 |         2598 |       0.68 |      86.60 |   24.409 |    55.087 | C:\iotest.dat (5120MB)
-----------------------------------------------------------------------------------------------------

and then create a new container - this time capping the max i/o:

docker run -it --name windowscorecapped  --io-maxiops 100 microsoft/windowsservercore cmd.exe

and again download and unzip diskspd and run the benchmark again with:

diskspd –b8K –d30 –o4 –t8 –h –r –w25 –L –Z1G –c5G C:\iotest.dat

Read IO
thread |       bytes     |     I/Os     |     MB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |         5390336 |          658 |       0.17 |      21.93 |   13.552 |   114.205 | C:\iotest.dat (5120MB)
     1 |         5849088 |          714 |       0.19 |      23.80 |   12.530 |   109.649 | C:\iotest.dat (5120MB)
     2 |         5619712 |          686 |       0.18 |      22.87 |   10.098 |    98.537 | C:\iotest.dat (5120MB)
     3 |         4931584 |          602 |       0.16 |      20.07 |   22.938 |   147.832 | C:\iotest.dat (5120MB)
     4 |         5758976 |          703 |       0.18 |      23.43 |   15.490 |   122.039 | C:\iotest.dat (5120MB)
     5 |         5611520 |          685 |       0.18 |      22.83 |   10.078 |    98.006 | C:\iotest.dat (5120MB)
     6 |         5283840 |          645 |       0.17 |      21.50 |   13.776 |   114.819 | C:\iotest.dat (5120MB)
     7 |         5169152 |          631 |       0.16 |      21.03 |   18.774 |   133.842 | C:\iotest.dat (5120MB)
-----------------------------------------------------------------------------------------------------
total:          43614208 |         5324 |       1.39 |     177.46 |   14.486 |   117.836

Write IO
thread |       bytes     |     I/Os     |     MB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |         1867776 |          228 |       0.06 |       7.60 |  487.168 |   493.073 | C:\iotest.dat (5120MB)
     1 |         1941504 |          237 |       0.06 |       7.90 |  464.425 |   492.295 | C:\iotest.dat (5120MB)
     2 |         1851392 |          226 |       0.06 |       7.53 |  500.177 |   493.215 | C:\iotest.dat (5120MB)
     3 |         1769472 |          216 |       0.06 |       7.20 |  487.016 |   492.672 | C:\iotest.dat (5120MB)
     4 |         1843200 |          225 |       0.06 |       7.50 |  480.479 |   492.992 | C:\iotest.dat (5120MB)
     5 |         1826816 |          223 |       0.06 |       7.43 |  507.046 |   492.908 | C:\iotest.dat (5120MB)
     6 |         1761280 |          215 |       0.06 |       7.17 |  516.741 |   492.909 | C:\iotest.dat (5120MB)
     7 |         1826816 |          223 |       0.06 |       7.43 |  484.961 |   492.556 | C:\iotest.dat (5120MB)
-----------------------------------------------------------------------------------------------------

As seen from the results above there is a significant difference in iops and latency between the two containers.

The other option is to throttle based on maximum bandwidth per second - for example based on 5mbps:

docker run -it --name windowscorebandwdith --io-maxbandwidth=5m microsoft/windowsservercore cmd.exe

0 comments:

Post a comment