Windows Task Manager, RAM and the Windows Metafile

I have sometimes noticed slight discrepancies between the available RAM presented in Windows Task Manager and the sum of private and shared bytes processes use - although on one particular server this discrepancy was a whopping 4GB!

Doubting the accuracy of what the Windows Task Manager was reporting I checked a few other tools, including Process Hacker - which provides me with a more granular look at the system's RAM consumption, private, shared memory etc. However the figures still did not add up - so I ended up coming across a tool called RAMMap by sysinternals.

This tool provides you with everything included by the above, but in addition detailed paging and kernel consumption information.

However there was an item called 'Metafile' - This metafile is used part of the Windows System cache and holds NTFS metadata - which is used to provide increased performance when accessing files.

For each file that is accessed on the disk there is a corresponding block of 1kb (or if a file has e.g. 3 attributes - this would be 3kb). Now on a system such as a fileserver that potentially has millions of files this could soon add up and could easily be 10GB plus.

You do have the ability to empty the metafile - however doing this could have adverse (but temporary) effects on the server - if they are cleared out of RAM - there will likely be a surge of activity reading the metafile again from the disk! If you really need to perform this you can click on the 'Empty' drop down on RAMMap and select 'System working-set.'

The recommended approach is to install the 'Dynamic Cache Service' - which can be downloaded from here.

Extract the archive and copy 'DynCache.exe' to '%SystemRoot%\System32'

and then cd to the Systemc32 directory and and create a service for it by running the following:

sc create DynCache binpath= %SystemRoot%\System32\DynCache.exe start= auto type= own DisplayName= "Dynamic Cache Service"

Now open regedit and go to the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DynCache\Parameters

and ensure the 'MaxSystemCacheMBytes' are set to (at least) half of the amount of physical RAM you have - for example if you had 8GB of RAM you could set it to: 4096 (mb)

and the minimum cache size ('MinSystemCacheMBytes') to 512 (mb).

Finally start the service with:

sc start DynCache

Note: You can increase / decrease the values above without having to restart the DynCache service!

Sources

Fixing High Memory Usage by Metafile on Windows Server 2008 R2: http://woshub.com/fixing-high-memory-usage-by-metafile-on-windows-server-2008-r2/