NFS server : increase worker threads

Posted on August 4, 2017  •  2 minutes  • 301 words  •  Suggest Changes

I have some NFS servers (read: the main function is to store data and share it over NFS) to maintain and mostly they simply work (like everything in Linux). Most variables have been battle tested for … well forever. So rarely do you need to check on the best trade-off. Well enter the NFS thread counter.

By vendor default, NFS will start with 8 threads. You can verify this by checking the running process or the statics in /proc.

You can check running processes (note that grep and the mother process is also there)

# ps aux | grep nfs | wc -l
11

Alternatively you can check using the proc value’s; the value behind th (for threads) is the running threads, the other value’s have been deprecated a while ago and are never populated.

# cat /proc/net/rpc/nfsd | grep th
th 8 0 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

Changing the value is simple, only the there is a trade-off, the threads will use some resources but this is in any current server trivial. The issue is that when there is only 1 request (RPC call) the other threads will try to read in advance, and hence all start accessing the disk. However the default 8 is to low, and will quickly become a bottleneck at heavy loads.  It’s therefor advised to increase this number. On a storage backend for a 8 head cluster I increased it to 128, this was the point where the workload did not go faster anymore.  (based on rudimentary checking of iostat)

To change it edit (Centos 6.x , 7.x) : /etc/sysconfig/nfs

# RPCNFSDCOUNT=8

to (for example)

# increased threadcount 04/08/2017 - should increase performance
RPCNFSDCOUNT=32

and restart NFS. (note : a reload won’t work)

service nfs-server restart

That’s it !