One interesting find I made while working with the Seneca students on Fedora ARM was that a loopback filesystem hosted on top of an NFS share can outperform the NFS share. Yes, it's counter-intuitive, because that would seem to introduce additional layers of processing, but I think it makes sense.
When using NFS, file metadata management is performed by NFS. When loopback-mounting a filesystem in a file hosted on NFS, the file metadata management takes place entirely on the local system -- NFS merely provides a data store. In this sense, it's not much different from iSCS, because the loopback filesystem can't be readily accessed by two separate hosts at once.
In fact, on a small ARM system such as an OpenRD-Client, loopback-ext3-on-NFS over GigE handily outperforms both Class 6 SD and a local USB-PATA drive.
Why not just use iSCSI? Well, for reasons I haven't yet determined, the Fedora iSCSI initiator doesn't work reliably on ARM-- under heavy load, it sends invalid opcodes to the target. This sounds like an alignment issue, but alignment fixups don't cure it. Investigation continues...
You seem to be using NFS version 3. Version 4 brings much improved metadata caching on client.
OTOH, I find ATA-over-Ethernet much more lightweight compared to iSCSI. Just install vblade on server, "modprobe aoe" on client and you are ready to go.
About Chris Tyler
I am a Christian, college professor, computer programmer, system administrator, author, and consultant. My specialty is open source, particularly Fedora Linux.
These are my first two books: X Power Tools, a thorough guide to the X Window System (O'Reilly, ISBN 9780596101954) and Fedora Linux: A Complete Guide to Red Hat's Community Distro, a practical hands-on book on Fedora (O'Reilly, ISBN 9780596526825).
Fedora Linux is also available for online reading through Safari and in downloadable PDF format from oreilly.com