nfs-crust on AWS EFS

Key results

Each measurement covers the whole operation as the application experiences it. A create-new write includes creating a temporary file, flushing it to durable storage, checking its size, making the finished file visible in one step, and cleaning up. The Linux comparison follows the same steps. Reads served from Linux's memory cache are labeled separately.

If a create-new write lost its final server reply, it counted as successful only after the benchmark proved that the destination contained the expected data, or proved that it was absent and then retried successfully. The report counts these recovered results separately.

How to read this report

• Amazon EFS is a network file system. This report compares two clients for it: nfs-crust and the Linux NFS client.
• KiB and MiB are binary data sizes: 1 KiB is 1,024 bytes and 1 MiB is 1,024 KiB. ops/s means completed operations per second; MiB/s means payload data completed per second.
• µs means microseconds, or millionths of a second. ms means milliseconds, or thousandths of a second. CPU-ms/op is the processor time used for each completed operation.
• p50 is the median or typical response time: half of operations completed faster and half slower. p99 is the slowest-1% threshold: 99 of every 100 operations completed at or below it. p99.9 is the equivalent threshold for 999 of every 1,000 operations.
• Concurrent workers repeatedly start an operation and wait for it to finish before starting the next. More workers allow more operations to be in progress at once.
• Scheduled-load tests start operations at a fixed target rate even when earlier operations are still unfinished. This exposes waiting and queue buildup that the worker tests can hide.
• 85% means 85% of a client's measured peak request rate. It does not mean processor use, EFS I/O use, or any other resource-utilization percentage.
• For response time and processor cost, lower is better. For throughput, higher is better.

Test setup and fair comparison

nfs-crust and Linux ran on the same EC2 virtual machine and accessed the same EFS file system through the same Availability Zone. Both used encrypted NFSv4.1 connections. The table shows how the two paths were made comparable.

Response time by object size

These tests run one operation at a time. Each chart compares the typical response time (p50) with the slowest-1% threshold (p99) as object size changes. Each plotted value is the median across 2 separate runs; samples from different runs are not mixed together. Both axes use logarithmic spacing, so each step represents multiplication rather than a fixed addition.

How to read each band: bold upper edge = slowest-1% threshold (p99) · faint lower edge = typical response time (p50) · tinted area = gap between them · hollow upper marker = fewer than 1,000 successful operations in at least one run.

Scroll charts sideways to see the full plot without shrinking the labels.

Throughput as concurrent work increases

These tests use 1, 8, or 32 workers. Each worker starts its next operation only after its previous operation finishes. Throughput is the median across repeated runs and counts payload bytes completed through the end of the operation, not bytes sent over the network.

Response time under scheduled load

These tests schedule 128 KiB known-size reads and create-new writes at a fixed rate, whether or not earlier operations have finished. Response time begins at the scheduled start time, so it includes time waiting for its turn to run as well as time doing the operation.

Same request rate gives both clients 85% of the slower client's measured peak request rate. Each client at its own rate gives each client 85% of its own measured peak. These percentages describe scheduled request rates—not processor use or EFS utilization.

The benchmark uses the demanding 85% load level. Client order was reversed across runs to reduce order bias.

Sample-size check: 88 of 128 runs have fewer than 1,000 successful operations for p99; 123 have fewer than 10,000 for p99.9. Those minimums provide about ten operations beyond each threshold. They do not, by themselves, guarantee statistical confidence.

Resource use

Processor cost includes every thread in the benchmark process and is divided by the number of successful operations. Host network counters cover every external network interface and may include unrelated traffic. Per-configuration values are in All results.

Data quality and correctness

• 6 create-new operations returned without a definite final outcome and required a follow-up check before being counted as successful.
• Host-wide TCP counters recorded 171 retransmitted segments and 15 established resets during the run.

• 128 runs across 64 configurations produced 197,574 successful operation samples. 6 create-new outcomes required a follow-up check after the final server reply was lost or inconclusive.
• Data-content checks run outside the timed operation. The benchmark verifies replaced files, a sample of newly created files, and deleted files by reading them through the other client.
• An error during untimed preparation, or a measurement with no successful operations, invalidates that test configuration. A follow-up check that is mismatched or inconclusive remains an error.
• Downloaded-file integrity: all 22 downloaded-file checksums verified. Raw success/error counts, follow-up-check flags, and response-time data agree with each run's summary.

Method

• The test created a fresh, encrypted Regional EFS file system using General Purpose performance mode and Elastic throughput. Its network endpoint was in the same subnet and Availability Zone as the EC2 test machine.
• Both clients connected to the same private EFS address from one dedicated c7g.xlarge. Timed operations did not pass through the developer workstation or AWS Systems Manager.
• Both clients used NFSv4.1 over TLS encryption with traditional Unix user/group credentials. nfs-crust verified the EFS server's DNS name against public certificate authorities; Linux used Amazon's EFS mount helper. EFS identity-based (IAM) authorization was off.
• nfs-crust requested up to 64 in-flight NFS requests per connection, split reads into 1 MiB chunks, and split writes into 504 KiB EFS-compatible chunks. Individual tests requested 128 KiB or 1 MiB at a time.
• Linux reads used o-direct mode to bypass the file-data memory cache. Linux file-metadata and directory operations used its normal mounted-file-system path. Results explicitly labeled as memory-cache context are never presented as comparable remote EFS reads.
• Results apply to this EC2 type, EFS configuration, region and Availability Zone, Linux kernel, and measurement window. Each configuration has 2 separate runs. The uncertainty intervals describe variation among those runs; they do not capture other machines, times, regions, or production environments.
• Read tests rotate among two files per worker. Linux bypassed its memory cache, but both clients still read files that had already been accessed. These are not first-read or production-traffic measurements.

All results

Each main value is the median across separate runs. Brackets show a 95% uncertainty interval calculated by resampling those run-level values, followed by the lowest and highest observed run. Operation samples from separate runs are never mixed together. “Reconciled” counts create-new writes that needed a follow-up check after an inconclusive final reply. † means at least one run had fewer than 1,000 successful operations for p99; ‡ means at least one had fewer than 10,000 for p99.9.