Buffered vs Unbuffered I/O on Unix

TL;DR

• Buffered I/O batches data in user-space (or library) buffers and performs fewer, larger system calls. Great for throughput.
• Unbuffered I/O sends data directly (or more directly) to the kernel/device, offering lower-latency and more predictable timing — at a throughput cost for many small ops.
• Use buffered I/O for normal file processing and high-throughput tasks. Use unbuffered or synchronous I/O when you need immediate visibility, determinism, or strict durability.

What ‘buffered’ and ‘unbuffered’ mean

• Buffered I/O: The runtime (for example, the C standard library’s FILE*) keeps an intermediate buffer and only calls read()/write() when the buffer is full, empty, or flushed. This reduces the number of system calls and generally improves throughput.
• Unbuffered I/O: Your operations are passed to the kernel (or device) with minimal user-space aggregation. Examples: calling POSIX read()/write() repeatedly with small sizes.

Two layers of buffering on Unix

1. User-space buffering (stdio, language runtime) — what most people mean by “buffered I/O” at the application level.
2. Kernel page cache — even after write() returns, the kernel often holds the content in RAM (the page cache) until it writes to disk. To make data durable to stable storage you need fsync()/fdatasync() or O_SYNC/O_DSYNC semantics.

Understanding both layers matters: you may fflush() (user-space) and still need fsync() (kernel/disk) if you require persistence.

Performance and correctness tradeoffs

• Buffered I/O
  • Pros: fewer system calls, higher throughput for many small operations, convenient formatted output.
  • Cons: data can sit in user-space buffers; if your process crashes before flushing, that data is lost.
• Unbuffered / synchronous I/O
  • Pros: lower-latency visibility of output and more predictable timing; can be required for device or real-time uses.
  • Cons: more system calls, often lower throughput for many small writes, and synchronous disk writes can be slow.

When to use buffered I/O

• Normal file processing (reading/writing files, copying data).
• Workloads that perform lots of small writes — let a buffer combine them into larger kernel writes.
• Typical application logging where you accept eventual flush to disk (or rely on log rotation / external shipper).

Use the runtime defaults in most cases — they’re tuned for common workloads.

When to use unbuffered or synchronous I/O

• Interactive output that must be visible immediately (e.g., REPLs, progress bars). For stdio, use line-buffering or call fflush(stdout) after important messages.
• Critical logs that must be on disk before continuing (use fsync() or open with O_SYNC, understanding the performance cost).
• Devices/databases that manage their own caching and require direct control.
• Real-time or low-latency systems where predictable timing matters.

C examples: buffered vs unbuffered

Below are two minimal C programs that demonstrate the difference. The first uses the C standard library (buffered stdio). The second uses POSIX open()/write() directly (unbuffered from the user-space point of view).

Buffered (stdio):

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char **argv) {
  /* cast to void to silence compiler warnings */
  (void)argc;
  (void)argv;

  FILE *f = fopen("buffered.out", "w");
  if (!f) {
    fprintf(stderr, "Unable to open: %s\n",strerror(errno));
    exit(EXIT_FAILURE);
  }

  for (int i = 0; i < 1000000; i++) {
    /* goes into stdio buffer */
    fprintf(f, "line %d\n", i);
  }

  /* flush user-space stdio buffer into the kernel */
  fflush(f);

  /* optionally ensure the kernel pushes to stable storage */
  fsync(fileno(f));

  fclose(f);
  return EXIT_SUCCESS;
}

Unbuffered (POSIX write):

#include <errno.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char **argv) {
  /* cast to void to silence compiler warnings */
  (void)argc;
  (void)argv;

  int file_descriptor = open("unbuffered.out", O_WRONLY | O_CREAT | O_TRUNC, 0644);
  if (file_descriptor < 0) {
    fprintf(stderr, "Unable to open: %s\n", strerror(errno));
    exit(EXIT_FAILURE);
  }

  for (int i = 0; i < 1000000; i++) {
    char buf[64];
    int n = snprintf(buf, sizeof buf, "line %d\n", i);

    /* each write() is a separate system call
     * and bypasses stdio buffering
     */
    if (write(file_descriptor, buf, n) != n) {
      fprintf(stderr, "Unable to write: %s\n", strerror(errno));
      exit(EXIT_FAILURE);
    }
  }

  /* ensure kernel writes to stable storage */
  fsync(file_descriptor);
  close(file_descriptor);
  return EXIT_SUCCESS;
}

Quick compile/run:

$ cc --version
cc (nb3 20231008) 10.5.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ cc -Wall -Werror -Wextra buffered.c -o buffered
$ cc -Wall -Werror -Wextra unbuffered.c -o unbuffered
$ time ./buffered
        0.18 real         0.00 user         0.13 sys
$ time ./unbuffered
        1.26 real         0.15 user         0.94 sys

Compare the timing and observe that the buffered version typically issues fewer system calls and will often be faster for many small writes.

Quick decision guide

• Throughput for many small ops → buffered I/O (default). See previous example.
• Must see output immediately or guarantee ordering/durability → flush + fsync or use synchronous/unbuffered I/O.
• Device or DB-managed caching → follow the device/library recommendation (often unbuffered/direct I/O).

Final thought

Buffered I/O is the sensible default: it gives better performance with low complexity. Use unbuffered or synchronous I/O deliberately when you need its properties — determinism, immediate visibility, or strong durability — and measure the cost. When in doubt, prefer the default high-level APIs and add explicit flush or sync where correctness requires it.