Same TIFF LZW. 13x parallel.
A C-ABI shared library registered via libtiff's own codec-plugin extension API. Every consumer of libtiff - GDAL, ImageMagick, Pillow, rasterio - gets the speedup with zero code changes, just by LD_PRELOAD-ing the TurboLZW shared library.
Drop-in for libtiff 4.7.1 LZW codec (Compression=5). SIMD backends: AVX2+BMI2, NEON, scalar.
Headline
Drop-in libtiff LZW - 1.15-1.20x faster on Intel, +33-35% on Apple Silicon, 13x via the parallel API.
Intel (was 0.59x at start); +33-35% on Apple NEON
1.07x to 11.0x on Intel x86
1000 × 64 KB tile batch — 9.78 GiB/s aggregate (reused buffers)
Single-thread decode
Every workload, every buffer size.
Speedup vs the libtiff C reference across the synthetic workload matrix — three data-character classes (highly compressible NoData / smooth structured / photographic noisy) at four buffer sizes from a small chunk up to a 1 MB strip. 12 of 12 cells faster than libtiff on Intel x86.
Decode speedup vs libtiff (single-thread)
The 64 KB and 256 KB cells are the 256² and 512² image-tile shapes RIPT and TurboLERC pipelines invoke single-thread. NEON on Apple Silicon tracks the same shape, with +33–35% gains on the photographic / 64 KB cell.
Multi-thread decode
Same chart. Multi-core baked in.
The same speedup-vs-libtiff view, but decoding the batch through the parallel batch API into a reused tile-cache buffer — how a real host (GDAL / rasterio) decodes many strips. Warm buffers pay the first-touch cost once, not per call, so the multi-core aggregate beats libtiff on every workload: 10–16× on compute-bound real imagery and gradients, and trivially-compressible NoData rides memory bandwidth even higher.
Decode speedup vs libtiff (multi-thread)
On a 24-core machine, each point is the parallel batch decode (into reused buffers) over libtiff decoding the same batch one strip at a time. Same batch shape per size across classes: 100 strips at 4 / 64 KB, 50 at 1 MB. Reusing buffers across the batch is what makes the difference — on real imagery and gradients the multi-core aggregate runs 10–16× faster than libtiff, and trivially-compressible NoData masks ride memory bandwidth even higher (up to 36× at 1 MB).
Parallel batch API
One API call. Multi-core scaling inside.
Real callers process many strips per TIFF. The parallel batch API bakes multi-core orchestration into the codec — one call instead of per-strip host plumbing. Decoding into reused buffers (the tile-cache shape real deployments use), every batch clears libtiff and a 24-core machine hits 13× over single-threaded. The win is cross-platform — Apple Silicon (16-core) lands the same shape at 12.3× / 6.51 GiB/s on a 1000 × 64 KB SAR batch.
Parallel batch API: throughput on a 24-core host· 24-core x86
decode_strips_par)Same shape on Apple Silicon (12 P + 4 E cores) reaches 9.81× on the 1000 × 64 KB tile batch and 9.77× on 50 × 1 MB strips. Photographic / noisy workloads shown; smooth and zeros workloads scale similarly. A parallel batch encode exists as the same shape on the write side.
Multi-thread vs the C reference
Every workload, 11–16× vs libtiff.
Same comparison, cut by batch shape instead of buffer size. libtiff has no in-codec parallelism — a host decoding a batch loops it one strip at a time. Point the parallel batch API at the same batch (reused buffers) and the end-to-end win against the C reference runs 11× to 16× across the real-imagery and gradient workloads.
Multi-thread decode vs libtiff — by batch shape· 24-core x86
Each bar is the parallel batch decode (into reused buffers) over libtiff decoding the same batch one strip at a time — the end-to-end speedup a GDAL / rasterio host sees by adopting the batch API into a reused tile cache. The 1000 × 64 KB SAR shape hits 9.78 GiB/s against libtiff's 0.65 GiB/s (15.1×). Photographic and structured classes shown; full per-class scaling is in the line chart above.
Why switch
A library swap. Not a migration.
- - Drop-in via libtiff's own TIFFRegisterCODEC extension API. The supported plug-in mechanism, not an unsupported symbol override.
- - 256x256 tile cell (the per-tile ingest shape RIPT and TurboLERC pipelines invoke) was 0.59x of libtiff at the start of the optimization cycle. Now 1.15-1.20x on Intel.
- - 12 of 12 synthetic decode cells beat libtiff (1.07x to 11.0x). 12 of 12 encode cells too (1.01x to 4.21x).
- - Parallel batch APIs bake multi-core orchestration inside the codec — decode or encode a whole batch in one call. Decoding into reused buffers, every cell beats libtiff and a 24-core machine hits 13x over single-threaded (9.78 GiB/s on a 1000 × 64 KB batch).
- - End-to-end on real Planet, ICEYE, and flood-survey imagery: full-file transcodes flipped from 3 of 8 to 7 of 8 cells beating gdal_translate.
Compatibility
Bit-exact on the wire.
- - Bit-exact wire compatibility with libtiff's tif_lzw.c (TIFF Compression=5, MSB-first early-clear variant).
- - Validated by bidirectional parity oracle on 26-test corpus + 400 proptest cases.
- - A memory-safe API for callers who want a dependency-free native path.
- - Runtime SIMD dispatch: scalar fallback always present; AVX2+BMI2 and NEON detected at process startup.
- - Failure mode is graceful fallback to libtiff's own LZW path - the swap is reversible.
libtiff LZW.
Just faster, safer.
Existing pipelines, existing data, existing tooling. Swap the implementation, keep everything else.
Comparisons on this page reference the LZW codec in libtiff, distributed by its maintainers under the libtiff (MIT-style) license; the name is used solely to identify the implementation being compared, and no affiliation, sponsorship, or endorsement is implied. The LZW patents expired worldwide in 2003–2004; the algorithm is freely usable. TurboLZW is an independent, clean-room implementation. Performance comparisons reflect our own measurements under the stated methodology; results vary by workload and hardware. Full third-party notices: Attributions.