Photo format converter, CoW and BindFS
(Photo of Lake Geneva, taken near Lausanne)
Introduction
I’ve been processing RAW photos taken with my camera on my Mac. One crucial thing to watch out for is HDR—if it’s enabled, you need to be extra careful; otherwise, the output can look terrible. The final step in my editing workflow is exporting RAW images to more shareable formats. PNG supports HDR, but uncompressed PNG files often take up even more space than the original RAW files. To save space, I usually convert PNG files to HEIC using Finder’s built-in photo format converter.
Problem and Analysis
However, problems arise when converting a large number of photos. As you might expect, the conversion will fail if there isn’t enough space on the source or destination disk for the HEIC files. What’s less obvious is that it will also fail if your system disk lacks sufficient space to temporarily accommodate all source files. By inspecting the open files of related processes, I discovered that Finder copies all source files to a temporary directory in the user’s home folder (/Users/YOURNAME/Library/Group Containers/group.com.apple.shortcuts/Temporary), performs the conversion there, and then writes the results back to the source directory.
This process is relatively fast and efficient if your source files are on the same disk as your system because macOS requires APFS, which supports Copy-on-Write (CoW). With CoW, duplicating a file within the same partition doesn’t actually create a second copy until the files differ. However, in my case, all my PNG files are stored on an external USB disk. Since CoW doesn’t work across different disks, macOS has to physically copy every file to the internal disk and then copy it back—an inefficient and frustrating process.
My Tests
To work around this, I considered using bindfs, a special file system that maps a directory (possibly on another disk) to a different path, redirecting all read and write operations to the mounted disk instead of the original one. In theory, this sounded like a clean solution. However, the available macOS-compatible implementation of bindfs is based on FUSE, which does not support CoW. If you mount the temporary file directory back to the source disk, the copying process is treated as a standard write operation, which means that while it avoids writing to the internal disk, it’s still slow.
Digging deeper, I found that macOS’s kernel (XNU) actually has built-in support for bindfs. A project called mount_nullfs exposes this feature, but unfortunately, I had no success running it on my relatively recent macOS version. Checking the XNU source code, I confirmed that bindfs is still present in the latest macOS kernel. However, Apple has explicitly disabled write support for bindfs, whereas other modern BSD systems do not have this restriction. Another dead end.
Summary
In summary, Apple’s conversion tool makes strong assumptions about the storage environment. To avoid the internal disk space requirement, some form of redirection is necessary—but without CoW, efficiency is compromised. Perhaps someone could implement a CoW-based solution for FUSE, which would provide an elegant fix for this issue.