Content List Toolkit

Content List Toolkit is a desktop tool for archival migration workflows. Point it at a folder, and it produces a CSV inventory of every file under that folder — relative path, size, modified date, and an optional hash. Output streams to disk so multi-million-row scans do not run out of memory.

Three additional workflows live alongside the scan: copy every email-related file under a tree into a curated destination, compare two drives' content lists to verify a clone, and check the Getty tags on a CONTENTdm export before it is uploaded.

Pick the build that matches your platform. All builds are unsigned at this stage and the OS may prompt before running.

Latest release: github.com/snyderb-de/content-list-toolkit/releases/latest

The GUI is organized as a vertical sidebar plus a workspace pane that swaps based on which workflow you pick.

1. Sidebar. Three workflow tabs — Content List, Email Copy, Clone Compare — plus theme toggle.
2. Source folder. Browse button. The folder you point this at is scanned recursively.
3. Output folder + filename. Where the CSV (or copy destination) lands. Defaults to the source.
4. Settings panel. Hash algorithm, exclusions, XLSX option, folders-only toggle.
5. Generate. Starts the work. Progress bar + live stats appear.
6. Done summary. Top extensions by count and by size, filtered samples, output paths.

Produces a recursive CSV inventory of every file under a folder.

1. Pick a source folder. Browse to the drive or folder you want inventoried. Network drives work too, but expect slower throughput.
2. Pick an output folder. Defaults to the source. Choose somewhere with enough free space — a CSV of 1M rows is roughly 200 MB.
3. Tune settings. See the Settings reference below — hash algorithm, exclusions, XLSX, etc.
4. Click Generate. The app first counts everything (so progress + ETA are real), then streams the scan to CSV.
5. Review the done summary. Top extensions, totals, sample of filtered rows.

What each toggle does

Finds every email file under a source folder and copies them into a destination, preserving the relative folder structure. Writes a manifest CSV of what got copied.

1. Pick the email-copy tab in the sidebar.
2. Source folder — the tree to search for email files.
3. Destination folder — where copies land. Folder structure under the source is mirrored.
4. Click Copy. The app walks the tree, copies files with matching extensions, and writes a manifest CSV.
5. Review the manifest. The destination contains email-copy-manifest-{timestamp}.csv with one row per copied file: Source Path, Destination Path, Relative Path, File Name, Extension, Size in Bytes.

Compares two drive content-list CSVs to verify a clone. Picks one of several verdicts and writes a diff CSV showing every difference.

1. Generate a content list for the original drive (use BLAKE3 or SHA-256 for confidence).
2. Generate a content list for the cloned drive — same hash algorithm.
3. Open the clone-compare tab, point it at both CSVs.
4. Optional: enable soft compare — accepts renamed paths with identical hashes as matches.
5. Click Compare. Get a verdict + diff CSV.

Soft compare (PDFs)

Off by default. When enabled and a PDF's hashes differ at the byte level, the comparator re-reads the last 2 KB of each PDF, masks the PDF /ID trailer (the random pair PDF writers stamp on every save), and accepts the pair as a soft match if the masked tails are identical. Use this when a tool re-saved every PDF without changing rendered content — the verdict becomes Metadata Clone instead of Not a Clone.

Soft compare only runs on .pdf files. Other file types fall through to strict hash equality.

Checks the Tags column of a CONTENTdm export before you upload it. It does two jobs: it removes the invisible characters that stop a tab-delimited file loading, and it checks every term against the Getty Art & Architecture Thesaurus. It writes a corrected copy and a report, and never changes the file you point it at.

1. Export the Access Main table to Excel, as the CONTENTdm process already has you do.
2. Open Getty Tags and browse to the export. .xlsx, .csv, and tab-delimited .txt all work — including the .txt you are about to upload.
3. Choose where terms are verified. See the table below.
4. Click Check Tags. Rows with nothing wrong are not listed; a clean sheet says so.
5. Fix what it found. Edit a tag directly, or click a suggested replacement.
6. Apply Fixes & Re-check. Repeat until nothing is left, then upload the cleaned copy.

What it writes

Both files land beside the export. The export itself is never modified — it is the record of what Access held.

Use Show in Folder to find them. A sheet with nothing to correct gets no cleaned copy — an identical file would only raise the question of which one to upload.

Where terms are verified

Three choices, and they are not equivalent. Pick the first one your network allows.

Getting a fresher list

Build list from a Getty archive turns a Getty relational archive (aat_rel_<mmyy>.zip, about 73 MB) that you already have into an English term list, written beside the archive and selected for you. The publication date is written into the filename.

The app does not download that archive itself: Getty serves it over an unencrypted connection, and this app makes none. Fetch it separately if you need one, then point the app at the file.

If Getty is blocked here, Getty network access lists every request the check makes, with hosts and ports, written for whoever approves the firewall exception.

Content list CSV columns

Creation-date note: copying, restoring, downloading, or migrating a file can change its filesystem creation time. It is metadata observed during the scan, not proof of when the document contents were authored.

Folders-only mode writes a single column: Path From Root Folder.

Companion files

Every scan also writes a -report.txt alongside the CSV. It contains totals, filtered samples, top extensions, and the settings used.

my-folder-content-list-2026-05-20T13-45-02.csv
my-folder-content-list-2026-05-20T13-45-02-report.txt
my-folder-content-list-2026-05-20T13-45-02.xlsx   (if XLSX enabled)

Default base name is {source-folder}-content-list-{YYYY-MM-DDTHH-MM-SS}.csv. In folders-only mode it becomes …-folder-list-….

Hashing reads every byte of every file. Choose based on speed vs. cryptographic strength.

CSV files over a few hundred thousand rows get unwieldy in Excel and many text editors. Content List Toolkit splits output every 300,000 rows by default.

XLSX files are split using the same rule. Each chunk includes its own header row.

Always-excluded files

These never appear in output regardless of settings:

Always-excluded directories

Recycle bins, indexer caches, and other OS infrastructure folders.

• $RECYCLE.BIN, System Volume Information
• .Spotlight-V100, .Trashes, .fseventsd
• .TemporaryItems, .DocumentRevisions-V100

User-toggleable filters

• Hidden files — names starting with . or with the hidden attribute set.
• System files — short list of OS metadata files.
• Extension list — comma-separated. Match is on the lowercased extension without the dot.

The Copy Email Files workflow looks for files with any of these extensions:

Compare results land on one of these verdicts. The diff CSV always lists every actual difference, regardless of verdict.

Every finding leads with what it wants from you.

What it looks for

If Getty cannot be reached

The indicator beside the live option turns red before you run anything, and names the cause — a blocked host, a DNS failure, or a proxy intercepting the connection. Cleaning and every structural check still work; only term verification stops. Terms are then reported as not checked, never as wrong, because sending someone to re-verify metadata that was correct is worse than saying nothing.

Switch to the built-in list and the check runs with no network at all.

On Windows, the app checks the X:\Apps folder for a newer signed content-list-generator.exe. Open About if support gives you a different release folder.

• The app only installs a newer signed executable from the configured release folder.
• When an update is ready, restart the app from the update prompt.

Which download do I need?

macOS: the .app. Linux: the binary. Windows: the .exe. There is one application now — the Python runtime was retired once the Windows .exe could do everything it did. The .exe needs no admin rights and no installer, and runs from a USB drive or network share, so it suits managed fleets as well as personal machines.

Does it work on network drives?

Yes. SMB / NFS / mapped drives all scan correctly. Throughput is limited by network round-trip time per file, so a network scan is typically 5–20× slower than a local scan. Run with hashing off first to validate the inventory, then re-run with BLAKE3 if you need verification.

How large a scan can it handle?

There is no fixed cap. CSVs auto-chunk every 300,000 rows, and the scan streams to disk so memory stays flat regardless of row count. Multi-million-row scans are routine. The dominant cost is hash mode (full byte read of every file) and disk throughput.

Why are $RECYCLE.BIN / System Volume Information not in the output?

These OS infrastructure folders are always excluded, with no user toggle, because they never contain archival content. The same applies to swap files (pagefile.sys, hiberfil.sys, swapfile.sys), macOS metadata (.Spotlight-V100, .Trashes, .fseventsd), Office lock files (~$*, .~lock.*), and resource forks (._*).

Can I resume an interrupted scan?

No — stopping a scan discards its partial CSV. Scans are fast enough that restart is the safer model. For very large drives, work folder-by-folder so a stop only loses one subtree.

Do any of the hash options need something installed?

No. BLAKE3, SHA-1, and SHA-256 are all built into the application via github.com/zeebo/blake3 and the Go standard library. This used to be a real question on the retired Python runtime, where BLAKE3 needed a separate package.

Will email-copy modify or open .pst / .ost files?

No. Email-copy is a pure file copy by extension — it never opens, parses, or mutates email containers. Outlook can keep .pst / .ost exclusively locked while open; close Outlook before copying or expect "file in use" errors.

What if soft compare returns Metadata Clone but I want to know which PDFs differed?

The diff CSV always lists every divergence, regardless of verdict. Look for rows where Difference Type indicates a hash mismatch — those are the PDFs that matched only after /ID normalization.

Why is the XLSX file split into multiple files?

Excel's .xlsx format caps at 1,048,576 rows per sheet, and Excel itself becomes sluggish well below that. Splitting at 300,000 rows keeps each file responsive and aligns with the matching CSV chunks for easy cross-referencing.

How do I reset all settings?

Close the app, delete the settings file at the path listed in About for your OS, relaunch. The app re-creates it with defaults on next save.

• Start with hashing Off to validate paths and counts, then re-run with BLAKE3 once you trust the inventory plan.
• Stop a scan with s or esc in the TUI; in the GUI use the Stop button.
• Filtered samples in the done summary tell you what got skipped. Glance over them to confirm exclusions did what you expected.
• For migrations, write the original and clone CSVs to the same output folder so the diff is easy to file alongside.
• The Windows .exe needs no admin rights and no installer — it runs from wherever you put it. Recommend it to less-technical users.
• Folders-only mode + a depth cap is the fastest way to map a drive before deciding what to scan in full.
• Settings persist across launches per-user. Reset them by deleting the settings file (path listed in About).

Diagnostic info you may need to share with whoever set up your install.