TL;DR

Claude Code cloud sessions don’t install the plugins a repository’s .claude/settings.json turns on, and their GitHub proxy only serves the repositories attached to the session. So a plugin from a private marketplace doesn’t load there. Its skill still can. Zip the skill folder, upload it under Customize > Skills on claude.ai, and turn it on. In a new cloud session with nothing attached, it showed up as anthropic-skills:<skill>. The trade-offs: you upload again for every release, and the skill loads in every session on your account. A missing skill doesn’t raise an error. Claude still answers, but without the skill’s process, so ask the session to list its skills.

I was working on a project spread across several private repositories, and I created a skill I wanted to share between them as a Claude Code plugin. My plan was a private GitHub repository that’s both the plugin and its marketplace, so the skill stays in sync on my laptop and in cloud sessions. Most of my work happens in Claude Code on the web, because I like that it keeps the work off my laptop, runs in isolation, and is there on my phone. I tested the install commands by hand in a cloud session, then put them in the environment’s setup script, and the test worked. In the next real session the skill never loaded, and I got no error, not even a small one. Here’s how I tracked down the cause, what I tried that didn’t work, and the two workarounds that did.

Skill doesn’t load in a cloud session, and there’s no error

The session starts normally. You paste the notes the skill is built for, and Claude gives a reasonable, generic answer instead of following the skill.

That’s what makes it easy to miss. When a skill doesn’t load, Claude still answers, and nothing hints that a skill should have run. The only clue I had was that the answer was good but didn’t follow the skill’s steps.

My test had passed only because of how that session was set up. The session I tested in had the skills repository attached as one of its sources. The real sessions had only the repository I was working in. I also never confirmed that the saved setup script still had the install commands in it. Its log only showed output from another installer.

A second session found the cause. I messaged the failing session from the one where I built the plugin and asked it to inspect its own environment: the plugin list, the contents of ~/.claude, and the settings files. It reported that no marketplace had ever been added.

Cloud session can’t reach a private repository that isn’t attached

A cloud session reaches GitHub through a proxy, and the cloud environments documentation (opens in a new tab) says it serves GitHub API requests only for the repositories attached to the session. Git behaved the same way for me. In a fresh session with only the consuming repository attached, I tried to reach the skills repository:

$ git ls-remote https://github.com/<owner>/<skills-repo>.git

fatal: could not read Username for 'https://github.com': terminal prompts disabled

No credentials were supplied for it, so git asked for a username and gave up, even though the Claude GitHub App had access to that repository on my account. I didn’t run the same command against the attached repository to compare. The same session hit the limit again, this time with nothing to do with plugins: npm ci couldn’t fetch the project’s private SDK, which lives in another private repository that wasn’t attached.

Here’s what that means for each way of installing the plugin:

• Repository settings: The documentation (opens in a new tab) lists plugins and marketplaces declared in a repo’s .claude/settings.json as not available in cloud sessions. I never put the plugin there, so this is what the docs say, not something I tested.
• Setup-script install: I never confirmed my install commands ran, and the log was empty. If the script can only reach attached repositories, like the session, it had nothing to fetch. I haven’t tested that.
• Plugin installed mid-session: In one session, installing from an attached copy left the plugin enabled but not loaded, and the cloud UI has /reload-skills but no /reload-plugins. In an earlier session the same install did make the skill appear. I don’t know what was different.

Plugin added on claude.ai never reaches the session

Your claude.ai account has a plugins page too. Under Customize > Plugins > Add > Add marketplace > Add from a repository, claude.ai takes a private GitHub repository once the Claude GitHub App can read it. The plugins overview (opens in a new tab) says a plugin you add there downloads to Claude Code as a synced plugin, which the plugin loading reference (opens in a new tab) names <name>@synced. That reference says synced plugins load in Cowork and in signed-in terminal sessions. It doesn’t mention cloud sessions, although mine did get Anthropic’s synced plugins.

The marketplace synced and the plugin showed as added, with no error. In the ten minutes I watched, it never showed up anywhere:

• Cloud session, started five minutes after the add: The synced manifest under ~/.claude/plugins/synced/ listed 10 plugins, all from Anthropic’s knowledge-work-plugins marketplace. Mine wasn’t there.
• Terminal on my Mac, started ten minutes after the add: It printed Plugins changed. Run /reload-plugins to activate., and the rewritten manifest listed 4 plugins from the same Anthropic marketplace. Mine wasn’t there either.
• claude plugin marketplace list: Its From claude.ai: section listed only My Uploads and the Anthropic Directory.

One caveat: my Mac already had the same plugin installed from the command line, which may have affected that result. The cloud session had no local copy, so its result is clean. Add > Upload plugin is a second way to put a plugin on your account. I didn’t try it.

An open issue, anthropics/claude-code#92031 (opens in a new tab), describes the same thing from another account: plugins installed on claude.ai never reach cloud sessions, while skills enabled on the account do. That’s the route in the next section.

Upload the skill to your claude.ai account

A skill doesn’t need to be a plugin to reach a cloud session. The skills documentation (opens in a new tab) says cloud sessions load the skills enabled for your claude.ai account, synced at session start.

1. On your machine, zip the skill folder, with its SKILL.md inside the folder. To build the zip from a release tag in the skills repository, run:

   Command

   $ git archive --format=zip --prefix=<skill>/ -o <skill>.zip \
       <tag>:plugins/<plugin>/skills/<skill>

2. On claude.ai, go to Customize > Skills > Add > Upload skill, upload the zip, and turn the skill on.

3. Start a new cloud session, and ask Claude to list its skills.

On upload, claude.ai previews the skill’s name and description and runs a security scan. In a new cloud session with only the consuming repository attached, the skill showed up as anthropic-skills:<skill>. I checked that in one session. My test of Claude actually using the skill, in the section on confirming it loaded, ran on the copied version from the next section, which has the same files.

There are two costs. claude.ai numbers the skill’s versions v1, v2 and so on, not by your git tags, and each release means another upload. It’s an account skill, so it loads in every session on the account, not only the repositories that use it. That includes terminal sessions: on my Mac it appeared beside the same skill from the installed plugin, so the session offered it twice. Uninstall the local plugin, or turn the account skill off, to keep one.

Copy the skill into the cloud session

If a session needs a specific release, copy the skill in from the skills repository instead. The skills documentation (opens in a new tab) says Claude Code watches ~/.claude/skills/ and picks up a skill added there during the session, without a restart.

1. Start the cloud session with two repositories attached: the one you’re working in and the skills repository.

2. Ask Claude where the skills repository was cloned. It was /home/user/<skills-repo> in my sessions.

3. Ask Claude to copy the skill folder into your user skills directory, changing nothing else:

   Command

   $ mkdir -p ~/.claude/skills
   $ cp -R /home/user/<skills-repo>/plugins/<plugin>/skills/<skill> \
       ~/.claude/skills/<skill>

4. Ask Claude to list its skills. If ~/.claude/skills/ didn’t exist when the session started, run /reload-skills first, because Claude Code wasn’t watching it.

The skill loads as plain <skill>, with no prefix. To copy a specific release, run git -C /home/user/<skills-repo> fetch --tags and git -C /home/user/<skills-repo> checkout <tag> before the copy. I haven’t tested that step. The copy doesn’t carry over to other sessions.

If every session in a repository needs the skill, commit a copy under that repository’s .claude/skills/ instead. The documentation lists the repository’s own skills as available in cloud sessions. The cost is a copy in every repository, which is what the plugin was meant to avoid. I built it once, checked it against the release tag, and didn’t merge it.

Both routes bring over the plugin’s skills only. Hooks, agents and MCP servers in the plugin don’t load this way, and I haven’t found a way to load them in a cloud session without /reload-plugins.

Confirm the skill loaded

There are two checks. The session’s skill list shows the skill loaded. Asking a question the skill is built for, without naming the skill, shows that Claude actually uses it. My first test was a request to look over a project and say where it could be better. The session announced the skill’s first step and worked through its process. If the answer is generic, ask which skill it used.

For the skill list, ask Claude to list its skills, and look for <skill> or anthropic-skills:<skill>.

A plugin that shows as enabled in claude plugin list isn’t proof. In one of my sessions it was enabled and never loaded. A folder in ~/.claude/skills/ isn’t proof either, until the session lists it.

Do this with a coding agent

What found my cause was an agent inspecting its own environment. Paste this into the coding agent in the cloud session you want to check. Run it once in a session where the skill works and once where it doesn’t, then compare the two reports. Replace the placeholders first.

Inspect your own environment, read-only, and report whether a skill is available and where it came from. Change nothing: do not install, enable, disable, add, remove, copy, or edit anything, and do not print any token, key, or password you come across.

Skill name: <skill-name>
Plugin name: <plugin-name>
Skills repository: <owner>/<skills-repo>

1. Print the Claude Code version.
2. List the installed plugins with their version, scope, and status. Note which ones are <name>@synced.
3. List the plugin marketplaces that have been added.
4. List ~/.claude/skills/ and ~/.claude/plugins/, including plugins/synced/.
5. List the repositories attached to this session, and say whether the skills repository is one of them.
6. Run git ls-remote https://github.com/<owner>/<skills-repo>.git and quote any error verbatim.
7. Report whether this repository's .claude/settings.json has enabledPlugins or extraKnownMarketplaces entries.
8. Report whether a setup script log exists for this session and whether it contains output from a plugin install.
9. List the skills available to this session, and say whether <skill-name> is among them and under what name.

Then answer in this form:
- Skill <skill-name> available: yes or no, and under what name
- Plugin <plugin-name> installed: yes or no, and whether it is loaded or only enabled
- Synced plugins: the names, or none
- Skills repository attached: yes or no
- Skills repository reachable with git: yes, or the error
- Plugin settings in the repository: the entries, or none
- Setup script install output: found, not found, or no log
- Anything you could not check, and why

Common failures

Git fails with terminal prompts disabled

fatal: could not read Username for 'https://github.com': terminal prompts disabled

In a Claude Code cloud session, git prints this for a private repository that isn’t attached to the session, because the GitHub proxy supplies no credentials for it. A mistyped repository name probably produces the same prompt. Check the name, then attach the repository or start a session with it attached. The documentation describes the proxy’s answer to a GitHub API request for an unattached repository as a 403 whose message starts GitHub access to and contains is not enabled for this session.

Setup-script install leaves no trace

First check that the saved setup script still contains the install commands. Mine was never checked. If it does, a private skills repository that isn’t attached is a likely cause. The documentation (opens in a new tab) also says later sessions can start from a cached snapshot and skip the setup script, so a missing log line doesn’t prove the script failed. Check the session itself with claude plugin list.

enabledPlugins in .claude/settings.json does nothing in a cloud session

A cloud session doesn’t install the plugins a repository turns on under enabledPlugins. The plugin loading reference (opens in a new tab) says it doesn’t add the marketplaces under extraKnownMarketplaces either, because that needs the workspace trust dialog, which a cloud session never shows. Those keys still work for local sessions. The open issue anthropics/claude-code#78119 (opens in a new tab) tracks this, and a comment there reports that plugins declared in organization-managed settings don’t install in cloud sessions either. I can’t test that on an individual account.

Plugin is enabled but the skill isn’t listed

A plugin installed after the session starts loads only when Claude Code restarts or runs /reload-plugins, and the cloud UI has no /reload-plugins. Upload the skill to your claude.ai account, or copy it into ~/.claude/skills/.

Install worked in a test session and fails in the real one

What’s attached to a session decides what it can reach. Test in a session set up like the one that will use the skill, with the same repositories attached and nothing more.

Still open: a way to load a whole private plugin, with its hooks, agents and MCP servers, into a cloud session without attaching its repository. Its skills have a route. I haven’t found one for the rest. If you’ve found a way that works, I’d like to hear about it. Reply to any newsletter issue.

TL;DR

Quit every coding agent and editor that launches MCP servers, then run uv cache prune. A running session holds the cache lock, so prune prints “Cache is currently in-use, waiting for other uv processes to finish” and, after uv’s lock timeout, gives up. Do not use --force to skip it. Expect much less back than prune reports: on this Mac it reported 111.1 GiB (about 119 GB) removed and the disk gained 27.5 GB, most likely because cloned files share blocks.

$ uv cache size -H
$ df -h /
$ diskutil info / | grep "Container Free Space"
$ lsof "$(uv cache dir)/.lock"
$ uv cache prune

My Mac was down to 38.4 GB free on a 1 TB disk. The largest directory on it was a 132.7 GiB uv cache that I had never thought about. The coding agents I use launch MCP servers with uvx, and that was enough to fill it.

uvx is uv’s command for running a Python tool from a temporary environment, without installing it. If you have never run uv by hand, an agent or editor ran it for you. This page is part of a series on cleaning up a developer Mac’s disk. Part one of the series, finding what’s filling the disk, is how to know the cache is the cause; you don’t need it to follow the steps here.

Check the size and location of the cache

The environments uvx builds go in uv’s cache. Ask uv for the size and the directory:

$ uv cache size -H

warning: `uv cache size` is experimental and may change without warning. Pass `--preview-features cache-size` to disable this warning.
132.7GiB

$ uv cache dir

/Users/<user>/.cache/uv

uv labels uv cache size as experimental, which is the reason for the warning. The number is still the one to start from. The --cache-dir option, the UV_CACHE_DIR and XDG_CACHE_HOME variables and the tool.uv.cache-dir setting move the directory, according to uv’s cache documentation (opens in a new tab). Every command below reads the directory from uv cache dir, so it follows a moved cache.

Find what fills the cache

Environments that uvx builds sit in the cache’s archive-v0 directory. Count its entries, then count the ones that are Python environments, which have a pyvenv.cfg at their root:

$ ls "$(uv cache dir)"/archive-v0 | wc -l
$ find "$(uv cache dir)"/archive-v0 -maxdepth 2 -name pyvenv.cfg | wc -l

On this Mac, archive-v0 held 4,005 entries, and 2,562 of them were environments. To see which MCP servers the environments belong to, count the executables with mcp in their names in the environments’ bin directories (a server named otherwise needs its own pattern):

$ ls -d "$(uv cache dir)"/archive-v0/*/bin/*mcp* | xargs -n1 basename | sort | uniq -c | sort -rn | head

Nearly all of the environments were copies of three MCP servers, with 1,150, 1,143 and 191 copies, plus 50 of a fourth. One server’s environments held only 158 distinct sets of installed packages, and environments built on the same day had the same installed packages.

Why each launch can add an environment

An MCP server configured as uvx <server>@latest asks uv to resolve the latest version on every launch. With uvx -v, the log shows uv fetching the latest version and revalidating the index. Launching one server did not explain more than a thousand copies, so I measured when a launch adds an environment:

When several sessions start together and the resolution isn’t cached yet, each launch builds its own environment. Once the resolution is cached, launches add nothing. That implies pinning a version, instead of @latest, would stop most of the copies. I didn’t test that. I also didn’t check whether a later release of uv reuses an environment that another process built moments earlier. All of this is uv 0.11.12.

Choose between uv cache prune and uv cache clean

uv cache prune removes the cache entries nothing uses, and all centralized project environments, which uv recreates as needed. uv cache clean removes every entry and empties the cache; uv cache clean <package> removes one package’s entries. Both are described in uv’s cache documentation (opens in a new tab). This page uses prune, which here took the cache from 132.7 GiB to 3.9 GiB and left 540 entries in place. I didn’t run clean.

Quit every session that holds the cache lock

Each running uvx process holds the cache lock, the .lock file in the cache directory, for as long as its server runs. Cache-modifying commands wait for the lock, so a prune started now waits too. List the holders:

$ lsof "$(uv cache dir)/.lock"

COMMAND   PID    USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
uv      <pid> <user>   9u   REG  <dev>        0 <inode> /Users/<user>/.cache/uv/.lock
uv      <pid> <user>   9u   REG  <dev>        0 <inode> /Users/<user>/.cache/uv/.lock

lsof prints nothing when the lock is free; empty output from a path that is not your cache directory is not a clear lock. Each row is one uvx process. To find the session that started it, look up the parent of each <pid>:

$ ps -o ppid= -p <pid>
$ ps -o comm= -p <ppid>

On this Mac, one holder was launched by a Claude Code session and another by the VS Code extension host. Quitting a session or an editor ends its uvx children, which release the lock:

1. Run lsof "$(uv cache dir)/.lock".
2. Look up the parent of each <pid>.
3. Quit that session or editor.
4. Run lsof again until it prints nothing.

Record free space before pruning

The comparison needs readings from before the prune, so take them now and save the output:

$ df -h /
$ diskutil info / | grep "Container Free Space"
$ uv cache size -H

Run the prune

This command deletes cached environments. With nothing running, it removed every environment in the cache, not only the ones that no process referenced. The next launch of each MCP server builds a fresh environment. Run it from a plain Terminal window, not from inside one of the sessions you quit:

$ uv cache prune

Pruning cache at: .cache/uv
Removed 7278379 files (111.1GiB)

It took 409 seconds. Do not interrupt it.

Check the space you got back

The free space went up by much less than 111.1 GiB. Run the same three commands again and compare them with the readings you saved:

The container gained 27.5 GB. uv reported 111.1 GiB removed, which is about 119 GB. uv prints binary units (GiB, 2^30 bytes) and diskutil prints decimal ones (GB), so do not compare the two figures as equals. Even in the same unit, the disk got back about a quarter of what uv reported.

The explanation that fits is the link mode; I didn’t test it directly. uv’s default on macOS is clone, per uv’s settings reference (opens in a new tab), which makes APFS copy-on-write clones. Environments share blocks with each other and with the cached wheels. uv 0.11.12 counts every cloned file at its full size, and deleting a clone frees nothing when another file still uses its blocks. uv’s cache documentation (opens in a new tab) describes a cache-physical-space preview feature that accounts for copy-on-write clones; I didn’t test it.

du -sh agreed with uv cache size at 3.9G after the prune. That is a drop of 128.8 GiB from 132.7GiB, while prune printed 111.1GiB removed; I did not find why the two differ.

A local snapshot could also hold space after a delete. On this Mac, tmutil listlocalsnapshots / listed none, and diskutil apfs listSnapshots showed none on the data volume, so no snapshot was holding any.

Confirm your MCP servers still start

The prune left archive-v0 with 540 entries and no environments, and emptied environments-v2. Start each agent or editor that uses MCP servers and open a session. Each server builds a fresh environment on its next launch; on this Mac, one appeared at the next session start. Check:

$ find "$(uv cache dir)"/archive-v0 -maxdepth 2 -name pyvenv.cfg | wc -l

A count above zero means at least one server built an environment. Check each agent’s MCP server list or status view for errors.

Do this with a coding agent

Paste this prompt into your own local coding agent. It measures and reports, and it proposes before anything is removed:

Investigate my uv cache on this Mac. Measure and report only; change nothing yet.

1. Report the cache directory (uv cache dir) and its size (uv cache size -H).
2. Report how many entries in archive-v0 are Python environments (a pyvenv.cfg
   at the root), and which tools they belong to (the executables in their bin
   directories).
3. Report free space two ways: df -h / and diskutil info / (Container Free
   Space).
4. List every process holding the cache lock (lsof on the .lock file in the
   cache directory), and for each one its parent process, so I can tell which
   agent or editor session to quit.
5. Separate what is safe from what is in use: running uv processes, open agent
   or editor sessions, and any project whose own virtual environment lives
   inside the cache. If this session is itself one of the lock holders, say so
   and stop; I will run the prune from a plain Terminal.

Then propose each removal as a separate item, with the command and the current
cache size as an upper bound. uv has no dry run, so do not state an expected
figure, and note that APFS clones make the real gain smaller. Remove nothing until I approve that item. Remove only
through uv's own command (uv cache prune). Never delete files under the cache
directory by hand and never use --force. Afterwards, report free space again
with the same two readings and say how far the figure uv printed is from the
space actually gained.

Common failures

Cache is currently in-use, waiting for other uv processes to finish

Cache is currently in-use, waiting for other uv processes to finish (use `--force` to override)

Another uv process holds the cache lock, and uv cache prune and uv cache clean wait for it. Run lsof "$(uv cache dir)/.lock", quit the session that owns each holder as described earlier, and run the command again. Nothing was changed while it waited: I stopped a prune that had waited eight seconds, and the cache was unchanged.

These uv cache commands give up after a timeout; uv’s cache documentation (opens in a new tab) gives its value and the UV_LOCK_TIMEOUT environment variable that changes it. Waiting longer does not help while a session still runs its server.

Do not use --force. The documentation reserves it for cases where no other uv process is reading or writing the cache, and a running uvx server is exactly such a process.

TL;DR

Record the free space, then narrow down one level at a time with du until a directory belongs to one tool. That tells you which cleanup you need. Clean it with that tool’s own command where it has one, never with rm on a cache a tool manages, and record the free space again; the cleanup’s own report is not what the Mac got back.

$ diskutil info / | grep "Container Free Space"
$ du -hd 1 ~ 2>/dev/null | sort -hr | head -15

My Mac had 37.7 GB free on a 1 TB disk. I expected Xcode to be the reason, since I keep a full set of iOS simulators, and Xcode did hold 79 GiB. The largest directory was a 132 GiB uv cache, which I had never thought about: most of the entries in it were environments uvx had built for MCP servers, covered in part 2. Which directory is largest on your Mac depends on what you build and which tools you run. The method for finding it is below, and the series covers each cause.

Record the free space before you start

On a modern Mac, df -h / reports the sealed system volume, so its Used column is misleading:

$ df -h /

Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/disk3s1s1   926Gi    13Gi    40Gi    24%    484k  423M    0%   /

/ is the sealed system snapshot. Your files live on the Data volume in the same APFS container. Every volume in a container draws on one pool of free space, so Used for / counts the system alone, and Avail is the pool. The container’s own figure comes from the startup disk’s container only, so an external APFS drive cannot be mistaken for it:

$ diskutil info / | grep "Container Free Space"

   Container Free Space:      65.8 GB (65847853056 Bytes) (exactly 128609088 512-Byte-Units)

This reading was taken after the cleanup, so it does not match the df output above, which was taken partway through, or the 37.7 GB before any cleanup. df -h prints GiB and diskutil prints GB, so the same moment looks different in the two units. Use one of them for every reading, or the difference between the units looks like progress. A Mac keeps writing while you work, so treat a change of a gigabyte or two as noise.

Find what’s taking up space with du

du reads sizes and changes nothing. 2>/dev/null hides the permission errors that protected directories print.

1. List the largest directories in your home directory:

   Command

   $ du -hd 1 ~ 2>/dev/null | sort -hr | head -15

2. Skip the first line, which is the total for the directory you ran it on. Run the same command on the largest directory below it, and repeat until the directory belongs to one tool. On this Mac, that took three levels: ~/.cache, then ~/.cache/uv, then ~/.cache/uv/archive-v0.

3. Check the two places that hide the most, even if they didn’t top the first list:

   Command

   $ du -hd 1 ~/Library/Caches 2>/dev/null | sort -hr | head -15
   $ du -hd 1 ~/Library/Developer 2>/dev/null | sort -hr

4. For a package manager, ask the tool where its cache is rather than assuming a path. pip on macOS uses ~/Library/Caches/pip, not ~/.cache/pip:

   Command

   $ uv cache dir

   Output

   /Users/<user>/.cache/uv

   Command

   $ python3 -m pip cache dir

   Output

   /Users/<user>/Library/Caches/pip

A du total is an upper bound on what deleting frees. APFS clones share their blocks with the original until one of them changes, and du counts a cloned file at full size wherever it appears.

Match the directory to its cause

These are the directories that were large on this Mac, and the part of this series that covers each one:

For the uv cache, read part 2 before you prune: a running coding session holds a lock that makes the prune wait, and the prune reports far more than the disk gets back. The other parts are listed on the series page.

Some directories stay whatever their size. A Docker volume holds an app’s data, not cache. An app’s Application Support directory holds its state.

Clear Homebrew downloads and Ollama logs

Two directories on the list need no part of their own.

brew cleanup -s removes old versions and the cached downloads of anything no longer installed. It keeps the downloads of what is installed now, so on this Mac it freed about 270 MB of the 1.1 GiB:

$ brew cleanup -s

An app can keep its data somewhere else and still write its logs to the internal disk. My Ollama models live on an external SSD, linked into place:

$ ls -ld ~/.ollama/models

lrwxr-xr-x  1 <user>  staff  ... /Users/<user>/.ollama/models -> /Volumes/<drive>/ollama/models

The logs aren’t linked. List them first:

$ ls -lh ~/.ollama/logs

Here they had reached 947 MB in two files, server-1.log and app-1.log. Yours may have other names. Ollama has no cleanup command for them, so quit Ollama and delete the files that list showed, by name:

$ rm ~/.ollama/logs/server-1.log ~/.ollama/logs/app-1.log

Deleting a log leaves the models and the settings as they are.

Ollama wasn’t in Open at Login, but App Background Activity, under System Settings > General > Login Items, allowed it to run in the background. I turned that off. I didn’t check whether background runs are what filled the logs.

Record the free space again

Run the same command you started with:

$ diskutil info / | grep "Container Free Space"

Here it went from 37.7 GB before any cleanup to 65.8 GB after the uv prune. The cleanup commands had reported more than 140 GB between them. A cleanup command reports what it removed from its own storage; only the container’s figure says what the Mac got back.

Do this with a coding agent

Paste this prompt into your own local coding agent. It finds and reports, and it deletes nothing:

Find what is taking up space on this Mac's internal disk. Measure and report
only; delete, prune or clean nothing. Stay in my home directory, and don't
follow links to external drives.

1. Report free space with diskutil info / (Container Free Space) and
   df -h /, and say which units each prints.
2. Run du -hd 1 on my home directory, then on the largest subdirectory it
   lists (not the total line for the directory itself), and keep going down
   until each large directory belongs to one tool. Also check
   ~/Library/Caches and ~/Library/Developer. Ask package managers for their
   cache paths (for example uv cache dir, python3 -m pip cache dir) instead of
   assuming them.
3. For each directory over 1 GB, report its size, which tool owns it, what
   fills it, and the tool's own cleanup command if it has one.
4. Mark anything that is in use or holds work: running processes using the
   directory (lsof), open coding-agent or editor sessions, uncommitted or
   unpushed changes in any git checkout, Docker volumes, app data under
   Application Support. Say what would be lost if each were removed.
5. Note that du counts APFS clones at full size, so a total is an upper bound
   on what deleting frees.

Then stop and wait. I will choose what to clean.

Common failures

Free space barely moved after a cleanup

Docker reports space inside its VM disk, and du and uv cache size count cloned files at full size. Measure the container with diskutil info / before and after; that is the only figure that says what the Mac got back.

With Time Machine on, a local snapshot can keep a deleted file’s space in use. Time Machine keeps hourly snapshots for 24 hours and deletes them as they age or as space is needed, per Apple’s Time Machine guide (opens in a new tab). List them; on this Mac the list was empty:

$ tmutil listlocalsnapshots /

uv cache prune waits, then times out

uv cache prune waits while any other uv process is running, and every uvx server counts. After five minutes by default (UV_LOCK_TIMEOUT) it gives up; the uv cache documentation (opens in a new tab) covers the lock. Quit the coding sessions that started the servers, then run it again. To see what holds the lock:

$ lsof "$(uv cache dir)/.lock"

Part 2 shows how to find which session holds it.

npm cache clean left ~/.npm large

npm cache clean --force empties npm’s package cache, _cacache. The space is in _npx, where npx keeps the packages it runs; part 4 covers it.

TL;DR

Move the world in this order: stop the old server, take one last backup of it, build the new server, restore that backup onto it, check the world, and only then give players the new address. Stopping first means nothing can be played on the old server after the backup that the new one doesn’t have. The world and the server’s settings are the only things the new machine can’t get from Steam, and the backup holds both. That holds for a Navezgane world. A random-gen world also keeps its terrain in GeneratedWorlds, which this method doesn’t move, and mods aren’t archived either.

On the old server, in an elevated PowerShell with Stop-GameServer from the part on starting and stopping the server defined:

$telnet = Read-Host 'Telnet password (Enter if none)' -AsSecureString
Stop-GameServer -Password $telnet
Start-ScheduledTask -TaskName 7DaysToDieServerS3Backup

Step 1 waits for the task, checks that the backup started after the stop, and turns off the old server’s tasks. On the new server, install the game and its prerequisites with the earlier parts. Then download the newest archive as SYSTEM, copy myserver.xml out of it, and run Restore-GameWorld from the part on restoring without merging. The game-specific parts are the save folder, the config file and the log lines that show the world loaded.

The step most likely to stop you is the download. The backup task wrote the archive with the credentials in SYSTEM’s profile, so the download has to run as SYSTEM too, and the backup user’s policy has to allow s3:GetObject. The CloudFormation template in the backup article creates a user whose policy does.

The backup article started from a goal: a server I can replace without a second thought. This part is the replacing: the migration of a dedicated server from one Windows machine to another, world and settings included. My server runs on Windows Server 2019, and I want it on 2025. I rehearsed the move on two Windows Server 2025 machines. The world was a new Navezgane that nobody had played, so what follows is the order and the commands, not a move with players in it.

1. Stop the old server and take the last backup

Do these on the old server, in this order. Stopping the server first closes the window in which a player could change the world after the backup. The backup then copies a world that nothing is writing to, which the backup article lists under “Saves are captured mid-write” as the case it can’t promise.

$task = '7DaysToDieServerS3Backup'
Disable-ScheduledTask -TaskName $task | Out-Null
while ((Get-ScheduledTask -TaskName $task).State -eq 'Running') { Start-Sleep -Seconds 5 }
$telnet = Read-Host 'Telnet password (Enter if none)' -AsSecureString
Stop-GameServer -Password $telnet
$stopped = Get-Date
$stopped = $stopped.AddTicks(-($stopped.Ticks % [TimeSpan]::TicksPerSecond))
Enable-ScheduledTask -TaskName $task | Out-Null
Start-ScheduledTask -TaskName $task
Start-Sleep -Seconds 5
while ((Get-ScheduledTask -TaskName $task).State -eq 'Running') {
    Start-Sleep -Seconds 1
}
$info = Get-ScheduledTaskInfo -TaskName $task
if ($info.LastRunTime -lt $stopped) {
    throw 'The backup that finished started before the server stopped. Run it again.'
}
$info.LastTaskResult
Disable-ScheduledTask -TaskName 7DaysToDieServer
Disable-ScheduledTask -TaskName $task

The password is read as a secure string, so it stays out of the console history. With no telnet password set, press Enter and nothing is sent. The last two lines turn off the old server’s tasks before anyone is told the new address. 7DaysToDieServer starts the server at every boot, so a reboot would otherwise bring the old world back where a player could still reach it. The backup task would write an archive of the old world with a newer name than any the new server makes. A restore that takes the newest archive would then pick it. Both tasks read Disabled afterwards.

Disabling the backup task first keeps its daily trigger from starting a run while the server stops, and the first loop waits out a run that had already begun, because starting a running task does nothing and a backup that began before the stop could hold a world that was still being written. The task is enabled again only to take this last backup. Task Scheduler records the last run time to the second, so $stopped is cut to the second too, and the check fails only for a run that started before the stop. If the daily time falls in this window, run the step again.

The result must be 0. The script reads the object back from S3 and compares its size before it exits 0, so a 0 means the archive is in the bucket. The newest log under backup-logs names it:

Upload successful: 1806071 bytes, STANDARD_IA

The backup took seven seconds here and the archive was 1.8 MB, because the world was new. Note the archive’s name, which is a timestamp. On a world people have played it is larger.

This step assumes the backup task from the backup article is already running on the old server, with its script archiving myserver.xml, as the configuration part says to change it. If the archive holds the shipped serverconfig.xml instead, your settings exist only on the old server, and step 4 has a way to copy them.

2. Build the new server

On the new server, in an elevated PowerShell, follow the earlier parts for what each owns:

1. Install SteamCMD and the server. It took about eight minutes here, which is most of the time the move takes.
2. Install the Visual C++ runtime from the configuration part, step 1, and open the firewall ports from its step 5. Skip its steps 2 to 4, because the settings come from the archive.
3. Install AWS Tools for all users, from step 1 of the backup article. The runtime and the AWS modules together took 25 seconds.
4. Write the backup user’s access key into SYSTEM’s profile, as step 2 of the backup article does, and the bucket name into C:\Scripts\backup-config.txt, as its step 3 does. The new server uses the same key as the old one.

This part moves a world on one of the maps the game ships, such as Navezgane. It doesn’t move a random-gen world: that one also needs its GeneratedWorlds folder, and Restore-GameWorld refuses an archive that holds one, so steps 3 and 4 stop there. For a random-gen world, copy Saves and GeneratedWorlds across by hand with both servers stopped instead; I didn’t test that. Don’t start the new server yet.

3. Download the latest archive as SYSTEM

The key is in SYSTEM’s profile, so the download runs as SYSTEM, the way the backup does. A scheduled task is the way the backup article runs something as SYSTEM. None of the earlier parts downloads an archive this way, so this step is new. Save this as C:\Scripts\Get-LatestBackup.ps1:

$ErrorActionPreference = 'Stop'
Import-Module AWS.Tools.S3
$bucket = (Get-Content 'C:\Scripts\backup-config.txt' -First 1).Trim()
$prefix = '7days-backups/'
$latest = Get-S3Object -BucketName $bucket -Prefix $prefix |
    Where-Object { $_.Key -match '7dtd-backup-[^/]+\.zip$' } |
    Sort-Object LastModified -Descending | Select-Object -First 1
if (-not $latest) { throw "No archive under $prefix in $bucket." }
New-Item -ItemType Directory -Path C:\backups -Force | Out-Null
$zip = Join-Path C:\backups (Split-Path $latest.Key -Leaf)
Read-S3Object -BucketName $bucket -Key $latest.Key -File $zip | Out-Null
$size = (Get-Item $zip).Length
if ($size -ne $latest.Size) { throw "Size mismatch: S3 $($latest.Size), local $size." }
"Downloaded $zip ($size bytes)"

It picks the archive that S3 says is newest, downloads it to C:\backups, and stops if the local size differs from the object’s. Then run it once as SYSTEM, wait for it and remove the task:

$command = '& C:\Scripts\Get-LatestBackup.ps1 *> C:\Scripts\get-latest.log'
$action = New-ScheduledTaskAction -Execute 'powershell.exe' `
    -Argument "-NoProfile -ExecutionPolicy Bypass -Command `"$command`""
Register-ScheduledTask -TaskName 'GetLatestBackup' -Action $action `
    -User 'SYSTEM' -RunLevel Highest | Out-Null
Start-ScheduledTask -TaskName 'GetLatestBackup'
Start-Sleep -Seconds 3
while ((Get-ScheduledTask -TaskName 'GetLatestBackup').State -eq 'Running') {
    Start-Sleep -Seconds 1
}
(Get-ScheduledTaskInfo -TaskName 'GetLatestBackup').LastTaskResult
Get-Content C:\Scripts\get-latest.log
Unregister-ScheduledTask -TaskName 'GetLatestBackup' -Confirm:$false

The result is 0 and the log reads:

Downloaded C:\backups\7dtd-backup-20261005-063440.zip (1806071 bytes)

The name is the one step 1 noted. The download took five seconds.

4. Copy the settings and restore the world saves

The restore function doesn’t touch settings: its archive’s serverconfig.xml or myserver.xml stays in the zip. A move needs them, so copy the settings file out of the archive first:

$zip = (Get-ChildItem C:\backups\7dtd-backup-*.zip |
    Sort-Object LastWriteTime -Descending | Select-Object -First 1).FullName
$unpacked = Join-Path $env:TEMP 'archive-config'
Expand-Archive -LiteralPath $zip -DestinationPath $unpacked -Force
Copy-Item (Join-Path $unpacked 'myserver.xml') C:\7DaysToDieServer\myserver.xml
Remove-Item $unpacked -Recurse

The copy from the archive had the same SHA-256 as the old server’s file, so the name, password, slots and UserDataFolder came across with it. UserDataFolder puts Saves beside the executable, and the restore function looks for it there. If the archive holds serverconfig.xml and not myserver.xml, it has the shipped settings: copy your myserver.xml from the old server by hand.

Then define Restore-GameWorld from the restore part and run it on the archive:

Restore-GameWorld -ArchivePath $zip

PS> Restore-GameWorld -ArchivePath $zip

Restored Saves from 7dtd-backup-20261005-063440.zip.

There was no old Saves on the new server, so the function printed no line about one, and it started nothing because no server was running. It finished in under a second. It compared every restored file with the archive by name and size before it returned.

5. Start the server and check the world

Register the task from step 1 of the part on starting at boot and start it with its step 3. The server was ready 54 seconds after the start here. Then read three things.

The log names the world and the game name from the settings you copied, and the world loading:

Select-String -Path C:\7DaysToDieServer\output_log.txt `
    -Pattern 'GamePref.GameName =', 'GamePref.GameWorld =', 'createWorld',
    'StartGame done'

Each log line starts with a timestamp, which is left out here.

GamePref.GameName = MyGame
GamePref.GameWorld = Navezgane
INF createWorld: Navezgane, Navezgane (src: GameData, DeviceLocal), MyGame, GameModeSurvival
INF createWorld() done
INF StartGame done

The build matches the old server’s, read on both machines:

Select-String -Pattern '"buildid"' `
    -Path C:\7DaysToDieServer\steamapps\appmanifest_294420.acf

Both read 24994542. If yours differ, Steam published a build between the two installs and the new server is the newer one. The update part covers how the server’s build is compared with the one Steam publishes. I didn’t test loading an older world on a newer build.

Then stop the new server with Stop-GameServer -Password $telnet, the function from step 4 of the start-at-boot part, with $telnet read as in step 1. It exited with code 0 in four seconds, the task read Ready with a last result of 0, and the log had SaveAndCleanupWorld. Run this on both machines, once the new server has stopped, and compare the two outputs:

$root = 'C:\7DaysToDieServer\Saves'
Get-ChildItem $root -Recurse -File | Where-Object Extension -ne '.bak' |
    Sort-Object FullName | ForEach-Object {
        '{0} {1} {2}' -f $_.FullName.Substring($root.Length), $_.Length,
            (Get-FileHash $_.FullName).Hash
    } | Out-File C:\saves-manifest.txt -Encoding ascii

Copy the old server’s file to the new one as C:\old-manifest.txt and run Compare-Object (Get-Content C:\old-manifest.txt) (Get-Content C:\saves-manifest.txt). No output means the two lists are identical, and here they were: 61 files, size for size and hash for hash. To see it report a difference, I changed one byte of sdcs_profiles.sdf on the new server and ran the command again. It printed that file’s line twice, once for each hash, with => on the new server’s line and <= on the old one’s, and printed nothing again once I put the byte back. The server had started and stopped on the new machine, and the old one had not run since its backup. The .bak files are left out because the server writes new ones when it starts, and the new machine’s differed from the old. On a world people played, I can’t say whether a start and a stop leaves every other hash alone.

6. Move the players and the backups

Start the new server again with Start-ScheduledTask -TaskName 7DaysToDieServer, and give players its address. Whether they can reach it depends on the network in front of it, which the configuration part leaves to you.

Then save the script and register the task on the new server with steps 4 and 5 of the backup article. Change the script’s $Config line to myserver.xml, as the configuration part says, or the new archives hold the shipped settings. Run the task once from the scheduler. Here it exited 0 and wrote a second archive, 1.8 MB, and a listing of the bucket as SYSTEM showed both archives.

The old server’s tasks have been off since step 1; leave it stopped. Deleting it is yours to do, after the new one has made good backups.

Check that the key reaches only its bucket

The key should reach one bucket and no other. As SYSTEM on the new server, a listing of a different bucket that exists, one the user’s policy doesn’t name, was refused. I tried one other bucket:

User: arn:aws:iam::<account>:user/<backup user> is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::<other bucket>" because no identity-based policy allows the s3:ListBucket action

A download of an archive in the right bucket succeeded with the same credentials, which is what step 3 did.

Common failures

Download fails with is not authorized to perform: s3:GetObject

User: arn:aws:iam::<account>:user/<backup user> is not authorized to perform: s3:GetObject on resource: "arn:aws:s3:::<bucket>/7days-backups/<archive>" because no identity-based policy allows the s3:GetObject action

The backup user’s policy lacks s3:GetObject. I removed it from the test user’s policy and ran the download as SYSTEM, and it failed this way while listing the bucket still worked. Add the permission, as the template in the backup article has it, and run step 3 again.

Backup on the old server fails with AWS Tools for PowerShell not available

ERROR: AWS Tools for PowerShell not available to this account: The specified module 'AWS.Tools.SecurityToken' was not loaded because no valid module file was found in any module directory.. Install AWS.Tools.S3 and AWS.Tools.SecurityToken with -Scope AllUsers

The task’s result was 1 and nothing reached S3. The old server had AWS.Tools.S3 and not AWS.Tools.SecurityToken; it is what step 1 of the backup article installs. Run step 1 of the backup article, which installs both for all users, and start the task again.

What I didn’t test

• A world people had played, or a client connecting to the new server, or the ports from outside the machine.
• A reboot of the new server, so the at-boot trigger is registered and not seen to fire.
• The old server’s backup task running after the move. Disable-ScheduledTask on both tasks was run, and both read Disabled.
• Steps 2 and 3 of the backup article on either machine. The access key was written into SYSTEM’s profile on both for me, and I took the bucket name from the stack’s output.

TL;DR

Register a scheduled task whose action is the server executable itself, with an at-startup trigger, the SYSTEM account and the execution time limit set to zero. Stop the server with its telnet shutdown command, which the server accepts on 127.0.0.1:8081 by default.

It is the only stop I found that finishes the shutdown. The process exited with code 0 and wrote every save file. Stop-ScheduledTask ended it in about a second with exit code 1067 and wrote about half of them. The default time limit is 72 hours, after which Task Scheduler stops the server the same abrupt way.

In an elevated PowerShell on the server, with your config file (myserver.xml here) in the install folder and UserDataFolder set in it:

$dir = 'C:\7DaysToDieServer'
$log = Join-Path $dir 'output_log.txt'
$flags = '-quit -batchmode -nographics -configfile=myserver.xml -dedicated'
$arguments = "-logfile `"$log`" $flags"
$action = New-ScheduledTaskAction -Execute (Join-Path $dir '7DaysToDieServer.exe') `
    -Argument $arguments -WorkingDirectory $dir
$trigger = New-ScheduledTaskTrigger -AtStartup
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit ([TimeSpan]::Zero) `
    -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries
Register-ScheduledTask -TaskName '7DaysToDieServer' -Action $action -Trigger $trigger `
    -Settings $settings -User 'SYSTEM' -RunLevel Highest

Then Start-ScheduledTask -TaskName 7DaysToDieServer starts it, and the Stop-GameServer function in step 4 stops it. The game-specific parts are the server’s flags, its ready line, and telnet as the way to stop it.

I didn’t restart the test server, so the at-startup trigger is confirmed to be registered, not confirmed to fire. Step 5 has the check to run after your first reboot.

The previous part left a server you start by hand. This part makes Windows start it when the machine boots, keep it running, and stop it without losing the world.

1. Register the task

The task runs the server executable directly. Nothing wraps it, so the task’s state says whether the server is running, and stopping the task stops the server.

The previous part’s UserDataFolder setting matters here. A task that runs as SYSTEM has no user profile of its own, and without that setting the server writes its world under the SYSTEM profile, at C:\Windows\System32\config\systemprofile\AppData\Roaming\7DaysToDie. Check that your myserver.xml has it set before you register the task.

This registers the task:

$dir = 'C:\7DaysToDieServer'
$log = Join-Path $dir 'output_log.txt'
$flags = '-quit -batchmode -nographics -configfile=myserver.xml -dedicated'
$arguments = "-logfile `"$log`" $flags"
$action = New-ScheduledTaskAction -Execute (Join-Path $dir '7DaysToDieServer.exe') `
    -Argument $arguments -WorkingDirectory $dir
$trigger = New-ScheduledTaskTrigger -AtStartup
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit ([TimeSpan]::Zero) `
    -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries
Register-ScheduledTask -TaskName '7DaysToDieServer' -Action $action -Trigger $trigger `
    -Settings $settings -User 'SYSTEM' -RunLevel Highest

What each part does:

• -WorkingDirectory sets the folder the server starts in. The server found myserver.xml next to its executable without it, so this is a precaution and not a requirement.
• -AtStartup runs the task when Windows starts, before anyone logs on.
• -User 'SYSTEM' with -RunLevel Highest runs it as the SYSTEM account, with no password to store. SYSTEM has machine-wide rights, so a flaw in the server or a mod it loads would have them too. If players you don’t trust can reach the server, register the task under a dedicated local account that can write only to the install folder. I didn’t test that.
• -ExecutionTimeLimit ([TimeSpan]::Zero) turns the time limit off. The next step shows why that is the setting to get right.
• -AllowStartIfOnBatteries and -DontStopIfGoingOnBatteries turn off two defaults that exist for laptops. Both are on by default, and a virtual machine has no battery, so these are harmless there.

Register-ScheduledTask prints an error if the name already exists, and leaves the first task as it was. Running the block twice is harmless. To change a task, remove it with Unregister-ScheduledTask -TaskName '7DaysToDieServer' -Confirm:$false and register it again.

Read the settings back:

PS> $task = Get-ScheduledTask -TaskName '7DaysToDieServer'
PS> $task.Settings.ExecutionTimeLimit

PT0S

PS> $task.Triggers[0].CimClass.CimClassName

MSFT_TaskBootTrigger

PS> $task.Principal.UserId

SYSTEM

PT0S is how Task Scheduler stores no limit.

2. Turn off the 72-hour execution time limit

A task registered with no settings read back PT72H. Task Scheduler ends a task that runs longer than its limit, and a game server is meant to run for weeks.

To see what it does to the server, I registered the same task with a limit of two minutes and started it. The server ran until the limit, and Task Scheduler then ended it, about 123 seconds after the start. The task went back to Ready with a last result of 267014, which is Task Scheduler’s code for a task that was terminated. The log shows the server beginning its shutdown, but the limit ends the process the same abrupt way Stop-ScheduledTask does in step 4, which is not a clean stop.

A longer limit only moves that moment. With PT72H the server would have been stopped three days after each start, and nothing would have restarted it. The -ExecutionTimeLimit ([TimeSpan]::Zero) in step 1 turns the limit off.

There is a second way to build this task, and I tested it too. The action starts a small script that launches the server detached, with Start-Process, and exits. The task then finishes in seconds, and the limit never applies. With a two-minute limit the server was still running after more than three minutes, the task itself had finished within seconds, and Stop-ScheduledTask left the server running. That design needs the telnet stop in step 4 anyway. I used the direct task because its state says whether the server is running.

3. Start the task and wait for the server

Start-ScheduledTask returns as soon as the task starts. The server takes about 40 seconds to be ready. This starts the task and waits for the server’s own ready line:

$log = 'C:\7DaysToDieServer\output_log.txt'
if (Get-Process 7DaysToDieServer -ErrorAction SilentlyContinue) {
    throw 'A 7DaysToDieServer process is already running. Stop it first.'
}
Remove-Item -LiteralPath $log -ErrorAction SilentlyContinue
Start-ScheduledTask -TaskName '7DaysToDieServer'
$deadline = (Get-Date).AddMinutes(5)
$ready = $false
while (-not $ready -and (Get-Date) -lt $deadline) {
    $ready = (Test-Path $log) -and
        (Select-String -Path $log -Pattern 'StartGame done' -Quiet)
    if (-not $ready) { Start-Sleep -Seconds 2 }
}
if (-not $ready) { throw 'The server was not ready after five minutes.' }
$task = Get-ScheduledTask -TaskName '7DaysToDieServer'
'Task state: {0}; last result: {1}' -f $task.State,
    (Get-ScheduledTaskInfo -TaskName '7DaysToDieServer').LastTaskResult

It stops if a 7DaysToDieServer process is already running, however it was started. A task in the Ready state would start a second server on the same ports and folder if you had started the first one by hand. And Start-ScheduledTask on a running task does nothing and returns no error. It does set the last result to 2147946720, the refusal from the default one-instance rule, and a later read of the result would mislead. It deletes the old log first. The check looks for StartGame done, and a leftover log would satisfy it at once.

While the server runs, the output is:

Task state: Running; last result: 267009

A state of Running and a last result of 267009 mean the task is running. That is a status code and not an error.

4. Stop the server with the telnet shutdown command

The server has a telnet console, on 127.0.0.1:8081 by default, and its shutdown command is the stop that works. Setting TelnetPassword makes the console listen on every interface instead, so keep Windows Firewall and any cloud network security group closed to that port. This function sends the command, with the password first if there is one, and waits for the process to exit:

function Stop-GameServer {
    param([securestring]$Password, [int]$Port = 8081,
        [int]$TimeoutSeconds = 90)
    $server = @(Get-Process 7DaysToDieServer -ErrorAction SilentlyContinue)
    if ($server.Count -ne 1) {
        throw "Expected one 7DaysToDieServer process, found $($server.Count)."
    }
    $null = $server[0].Handle
    $client = New-Object Net.Sockets.TcpClient('127.0.0.1', $Port)
    try {
        $stream = $client.GetStream()
        $stream.ReadTimeout = 2000
        $buffer = New-Object byte[] 4096
        $reply = {
            $text = ''
            $until = (Get-Date).AddSeconds(3)
            try {
                while ((Get-Date) -lt $until -and
                    ($count = $stream.Read($buffer, 0, $buffer.Length)) -gt 0) {
                    $text += [Text.Encoding]::ASCII.GetString($buffer, 0, $count)
                }
            }
            catch [IO.IOException] { }
            $text
        }
        $send = {
            param($line)
            $bytes = [Text.Encoding]::ASCII.GetBytes("$line`r`n")
            $stream.Write($bytes, 0, $bytes.Length)
        }
        if ($Password -and $Password.Length -gt 0) {
            & $send ([Net.NetworkCredential]::new('', $Password).Password)
            $login = & $reply
            if ($login -match 'Password incorrect') {
                throw 'The telnet password is missing or wrong.'
            }
            if ($login -notmatch 'Logon successful') {
                throw 'The server did not confirm the telnet login.'
            }
        }
        & $send 'shutdown'
        if ((& $reply) -match 'Password incorrect') {
            throw 'The telnet password is missing or wrong.'
        }
    }
    finally { $client.Close() }
    if (-not $server[0].WaitForExit($TimeoutSeconds * 1000)) {
        throw "The server was still running after $TimeoutSeconds seconds."
    }
    'Stopped, exit code {0}.' -f $server[0].ExitCode
}

It stops with an error and doesn’t touch the process in two cases. The telnet port refuses the connection, which is what TelnetEnabled set to false does. Or the server answers Password incorrect. It never forces the process to end. Run it as Stop-GameServer. With a telnet password, read it as a secure string so it stays out of your console history and any transcript:

$telnet = Read-Host 'Telnet password' -AsSecureString
Stop-GameServer -Password $telnet

The function turns it back into text only to send it. The game’s telnet console takes it unencrypted, one more reason to keep that port closed.

On the task’s server it stopped the process with exit code 0. The task went back to Ready with a last result of 0, because the process ended by itself. The server wrote 9 to 14 files under Saves and its log ran on to the end of its shutdown, after SaveAndCleanupWorld and Saving 0 of chunks, with 47 more lines. It took two to eight seconds.

With a telnet password set, I connected three ways and sent shutdown each time. With no password, or a wrong one, the server answered Password incorrect, please enter password: and kept running, and the function reports that at once. With the right password it stopped in two seconds, exit code 0, and the log held SaveAndCleanupWorld.

Each wrong attempt counts toward the server’s TelnetFailedLoginLimit, which was 10 on this server. The function sends the password first and reads the reply, for at most three seconds, and sends shutdown only after the server has said Logon successful, so a wrong password uses one attempt. Sent together, the shutdown after a wrong password would count as a second one. With the limit set to 3, a wrong password and then a missing one still left the right password working.

If the reply is late or says neither, the function stops with The server did not confirm the telnet login. The server also prints its log to a telnet connection, so a read that waited only for silence could go on as long as the server logged something every two seconds.

Why Stop-ScheduledTask isn’t a clean stop

Stop-ScheduledTask -TaskName '7DaysToDieServer' is the obvious stop for a task, and it ends the process, but not cleanly. I ran it twice on the task’s server:

• The process exited with code 1067 after about a second. 1067 is ERROR_PROCESS_ABORTED, “The process terminated unexpectedly”, and it is the exit code the process had both times Task Scheduler ended it here.
• The log began the shutdown, with SaveAndCleanupWorld and Stopping server, and then ended at Cleared queues. instead of running on for another 40 or so lines.
• It wrote three to five files under Saves, among them main.ttw, players.xml and decoration.7dt. The telnet stop also wrote vehicles.dat, power.dat, turrets.dat, drones.dat and blockLimits.dat, and none of those five were written in either Stop-ScheduledTask run.
• The task read Ready with a last result of 267014, Task Scheduler’s code for a terminated task.

I ran these on a new world with nobody connected, so I can’t say what the missing files hold in a world that people are playing. Their names suggest vehicles, power, turrets, drones and block limits. Use Stop-ScheduledTask as the fallback when telnet isn’t available.

Other ways to stop it: taskkill and the close request

With telnet off, I tried the other ways a script might stop the process. The first two behave differently when a console session started the server and when the task did, so the table has both:

The server runs with -batchmode -nographics, and the two polite requests had no effect on it. Stop-Process -Force is the last resort. After it, the server had logged no save, so whatever happened since the last save is gone.

5. Check the task after the first reboot

The at-startup trigger was registered and read back as a boot trigger, and the task starts on command. I didn’t restart the test server, so I haven’t seen the trigger fire. After your first reboot, check that it did:

Get-ScheduledTaskInfo -TaskName '7DaysToDieServer' |
    Select-Object LastRunTime, LastTaskResult
Get-Process 7DaysToDieServer -ErrorAction SilentlyContinue

A LastRunTime near the boot time and a running process mean it worked. Stop the server with step 4 before you restart the machine on purpose. I haven’t tested what Windows does to the server when it shuts down with the task running, and if it ends the process the way Stop-ScheduledTask does, the save is only partial.

Nothing here restarts the server if it crashes. The task has no restart settings.

Common failures

Server stops by itself after three days

The task has the default 72-hour limit, or whatever limit you set. Read it with (Get-ScheduledTask -TaskName '7DaysToDieServer').Settings.ExecutionTimeLimit. If it isn’t PT0S, remove the task and register it again with step 1.

Task starts and the server isn’t running

The log in -logfile has the reason. Two causes seen on this server: a missing Visual C++ runtime, and a ServerPort outside 1024 to 65530.

Stop-GameServer says the connection was refused

Exception calling ".ctor" with "2" argument(s): "No connection could be made because the target machine actively refused it 127.0.0.1:8081"

Telnet is off, or on another port. Check TelnetEnabled and TelnetPort in myserver.xml. Until telnet works, Stop-ScheduledTask is the fallback, with the cost step 4 describes.

Last result 267014, or exit code 1067, after a stop

Task Scheduler uses it for a task that was ended from outside, which is what Stop-ScheduledTask and the time limit do. The process itself exits with 1067 when Task Scheduler ends it. A telnet stop leaves 0 for both. Step 4 shows what the difference cost.

I wanted every crossing of the I-90 floating bridge in 240 drive folders of dashcam footage, 1,958 GB, for a montage. I wasn’t going to watch the footage to find them, and my Rivian R1T’s GPS was too coarse to. What found them was the color of the lights in the Mount Baker tunnel, which sits at the Seattle end of the bridge. Brightness couldn’t see that tunnel. A red/blue ratio found it on 71 of the 79 bridge drives scanned, and a person looking at one frame per candidate confirmed 68 crossings. How to sort months of dashcam footage with Claude Code has the rest of the job this was one stage of.

Signals instead of pixels

The footage already describes itself if you read the right parts of it. Each 15-minute segment carries one GPS fix, at its start, in the container’s location tag. That sorts drives by area, but one fix per 15 minutes can’t place a bridge crossing to the minute, let alone the second.

The finer signal is in the keyframes. The truck writes one keyframe per second. Decoding only those, downscaled to 64 × 48, gives a per-second record of a drive and skips the 29 frames between them. Each second becomes four numbers: the mean change in green from the previous keyframe, which I called motion, and the mean red, green and blue. That record was computed once, cached as CSV, and every detector afterward read the cache instead of the video.

The first calibration read a motion of zero everywhere, parked or moving. The cause was ffmpeg duplicating the keyframes back up to 30 fps, so every frame was compared with a copy of itself. -fps_mode passthrough fixed it. The bug showed up only because the check ran on a known parked segment and a known moving one before it ran on anything else. A detector that reads zero on both cases is measuring itself.

Why brightness failed

A tunnel should be the easiest thing in a drive to find: the outside goes away and the picture gets darker. In this footage it doesn’t get darker. The camera’s auto-exposure raises the gain as the light drops and holds the frame’s brightness close to what it was in daylight. That’s what the exposure control is for, and it erases the one property a brightness threshold depends on.

Why color worked

Exposure scales brightness, and the hue of the lamps comes through. The Mount Baker tunnel is lit by sodium lamps, and sodium light is orange. Inside the tunnel, the front camera’s mean red rises above its mean blue. That ratio is the whole detector.

It runs with three guards:

• The red/blue ratio has to stay above 1.0 for 25–150 seconds, so a single orange frame or a long orange stretch doesn’t count.
• A rolling median smooths the ratio first, so one bright frame inside a tunnel doesn’t break the run in two.
• The first and last 3 minutes of every drive are ignored, because parking garages look the same to the ratio.

On the 79 bridge drives scanned, it found a tunnel on 71. Seven of the eight misses were after dark. My reading is that the ratio describes light, and once the light outside is artificial too, a tunnel stops standing out from the road around it.

Detector proposes, person confirms

Finding a tunnel isn’t the same as finding the right one. A drive on I-90 passes more than one stretch that reads as a tunnel, including the Mercer Island lid, and most drives had several candidates. The obvious rule, pick one tunnel by its position in the drive, was right about two times in three.

I could have spent the effort on a better rule. Instead the tool writes a guess for every drive and a contact sheet to check it against, then stops. Each row of the sheet is one drive, and each tile is a frame from the bridge side of one candidate. The bridge is unmistakable in a frame, so the right column is obvious, and the edit is one number in a picks file, which the cut turns into synced clips. Reviewing that way confirmed 68 crossings. Two night drives couldn’t be confirmed, because a dark frame of a dark bridge proves nothing, and one drive had no crossing among its candidates.

The layout of the sheet mattered as much as having one. The first version was a flat grid of every candidate frame, and it was easy to match a tile to the wrong drive. One row per drive makes that mistake visible rather than merely possible.

The sheet is also an instrument, and an instrument can be wrong. At one point I announced that the picks for some drives were the Mercer Island lid, not the Mount Baker tunnel. The sheets had been built with a seek that ffmpeg applied after tiling, so they showed a different window than the one they claimed. Extracting the same frame two ways exposed it, and the picks had been right all along. When a result contradicts work that was already verified, check the instrument before the conclusion.

The review step reads like an admission that the detector wasn’t good enough. I think it’s the reason the detector could stay simple. Take a rule that’s right two times in three, and add a sheet a person reads at a glance. That beats a rule that’s right more often and still needs someone to check it.

Signals that could not separate the cases

The same per-second record answered four other questions, and each answer was no for the same reason: the signal moved for the thing I wanted and for something I didn’t.

• Parked footage. Near-zero motion found parked time reliably, but there was little to find: 23 GB of the 1,760 GB scanned, because with Gear Guard turned off, the truck stops recording seconds after it parks.
• Hard stops. A sudden change in the picture marks braking, but turning changes the picture as much. Of 875 candidates, all 12 at the top were turns into parking lots or ordinary stops.
• Traffic jams. Sustained low motion should mark crawling traffic, but the neighboring cars keep moving, so the picture keeps changing and a jam looks like driving.
• Garage boundaries. A gray band at the top of the frame marks a concrete ceiling on a clear day, but an overcast sky is the same gray. The boundary was set by timing instead.

None of these is fixed by tuning a threshold. When a signal can’t tell two cases apart, a better number for the threshold still can’t.

Close passes did work once the frame’s overall brightness change was subtracted from the side cameras: 37 of the top 60 were real passes. A person still had to sort them.

Signals the camera leaves alone

A camera processes its own picture before anything reads it, and that processing exists to make the picture look normal. Auto-exposure made a tunnel look like daylight and left the lamp color intact. Before measuring anything in recorded video, ask which properties the camera flattens and which it passes through, and build on the second kind.

TL;DR

When a video has one keyframe per second, decode only the keyframes and give each one a frame at 30 fps. The result plays 30 times faster than the drive:

$ ffmpeg -skip_frame nokey -i <clip>.mp4 -vf "setpts=N/(30*TB)" -r 30 \
    -map_metadata -1 -c:v hevc_videotoolbox -q:v 65 -tag:v hvc1 \
    -color_range pc -an <clip>_30x.mp4

-skip_frame nokey goes before -i, because it tells the decoder to discard every frame that isn’t a keyframe. setpts=N/(30*TB) gives frame N the timestamp N/30 seconds, so one second of driving becomes one frame of output.

To keep several cameras in sync, render each recording segment separately, pad every camera’s segment with black to the front camera’s length, and join the segments with a stream copy. The script later in this article does that.

A Rivian R1T films five directions for as long as it’s driven. I wanted a hyperlapse of every drive from a summer of footage, one file per camera, so a drive could be cut as a multi-camera edit at 30× speed. An ordinary hyperlapse decodes 30 frames to keep one, and this footage didn’t need that: the truck writes one keyframe every second. Reading only those, a 36-minute drive became 72 seconds of video and took 4.4 minutes to render, with one drive rendering at a time. The hyperlapses for 218 drives, five cameras each, came to 107 GB.

This is part of a series about reading dashcam video without watching it. How to sort months of dashcam footage with Claude Code is the hub, with the prompts and the checks that drove the whole job.

Why keyframes are enough

The Rivian’s clips are H.264 at 30 fps with I- and P-frames only, and one keyframe (an I-frame) every second. A keyframe decodes on its own. A P-frame needs the frames before it, so taking every 30th frame of an ordinary decode still means decoding the other 29.

With -skip_frame nokey the decoder drops the P-frames before decoding them. What reaches the filter is one picture per second of driving, already spaced evenly in time. The setpts=N/(30*TB) filter discards their timestamps and numbers them off at 30 per second. N is the frame’s index and TB is the stream’s time base. Without setpts, the output has the same frames, each still held for a second, and plays at the speed of the drive.

The speed this produces is fixed by the footage: one keyframe per second, shown at 30 fps, is 30×. Footage with a keyframe every two seconds would come out at 60× from the same command.

Check the keyframe rhythm

Before relying on it, count the keyframes in one clip. In a terminal, run:

$ ffprobe -v error -skip_frame nokey -select_streams v \
    -show_entries frame=pts_time -of csv=p=0 <clip>.mp4 | head -3

0.000000
1.000000
2.000000

Pipe the same command to wc -l instead of head for the count. On a 351-second clip it printed 351, and a full 15-minute segment from the truck has 900. On some drives one camera’s keyframes fall at 0.967 s past each second rather than on it, and which camera varies by drive. The spacing is still one second, which is what the speed depends on.

If the times are not one second apart, the hyperlapse still renders, but its speed is the keyframe interval times 30 rather than 30×.

Render one clip

In the folder that holds the clip, run:

$ ffmpeg -nostdin -v error -skip_frame nokey -i <clip>.mp4 \
    -vf "setpts=N/(30*TB)" -r 30 -map_metadata -1 \
    -c:v hevc_videotoolbox -q:v 65 -tag:v hvc1 \
    -color_range pc -an <clip>_30x.mp4

It writes <clip>_30x.mp4 beside the source and changes nothing else. On a 351-second clip of a bridge crossing, the output was 351 frames and 11.7 seconds long, 11.2 MB from a 438.8 MB source.

The encoder options are the part to change on another machine:

• hevc_videotoolbox is the Mac’s hardware HEVC encoder. The Homebrew build of ffmpeg used here includes it; ffmpeg -encoders | grep videotoolbox lists it if yours does.
• -q:v 65 asks it for constant quality rather than a bitrate.
• -tag:v hvc1 labels the HEVC stream the way QuickTime and Apple’s players expect.
• -color_range pc labels the output full range like the footage, which the black filler and the join depend on. Apple’s encoder ignores -pix_fmt yuvj420p with a warning, so it isn’t set here.
• -map_metadata -1 drops the source’s tags, including its GPS fix.
• -an writes no audio track. The truck records none.

Pad each segment to the front camera’s length

The truck splits a drive into 15-minute segments, one file per camera per segment. All five cameras’ files for a segment have the same length, unless a camera stopped recording. If one did, that camera’s segment has fewer keyframes, its hyperlapse has fewer frames, and every later frame of that angle plays early. A camera that is missing a whole segment is worse.

When I first rendered every camera of one drive, two came out at 1,303 frames against the front camera’s 2,170. The rear camera had stopped 33 seconds into a segment and recorded the rest of the drive in a sibling folder.

Pad every camera’s segment to the front camera’s frame count with black, so each segment ends at the same frame on every camera. The numbers in these steps are an example, a 351-keyframe front segment and a camera that stopped 33 seconds in:

1. Count the front camera’s keyframes for the segment, as in the first step. On a 351-second segment, that’s 351.

2. Count the other camera’s keyframes for the same segment. A clip that stopped 33 seconds in has 33.

3. Render that camera’s segment with tpad adding the difference as black frames, and -frames:v capping it at the front camera’s count:

   Command

   $ ffmpeg -nostdin -v error -skip_frame nokey -i <rear-segment>.mp4 \
       -vf "setpts=N/(30*TB),tpad=stop=318:color=black" -r 30 -frames:v 351 \
       -map_metadata -1 -c:v hevc_videotoolbox -q:v 65 -tag:v hvc1 \
       -color_range pc -an <rear-segment>_30x.mp4

The -frames:v cap also trims a camera that recorded slightly longer than the front one.

Fill a missing segment with full-range black

When a camera has no file at all for a segment, render a black part of the same size and frame count. The front, rear and side cameras are 1920 × 1536; the Gear Guard bed camera is 1280 × 720.

$ ffmpeg -nostdin -v error -f lavfi -i "color=black:s=1920x1536:r=30" \
    -frames:v 351 -map_metadata -1 -c:v hevc_videotoolbox -q:v 65 -tag:v hvc1 \
    -color_range pc -an <missing-segment>_30x.mp4

The filler must be full range, the same as the footage. -color_range pc is what makes it so: without it, generated black came out labeled limited range (tv), and -pix_fmt yuvj420p doesn’t help, because Apple’s encoder ignores it. The decoded footage is already full range, so the same options are safe on every part.

A mismatch doesn’t make the next step fail. The join is a stream copy, which converts nothing, and the joined file takes its range label from the first part. A camera whose file opens on limited-range filler is labeled limited range throughout, real footage included. That happened to 11 of the more than a thousand hyperlapse files in my batch, all in folders that held only a fragment of a drive, and they had to be rendered again. The script checks each part’s range before joining, because the joined file shows only the first part’s.

Join the segments without re-encoding

List the parts in order in a text file, one file line each, then join them with the concat demuxer and a stream copy:

$ printf "file '%s'\n" <part-001>.mp4 <part-002>.mp4 > list.txt
$ ffmpeg -nostdin -v error -f concat -safe 0 -i list.txt -c copy <camera>_30x.mp4

Every part was encoded with the same options, so the copy only rewrites the container. It takes seconds.

Script the whole drive

The steps combine into one script per camera. It reads the segment list from the front camera’s files, so a camera missing a segment gets black filler for it. The macOS ._ files beside the clips on the truck’s exFAT drive don’t match its pattern, so they’re skipped. The front camera sets every segment’s length, so all five files come out the same length.

#!/usr/bin/env bash
set -euo pipefail

folder=$1
camera=$2
size=${3:-1920x1536}
name=$(basename "$folder")
encode=(-map_metadata -1 -c:v hevc_videotoolbox -q:v 65 -tag:v hvc1
        -color_range pc -an)

keyframes() {
  ffprobe -v error -skip_frame nokey -select_streams v \
    -show_entries frame=pts_time -of csv=p=0 "$1" | wc -l | tr -d ' '
}

work=$(mktemp -d)
: > "$work/list.txt"

for front in "$folder/${name}_frontCenter_"*.mp4; do
  segment=${front##*_}
  frames=$(keyframes "$front")
  source="$folder/${name}_${camera}_${segment}"
  part="$work/$segment"

  if [[ -f $source ]]; then
    shortfall=$(( frames - $(keyframes "$source") ))
    if (( shortfall < 0 )); then shortfall=0; fi
    ffmpeg -nostdin -v error -skip_frame nokey -i "$source" \
      -vf "setpts=N/(30*TB),tpad=stop=${shortfall}:color=black" -r 30 \
      -frames:v "$frames" "${encode[@]}" "$part"
  else
    ffmpeg -nostdin -v error -f lavfi -i "color=black:s=${size}:r=30" \
      -frames:v "$frames" "${encode[@]}" "$part"
  fi
  range=$(ffprobe -v error -select_streams v -show_entries stream=color_range \
    -of csv=p=0 "$part")
  if [[ $range != pc ]]; then
    echo "$part: color_range=$range, expected pc" >&2
    exit 1
  fi
  echo "file '$part'" >> "$work/list.txt"
done

ffmpeg -nostdin -v error -f concat -safe 0 -i "$work/list.txt" \
  -c copy "${name}_${camera}_30x.mp4"
rm -r "$work"

It writes <drive>_<camera>_30x.mp4 in the current folder. Run it once per camera, from the folder where the hyperlapses belong, and pass 1280x720 as the third argument for the Gear Guard camera:

$ for camera in frontCenter rearCenter sideLeft sideRight; do \
    bash hyperlapse.sh <drive-folder> "$camera"; done
$ bash hyperlapse.sh <drive-folder> gearGuard 1280x720

The script assumes the front camera recorded every segment. If it didn’t, the segment is missing from every camera’s hyperlapse.

Confirm every camera has the same frame count

Same length is the whole point of the padding, so count the frames in each output, and read the pixel format while you’re there:

$ for f in *_30x.mp4; do echo "$f $(ffprobe -v error -select_streams v \
    -count_packets -of compact=p=0 \
    -show_entries stream=nb_read_packets,pix_fmt,color_range "$f")"; done

<drive>_frontCenter_30x.mp4 pix_fmt=yuvj420p|color_range=pc|nb_read_packets=351
<drive>_gearGuard_30x.mp4 pix_fmt=yuvj420p|color_range=pc|nb_read_packets=351
<drive>_rearCenter_30x.mp4 pix_fmt=yuvj420p|color_range=pc|nb_read_packets=351
<drive>_sideLeft_30x.mp4 pix_fmt=yuvj420p|color_range=pc|nb_read_packets=351
<drive>_sideRight_30x.mp4 pix_fmt=yuvj420p|color_range=pc|nb_read_packets=351

Every camera of a drive prints the same count, and every file prints yuvj420p and pc. A camera with fewer frames was not padded. The range here is only the first part’s, because a stream-copy join takes its label from the first part, so the script checks every part’s range before it joins them and stops on one that isn’t pc.

Remove the location before publishing

A hyperlapse runs from the start of a drive to its end. Trim both ends before you publish one, too.

Keyframes vs a setpts timelapse of every frame

Reading only keyframes is what makes this fast, and it fixes what the hyperlapse can be.

On the same 351-second clip, I rendered the 30× hyperlapse twice: once from keyframes, and once by decoding every frame and keeping every 30th with select='not(mod(n,30))'. Both came out at 351 frames. The keyframe render took 4.4 seconds and used about one CPU core. The full decode finished in 7.0 seconds only because it spread across the cores: it used 57 seconds of CPU time, roughly 13 times the work. On one clip that’s a few seconds of waiting. Across a batch of drives, the CPU time is what adds up.

What the full decode buys you is choice:

For dashcam footage with a steady one-second keyframe, 30× was the speed I wanted, so keyframes were the right choice. If the speed has to be something else, decode everything and accept the cost.

The same -skip_frame nokey read is the cheap way to analyze footage, not only to watch it. When you extract keyframes as raw frames rather than encoding a video, add -fps_mode passthrough. Without it, ffmpeg duplicates each keyframe back up to 30 fps, and every frame-to-frame difference you compute reads zero.

TL;DR

Cut each camera with one pass of ffmpeg’s concat demuxer over its source segments, with -c copy. In the listing, set the first inpoint to that camera’s own keyframe at or before the moment you want, not to the moment itself:

file '<drive-folder>/07_16_26_120000_sideLeft_002.mp4'
inpoint 889.966667
outpoint 900
file '<drive-folder>/07_16_26_120000_sideLeft_003.mp4'
inpoint 0
outpoint 30

$ ffmpeg -f concat -safe 0 -i sideLeft.txt -c copy sideLeft.mp4

Snapping makes an offset camera’s file start a few frames early. Confirm that every file has a start_time of 0 and the same number of frames plus that lead. In DaVinci Resolve, set each clip’s In point that many frames in, then build the multicam with the angles synced by In point.

I wanted the same few minutes of a drive, the crossing of the I-90 floating bridge, from all five cameras of a Rivian R1T, as five files for a multicam edit. Re-encoding five streams to get exact cuts costs time and picture quality, so the cut is a stream copy. A stream copy is exact only where the source allows it. My first three attempts came out of sync, and each failure is the reason for one step in the procedure.

How the truck records

The truck writes one folder per drive, named MM_DD_YY_HHMMSS in local time. It holds one file per camera per 15-minute segment, <drive>_<camera>_<NNN>.mp4, and every segment except the last runs 900 s. The cameras are frontCenter, rearCenter, sideLeft, sideRight, and gearGuard.

Three properties of the files decide how they can be cut and synced:

• The video is H.264 at 30 fps, with I-frames and P-frames only and one keyframe per second.
• The files have no audio track and no timecode track.
• All five cameras of a segment have the same 900 s duration; their creation_time can differ by a couple of seconds, so it can’t sync them.

With no audio and no timecode, an editor has nothing to sync the angles on but each file’s first frame. The cut has to start every camera on a known frame, so the edit can line the first frames up.

Why stream-copied cameras drift

A stream copy can’t start a file on a P-frame, because a P-frame is stored as a difference from earlier frames that the new file won’t contain. It starts on a keyframe. Two details of this footage turn that rule into drift:

• Keyframes don’t fall at the same instant on every camera. On most cameras they fall on whole seconds. On some drives, one or more cameras put them 0.967 s after each second, one frame before the next whole second.
• Some cameras’ groups of pictures run across segment files. A segment can open with frames that come before its first keyframe. Those frames reference frames at the end of the previous file.

Each attempt that failed ran into one of these, and the checks in the last step of the procedure catch each one:

Equal durations aren’t enough. Two files can be the same length and still start at different moments, so the checks look at frame counts and at start_time together.

Find the segments the window touches

Express the window in seconds from the start of the drive. Segment NNN covers drive seconds (NNN − 1) × 900 to NNN × 900. The examples here use a window chosen to straddle a join, because that’s where a cut goes wrong: from 1790 to 1830 s, it touches two segments:

Every segment after the first starts at inpoint 0, and every segment before the last ends at outpoint 900. Only the first segment’s in-point is snapped.

Find each camera’s keyframe at the in-point

Each camera writes a keyframe about once a second, but not at the same fraction of a second as the others, and not always at the same fraction for a whole segment. On one drive, the Gear Guard camera’s keyframes start on whole seconds and slip to whole seconds plus 0.933 by the end of the segment:

$ ffprobe -v error -select_streams v -skip_frame nokey \
>   -show_entries frame=pts_time -of csv=p=0 -read_intervals %+3 \
>   08_02_26_120000_gearGuard_001.mp4

0.000000
1.000000
1.966667
2.966667

$ ffprobe -v error -select_streams v -skip_frame nokey \
>   -show_entries frame=pts_time -of csv=p=0 -read_intervals 887%+3.001 \
>   08_02_26_120000_gearGuard_001.mp4

887.933333
888.933333
889.933333

Across the drives I checked, keyframes sat on whole seconds or at whole seconds plus anything from 0.733 to 0.967, and most cameras that weren’t on whole seconds drifted within a segment. Which camera is offset, and by how much, changes from drive to drive. Test your own footage rather than assuming any of these numbers: list the keyframes in the 3 seconds before the in-point, in the segment that contains it, for every camera.

Snap the in-point to the camera’s own keyframe

For each camera, move the first in-point back to the latest keyframe at or before it, from the listing near the in-point. On the drive above, 890 becomes 889.933333 for the Gear Guard camera and stays 890 for the others. Measuring the offset at the start of the segment and applying it at 890 would give 889.966667, which is after the real keyframe.

Snap in drive time, not segment time. When the in-point falls before the first keyframe of its segment, start on the last keyframe of the previous segment, because the first frames of a segment reference the tail of the one before it. Only at the start of a drive, with nothing before it, does such an in-point become 0.

Rounding back rather than forward is the point. On the drive these examples come from, a stream copy given 890 on the offset side-left camera started at its next keyframe, 890.967, so the file opened 0.967 s late and carried that as its start_time. Given 889.966667, its keyframe there, it started on the keyframe exactly, at time 0. The snapped camera’s first frames come before the other cameras’ first frame, by the gap between its keyframe and the in-point: 1 frame at 0.967, 8 frames at 0.733. A stream copy can’t remove them, so the edit does: set that clip’s In point that many frames in.

Write one concat listing per camera

Write a listing for each camera with one file, inpoint, and outpoint entry per segment. <drive-folder> stands for the drive folder’s absolute path, which is why the cut later needs -safe 0:

file '<drive-folder>/07_16_26_120000_sideLeft_002.mp4'
inpoint 889.966667
outpoint 900
file '<drive-folder>/07_16_26_120000_sideLeft_003.mp4'
inpoint 0
outpoint 30

One listing for both segments is what keeps the frames after the join. The concat demuxer reads segment 003 straight after the tail of 002, so the frames at the start of 003 that reference 002 still have their reference. Cutting 003 on its own drops them, and joining the two pieces afterwards can’t bring them back.

Cut each camera with a stream copy

Run the concat demuxer once per camera. This writes a new file and leaves the source segments untouched:

$ ffmpeg -f concat -safe 0 -i 07_16_26_120000_sideLeft.txt \
>   -c copy 07_16_26_120000_sideLeft.mp4

The following script does the last four steps for all five cameras. It takes the drive folder, the window’s start and end in drive seconds, and an output folder, and writes each camera’s listing and clip there:

#!/bin/bash
set -euo pipefail
folder=$(cd "$1" && pwd) start=$2 end=$3 out=$4
drive=$(basename "$folder")
segment_seconds=900
mkdir -p "$out"

# Stop with the camera and the file named, rather than ffprobe's bare error.
require() {
  if [[ ! -f $1 ]]; then
    echo "$camera: no file $(basename "$1")" >&2
    exit 1
  fi
}

# Print the latest keyframe at or before $3 seconds into file $1, looking back
# $2 seconds; print nothing if there is none in that span.
keyframe_at_or_before() {
  local file=$1 back=$2 at=$3 from
  from=$(awk -v a="$at" -v b="$back" 'BEGIN { f = a - b; if (f < 0) f = 0; print f }')
  ffprobe -v error -select_streams v -skip_frame nokey \
    -show_entries frame=pts_time -of csv=p=0 \
    -read_intervals "${from}%$(awk -v a="$at" 'BEGIN { print a + 0.001 }')" "$file" |
    cut -d, -f1 | awk -v a="$at" -v f="$from" \
      '$1 >= f - 0.0005 && $1 <= a + 0.0005 { k = $1 } END { if (k != "") print k }'
}

for camera in frontCenter rearCenter sideLeft sideRight gearGuard; do
  segment=$(( start / segment_seconds + 1 ))
  local_in=$(( start - (segment - 1) * segment_seconds ))
  file="$folder/${drive}_${camera}_$(printf '%03d' "$segment").mp4"
  require "$file"
  length=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$file")
  if awk -v l="$length" -v i="$local_in" 'BEGIN { exit !(l <= i) }'; then
    echo "$camera: $(basename "$file") ends before ${local_in}s" >&2
    exit 1
  fi
  keyframe=$(keyframe_at_or_before "$file" 3 "$local_in")
  if [[ -n $keyframe ]]; then
    snapped=$(awk -v k="$keyframe" -v b=$(( (segment - 1) * segment_seconds )) \
      'BEGIN { printf "%.6f", b + k }')
  elif (( local_in >= 3 )); then
    # No keyframe this far into the segment means the file ends before it.
    echo "$camera: $(basename "$file") ends before ${local_in}s" >&2
    exit 1
  elif (( segment > 1 )); then
    # The in-point falls before this segment's first keyframe, so the frames
    # there depend on the previous segment: start on its last keyframe.
    previous="$folder/${drive}_${camera}_$(printf '%03d' $(( segment - 1 ))).mp4"
    require "$previous"
    keyframe=$(keyframe_at_or_before "$previous" 3 "$segment_seconds")
    if [[ -z $keyframe ]]; then
      echo "$camera: segment $(( segment - 1 )) ends early" >&2
      exit 1
    fi
    snapped=$(awk -v k="$keyframe" -v b=$(( (segment - 2) * segment_seconds )) \
      'BEGIN { printf "%.6f", b + k }')
  else
    snapped=0
  fi
  echo "$camera starts $(awk -v s="$snapped" -v a="$start" \
    'BEGIN { printf "%d", (a - s) * 30 + 0.5 }') frames early"
  first=$(awk -v s="$snapped" -v n="$segment_seconds" 'BEGIN { print int(s / n) + 1 }')
  last=$(( (end - 1) / segment_seconds + 1 ))
  listing="$out/${drive}_${camera}.txt"
  : > "$listing"
  for (( segment = first; segment <= last; segment++ )); do
    source="$folder/${drive}_${camera}_$(printf '%03d' "$segment").mp4"
    require "$source"
    segment_start=$(( (segment - 1) * segment_seconds ))
    clip_in=0
    (( segment == first )) && clip_in=$(awk -v s="$snapped" \
      -v b="$segment_start" 'BEGIN { printf "%.6f", s - b }')
    clip_out=$(( end - segment_start ))
    (( clip_out > segment_seconds )) && clip_out=$segment_seconds
    length=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$source")
    if awk -v l="$length" -v o="$clip_out" 'BEGIN { exit !(l < o - 0.05) }'; then
      echo "$camera: $(basename "$source") ends before ${clip_out}s" >&2
      exit 1
    fi
    printf "file '%s'\ninpoint %s\noutpoint %s\n" \
      "$source" "$clip_in" "$clip_out" >> "$listing"
  done
  ffmpeg -nostdin -v error -f concat -safe 0 -i "$listing" \
    -c copy "$out/${drive}_${camera}.mp4"
done

$ bash cut-window.sh <drive-folder> 1790 1830 cuts

For each camera, the script prints how many frames before the window’s start its file begins: 0 for a camera with a keyframe on the start, more for an offset one. Keep that list for the edit.

If a camera has no file for a segment the window touches, or its file ends before the window does, the script stops and names the camera and the file rather than writing a short clip. The camera usually restarted into a sibling folder partway through the drive.

Verify the start and frame count of every camera

In the output folder, print each file’s start_time, the number of frames that decode, and the number of packets in the file:

$ for clip in *.mp4; do
>   printf '%s ' "$clip"
>   ffprobe -v error -select_streams v -count_frames -count_packets \
>     -show_entries stream=start_time,nb_read_frames,nb_read_packets \
>     -of csv=p=0 "$clip"
> done

07_16_26_120000_frontCenter.mp4 0.000000,1200,1200
07_16_26_120000_gearGuard.mp4 0.000000,1200,1200
07_16_26_120000_rearCenter.mp4 0.000000,1201,1201
07_16_26_120000_sideLeft.mp4 0.000000,1201,1201
07_16_26_120000_sideRight.mp4 0.000000,1200,1200

The cut is good when all three hold:

Start

Every file’s start_time, the first column, is 0.000000.

Length

The frame counts differ only by each camera’s lead, the frames the script reported it starts early.

Join

The decoded frames equal the packets on every file, so every frame after the segment join decoded.

The third check is the one that sees the join. -count_frames decodes the whole file and counts only the frames that decoded; a frame whose reference is missing is dropped without an error. The same command on the source segment 003 on its own, outside the concat pass, shows the frames that depend on 002:

$ ffprobe -v error -select_streams v -count_frames -count_packets \
>   -show_entries stream=nb_read_frames,nb_read_packets -of csv=p=0 \
>   07_16_26_120000_sideLeft_003.mp4

26971,27000

Those 29 frames are the ones a per-segment cut loses. Decoding with ffmpeg -v error … -f null - prints nothing for them, so a silent decode is not evidence of a clean join. Compare the counts.

Build the multicam clip

In DaVinci Resolve, the leads the script printed are the only alignment the five files need:

1. Import the five files into the Media Pool.
2. For each camera with a lead, open its file in the source viewer, step forward that many frames from the first, and mark an In point.
3. Select all five, right-click, and choose Create New Multicam Clip Using Selected Clips.
4. Set Angle Sync to In.
5. Create the multicam clip, and put it on a timeline.
6. Switch the Edit page’s source viewer to Multicam and cut between angles during playback.

In lines up each file’s In point, which is its first frame unless you mark one. The other sync methods need something these files don’t carry: Timecode needs a timecode track and Sound needs audio. A file whose first frame is 0.967 s later than the others’ is 0.967 s out of sync after an In-point sync, whatever its start_time says. That’s why the verification step checks the start before the files reach Resolve.

Check each clip for a location before you share it

The concat demuxer drops the truck’s GPS fix, but other ways of cutting keep it; Read the GPS location from Rivian dashcam video with ffprobe covers which. Check the file you’re about to share:

$ exiftool -G1 -a "-gps*" "-location*" 07_16_26_120000_sideLeft.mp4

Empty output means no location field was found.

TL;DR

The camera didn’t stop recording. It restarted seconds later into a new drive folder that holds only the restarted cameras. Look for a folder named a few seconds after the broken one, with fewer than five cameras in it.

Don’t take the offset from the two folder names. Measure it by correlating the motion of two cameras that face the same way, such as the Gear Guard camera and the rear camera. Then re-encode the restarted cameras from the sibling folder at that offset, so that they start on the exact frame.

I cut a five-camera clip from a drive on August 7 and dropped it into a multicam timeline. Three angles played to the end. The rear camera and the side-left camera went black after 33 seconds. Every file was readable and nothing reported an error. The footage I thought was lost was in the next folder.

Rear or side camera stopped recording after a few seconds

The symptom has a few forms, depending on what you build from the drive:

• A camera’s only file in the drive folder is a few seconds long, while the other cameras have a full set of 15-minute segments.
• A multicam clip or a grid of the five cameras shows black tiles for one or two angles for most of the drive.
• A hyperlapse of one angle is much shorter than the others, or ends in black.

On the August 7 drive, the folder 08_07_26_140000 holds full segments for the front, Gear Guard and side-right cameras, and one 33-second file each for the rear and side-left cameras:

$ ffprobe -v error -show_entries format=duration -of csv=p=0 \
    08_07_26_140000/08_07_26_140000_rearCenter_001.mp4

33.000000

Cause: the camera restarted into a sibling folder

The truck writes one folder per drive, named MM_DD_YY_HHMMSS in local time, and one file per camera per 15-minute segment inside it. When a camera stops and restarts partway through a drive, its recording doesn’t continue in the drive’s folder. A new folder is created a few seconds later, and it holds only the cameras that restarted. The rest of the drive stays in the original folder.

The two folders together hold the whole drive. Nothing was lost, but the drive is now split across two clocks.

Find the sibling folder

Run this in the DriveStream folder of the truck’s drive, or of your copy of it. It prints each drive folder with the cameras it holds:

for d in */; do
  printf '%s ' "${d%/}"
  ls "$d" | grep -v '^\._' \
    | sed -E 's/^[0-9_]+_([A-Za-z]+)_[0-9]{3}\.mp4$/\1/' \
    | sort -u | tr '\n' ' '
  echo
done

The grep -v '^\._' drops the AppleDouble files that macOS leaves beside every clip on the truck’s exFAT drive. The August 7 folders look like this:

08_07_26_100000 frontCenter gearGuard rearCenter sideLeft sideRight
08_07_26_140000 frontCenter gearGuard rearCenter sideLeft sideRight
08_07_26_140034 rearCenter sideLeft
08_07_26_160000 frontCenter gearGuard rearCenter sideLeft sideRight

A sibling folder has all three of these properties:

• Its name is seconds, not minutes, after the previous folder’s name.
• It holds fewer than five cameras.
• Its cameras are the ones that stop early in the previous folder.

A short folder with one camera and no partner is a different case: a stray fragment, not half of a drive. Across 233 drive folders, one was a 21-second fragment from a single camera.

Folder name offset is wrong

The folder names differ by 34 seconds, so the obvious fix is to place the restarted cameras 34 seconds into the drive. Measured from the footage, the restarted cameras begin 32.7 seconds into the drive. The names are wrong by more than a second. The files’ creation_time tags are no better: the two folders' differ by 35 seconds, a third answer.

A folder name is a label, not a clock: nothing ties it to the first frame of the video inside. An error of more than a second is visible in a multicam edit. A car that passes the truck crosses the rear view and the side view at moments that no longer match. The truck’s files carry no timecode track to correct it, so the offset has to come from the pictures.

Measure the offset by motion correlation

Two cameras that see the same scene change at the same moments. When the truck hits a bump or a car passes behind it, both backward-facing pictures move at once. The script reduces each camera to a motion signal and slides one signal along the other until they match best:

1. Decode a window of each clip at a fixed sample rate, scaled to a tiny grayscale frame.
2. Take the mean absolute change between consecutive frames. That’s the motion signal: high when the picture changes, low when it holds still.
3. Standardize the reference, and each stretch of the other signal it’s compared with, so that a brighter or busier camera or stretch doesn’t outweigh the rest.
4. Try every lag within a search range around your guess, and keep the lag with the highest correlation.

It needs Python 3 and ffmpeg on your path, and nothing else:

"""Measure the time offset between two dashcam clips by correlating their motion.

Usage:
    python3 measure_offset.py REFERENCE OTHER --at SECONDS [--guess SECONDS]

Reads a window of REFERENCE starting at --at, and a slightly wider window of
OTHER around --at minus --guess. Prints the offset that lines the two up:
a moment at time t in REFERENCE is at time t - offset in OTHER.
"""
import argparse
import subprocess

WIDTH, HEIGHT = 32, 24
SAMPLES_PER_SECOND = 10


def motion(clip: str, start: float, seconds: float) -> list[float]:
    """Mean absolute change between consecutive tiny grayscale frames."""
    raw = subprocess.run(
        ["ffmpeg", "-nostdin", "-v", "error",
         "-ss", f"{start:.3f}", "-t", f"{seconds:.3f}", "-i", clip,
         "-vf", f"fps={SAMPLES_PER_SECOND},scale={WIDTH}:{HEIGHT},format=gray",
         "-f", "rawvideo", "-"],
        capture_output=True, check=True,
    ).stdout
    size = WIDTH * HEIGHT
    frames = [raw[i:i + size] for i in range(0, len(raw) - size + 1, size)]
    return [sum(abs(a - b) for a, b in zip(before, after)) / size
            for before, after in zip(frames, frames[1:])]


def standardize(values: list[float]) -> list[float]:
    mean = sum(values) / len(values)
    variance = sum((v - mean) ** 2 for v in values) / len(values)
    spread = variance ** 0.5 or 1.0
    return [(v - mean) / spread for v in values]


def best_lag(reference: list[float], other: list[float]) -> tuple[float, int]:
    """Slide reference along other; return the best correlation and its lag."""
    scores = []
    for lag in range(len(other) - len(reference) + 1):
        window = standardize(other[lag:lag + len(reference)])
        score = sum(a * b for a, b in zip(reference, window)) / len(reference)
        scores.append((score, lag))
    return max(scores)


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("reference")
    parser.add_argument("other")
    parser.add_argument("--at", type=float, required=True,
                        help="window start in REFERENCE, seconds")
    parser.add_argument("--guess", type=float, default=0.0,
                        help="expected offset, seconds")
    parser.add_argument("--length", type=float, default=120.0,
                        help="window length, seconds")
    parser.add_argument("--search", type=float, default=5.0,
                        help="how far either side of the guess to look")
    args = parser.parse_args()

    reference = standardize(motion(args.reference, args.at, args.length))
    # The other window starts --search seconds early, so lag 0 means guess + search.
    other_start = args.at - args.guess - args.search
    other_length = args.length + 2 * args.search
    other = motion(args.other, other_start, other_length)

    score, lag = best_lag(reference, other)
    offset = args.guess + args.search - lag / SAMPLES_PER_SECOND
    print(f"window at {args.at:g} s: correlation {score:.2f}, offset {offset:+.1f} s")


if __name__ == "__main__":
    main()

Give it a camera from the original folder as the reference, a camera from the sibling folder as the other, and the folder-name difference as the guess. Both folders number their segments from 001 and every segment but the last runs 900 seconds, so segment 002 of the sibling starts the same offset after segment 002 of the original. Compare files with the same segment number, and keep the window inside one segment.

In the DriveStream folder, run:

$ python3 measure_offset.py \
    08_07_26_140000/08_07_26_140000_gearGuard_001.mp4 \
    08_07_26_140034/08_07_26_140034_rearCenter_001.mp4 \
    --at 300 --guess 34

window at 300 s: correlation 0.84, offset +32.6 s

One window is one measurement. Repeat it in windows that don’t overlap, from different segments, and trust the offset only when they agree. Four windows on the August 7 drive gave this:

That’s 32.7 seconds, plus or minus 0.1, from four independent windows. A moment at time t in the original folder is at t − 32.7 in the sibling folder.

Compare cameras that face the same way

The front camera was my first choice for the reference, because it recorded the whole drive. Against the restarted rear camera it gave two answers:

The two windows disagree by 0.9 seconds, and each one looks plausible on its own. The front camera sees the road ahead and the rear camera sees the road behind. Their pictures change for different reasons most of the time, so the best match moves from one window to the next. The Gear Guard camera looks backward over the bed, the same direction as the rear camera, and the two agree.

Pick a reference camera that kept recording in the original folder and faces the same direction as one of the restarted cameras. The files in a sibling folder share one start time and the same segment lengths, so the offset you measure for one of its cameras applies to all of them.

Stitch the restarted cameras

A stream copy can only start on a keyframe, and the truck writes one keyframe per second. The stream-copied angles from the original folder start on their own keyframes. An offset of 32.7 seconds isn’t a whole number of seconds, so no keyframe in the sibling folder falls on the same instant. Re-encode only the restarted cameras, cut at the exact frame, and leave the rest as stream copies.

Suppose your clip covers START to END seconds of the original folder, and OFFSET is the measured offset. The window has to begin after the restart, so START is greater than OFFSET. In the sibling folder, the clip begins at START minus OFFSET and lasts END minus START seconds.

In the DriveStream folder, the following commands write a concat list of the restarted camera’s segments in the sibling folder, and then cut the clip from the joined segments, re-encoding to HEVC. They create two files and change nothing in the drive folders:

printf "file '%s'\n" "$PWD"/08_07_26_140034/08_07_26_140034_rearCenter_*.mp4 \
  > rearCenter.txt
ffmpeg -f concat -safe 0 -i rearCenter.txt -ss <sibling-start> -t <duration> \
  -c:v hevc_videotoolbox -q:v 80 -tag:v hvc1 -color_range pc -an \
  <clip-folder>/rearCenter.mp4

Run the same two commands for every other camera in the sibling folder.

-ss after -i decodes from the start of the list and discards frames up to the exact time, so the output starts on that instant rather than on the nearest keyframe. It’s slow for a window late in the drive, because ffmpeg decodes everything before the window first. Joining the segments in one concat pass, rather than cutting each file and joining the cuts, keeps the frames whose group of pictures runs across a segment boundary. Cut multi-camera dashcam clips with ffmpeg covers the stream-copied cameras and that join in full.

The encoder hevc_videotoolbox is the hardware HEVC encoder on macOS. On another system, use the encoder you’d use for the rest of the project. The option -color_range pc labels the re-encoded angle full range like the truck’s footage, so it matches the stream-copied ones; check it with ffprobe -show_entries stream=color_range. The truck records no audio, so -an drops nothing.

Every file now starts at time 0. The re-encoded angles start on the exact frame, and a stream-copied one starts early by the lead the cut script printed for it. In DaVinci Resolve, mark each lead as an In point, then create the multicam clip with angle sync set to the In point.

Confirm the stitch

Run the same script on the finished clip, with the guess left at zero. In the clip’s folder, compare the Gear Guard camera, which was stream-copied from the original folder, with the re-encoded rear camera:

$ python3 measure_offset.py 08_07_26_140000_gearGuard.mp4 \
    08_07_26_140000_rearCenter.mp4 --at 300 --length 90

window at 300 s: correlation 0.79, offset +0.1 s

An offset of +0.1 seconds is one sample step of SAMPLES_PER_SECOND, the resolution of the method. The angles are in sync. The same script reported 32.7 seconds on the unstitched folders, so it does notice an offset when one exists. A zero from a check that has never reported anything else proves nothing.

TL;DR

The GPS position is in the MP4 container’s location tag. Read it with ffprobe:

$ ffprobe -v error -show_entries format_tags=location -of default=nw=1:nk=1 \
    <clip>.mp4

47.589462-122.2655

The value is a signed latitude immediately followed by a signed longitude, in decimal degrees: 47.589462 north, -122.2655 west. That clip starts on the I-90 floating bridge. 0,0 means the truck had no fix.

There is one fix per file, taken where the 15-minute segment began. There is no per-second GPS track.

Spellings of the tag

The value has no separator between the two numbers. The sign of the longitude is the separator, so a parser has to split on the second + or -, not on a comma or a space. The truck never writes a leading plus sign. Across 475 front-camera segments, all from encoder tag Rivian Firmware ver. 2022-47, three spellings turned up, and 0,0 stands in when there is no fix:

Usual

47.589462-122.2655, on 439 of the 475 segments.

Trailing slash

the same form ending in /, on the other 36.

After ffmpeg copies the file

+47.5894-122.2655/, the ISO 6709 form, with a leading plus sign and four decimal places. This is ffmpeg’s rewrite, not a firmware variant.

No fix

0,0, which is a position in the Gulf of Guinea, not a reading.

ffprobe also prints a location-eng tag with the same value. Either one works; the commands here read location.

What one fix per segment rules out

The truck records one file per camera per 15-minute segment, and each file carries a single fix from the moment that segment started. Nothing inside the file records where the truck was a minute later. That rules out:

• A route drawn from the video. A segment gives you a point every 15 minutes, which is too coarse to follow a road.
• Speed. Two fixes 15 minutes apart give an average over the whole segment, not the speed at any moment.
• Finding a place by GPS alone. A drive across the floating bridge takes less than a segment, so the bridge usually falls between two fixes.

The five cameras of a segment each carry their own fix. In the clip earlier, the front and rear files read 47.589462-122.2655 and the two side cameras and the Gear Guard camera read 47.589474-122.2659. Treat any one of them as the segment’s position.

To place a moment inside a segment, read the picture instead of the container. Finding a bridge in dashcam video by the color of tunnel lights locates the I-90 floating bridge crossing from the picture, where the GPS can’t.

Parse the tag in Python

parse_location returns a (latitude, longitude) tuple, or None when the tag is missing, unreadable, or 0,0. read_location runs ffprobe on one file. The standard library is enough; Python 3.10 or later, for the | in the type hints.

import re
import subprocess
import sys

# A signed latitude immediately followed by a signed longitude. The leading
# plus sign and the trailing slash are optional because not every writer
# emits them.
LOCATION = re.compile(r"([+-]?\d+(?:\.\d+)?)([+-]\d+(?:\.\d+)?)")


def parse_location(tag: str) -> tuple[float, float] | None:
    match = LOCATION.search(tag)
    if not match:
        return None
    latitude, longitude = float(match[1]), float(match[2])
    # The truck writes 0,0 when it has no GPS fix.
    if latitude == 0 and longitude == 0:
        return None
    return latitude, longitude


def read_location(clip: str) -> tuple[float, float] | None:
    tag = subprocess.run(
        ["ffprobe", "-v", "error", "-show_entries", "format_tags=location",
         "-of", "default=nw=1:nk=1", clip],
        capture_output=True, text=True, check=True,
    ).stdout
    return parse_location(tag)


if __name__ == "__main__":
    for clip in sys.argv[1:]:
        point = read_location(clip)
        print(clip, "no fix" if point is None else f"{point[0]}, {point[1]}")

Pass it one or more clips:

$ python3 rivian_location.py <clip>.mp4

<clip>.mp4 47.589462, -122.2655

The tests cover every spelling in the list earlier, a southern and eastern position so both signs are exercised, and each way the tag can mean “no fix”:

import pytest

from rivian_location import parse_location


@pytest.mark.parametrize(("tag", "expected"), [
    ("47.589462-122.2655", (47.589462, -122.2655)),
    ("47.589462-122.2655/", (47.589462, -122.2655)),
    ("+47.5894-122.2655/", (47.5894, -122.2655)),
    ("-33.8688+151.2093/", (-33.8688, 151.2093)),
])
def test_reads_each_spelling_of_the_tag(tag, expected):
    assert parse_location(tag) == expected


@pytest.mark.parametrize("tag", ["", "\n", "garbage", "+0.0000+0.0000/", "0-0"])
def test_returns_none_for_no_fix(tag):
    assert parse_location(tag) is None

$ python3 -m pytest -q

.........                                                                [100%]
9 passed in 0.01s

Dropping the optional sign from the latitude’s half of the pattern fails the southern-hemisphere case, so the tests do notice a broken pattern.

Remove the GPS location from a clip before sharing it

The tag travels with the file. A raw clip posted online tells anyone who downloads it where it was recorded, and a folder of them maps where the truck goes every day. Trimming the clip doesn’t necessarily remove it: a stream copy with ffmpeg kept the tag, rewritten in the +…/ form, and exiftool reports it as a GPS position:

$ ffmpeg -i <clip>.mp4 -t 2 -c copy copy.mp4
$ exiftool -G1 -a "-gps*" "-location*" copy.mp4

[Composite]     GPS Altitude                    : 0 m
[Composite]     GPS Altitude Ref                : Above Sea Level
[Composite]     GPS Latitude                    : 47 deg 35' 22.02" N
[Composite]     GPS Longitude                   : 122 deg 15' 55.76" W
[Composite]     GPS Position                    : 47 deg 35' 22.02" N, 122 deg 15' 55.76" W
[UserData]      Location Information            : (none) Role=shooting Lat=47.58945 Lon=-122.26549 Alt=0.00 Body=earth Notes=

Other ways of writing a file behave differently. The multi-camera clips cut with ffmpeg’s concat demuxer came out with no location tag at all, because the demuxer drops file-level metadata. The hyperlapse tool I used kept it, because it copies the first source file’s metadata on purpose. Check each file rather than trusting the tool. To drop the container’s metadata while copying, add -map_metadata -1; the same exiftool command then prints nothing:

$ ffmpeg -i <clip>.mp4 -t 2 -c copy -map_metadata -1 strip.mp4
$ exiftool -G1 -a "-gps*" "-location*" strip.mp4

Empty output is the result you want. Run the check on the file you’re about to upload, not on its source: each tool between the two can keep the tag or drop it.

TL;DR

Create a role with no permissions that your account can assume, switch the session to its temporary keys, and run the script. Every call it makes that needs a permission is denied for real:

$identity  = Get-STSCallerIdentity
$partition = $identity.Arn.Split(':')[1]
$account   = $identity.Account
$trust = '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:' + $partition + ':iam::' + $account + ':root"},"Action":"sts:AssumeRole"}]}'
New-IAMRole -RoleName denytest -AssumeRolePolicyDocument $trust
$previous = Get-AWSCredential
try {
    # A new role took about eight seconds to become assumable in testing.
    Start-Sleep -Seconds 8
    $delay = 1
    foreach ($attempt in 1..6) {
        try {
            $keys = (Use-STSRole -RoleArn "arn:${partition}:iam::${account}:role/denytest" -RoleSessionName denytest -ErrorAction Stop).Credentials
            break
        }
        catch {
            # Retry only the denial a new role gives; rethrow anything else, and the last denial.
            if ($_.Exception.InnerException.ErrorCode -ne 'AccessDenied' -or $attempt -eq 6) { throw }
            Start-Sleep -Seconds $delay
            $delay *= 2
        }
    }
    Set-AWSCredential -Credential $keys
    ./Your-Script.ps1
}
finally {
    # Restore your own credentials first; deleting the role needs them.
    if ($previous) { Set-AWSCredential -Credential $previous } else { Clear-AWSCredential }
    Remove-IAMRole -RoleName denytest -Force
}

AWS.Tools wraps the AWS error in an InvalidOperationException; the AccessDenied code is on $_.Exception.InnerException, not on $_.Exception.

A script that calls AWS will one day run with credentials that can’t do what it asks: a new role missing one permission, a scheduled task under the wrong profile. What it does then is usually untested, because the identity the script was written with can do everything.

I hit this while writing How to Publish a Custom CloudWatch Metric from PowerShell. Its common failures needed the exact message Write-CWMetricData prints when the caller isn’t allowed to publish. Removing a permission from my account to see one error was out, and so was publishing a real metric to watch it fail. This is the trick I used instead, and the exception details in this article come from those Write-CWMetricData runs.

A role with no permissions is a second identity that IAM denies for every action that needs a permission. A call made with its keys comes back with the genuine AccessDenied, and nothing is written or changed. The IAM Policy Simulator says whether a call would be denied; it doesn’t produce the exception a catch block has to handle.

Install the IAM and STS modules

New-IAMRole and Remove-IAMRole are in AWS.Tools.IdentityManagement; Get-STSCallerIdentity and Use-STSRole are in AWS.Tools.SecurityToken.

Install-Module -Name AWS.Tools.IdentityManagement, AWS.Tools.SecurityToken -Scope CurrentUser -Force
Import-Module AWS.Tools.IdentityManagement, AWS.Tools.SecurityToken

Create a role with no permissions

The trust policy decides who can assume the role. The partition and account come from your own ARN, so the same lines work in the standard, China and GovCloud partitions. Naming your account’s root lets any identity in the account with sts:AssumeRole permission assume the role. Yours can if its policy allows sts:AssumeRole on this role; an administrator’s does. No permission policy is attached, so the role is denied everything except calls that need no permission, such as Get-STSCallerIdentity, and resources whose own policy grants access to anyone.

$identity  = Get-STSCallerIdentity
$partition = $identity.Arn.Split(':')[1]
$account   = $identity.Account
$trust = '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:' + $partition + ':iam::' + $account + ':root"},"Action":"sts:AssumeRole"}]}'
New-IAMRole -RoleName denytest -AssumeRolePolicyDocument $trust

Assume the role for temporary keys

Use-STSRole returns temporary credentials for the role; keep the Credentials property. A new role isn’t assumable straight away: in my test the first attempt was denied and the role became assumable about eight seconds later. Wait that long before the first call, which would otherwise be denied for certain, then retry with a backoff to cover a slower day. The loop waits 1, 2, 4, 8 and 16 seconds between attempts, retries only AccessDenied, and rethrows after the sixth attempt so a role you can’t assume fails instead of looping:

# A new role took about eight seconds to become assumable in testing.
Start-Sleep -Seconds 8
$delay = 1
foreach ($attempt in 1..6) {
    try {
        $keys = (Use-STSRole -RoleArn "arn:${partition}:iam::${account}:role/denytest" -RoleSessionName denytest -ErrorAction Stop).Credentials
        break
    }
    catch {
        # Retry only the denial a new role gives; rethrow anything else, and the last denial.
        if ($_.Exception.InnerException.ErrorCode -ne 'AccessDenied' -or $attempt -eq 6) { throw }
        Start-Sleep -Seconds $delay
        $delay *= 2
    }
}

Catch AccessDenied in the script

The cmdlet throws a System.InvalidOperationException whose InnerException is the AWS exception. That inner exception is an Amazon.Runtime.AmazonServiceException carrying ErrorCode AccessDenied, StatusCode Forbidden and ErrorType Sender. Test the inner exception, handle the denial, and rethrow anything else. $datum is the metric from your own script:

try {
    Write-CWMetricData -Namespace 'MyApp/Queue' -MetricData $datum -ErrorAction Stop
}
catch {
    $awsError = $_.Exception.InnerException
    if ($awsError -is [Amazon.Runtime.AmazonServiceException] -and $awsError.ErrorCode -eq 'AccessDenied') {
        Write-Warning "No permission to publish the metric: $($awsError.Message)"
    }
    else {
        throw
    }
}

Run the script as the role

Set-AWSCredential -Credential makes the role’s keys the default for the session, so the script runs unmodified. Every call in it that needs a permission is denied. Get-AWSCredential with no arguments returns the session’s current credentials, so save them first and put them back after. Clear-AWSCredential alone would only remove the role’s keys, not restore what was there before, and the Remove-IAMRole that follows needs your own access. When nothing was set, clearing is right: later calls fall back to your default profile or environment. The restore sits in finally because the script is expected to fail: without it, an unhandled denial skips the restore, the session stays on the role’s keys, and the cleanup that follows is denied too. The script’s error still reaches you.

$previous = Get-AWSCredential
Set-AWSCredential -Credential $keys
try { ./Your-Script.ps1 }
finally { if ($previous) { Set-AWSCredential -Credential $previous } else { Clear-AWSCredential } }

To deny a single call instead, pass the keys to that cmdlet with -Credential $keys.

Delete the role

Remove-IAMRole -RoleName denytest -Force