AWS CLI sync vs. Write-S3Object: What Changes on the Second Run

TL;DR

aws s3 sync compares a source and destination before selecting transfers. Write-S3Object -Folder -Recurse uploads every file in the folder hierarchy on every run; recursion adds subfolders, not a destination comparison or deletion pass.

Keep sync when your script depends on skipping files that already match. Use Write-S3Object when uploading every selected file is the intended operation. Changing tools also means changing error handling: native exit codes for the CLI, PowerShell errors for the cmdlet.

The refactor looks harmless. Replace one external command with a PowerShell cmdlet, run it, and confirm that the files appear in S3.

The first run proves little. The second run is where the behaviors separate.

Two commands that can produce the same first upload

The following are alternatives, run from PowerShell. They assume an existing source folder, a bucket you control, and configured AWS access. Replace the bucket placeholder. The first needs AWS CLI; the second needs AWS.Tools.S3.

PowerShell
$Source = 'C:\Data\exports'
$Bucket = 'YOUR-BUCKET'

$null = Get-Command aws -CommandType Application -ErrorAction Stop
aws s3 sync $Source "s3://$Bucket/exports/"
$Code = $LASTEXITCODE
if ($Code -ne 0) { throw "S3 sync exited with code $Code" }

PowerShell
$Source = 'C:\Data\exports'
$Bucket = 'YOUR-BUCKET'

Import-Module AWS.Tools.S3 -ErrorAction Stop
Write-S3Object -BucketName $Bucket -Folder $Source `
    -KeyPrefix 'exports/' -Recurse -ErrorAction Stop

For a source containing report.json and daily\summary.csv, the intended keys are exports/report.json and exports/daily/summary.csv. Keep the prefix fixed when comparing repeated runs; a new prefix gives either command an empty destination namespace.

AWS describes the cmdlet as an uploader for files, content, or folder trees. -Recurse includes subfolders. It doesn’t implement the CLI’s sync algorithm. See the Write-S3Object reference (opens in a new tab).

What each command does on a repeat run

Behavior	`aws s3 sync`	`Write-S3Object -Folder -Recurse`
Select transfers by comparing destination	Yes	No sync comparison
Upload an unchanged selected file again	Usually skipped by comparison	Uploaded again
Traverse subfolders	Built in	Requires `-Recurse`
Remove destination-only objects	Only with `--delete`	No reconciliation of deletions
Preview a sync plan	`--dryrun`	No equivalent sync plan
Report failure to PowerShell	Native exit code	PowerShell error

For local-to-S3 sync, the default upload criteria are a missing destination object, a different size, or a local modification time newer than the S3 object’s last-modified time. That time is when the object was uploaded. It isn’t a byte-for-byte scan of every file. A same-size file restored or copied with its original, older timestamp is skipped, and S3 keeps the stale bytes. --size-only discards the time criterion; --exact-timestamps applies to S3-to-local sync. These rules and directions are documented in the AWS CLI sync reference (opens in a new tab).

This is the migration question: did your script promise to upload a collection of files, or did it promise to transfer only what its comparison considers changed? Similar-looking final keys don’t answer that question.

Deletions are a separate decision

Deleting a local file doesn’t remove its S3 object in either example above. For the CLI, --delete explicitly adds destination cleanup. Preview that choice before applying it:

PowerShell
aws s3 sync 'C:\Data\exports' 's3://YOUR-BUCKET/exports/' --delete --dryrun
$Code = $LASTEXITCODE
if ($Code -ne 0) { throw "Sync preview exited with code $Code" }

The preview reports proposed operations; it isn’t proof that subsequent writes or deletions will be authorized or succeed. Review the exact bucket and prefix before removing --dryrun.

Caution

A destination prefix shared by unrelated jobs is a poor place for --delete. A file absent from one job’s source may still belong to another, and the reconciliation will remove it. Give each source a deliberate namespace before enabling deletion. In a versioned bucket the delete adds a delete marker, so the object can be recovered until lifecycle rules expire its noncurrent versions; in an unversioned bucket it’s gone.

For backups, destination-only files create another issue: restoring all current objects can resurrect files that were intentionally removed locally. Decide whether you need a current mirror or recoverable snapshots before choosing how to handle that case.

Repeated uploads in a versioned bucket

In a bucket with versioning enabled, another successful overwrite creates another object version. Each stored version is a complete object, not only the changed bytes. AWS explains this in its S3 Versioning guide (opens in a new tab).

Consequently, repeatedly uploading an unchanged tree can create redundant versions as well as extra transfer work and requests. How much that matters depends on your data and schedule. A small configuration directory and a large export tree need different tradeoffs.

Conversely, a fresh timestamped prefix intentionally stores a new set of keys. There’s no previous tree there for sync to reuse. For that design, uploading every file may be exactly what you want.

Neither approach makes a live folder an atomic snapshot. If several related files change during the transfer, the destination can contain a mixture of moments. Use an application-consistent export or stable staging directory when a restore requires the files to agree with each other.

Preserve the failure contract when changing tools

The CLI example checks its result immediately. A final check after several native commands can miss an earlier failure. AWS’s return-code documentation (opens in a new tab) includes failed and skipped S3 transfers among nonzero outcomes.

For the cmdlet, add context and propagate the error if needed:

PowerShell
try {
    Write-S3Object -BucketName $Bucket -Folder $Source `
        -KeyPrefix 'exports/' -Recurse -ErrorAction Stop
}
catch {
    Write-Warning ("Export upload failed: {0}" -f $_.Exception.Message)
    throw
}

Don’t leave an old $LASTEXITCODE check after the cmdlet. The separate PowerShell error-handling article shows why that can report success for a failed upload.

Check a migration before trusting the replacement

Use a disposable source folder and dedicated destination prefix:

Upload two files and record their destination keys.
Repeat without editing either file; inspect actual uploads or object versions.
Change one file’s size and repeat; check which objects are written.
Remove one local file; inspect the destination-only object.
Use a missing source path and verify the overall script fails.
Use a -SearchPattern that matches nothing and confirm the script notices. Write-S3Object uploads nothing and reports no error.

Those checks exercise the behavior a syntax translation can overlook. They also make your deletion and repeat-run expectations explicit.

When each is the wrong choice

aws s3 sync is the wrong choice when:

A file can come back with its size and an older timestamp, as a restore or a copy that keeps times does. The comparison skips it, and the destination keeps the old bytes.
You add --delete on a prefix that other jobs also write to. The reconciliation removes their objects because they are absent from your source.
Every run writes to a fresh prefix, such as a dated snapshot. There’s nothing there to compare against, so the comparison buys nothing and the script still depends on the AWS CLI.

Write-S3Object -Folder -Recurse is the wrong choice when:

The script relies on skipping files that haven’t changed. Every run uploads the whole tree again, and in a versioned bucket each upload is another complete version.
The destination has to mirror the source, deletions included. The cmdlet never removes destination-only objects, and adding that means owning comparison rules, key mapping, filters, deletions and partial failures yourself.
You need to see what a run will do before it runs. There’s no equivalent of --dryrun.

Choose based on the repeat run and the restore you need.