restic - failed: nonce is invalid error

I’m using restic to back up a few of my servers to Backblaze’s b2 storage. Occasionally I also clean up after old snapshots to keep the bill low for my files. For some reason, restic wasn’t able to delete a snapshot, and I just got this error message when running restic prune to remove all unreferenced files from snapshots:

repository 112911b9 opened successfully, password is correct
loading indexes...
loading all snapshots...
finding data that is still in use for 95 snapshots
[16:29] 3.16%  3 / 95 snapshots
decrypting blob bc103b4a84971ef6459b294a2b98568a2bfb72cded09d4acd1e16366a401f95b failed: nonce is invalid
github.com/restic/restic/internal/repository.(*Repository).LoadBlob
        /restic/internal/repository/repository.go:204
github.com/restic/restic/internal/repository.(*Repository).LoadTree
        /restic/internal/repository/repository.go:728
github.com/restic/restic/internal/restic.loadTreeWorker
        /restic/internal/restic/tree_stream.go:37
github.com/restic/restic/internal/restic.StreamTrees.func1
        /restic/internal/restic/tree_stream.go:164
golang.org/x/sync/errgroup.(*Group).Go.func1
        /home/build/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57
runtime.goexit
        /usr/local/go/src/runtime/asm_arm.s:841

I don’t know what exactly happened, but apparently restic wasn’t able to decrypt a blob stored in b2. Does it mean they had a failure and restic wasn’t able to recover? Did restic make some mistake? Did I? Don’t know. It’s worrisome, and I couldn’t find anything meaningful online.

Eventually I got it fixed, I’m still not convinced that everything went right, and I’m not sure about the reliability of my system, restic and b2. 🤷‍♂️

Delete the snapshot / files

In order to clean this mess up instead of deleting the whole bucket and re-uploading everything, I did the following steps:

Find the blob from the error message above. Take the checksum from the blob error message and run this command:

restic find --blob bc103b4a84971ef6459b294a2b98568a2bfb72cded09d4acd1e16366a401f95b -r b2:your-bucket-name

It’ll output another checksum, so you can now forget this file/snapshot/blob(?) with this command (take the new checksum restic gave you)

restic forget 9aff6ba4b042b9d09991a9fbf8c80ddbd2a9c433638339cd831bed955e39f106 -r b2:your-bucket-name

After this is done, you should be able to prune all unreferenced files from the backup like this:

restic -r b2:your-bucket-name prune