Duplicity out of memory

For years I’am using duplicity to create encrypted backups and store them on amazon s3. I’m using a bash wrapper script for automated backups with duplicity supporting Amazon’s S3 online storage as well as other storage destinations (ftp, rsync, sftp, local storage…).

Backup error

But I received a message on my backup notification slack channel that one backup failed. The message buffer dmesg on an ubuntu 16.04 machine showed an out of memory message.

1
2
[29152765.948135] Out of memory: Kill process 2519 (python2) score 606 or sacrifice child
[29152765.949376] Killed process 2519 (python2) total-vm:1331852kB, anon-rss:845472kB, file-rss:488kB

Duplicity version upgrade needed

This was due to a bug which is resolved in version 0.7.15. (bug report)

1
2
3
* Fixed bug #1720159 - Cannot allocate memory with large manifest file since 0.7.03
  - filelist is not read if --file-changed option in collection-status not present
  - This will keep memory usage lower in non collection-status operations

The server was using a version from the default ubuntu repo, which is version 0.7.06.

$ apt show duplicity
Package: duplicity
Version: 0.7.06-2ubuntu2
Priority: optional
Section: utils
Origin: Ubuntu
...

Install duplicity package from duplicity-team repo

Install the duplicity package from ppa:duplicity-team/ppa. I also suggest to add this to your ansible roles or configuration management tool you are using.

sudo add-apt-repository ppa:duplicity-team/ppa
sudo apt-get update
sudo apt-get purge duplicity
sudo apt-get install duplicity

Check the package version is at least 0.7.15.

$ duplicity --version
duplicity 0.7.18.2

Thanks for reading

Feel free to leave a comment if you have remarks or like this post