Nov 3, 2018

How to backup using pipe without intermediate save

Pipe large numbers of small files to Google cloud storage

For example, if you are copying from one server to another via SSH, you can also pipe it over with:

$ tar cjf - . | ssh user@backupserver 'cat > backup.tar.bz2'

To backup with compression directly to Google Cloud storage:

$ sudo tar cjf - /export/backup | gsutil cp - gs://i88-ca-backup/i88.ca.`date +%F`.tar.bz2
Copying from <STDIN>...
Uploading   gs://i88-ca-backup/i88.ca.2015-12-18.tar.bz2: