Apr 30, 2018

How to use gsutil from Google to copy files to Amazon S3

( The recommended way is still to use AWS Command Line Interface )
After gsutil is installed in your home directory, you cd into gsutil, edit the gsutil file and search for the line:

def SanityCheckXmlParser(cmd):

Then, right below the line, add a line with two spaces and the word

  return

That will make it work.  You also need to setup ~/.boto file like this


[Credentials]
aws_access_key_id = your id key
aws_secret_access_key = your secret key

[Boto]
debug = 0
num_retries = 10

gsutil will ask you for GS only if you don't have anything.

The documentation says gs:// but it will also works for s3://, exactly the same.  The syntax is similar to s3cmd but more like Linux.

To copy the whole directory (and subs):

gsutil cp -R mydir s3://my-unique-bucket

To copy subdirectories, -R option is necessary for S3.