Small tips I’ve learned when using cloud storage:
gsutil is a great little tool for managing Google Cloud Storage. In fact, if you look in ~/.boto it lets you also configure it’s usage with AWS. This is great for me because I am often using both to test things (like performance) and it’s nice to use one tool for “cloud storage stuff” instead of tools specific to one storage provider.
I wanted to see how Google Storage works for hosting my blog so I copied it over.
gsutil -m cp -R s3://www.jorgecastro.org gs://www.jorgecastro.org
The -m is a nice little feature of gsutil. It turns on multithreaded copies, very nice if you have a fast connection. I now have my blog on two cloud providers, not bad for backups!
So does this mean I’ll never need s3cmd again? One bummer is that gsutil does copies, it doesn’t do an equivalent of an rsync. So if I am automating things I’m recopying a bunch of things over that don’t need to be.
s3cmd sync local_dir s3://www.jorgecastro.org
Ok so that means, if you’re doing a huge copy into S3 you can use gsutil’s awesome multithreaded uploading to speed that up, then use s3cmd in subsequent copies to save bandwidth. Not bad!
I suspect as more and more Cloud Storage providers pop up that both gsutil and s3cmd will start to become more generalized tools, as a cloud user I don’t really care where my bucket is, I do however want to have one tool to manipulate it.
References and Stuff you should check out: