s3cmd is an usual and handy tool for dealing with data in S3 from the command line. Usage Doc.
For simple data copy between two buckets, I would generally use the “s3cmd cp” command.
s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
However, recently I came across a two different needs while addressing this seemingly mundane task.
1. The folders to be copied were partially copied to the destination bucket; and it would have been expensive to simply copy them again.
2. The folders were strangely named with spaces in their name. eg. <source folder 1> instead of <source_folder_1> or other better naming practices.
The solution was to use “s3cmd sync” command between the folders and use double quotes (“) to wrap the complete folder name (S3 url).
s3cmd sync "s3://source-bucket/source folder 1/" "s3://destination-bucket/source folder 1/"
This is also a quick method to verify if both folders are in sync. If so, you get an output:
“Summary: 0 source files to copy, 0 files at destination to delete”