Recently, I came across a problem of dividing a folder with N number of files into sub-folders with almost equal number of files each. While with small number of file you may do it easily, when things come to Big Data, its always wiser to run a script.
The requirement can be of two types:
- Divide N number of files into sub-folders with X number of files each. Number of sub-folders doesn’t matter.
cd /path/to/folder/with/files
export num_file=1000 #Assuming 1000 files into each sub-folder
i=0;
for file_name in *;
dodir_name=dir_$(printf %03d $((i/$num_file)));
#Direcotry names would be dir_000 … dir_999 and so on.
mkdir -p $dir_name;
mv “$file_name” $dir_name;
let i++;done
- Divide N number of files into X number of sub folders with almost equal number of files. Number of files in each folder doesn’t matter.
cd /path/to/folder/with/files
export tot_file=$(ls | wc -l)
echo $tot_file
export num_folder=24 #Assumming 24 sub-folders, each for an hour.
export num_file=($(($tot_file/$num_folder)) + 1)
i=0;
for file_name in *;
dodir_name=hr_$(printf %02d $((i/$num_file)));
#Directory names would be hr_00, hr_01 … hr_23
mkdir -p $dir_name;
mv “$file_name” $dir_name;
let i++;done