As a Data Engineer, one faces the need to share files securely over the internet. An easy way of doing … More
Category: Scripts
How to copy a file, recreating the directory structure, using python?
To copy “Folder1/Folder2/file1” to “Folder3” source structure: Folder1/FolderA ………………….(not to be copied)Folder1/fileX…………………………(not to be copied)Folder1/Folder2/file1 desired destination structure: Folder3/Folder1/Folder2/file1 P.S. … More
How to exclude certain column(s) while exporting a Hive Table to local file?
In a previous post, I documented How to export complete Hive table to a local file? While being required to get … More
How to export complete Hive table to a local file?
Generally, solutions provided over the internet point towards: hive -e ‘select * from dbname.tablename;’ > /path/to/datadump.csv However, this will not … More
How to color code ‘diff’ output on CLI?
TL;DR : vim -d <file1> <file2> Comparing changes in files from time to time is very much the part and … More
Dividing folder contents into subfolders
Recently, I came across a problem of dividing a folder with N number of files into sub-folders with almost equal … More
