Skip to content

Piyush Routray

  • Twitter
  • LinkedIn
  • About Piyush

Category: PySpark

PySpark DataFrame ordering based on multiple columns

How to deal with the requirement to order a DataFrame using more than one column simultaneously? Also, consider some values … More

Data Engineering, Databricks, PySpark

How to retain the first row of each ‘group’ in a PySpark DataFrame?

For a given dataframe, with multiple occurrence of a particular column value, one may desire to retain only one (or … More

Databricks, PySpark

How to split contents of a column into *only two* by a delimiter?

While working with PySpark, I came across a requirement, where data in a column had to be split using delimiters … More

PySpark

Browse by Category

ambari AWS Firehose AWS S3 centOS 7 cloudera cron Databricks Data Engineering datanode DevOps docker hadoop HDP hive Hortonworks java jenkins kubernetes linux OpenJDK OracleJDK PGP platform administration PySpark Python s3cmd security Snowflake stackoverflow Windows

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9 other subscribers
My Tweets
Create a website or blog at WordPress.com
  • Subscribe Subscribed
    • Piyush Routray
    • Already have a WordPress.com account? Log in now.
    • Piyush Routray
    • Subscribe Subscribed
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar