Ask Your Question
1

How can the columns of a Pyspark data frame be changed to lowercase?

asked 2023-06-04 16:13:08 +0000

scrum gravatar image

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
3

answered 2023-06-04 16:38:02 +0000

devzero gravatar image

To change the columns of a Pyspark data frame to lowercase, you can use the toDF() and withColumnRenamed() functions. Here's an example:

from pyspark.sql.functions import lower

# create example data frame
df = spark.createDataFrame([(1, "A", 10), (2, "B", 20), (3, "C", 30)], ["ID", "Name", "Score"])

# change column names to lowercase
df = df.toDF(*[c.lower() for c in df.columns])

# display the new data frame
df.show()

In this example, the toDF() function is used to create a new data frame with the same data but with lowercased column names. The withColumnRenamed() function is not used because it only renames one column at a time, so a list comprehension is used to apply the lower function to each column name. The show() function is used to display the new data frame.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss

Add Answer


Question Tools

Stats

Asked: 2023-06-04 16:13:08 +0000

Seen: 12 times

Last updated: Jun 04 '23