To change the columns of a Pyspark data frame to lowercase, you can use the toDF()
and withColumnRenamed()
functions. Here's an example:
from pyspark.sql.functions import lower
# create example data frame
df = spark.createDataFrame([(1, "A", 10), (2, "B", 20), (3, "C", 30)], ["ID", "Name", "Score"])
# change column names to lowercase
df = df.toDF(*[c.lower() for c in df.columns])
# display the new data frame
df.show()
In this example, the toDF()
function is used to create a new data frame with the same data but with lowercased column names. The withColumnRenamed()
function is not used because it only renames one column at a time, so a list comprehension is used to apply the lower function to each column name. The show()
function is used to display the new data frame.
Asked: 2023-06-04 16:13:08 +0000
Seen: 12 times
Last updated: Jun 04 '23