You can use the selectExpr
method in Spark to add multiple columns with the same WHEN
condition. Here's an example:
val df = Seq((1, "A"), (2, "B"), (3, "C")).toDF("id", "letter")
val newDf = df.selectExpr("*",
"CASE WHEN id < 2 THEN 'Low' ELSE 'High' END as id_category",
"CASE WHEN id < 2 THEN 'Small' ELSE 'Big' END as id_size")
newDf.show()
In this example, we're adding two columns (id_category
and id_size
) with the same WHEN
condition (id < 2
). The selectExpr
method allows us to pass in multiple expressions separated by commas, which are evaluated and added as new columns to the DataFrame.
Asked: 2023-05-31 19:46:56 +0000
Seen: 13 times
Last updated: May 31 '23