Revision history [back]

You can use the pivot function in Apache Spark to convert multiple columns into rows. Here's an example:

import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;

Dataset<Row> df = // your dataset with multiple columns

// specify the column to use as the keys for the new rows
String[] pivotColumns = {"column1", "column2"};

// select the columns to pivot
String[] pivotColValues = {"column3", "column4", "column5"};

// pivot the columns into rows
df.groupBy(pivotColumns).pivot("pivotColValues").agg(functions.sum("value"));

This will group the dataset by the specified pivot columns and pivot the selected columns into rows. The resulting dataset will have one row for each unique combination of the pivot columns, with the pivoted values as columns.