Ask Your Question
0

How can I convert several columns into rows in a Java Dataset?

asked 2023-05-22 07:03:04 +0000

scrum gravatar image

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2023-05-22 07:30:01 +0000

ladyg gravatar image

You can use the pivot function in Apache Spark to convert multiple columns into rows. Here's an example:

import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;

Dataset<Row> df = // your dataset with multiple columns

// specify the column to use as the keys for the new rows
String[] pivotColumns = {"column1", "column2"};

// select the columns to pivot
String[] pivotColValues = {"column3", "column4", "column5"};

// pivot the columns into rows
df.groupBy(pivotColumns).pivot("pivotColValues").agg(functions.sum("value"));

This will group the dataset by the specified pivot columns and pivot the selected columns into rows. The resulting dataset will have one row for each unique combination of the pivot columns, with the pivoted values as columns.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss

Add Answer


Question Tools

Stats

Asked: 2023-05-22 07:03:04 +0000

Seen: 19 times

Last updated: May 22 '23