Ask Your Question
0

What is the method for displaying the parquet file's (including type) structure through the command line or spark shell?

asked 2021-11-10 11:00:00 +0000

lalupa gravatar image

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
2

answered 2022-12-02 06:00:00 +0000

lakamha gravatar image

To display the structure of a parquet file, including the data type of each column, you can use the printSchema() method in Apache Spark. Here's an example:

val df = spark.read.parquet("path/to/parquet/file")
df.printSchema()

Alternatively, you can use the parquet-tools command-line tool to display the schema of a parquet file. Here's an example:

parquet-tools schema path/to/parquet/file

This will output the schema of the parquet file in JSON format.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss

Add Answer


Question Tools

Stats

Asked: 2021-11-10 11:00:00 +0000

Seen: 8 times

Last updated: Dec 02 '22