Ask Your Question
1

What is the method for using Pandas to display a list of all csv files in an s3 bucket?

asked 2021-08-14 11:00:00 +0000

huitzilopochtli gravatar image

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
2

answered 2022-07-21 12:00:00 +0000

djk gravatar image

To use Pandas to display a list of all csv files in an s3 bucket, you can use the following code:

import pandas as pd
import boto3

# Create S3 client
s3 = boto3.client('s3')

# List all objects in the bucket
objects = s3.list_objects(Bucket='your_bucket_name')

# Filter the list to include only csv files
csv_files = [file['Key'] for file in objects['Contents'] if file['Key'].endswith('.csv')]

# Convert the list of csv file names to a pandas DataFrame
df = pd.DataFrame(csv_files, columns=['File Name'])

# Display the DataFrame
print(df)

Note that you will need to replace "yourbucketname" with the actual name of the s3 bucket you want to use.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss

Add Answer


Question Tools

Stats

Asked: 2021-08-14 11:00:00 +0000

Seen: 24 times

Last updated: Jul 21 '22