To use Pandas to display a list of all csv files in an s3 bucket, you can use the following code:
import pandas as pd
import boto3
# Create S3 client
s3 = boto3.client('s3')
# List all objects in the bucket
objects = s3.list_objects(Bucket='your_bucket_name')
# Filter the list to include only csv files
csv_files = [file['Key'] for file in objects['Contents'] if file['Key'].endswith('.csv')]
# Convert the list of csv file names to a pandas DataFrame
df = pd.DataFrame(csv_files, columns=['File Name'])
# Display the DataFrame
print(df)
Note that you will need to replace "yourbucketname" with the actual name of the s3 bucket you want to use.
Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss
Asked: 2021-08-14 11:00:00 +0000
Seen: 24 times
Last updated: Jul 21 '22
How to remove rows from a CSV file stored in a Google Cloud Storage bucket?
How can Deny be set on PutObject for only files that already exist in an s3 bucket?
What does the error message "NoSuchKey... The specified key does not exist" mean in a GCP bucket?
How can a png/plot be uploaded to an s3 bucket?
What is the process for configuring Amazon S3 replication in LocalStack?
How can one generate a list by using a portion of another list?
How can Swagger V3 use Schema Annotation to transform a List of Long to look like a List of String?
How can I modify the width of the column filter in an R shiny datatable?