To perform BQ load on a CSV file saved in storage using Java client, you can follow these steps:
Set up the Google Cloud Storage Java client library in your project.
Create a Storage object that points to the bucket containing the CSV file. Storage storage = StorageOptions.getDefaultInstance().getService(); Bucket bucket = storage.get("my-bucket");
Create a Blob object that represents the CSV file. Blob blob = bucket.get("path/to/myfile.csv");
Create a TableId object that represents the destination table in BigQuery. TableId tableId = TableId.of("my-project", "mydataset", "mytable");
Create a LoadJobConfiguration object that specifies the details of the load job, such as source format, destination table, schema, and other parameters. LoadJobConfiguration configuration = LoadJobConfiguration.builder(tableId, blob) .setFormatOptions(FormatOptions.csv()) .setSchema(mySchema) .setWriteDisposition(JobInfo.WriteDisposition.WRITE_TRUNCATE) .build();
Submit the job to BigQuery using the BigQuery client library. BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService(); Job loadJob = bigquery.create(JobInfo.of(configuration));
Wait for the job to complete and check the result. loadJob = loadJob.waitFor(); if (loadJob.getStatus().getError() != null) { throw new RuntimeException("Failed to load CSV file to BigQuery: " + loadJob.getStatus().getError()); }
That's it! Once the job completes successfully, the CSV file data will be loaded into the specified BigQuery table.
Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss
Asked: 2023-07-09 23:08:27 +0000
Seen: 9 times
Last updated: Jul 09 '23
How can set the Project Title in the Doxygen Configuration File?
How can I convert Double to Long in Java?
Can I add a default Parameter for a Method in Java like int calculate(int x, int y=2)?
How can the rejection of the class text_plain from JavaMail API due to a VerifyError be confirmed?
Is it feasible to utilize a Toggle Button to switch among multiple classes exceeding 2?
How can I set up Environment variables on my M1 MacBook and resolve the ZSH permission denied error?
What is the process for generating a dynamic subdomain/URL using vue.js?