Ask Your Question
1

When transferring data from mysql to bigquery using Google Datastream, how can tables without a primary key's is_deleted field be handled?

asked 2023-05-16 04:56:14 +0000

ladyg gravatar image

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
2

answered 2023-05-16 05:16:01 +0000

david gravatar image

When transferring data from MySQL to BigQuery using Google Datastream, tables without a primary key's is_deleted field can be handled in the following ways:

  1. The data can be loaded directly into BigQuery without any changes. In this case, Datastream will create a surrogate key for the table and insert a timestamp column to indicate when each record was deleted.

  2. If the table does not have a primary key, you may consider adding one to make the data replication easier. This can be done by selecting a unique grouping of columns to act as the primary key.

  3. If the table cannot have a primary key, you may consider adding a new column to the table with a default value of false. This column can be used to track the is_deleted flag for each record.

  4. Another option is to add a separate table to the database to track the is_deleted flag for each record. This table can be joined to the primary table to retrieve the latest state of each record.

Ultimately, the best solution will depend on the specific needs of your application and data structure.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss

Add Answer


Question Tools

Stats

Asked: 2023-05-16 04:56:14 +0000

Seen: 10 times

Last updated: May 16 '23