When transferring data from MySQL to BigQuery using Google Datastream, tables without a primary key's is_deleted field can be handled in the following ways:
The data can be loaded directly into BigQuery without any changes. In this case, Datastream will create a surrogate key for the table and insert a timestamp column to indicate when each record was deleted.
If the table does not have a primary key, you may consider adding one to make the data replication easier. This can be done by selecting a unique grouping of columns to act as the primary key.
If the table cannot have a primary key, you may consider adding a new column to the table with a default value of false. This column can be used to track the is_deleted flag for each record.
Another option is to add a separate table to the database to track the is_deleted flag for each record. This table can be joined to the primary table to retrieve the latest state of each record.
Ultimately, the best solution will depend on the specific needs of your application and data structure.
Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss
Asked: 2023-05-16 04:56:14 +0000
Seen: 10 times
Last updated: May 16 '23
How can multiple queries be merged into a single stored procedure in MySQL?
What does "waiting for handler commit" mean in relation to the slow writes experienced in MySQL 8?
How can Spring Boot and Mysql be utilized for CRUD operations?
How can I use sqlalchemy to merge JSON within a column of mysql?
Retrieve all rows in which two IDs appear for the first time.
What are the benefits of choosing sqlalchemy.types instead of sqlalchemy.dialects.mssql?