This article was shared from Huawei Cloud CommunityGaussTech Technology Column] Principles of GaussDB Logic Decoding Technology., authored by GaussDB Database.
1. Background
With the accelerated pace of digital transformation in major domestic industries, the demand for heterogeneous database data synchronization scenarios is increasing.
Heterogeneous database synchronization, i.e., synchronizing data between databases of different types and structures to ensure the consistency of data between different databases. For example, migrating data from the current database to other types of databases, or backing up data from the current database to another database in real time, so as to improve the security and reliability of data.
Huawei Cloud provides DRS service, which covers migration, synchronization, disaster recovery, subscription, and recording and playback between heterogeneous databases. Currently, DRS supports more than 20 relational and non-relational databases as the source, including GaussDB. the principle of DRS data synchronization with GaussDB as the source database is shown in the following figure.
DRS drives the source database GaussDB to parse the WAL logs in real time to generate logical logs, and then the DRS service receives and parses the logical logs, converts them into SQL statements for the target database, and drives the target database to execute the SQL statements, a process known as logical replication.
For the source database, the core problem to be solved is how to convert WAL logs into logical logs, the process is called logical decoding. In this article we introduce the technical principles of GaussDB logical decoding.
2. GaussDB logic decoding
The WAL log contains all data changes that occur in the database, including operations such as inserts, updates, and deletes, as well as many internal database details and unique implementations.
Logical decoding is used to parse WAL logs into logical log formats that are easy to understand and process, including JSON, binary, or fixed text formats. Users and logical replication tools such as DRS can parse and process these logical logs according to their needs.
When logical decoding is enabled, GaussDB records a small amount of decoding-assisted information (such as csn snapshots for visibility determinations during the decoding phase) to the WAL log in addition to writing the basic operations of each transaction to the WAL log to support the logical decoding process. It is also necessary to create a logical replication slot. The purpose of the logical replication slot is to prevent the database from deleting WAL logs that have fallen off the disk and to prevent the system table records required for decoding from being purged.
As shown in the above figure, logical decoding mainly includes several modules of data source, reading/loading, decoding, rearranging/sending, etc. WAL log and table metadata stored in the system table are the content source of logical decoding. Logical decoding captures the change records of user table DML from WAL log, loads the metadata of the table corresponding to the system table based on the physical storage identifier (block number and offset, etc.) and the commit sequence number (csn), converts the strongly coupled internal information in the physical change records into user-understandable table content, generates logical change records that are independent of database implementation, and finally rearranges and sends the logical change records, and finally rearranges and sends the logical change records. Then convert the strongly coupled internal information in the physical change records into user-understandable table contents, generate logical change records that are independent of the database implementation, and finally rearrange and send the logical change records.
There are two ways to decode GaussDB logic, serial decoding and parallel decoding. Serial decoding process is divided into read, decode, send three steps, the entire serial decoding process are completed in the same thread, in which the decoding time consumed more than 70% of the whole process. The performance of serial decoding is about 3-5M/s, while parallel decoding is executed in parallel by multiple threads, which greatly compresses the decoding process. Because of the low efficiency of serial decoding, we do not recommend serial decoding, but recommend the more efficient parallel decoding, and the current GaussDB logic decoding tasks initiated by Huawei Cloud DRS service are all parallel decoding, and we focus on parallel decoding below.
3. Parallel decoding
Parallel decoding is used to improve logic decoding performance by concurrent execution of multiple threads.
As shown above, parallel logic decoding contains three types of threads:
Reader thread
Read the WAL log, extract the business DML operation and the record corresponding to the decoding auxiliary information, construct the LogicalLogChange memory object, if the DML involves toast data, it is also necessary to splice the toast data into the LogicalLogChange object of the dml operation, and then distribute it to the decoding input queue according to the log sequence number (LSN. The unique identifier of a WAL log, which is an ordered value) is distributed to the decode input queue in rotating order.
If a DDL is decoded and the DDL updates the system table, the Reader thread invalidates the localsyscache at this point and adds the invalidation message to the Decoder thread input queue and broadcasts a notification to the Decoder thread to invalidate the local cache. Subsequent decoder threads will also invalidate the localsyscache in the thread when they decode the invalidated message.
Decoder thread
Get the LogicalLogChange object from the decoding input queue, load the meta-information of the data table according to the content of the log version, convert the physical data in the log into logical data that is easy to be understood by the user such as the table name, column name, column data, and so on, and the decoded content is stored in the LogicalLog memory object, and the LogicalLog object is added to the decoding output queue.
As introduced earlier, the Reader thread will put the LogicalLogChange object into the decoding input queue according to the LSN order, and the decoder will also put the LogicalLog into the decoding output queue according to the LSN order after the decoding is completed, for the later Sender thread to read.
Sender thread
In accordance with the DML log generation order, take LogicalLog objects from the decoding output queue, construct a hash bucket according to the transaction ID of each object, according to the order of each LogicalLog object in the decoding output queue (the queue has been sorted in LSN order), categorize the LogicalLog objects of the same transaction into corresponding hash buckets, and then send all LogicalLog objects of each transaction to the logical log receiver (e.g., DRS) in the The commit order of the transaction sends all LogicalLog objects of each transaction to the LogicalLog receiver (e.g., DRS).
Parallel decoding can be configured on-demand decoding concurrency, the concurrency parameter value range is 1-20, the maximum number of concurrent decoding threads can be configured 20.
GaussDB parallel decoding can greatly improve the decoding performance. In a typical tpcc scenario, the decoding rate can reach 100MB/s.
4. DDL decoding
GaussDB logical decoding supports DDL decoding. If logical decoding is enabled in GaussDB, the parse tree of the DDL statement is parsed during the DDL SQL execution phase, and the result of the parsing is assembled into a string in Json format (example), and a new WAL log type is added to write the Json string to the WAL log. When the logical decoding thread parses to this WAL log type, it outputs the decoded logical log of the DDL in the original Json format. the Json format decoding result of the DDL statement alter table t1 add column col3 varchar(64) is shown in the following figure.
5. Multi-version data dictionaries and finger-point decoding
Logical decoding relies on table metadata information during the decoding of DML, so it requires access to the system table (also called the data dictionary), and this decoding schema is also called the online catalog schema.GaussDB caches the system table information in a syscache, which needs to be invalidated when a DDL statement updates the system table. When the DDL is executed, an invalidation message log will be written to the WAL. Each thread that logically decodes the DDL saves a copy of the syscache, i.e., the localcache, and when it decodes the invalidation message log, it will invalidate the localcache accordingly.
The logical decoding of the online catalog schema relies on the historical metadata information of the tables, so it needs to utilize the logical replication slots to retain the old version tuples of the system tables. If you want to decode any historical logs at a given location, the old version of the system table tuple on which the logical decoding depends needs to be retained all the time, which will seriously affect the performance of the database.
Therefore, GaussDB adds a set of system tables dedicated to logical decoding, called Logical Decoding Multi-Version Data Dictionary. Logical decoding multi-version data dictionary refers to the kernel system table, the table structure is basically consistent with the kernel system table structure, retains all the historical versions of the metadata of the table during the decoding process, and implements a separate cleanup mechanism.
Newly installed instances automatically create the logical decoding multi-version data dictionary. For non-newly installed instances, when initializing the logical decoding multi-version data dictionary for the first time, you need to call a system function to initialize it, which synchronizes the data from the kernel system table to the logical decoding multi-version data dictionary. When the DDL statement is executed and before the transaction commits, the metadata is inserted into the kernel system table and at the same time the metadata is inserted into the logical decoding multi-version data dictionary. When decoding, the decoding thread reads the metadata information from the logical decoding multi-version data dictionary into the localcache cache according to the current csn snapshot. Meanwhile, logical decoding implements a timed task for multi-version data dictionary cleanup, and through the guc parameter, the retention time is configured to remove tuples that exceed the retention time. Moreover, with this guc parameter, we can retain old tuples for an arbitrary time, thus realizing logical decoding from a finger locus (i.e., the physical location of the WAL) without affecting the database performance, and the locus can be any location including the archived WAL logs.
By adding a separate set of multi-version data dictionaries for logical decoding, the logical decoding can be decoupled from the kernel system table, and the kernel system table does not need to keep a large number of old version tuples for logical decoding, which guarantees the kernel's operational performance.
6. Distributed CN decoding
GaussDB supports distributed decoding capability. Distributed decoding is divided into directly connected DN decoding and CN decoding. Directly connected DN decoding that is not through the CN, the decoding tool is directly connected to the DN node for DN decoding, the decoding method and centralized decoding principle is the same, can only decode the DN slice of the data. Distributed decoding mainly refers to connecting CN for decoding, which can completely decode the incremental data of the whole cluster.
CommitCSN represents the order in which transactions are completed. For two dependent transactions, the CommitCSN of the later executing transaction is greater than the CommitCSN of the first executing transaction.Distributed Logical Decoding returns the logical logs of the transactions in the order in which they were committed.
As shown in the figure, the DNs each return the logical logs of the local transactions in the order of transaction commits, and the CN coordinates the aggregation of the transaction logical logs from each DN through heap sorting, and outputs the logical logs in the order of heap sorting.
Compared to direct-connect DN decoding, it has the advantages of having fewer links, being able to ensure strong consistency, and being easier to use.
7. Summary
GaussDB kernel provides logical decoding technology , with DRS services , you can realize the logical replication between heterogeneous databases . GaussDB logical decoding not only supports DML decoding , but also supports DDL decoding , which greatly improves the ease of use of logical replication . It supports independent multi-version data dictionary schema for decoding and supports finger point decoding and data retrieval. By enabling parallel decoding, the WAL log decoding rate can be greatly improved to ensure the high performance of data synchronization.
Huawei Developer Space, which gathers development resources and tools from various root technologies such as Hongmeng, Rise, Kunpeng, GaussDB, and Euler, is committed to providing each developer with a cloud host, a set of development tools, and storage space on the cloud, so that developers can innovate based on Huawei's root ecosystem.
Click on the link to get your exclusive cloud hosting for free~!