2PC
For performance reasons, each time you commit a transaction, you only need to place redo and undo to indicate that the transaction has been persisted, and you do not need to wait for the data to be placed. This can already guarantee the roll forward or roll back of the transaction crash. Since the undo information is also written to redo, in fact, we only need to decide whether to redo or rollback the crash recovrey according to whether the redo is placed. As mentioned above, after starting binlog, you also need to consider whether binlog is placed (binlog involves the master-slave data consistency and the site of full recovery). According to whether the transaction is successfully written to binlog to decide whether to redo or rollback the transaction.
2PC is innodb's two-phase commit mechanism for transactions. When mysql starts binlog, there will be an internal XA problem: the order of transactions in the storage engine layer (redo) commit and the order of submission in binlog are inconsistent.
2PC principle
The transaction commit is divided into two stages: prepare and commit:
1. prepare phase: redo persists to disk (redo group commit), and puts the rollback segment into the prepared state, at this time binlog does not operate.

2. Commit phase: innodb releases the lock, releases the rollback segment, sets the commit status, persists the binlog to disk, and then commits to the storage engine layer

After having an understanding of 2PC, we can find that redo and binlog can be group commit in the above picture, then let ’s understand what is group commit
Group Commit
background
As we know, log writing is basically sequential IO. WAL (Write-Ahead-Logging) uses sequential log writing instead of random IO for data to achieve transaction persistence. However, despite this, every transaction commit requires log flushing, which is still limited to disk IO. The emergence of group commit is to merge the action of redo / binlog brushing, thereby improving IO performance
The 2PC mechanism can only solve the problem that the redo / binlog order of a single transaction is consistent. What if it is concurrent? (There may be a case: binlog submission order (T1, T2, T3), innodb commit order (T2, T3, T1)).

As shown above, the transaction starts in T1, T2, and T3 order, writes the binary log (in T1, T2, and T3 order) to the log file system buffer, and calls fsync () to perform a group Commit writes log files to disk permanently, but the order of storage engine submission is T2, T3, and T1. After T2 and T3 commit the transaction, if the online physical backup is used for database recovery to establish replication, because the InnoDB storage engine layer will detect the transaction T3 has completed the transaction submission at both the upper and lower layers, there is no need to restore it at this time The primary and secondary data are inconsistent
Before MySQL version 5.6, the prepare_commit_mutex lock was used to ensure the serial commit order of the upper binary log of the MySQL database and the Innodb storage engine layer in a serial manner, and then the group commit would not take effect

As shown in the figure above, MySQL uses prepare_commit_mutex and sync_log when binary log is turned on to ensure that the order of binary logs and storage engines is consistent. The lock mechanism of prepare_commit_mutex results in high concurrent commit transactions. Nor can group commit.
Redo Group Commit
In the prepare phase of 2PC, a redo operation will be performed on redo (innodb_flush_log_at_trx_commit = 1). At this time, the redo group commit process is as follows:

1. Get log_mutex
2. If flushed_to_disk_lsn> = lsn, it means the log has been flashed, jump to 5
3. If current_flush_lsn> = lsn, it means that the log is being flushed, and it will enter the waiting state after jump 5
4. Flush and sync logs smaller than LSN
5. Exit log_mutex
   This process is merged according to the order of LSN, that is to say, a redo group commit process may talk about lsn in other uncommitted transactions.
   Redo Group Commit optimization
   When each transaction is submitted, a redo flush action will be triggered. Due to the slow disk read and write, the system throughput is greatly affected. Taobao made an optimization, and moved the redo action of the prepare phase to the flush phase of the commit (flush-sync-commit), to ensure that the redo will be brushed before the binlog. This will not violate the original fault recovery logic. The advantage of moving to the commit phase is that instead of brushing every transaction, the leader thread helps to brush a batch of redo. How to achieve it is very simple, because log_sys-> lsn always maintains the current largest lsn, as long as we brush redo to the current log_sys-> lsn, we can ensure that the transaction redo log that will be binlog must have been placed. By delaying the writing of redo, the purpose of redo log group submission is achieved, and the competition of log_sys-> mutex is reduced. At present, this strategy has been introduced by the official mysql5.7.6.
   Binlog Group Commit
   The basic idea of ​​Binlog Group Commit is to introduce a queue mechanism to ensure that the order of innodb commit is the same as the order in which binlog is placed, and to group transactions into a transaction. Proceed to achieve the purpose of group submission. The first transaction in the queue is called the leader, and the other transactions are called followers. Let the leader do everything. The process is divided into three stages
   Flush Stag
   Write the binlog of each transaction to memory

1. Hold Lock_log mutex [leader holds, follower waits].
2. Get a set of binlogs in the queue (all transactions in the queue).
3. Transfer binlog buffer to I / O cache.
4. Notify the dump thread of dump binlog.
   Sync Stage
   Refresh the binary log in memory to disk. If there are multiple transactions in the queue, then the binary log is written in only one fsync operation, which is BLGC.
5. Release Lock_log mutex, hold Lock_sync mutex [leader holds, follower wait].
6. Place a set of binlogs (sync action, the most time-consuming, assuming sync_binlog is 1).
   Commit Stage
   The leader invokes the commit of the storage engine layer transaction according to the order. Innodb itself supports group commit, so the original problem of group commit due to lock prepare_commit_mutex is fixed.
7. Release Lock_sync mutex, hold Lock_commit mutex [leader holds, follower wait].
8. Traverse the transactions in the queue and perform innodb commit one by one.
9. Release Lock_commit mutex.
10. Wake up the waiting thread in the queue.
    Since there are multiple queues, each queue is protected by a mutex, and the queues are in order. The thread that enters the queue is the leader, so the leader in the FLUSH phase may be in the SYNC phase follower, but the follower is always the follower.
    When there is a group of transactions in the commit phase, other new things can be in the Flush phase, so that the group commit continues to take effect. The effect of group commit is determined by the number of transactions in the queue. If there is only one transaction in the queue at a time, the effect may be the same as before, or even worse. But when more transactions are submitted, the effect of group commit is more obvious, and the performance of the database is improved.
    One more thing to explain: sync_binlog = 1 This parameter unit represents a group of transactions, not a transaction
    Crash Recovery
    Crash Recovery(no binlog)
    Since uncommitted and rolled back transactions will also be recorded in the redo log, these transactions will need special treatment when recovering
    Innodb's processing strategy is: when recovering, start from checkpoint, redo all transactions (including uncommitted transactions and rolled back transactions), and then roll back those undo logs through undo log Committed transactions
    XA Crash Recovery
    1. Scan the last binlog to extract xid (identify the number of events in the binlog)
    2. The xid will also be written to redo, compare the xid in the prepare state in redo with the xid in the last binlog, if it exists in the binlog, submit it, otherwise roll back
    Why only scan the last binlog? Because when binlog rotates, all the previous binlogs will be flashed, and the transaction will not cross binlog