Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions best-practices-for-security-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,9 @@ By default, TiDB installation includes several privileged interfaces for inter-c
| TiFlash | 20170 | Protocol |
| TiFlash | 20292 | HTTP |
| TiFlash | 8234 | HTTP |
| DM master | 8261 | HTTP |
| DM master | 8291 | HTTP |
| DM worker | 8262 | HTTP |
| DM-master | 8261 | HTTP |
| DM-master | 8291 | HTTP |
| DM-worker | 8262 | HTTP |
| TiCDC | 8300 | HTTP |
| TiDB Lightning | 8289 | HTTP |
| TiDB Operator | 6060 | HTTP |
Expand Down
6 changes: 3 additions & 3 deletions dm/dm-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,12 +211,12 @@ The following are two solutions to this issue:
2. Remove all files in the directory of exported data.
3. Delete the task using dmctl and run the command `start-task --remove-meta` to create a new task.

After the new task starts, it is recommended to ensure that there is no redundant DM worker node and avoid restarting or upgrading the DM cluster during the full import.
After the new task starts, it is recommended to ensure that there is no redundant DM-worker node and avoid restarting or upgrading the DM cluster during the full import.

- If the data volume is large (more than 1 TB), take these steps:

1. Clean up the imported data in the downstream database.
2. Deploy TiDB-Lightning to the DM worker nodes that process the data.
2. Deploy TiDB-Lightning to the DM-worker nodes that process the data.
3. Use the Local-backend mode of TiDB-Lightning to import data that DM dump units export.
4. After the full import completes, edit the task configuration file in the following ways and restart the task:
- Change `task-mode` to `incremental`.
Expand Down Expand Up @@ -364,7 +364,7 @@ To solve this issue, you are recommended to maintain DM clusters using TiUP. In

## Why DM-master cannot be connected when I use dmctl to execute commands?

When using dmctl execute commands, you might find the connection to DM master fails (even if you have specified the parameter value of `--master-addr` in the command), and the error message is like `RawCause: context deadline exceeded, Workaround: please check your network connection.`. But after checking the network connection using commands like `telnet <master-addr>`, no exception is found.
When using dmctl execute commands, you might find the connection to DM-master fails (even if you have specified the parameter value of `--master-addr` in the command), and the error message is like `RawCause: context deadline exceeded, Workaround: please check your network connection.`. But after checking the network connection using commands like `telnet <master-addr>`, no exception is found.

In this case, you can check the environment variable `https_proxy` (note that it is **https**). If this variable is configured, dmctl automatically connects the host and port specified by `https_proxy`. If the host does not have a corresponding `proxy` forwarding service, the connection fails.

Expand Down
2 changes: 1 addition & 1 deletion dm/dm-safe-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ You can enable safe mode either automatically or manually. This section describe

### Automatically enable

When DM resumes an incremental replication task from a checkpoint (For example, DM worker restart or network reconnection), DM automatically enables safe mode for a period (60 seconds by default).
When DM resumes an incremental replication task from a checkpoint (For example, DM-worker restart or network reconnection), DM automatically enables safe mode for a period (60 seconds by default).

Whether to enable safe mode is related to `safemode_exit_point` in the checkpoint. When an incremental replication task is paused abnormally, DM tries to replicate all DML statements in the memory to the downstream and records the latest binlog position among the DML statements as `safemode_exit_point`, which is saved to the last checkpoint.

Expand Down
2 changes: 1 addition & 1 deletion dm/dm-worker-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ summary: Learn the configuration file of DM-worker.

# DM-worker Configuration File

This document introduces the configuration of DM worker, including a configuration file template and a description of each configuration parameter in this file.
This document introduces the configuration of DM-worker, including a configuration file template and a description of each configuration parameter in this file.

## Configuration file template

Expand Down
2 changes: 1 addition & 1 deletion dm/feature-shard-merge-pessimistic.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ But when a DM-worker coordinates the migration of the sharding group within itse
- When the DM-worker receives the DDL statement of `table_1`, it cannot pause the migration and needs to continue parsing the binlog to get the subsequent DDL statements of `table_2`. This means it needs to continue parsing between `t2` and `t3`.
- During the binlog parsing process between `t2` and `t3`, the DML statements of `schema V2` from `table_1` cannot be migrated to the downstream until the sharding DDL statement is migrated and successfully executed.

In DM, the simplified migration process of sharding DDL statements within the DM worker is as follows:
In DM, the simplified migration process of sharding DDL statements within the DM-worker is as follows:

1. When receiving the DDL statement of `table_1` at `t1`, the DM-worker records the DDL information and the current position of the binlog.
2. DM-worker continues parsing the binlog between `t2` and `t3`.
Expand Down
2 changes: 1 addition & 1 deletion releases/release-5.3.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ TiDB version: 5.3.2
- Fix the issue that DM occupies more disk space after the task automatically resumes [#3734](https://github.com/pingcap/tiflow/issues/3734) [#5344](https://github.com/pingcap/tiflow/issues/5344)
- Fix an issue that the uppercase table cannot be replicated when `case-sensitive: true` is not set [#5255](https://github.com/pingcap/tiflow/issues/5255)
- Fix the issue that in some cases manually executing the filtered DDL in the downstream might cause task resumption failure [#5272](https://github.com/pingcap/tiflow/issues/5272)
- Fix the DM worker panic issue that occurs when the primary key is not first in the index returned by the `SHOW CREATE TABLE` statement [#5159](https://github.com/pingcap/tiflow/issues/5159)
- Fix the DM-worker panic issue that occurs when the primary key is not first in the index returned by the `SHOW CREATE TABLE` statement [#5159](https://github.com/pingcap/tiflow/issues/5159)
- Fix the issue that CPU usage may increase and a large amount of log is printed when GTID is enabled or when the task is automatically resumed [#5063](https://github.com/pingcap/tiflow/issues/5063)
- Fix the issue that the relay log may be disabled after the DM-master reboots [#4803](https://github.com/pingcap/tiflow/issues/4803)

Expand Down
2 changes: 1 addition & 1 deletion releases/release-5.4.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,5 +170,5 @@ TiDB v5.4.1 does not introduce any compatibility changes in product design. But
- Fix the issue that execution errors of the update statement in safemode may cause the DM-worker panic [#4317](https://github.com/pingcap/tiflow/issues/4317)
- Fix the issue that in some cases manually executing the filtered DDL in the downstream might cause task resumption failure [#5272](https://github.com/pingcap/tiflow/issues/5272)
- Fix a bug that no data is returned for the `query-status` command when binlog is not enabled in the upstream [#5121](https://github.com/pingcap/tiflow/issues/5121)
- Fix the DM worker panic issue that occurs when the primary key is not first in the index returned by the `SHOW CREATE TABLE` statement [#5159](https://github.com/pingcap/tiflow/issues/5159)
- Fix the DM-worker panic issue that occurs when the primary key is not first in the index returned by the `SHOW CREATE TABLE` statement [#5159](https://github.com/pingcap/tiflow/issues/5159)
- Fix the issue that CPU usage may increase and a large amount of log is printed when GTID is enabled or when the task is automatically resumed [#5063](https://github.com/pingcap/tiflow/issues/5063)
2 changes: 1 addition & 1 deletion releases/release-6.1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,7 @@ In 6.1.0, the key new features or improvements are as follows:
- Fix the problem that checkpoint flush may cause the data of failed rows to be skipped [#5279](https://github.com/pingcap/tiflow/issues/5279)
- Fix the issue that in some cases manually executing the filtered DDL in the downstream might cause task resumption failure [#5272](https://github.com/pingcap/tiflow/issues/5272)
- Fix an issue that the uppercase table cannot be replicated when `case-sensitive: true` is not set [#5255](https://github.com/pingcap/tiflow/issues/5255)
- Fix the DM worker panic issue that occurs when the primary key is not first in the index returned by the `SHOW CREATE TABLE` statement [#5159](https://github.com/pingcap/tiflow/issues/5159)
- Fix the DM-worker panic issue that occurs when the primary key is not first in the index returned by the `SHOW CREATE TABLE` statement [#5159](https://github.com/pingcap/tiflow/issues/5159)
- Fix the issue that CPU usage may increase and a large amount of log is printed when GTID is enabled or when the task is automatically resumed [#5063](https://github.com/pingcap/tiflow/issues/5063)
- Fix the offline option and other usage issues in DM WebUI [#4993](https://github.com/pingcap/tiflow/issues/4993)
- Fix the issue that incremental tasks fail to start when GTID is empty in the upstream [#3731](https://github.com/pingcap/tiflow/issues/3731)
Expand Down
2 changes: 1 addition & 1 deletion releases/release-6.1.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,4 +83,4 @@ Quick access: [Quick start](https://docs-archive.pingcap.com/tidb/v6.1/quick-sta
- Fix the issue that when `collation_compatible` is set to `"strict"`, DM might generate SQL with duplicated collations [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716)
- Fix the issue that DM tasks might stop with an `Unknown placement policy` error [#7493](https://github.com/pingcap/tiflow/issues/7493) @[lance6716](https://github.com/lance6716)
- Fix the issue that relay logs might be pulled from upstream again in some cases [#7525](https://github.com/pingcap/tiflow/issues/7525) @[liumengya94](https://github.com/liumengya94)
- Fix the issue that data is replicated for multiple times when a new DM worker is scheduled before the existing worker exits [#7658](https://github.com/pingcap/tiflow/issues/7658) @[GMHDBJD](https://github.com/GMHDBJD)
- Fix the issue that data is replicated for multiple times when a new DM-worker is scheduled before the existing worker exits [#7658](https://github.com/pingcap/tiflow/issues/7658) @[GMHDBJD](https://github.com/GMHDBJD)
2 changes: 1 addition & 1 deletion releases/release-6.5.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ Starting from v6.5.0, the `AMEND TRANSACTION` mechanism introduced in v4.0.7 is
+ TiDB Data Migration (DM)

- Fix the issue that a `task-mode:all` task cannot be started when the upstream database enables the GTID mode but does not have any data [#7037](https://github.com/pingcap/tiflow/issues/7037) @[liumengya94](https://github.com/liumengya94)
- Fix the issue that data is replicated for multiple times when a new DM worker is scheduled before the existing worker exits [#7658](https://github.com/pingcap/tiflow/issues/7658) @[GMHDBJD](https://github.com/GMHDBJD)
- Fix the issue that data is replicated for multiple times when a new DM-worker is scheduled before the existing worker exits [#7658](https://github.com/pingcap/tiflow/issues/7658) @[GMHDBJD](https://github.com/GMHDBJD)
- Fix the issue that DM precheck is not passed when the upstream database uses regular expressions to grant privileges [#7645](https://github.com/pingcap/tiflow/issues/7645) @[lance6716](https://github.com/lance6716)

+ TiDB Lightning
Expand Down
2 changes: 1 addition & 1 deletion releases/release-7.0.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ In v7.0.0-DMR, the key new features and improvements are as follows:

+ TiDB Data Migration (DM)

- Fix the issue that when a DM worker node uses Google Cloud Storage, due to too frequent breakpoints, the request frequency limit of Google Cloud Storage is reached and the DM worker cannot write the data into Google Cloud Storage, thus causing the full data to fail to load [#8482](https://github.com/pingcap/tiflow/issues/8482) @[maxshuang](https://github.com/maxshuang)
- Fix the issue that when a DM-worker node uses Google Cloud Storage, due to too frequent breakpoints, the request frequency limit of Google Cloud Storage is reached and the DM-worker cannot write the data into Google Cloud Storage, thus causing the full data to fail to load [#8482](https://github.com/pingcap/tiflow/issues/8482) @[maxshuang](https://github.com/maxshuang)
- Fix the issue that when multiple DM tasks replicate the same downstream data at the same time and all use the downstream metadata table to record the breakpoint information, the breakpoint information of all tasks is written to the same metadata table and uses the same task ID [#8500](https://github.com/pingcap/tiflow/issues/8500) @[maxshuang](https://github.com/maxshuang)

+ TiDB Lightning
Expand Down
Loading