PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

[poorbarcode]

Discussion: https://lists.apache.org/thread/tp25lom2ggztljlo76krsldo270f293j

Motivation

Just auto-create no-partitioned DLQ/Retry Topic

If enabled the config allowAutoTopicCreation, Pulsar will auto-create a topic when the client loads it; After setting config allowAutoTopicCreationType=partitioned, defaultNumPartitions=2, Pulsar will auto-create a partitioned topic(which have two partitions) when the client loads it.

After the above, if using the feature Retry Topic and DLQ enable topic auto-creation, we will get a partitioned DLQ and a partitioned Retry Topic like this:

• {primary_topic_name}-{sub_name}-DLQ
  -{primary_topic_name}-{sub_name}-DLQ-partition-0
  -{primary_topic_name}-{sub_name}-DLQ-partition-1
• {primary_topic_name}-{sub_name}-RETRY
  -{primary_topic_name}-{sub_name}-RETRY-partition-0
  -{primary_topic_name}-{sub_name}-RETRY-partition-1

---

I feel that almost all users will not use the multi-partitioned DLQ or multi-partitioned Retry topic because there is a bug that causes the above behavior to be incorrect, but we have yet to receive any issues about it. This bug causes the above behavior to look like this: When the partitioned DLQ is auto-created for the topic tp1-partition-0, Pulsar will create a partitioned topic meta which has two partitioned but only create a topic named {primary_topic_name}-{sub_name}-DLQ, there is no topic named {primary_topic_name}-{sub_name}-DLQ-partition-x. Please look at this PR for a detailed bug description.

So I want to change the behavior to Just auto-create no-partitioned DLQ/Retry Topic.

---

Prevent auto-create the DLQ for a DLQ

If we use regex-topic(not filter out the Retry topics) consumer and enable retry, and after several times restart the client. it is possible to create such a topic persistent://public/default/tp1-sub1-RETRY-sub2-RETRY-sub3-RETRY.....
Please look at this Discussion for the detail.

---

Goal

• Just auto-create no-partitioned DLQ/Retry Topic(with the other words: prevent auto-create partitioned DLQ)
• DLQ/Retry topic should not create for a DLQ/Retry Topic
  • roles:
    • DLQ will not auto-create for a DLQ
    • Retry Topic will not auto-create for a Retry Topic
    • DLQ will not auto-create for a Retry Topic
    • Retry Topic will not auto-create for a DLQ
  • client changes: Clients will not create a DLQ for a DLQ
  • broker changes: rejected the request which wants to auto-create a DLQ for a DLQ

---

API Changes

CommandSubscribe.java

/**
  * This is an enumeration value with tree options: "standard", "dead letter", "retry letter".
  */
private String topicPurpose;

Properties of Topic

"purposeOfAutoCreatedTopic": value with tree options: "standard", "dead letter", "retry letter"

Why not use two properties: isAutoCreated and topicPurpose?
Because there is a scenario like this: auto-create a topic, use it as a DLQ after a few days, and not use it as a DLQ after a few days, this Topic will be allowed to have DLQ/Retry Topic. We only mark the topics created for DLQ/Retry purposes.

[github-actions]

The issue had no activity for 30 days, mark with Stale label.

[poorbarcode]

instead of #20445