Apache Pulsar 2.11.1
2023-04-19​
Broker​
- AbstractBatchedMetadataStore - use AlreadyClosedException instead of IllegalStateException #19284
- Add ref count for sticky hash to optimize the performance of Key_Shared subscription #19167
- Pass subName for subscription operations in ServerCnx #19184
- Enable custom metadata stores #19208
- Topics stats shows msgBacklog but there reality no backlog #19275
- Fix open cursor with null-initialPosition result with earliest position #18416
- Added isActive in ManagedCursorImpl #19341
- Replaced checkBackloggedCursors with checkBackloggedCursor(single subscription check) upon subscription #19343
- Close TransactionBuffer when MessageDeduplication#checkStatus failed #19289
- Fix the incorrect total size if use ML interceptor #19404
- Support deleting partitioned topics with the keyword -partition- #19230
- Correct the prompt message #17009
- Expect msgs after server initiated CloseProducer #19446
- Fix Npe thrown by splitBundle #17370
- Fix executeWithRetry result is null #17694
- ManagedLedger: move to FENCED state in case of BadVersionException #17736
- Fix can not revoke permission after update topic partition #17393
- Allow deletion of a namespace that was left in deleted status #17592
- Add UncaughtExceptionHandler for every thread pool #18211
- Support setting ForceDeleteNamespaceAllowed dynamically #18181
- Support setting forceDeleteTenantAllowed dynamically #18192
- Remove locallyAcquiredLock when removeOwnership #18197
- Update the log print content of createSubscriptions #18024
- Unify time unit at dropping the backlog on a topic #17957
- Fix mutex never released when trimming #17911
- Fix getChildren in MemoryMetadataStore and EtcdMetadataStore #18172
- Fix the order of resource close in the InMemoryDelayedDeliveryTracker #18000
- Allow to configure and disable the size of lookahead for detecting fixed delays in messages #17907
- Make BookieId work with PulsarRegistrationDriver #17922
- In the trimDeletedEntries method, release the removed entry #18305
- Make Consumer#equals more effective #18662
- PulsarRegistrationClient - implement getAllBookies and follow BookieServiceInfo updates #18133
- Fix PulsarRegistrationClient and ZkRegistrationClient not aware rack info problem #18672
- Simplify extract entryMetadata code in filterEntriesForConsumer #18729
- Fix uncompleted future when getting the topic policies of a deleted topic #18824
- Close transactionBuffer after MessageDeduplication#checkStatus failed #19157
- GetLastMessageId returns a wrong batch index of last message if enabled read compacted #18877
- Fix multi invocation for ledger createComplete #18975
- Fix txn coordinator recover handle committing and aborting txn race condition #19201
- Reset individualDeletedMessagesSerializedSize after acked all messages #19428
- Fix potential NPE cause future never complete #19415
- Fix the return value of canConsumeAsync #19412
- Close TransactionBuffer when creating persistent topic timeout #19384
- Handle changeToReadyState failure correctly in TC client #19308
- The atomicity of multiple fields of ml is broken #19346
- MessagesConsumedCounter of NonDurableCursor was initialized incorrectly #19355
- Improve PersistentMessageExpiryMonitor expire speed when ledger not existed #17842
- Always send correct transaction id in end txn response #19137
- Allow superusers to abort transactions #19467
- Make ServerCnx#originalAuthData volatile #19507
- Validate originalPrincipal earlier in ServerCnx #19270
- Require authRole is proxyRole to set originalPrincipal #19455
- ServerCnx broken after recent cherry-picks #19521
- ServerCnx: go to Failed state when auth fails #19312
- Add test to verify authRole cannot change #19430
- Make authentication refresh threadsafe #19506
- OneStageAuth State: move authn out of constructor #19295
- Allow proxy to pass same role for authRole and originalRole #19557
- Copy command fields and fix potential thread-safety in ServerCnx #19517
- Topic load fail by ledger lost #19444
- Incorrect service name selection logic #19505
- Make the service name resolver cache of PulsarWebResource expire after access #19532
- Simplify enabling Broker, WS Proxy hostname verification #19674
- Improve tls config on replication client and cluster cli #19588
- Fix geo-replication admin #19686
- Filter system topic when getting topic list by binary proto #19667
- Fixed history load not releasing #19560
- Fix loadbalance score calculation problem #19420
- Use shrink map for trackerCache #19534
- Fix individual ack batch message with transaction abort redevlier duplicate messages #14327
- Fix receive duplicated messages due to pendingAcks in PendingAckHandle #19581
- Fail to update partition meta of topic due to ConflictException: subscription already exists for topic #17251
- Fix catching ConflictException when update topic partition #17374
- Only create extended partitions when updating partition number #17349
- Create replicated subscriptions for new partitions when needed #18659
- Fix race condition while updating partition number #19199
- Topic close failure leaves subscription in a permanent fence state #19692
- Fix potential exception cause the policy service init fail #19746
- Allow version to start positive and grow by more than one #19503
- Fix deadlock causes session notification not to work #19754
- Fix issue where msgRateExpired may not refresh forever #19759
- Delete topic failed if disabled system topic #19735
- Fix index generator is not rollback after entries are failed added #19727
- Store the original authentication data #19519
- Copy subscription properties during updating the topic partition number #19800
- Fix close borrowed executor #19761
- Fixed race condition between ResourceLock update and invalidation #19817
- Counter of pending send messages in Replicator incorrect if schema future not complete #19242
- Add getState in transaction for client API #19834
- Follow up #19230 to tighten the validation scope #19234
- Authorize originalPrincipal when provided #19830
- Follow up #19817, Fix race condition between ResourceLock update and invalidation #19859
- Fix delete system topic clean topic policy #19835
- Fix NPE when update topic policy #19875
- Make the result of delete cursor is success if cursor is deleted #19825
- Only validate superuser access if authz enabled #19989
- Cleanup how superusers abort txns #19976
- Return if AbstractDispatcherSingleActiveConsumer closed #19934
- Use AuthenticationParameters for rest producer #20046
- Fix uncompleted future when removing cursor #20050
- Fix estimateBacklogFromPosition if position is greater than the greatest ledgerId #20069
Clients​
- [Java] Fix reader listener can't auto ack with pooled message #19354
- [Java] Prevent DNS reverse lookup when physical address is an IP address #19028
- [Java] Set fields earlier for correct ClientCnx initialization #19327
- [Java] Improve docs and code quality about KeyValueSchema usages #17256
- [Java] Add init capacity for messages in BatchMessageContainerImpl #17822
- [Java] Support MAX_ACK_GROUP_SIZE configurable #18107
- [Java] Fix exception when calling loadConf on a ConsumerBuilder that has a KeySharedPolicy #18345
- [Java] Support LocalDateTime Conversion #18334
- [Java] Fix pendingLookupRequestSemaphore leak #18219
- [Java] Unify the acknowledge process for batch and non-batch message IDs #17833
- [Java] Set authentication when using loadConf in client and admin client #18358
- [Java] Fixes batch_size not checked in MessageId#fromByteArrayWithTopic #18405
- [Java] Fix possible npe #18406
- [Java] Change the get lastMessageId to debug level #18421
- [Java] Fix failover/exclusive consumer with batch cumulate ack issue #18454
- [Java] Avoid redelivering duplicated messages when batching is enabled #18486
- [Java] For exclusive subscriptions, if two consumers are created repeatedly, the second consumer will block #18633
- [Java] Fix the Windows absolute path not recognized in auth param string #18403
- [Java] Broker address resolution wrong if connect through a multi-dns names proxy #19597
- [Java] Fix memory leak if enabled pooled messages #19585
- [Java] Fix authentication not update after changing the serviceUrl #19510
- [Java] Fix async completion in ConsumerImpl#processPossibleToDLQ #19392
- [Java] Moving get sequenceId into the sync code segment #19837
- [Java] Fix DeadLetterProducer creation callback blocking client io thread #19930
Pulsar IO and Pulsar Functions​
- [Functions] Ensure prometheus metrics are grouped by type #15558
- [Functions] Fix k8s merge runtime opts bug #19481
- [Functions] Use AuthorizationServer more in Function Worker API #19975
- [IO Connector] Debezium sources: Support loading config from secrets #19205
- [IO Connector] Elasticsearch sink: Support loading config from secrets #19206
- [IO Connector] Update Elasticsearch sink idle cnx timeout to 30s #19377
- [IO Connector] DBC sink: allow any jdbc driver #17951
- [IO Connector] ElasticSearch sink: align null fields behaviour #18577
- [IO Connector] KCA sink: handle null values with KeyValue<Avro,Avro> schema #19861
- [IO Connector] KCA: option to collapse partitioned topics #19923
- [IO Connector] KCA: flag to force optional primitive schemas #19951
Observability​
- [Broker] Fix wrong metrics text generated when label_cluster specified #17704
- [Broker] Add metrics for transaction #15140
- [Broker] Remove timestamp from Prometheus metrics #17419
- [Broker] Fix incorrect bundle split count metric #17970
- [Broker] Update authentication failed metrics report #17787
- [Function] Fix invalid metric type "gauge " #18129
CLI​
- [Pulsar Admin] Returns 4xx error when pulsar-worker-service is disabled and trying to access it #17901
- [Pulsar Admin] PulsarAdminBuilderImpl overrides timeout properties passed through config map #17375
- [Pulsar Admin] Improve partitioned-topic condition evaluation #19015
- [Pulsar Admin] Make the default value of param --get-subscription-backlog-size of admin API topics stats true #19302
- [Pulsar Admin] Make response code to 400 instead of 500 when deleting topic fails due to enabled geo-replication #19879
- [Pulsar Admin] Fix delete tenant #19925
- [Pulsar Admin] Fix incorrect state replication.connected on API partitioned-topic stat #19942
- [Pulsar CLI] Quit PerformanceConsumer after receiving numMessages messages #17750
- [Pulsar CLI] Check numMessages after incrementing counter #17826
- [Pulsar CLI] Fix Pulsar admin tool is ignoring tls-trust-cert path arg #19696
- [Pulsar CLI] Fix Broker crashed by too much memory usage of pulsar tools #20031
Others​
- [Proxy] Only go to connecting state once #19331
- [Proxy] Fix using wrong client version in pulsar proxy #19569
- [Proxy] Implement graceful shutdown for Pulsar Proxy #20011
- [Proxy] Fix connection read timeout handling in Pulsar Proxy #20014
- [WebSocket] Add ping support #19255
- [Schema] Do not print error log with stacktrace for 404 #19130
- [Tiered Storage] Don't cleanup data when offload met Metastore exception #17512
- Fix transitive critical CVEs in file-system tiered storage #19957