The Apache Pulsar community releases version 2.7.4! 32 contributors provided improvements and bug fixes that delivered 98 commits.
Highlights of this release are as below:
ManagedLedgercan be referenced correctly when
OpAddEntryis recycled. PR-12103
NPE does not occur on
OpAddEntrywhile ManagedLedger is closing. PR-12364
This blog walks through the most noteworthy changes grouped by the affected functionalities. For the complete list including all enhancements and bug fixes, check out the Pulsar 2.7.4 Release Notes.
Notable bug fixes and enhancements
A serious vulnerability was reported regarding Log4j that can allow remote execution for attackers. The vulnerability issue is described and tracked under CVE-2021-44228.
Pulsar 2.7.4 upgraded Log4j to 2.17.0.
ManagedLedger can be referenced correctly when
OpAddEntry is recycled. PR-12103
Previously, after a write failure, a task was scheduled in the background to force close the ledger and trigger the creation of a new ledger. If the
OpAddEntryinstance was already recycled, that could lead to either an NPE or undefined behavior.
ManagedLedgerImplobject reference is copied to a final variable so the background task will not be dependent on the lifecycle of the
No potential race condition in the
BlobStoreBackedReadHandlerentered an infinite loop when reading an offload ledger. There was a race condition between the operation of reading entries and closing BlobStoreBackedReadHandler.
Added a state check before reading entries and made the
BlobStoreBackedReadHandlerexit loop when the
entryIDis bigger than the
NPE does not occur on
OpAddEntry while ManagedLedger is closing. PR-12364
Previously, the test
ManagedLedgerBkTest#managedLedgerClosedclosed ManagedLedger object on some
asyncAddEntryoperations and failed with NPE.
OpAddEntryto fail. In this way, the
OpAddEntryobject was correctly recycled and the failed callback was correctly triggered.
Previously, the topic name of a partition could not be used to set a topic policy.
Allowed setting a topic policy through a topic name of a partition by converting the topic name of a partition in
Previously, dispatch rate limiter did not take effect in cases where all consumers started reading in the next second since
acquiredPermitswas reset to 0 every second.
Changed the behaviour of
permitsevery second instead of reset
acquiredPermitsto 0. Consumers stopped reading entries temporarily until
acquiredPermitsreturned to a value less than
When performing pressure tests on persistent partitioned topics, NPE occurred when executing unload bundles operations. Concurrently, producers did not write messages.
Added more safety checks to fix this issue.
Previously, namespace bundle cache was not invalidated after a namespace was deleted.
Invalidated namespace policy cache when bundle cache was invalidated.
Previously, the replicator and the replication client were not closed after a cluster was deleted. The producer of the replicator would then try to reconnect to the deleted cluster continuously.
Closed the relative replicator and replication client.
Previously, there were various issues if
preciseTopicPublishRateLimiterEnablewas set to
truefor rate limiting:
Updating the limits did not set a boundary when changing the limits from a bounded limit to an unbounded limit.
Each topic created a scheduler thread for each limiter instance.
Topics did not release the scheduler thread when the topic was unloaded or the operation closed.
Updating the limits did not close the scheduler thread related to the replaced limiter instance
Cleaned up the previous limiter instances before creating new limiter instances.
brokerService.pulsar().getExecutor()as the scheduler for the rate limiter instances.
Added resource cleanup hooks for topic closing (unload).
When updating a ZNode list, ZooKeeper threw an exception and did not clean up the created ledger. Newly created ledgers were not indexed to a topic
managedLedgerlist and could not be cleared up as topic retention. Also, ZNode numbers increased in ZooKeeper if the ZNode version mismatch exception was thrown out.
Deleted the created ledger from broker cache and BookKeeper regardless of exception type when the ZNode list failed to update.
If you are interested in learning more about Pulsar 2.7.4, you can download and try it out now!
Pulsar Summit Asia 2021 will take place on January 15-16, 2022. Register now and help us make it an even bigger success by spreading the word on social media!