Skip to main content

What’s New in Apache Pulsar 2.10.1

codelipenghui, momo-jun
· 4 min read

The Apache Pulsar community releases version 2.10.1! 50 contributors provided improvements and bug fixes that delivered 200+ commits. Thanks for all your contributions.

The highlight of the 2.10.1 release is introducing 30+ transaction fixes and improvements. Earlier-adoption users of Pulsar transactions have documented long-term use in their production environments and reported valuable findings in real applications. This provides the Pulsar community with the opportunity to make a difference.

This blog walks through the most noteworthy changes. For the complete list including all feature enhancements and bug fixes, check out the Pulsar 2.10.1 Release Notes.

Fixed ineffective load manager due to broker’s zero resource usage. PR-15314​

Issue​

Introduced in 2.10.0, the leader broker’s resource usage (CPU, memory, direct memory…) was always 0 when performing load balance. The root cause is that deserializing the JSON data to ResourceUsage POJO didn’t use the constructor ResourceUsage (double usage, double limit), so the percentage was always 0.

Allow users with produce/consume privileges to get topic schema. PR-15956​

Issue​

In earlier versions, only users with admin privileges were able to get topic schema, which made schema inconvenient to use.

Resolution​

Allow users who have metadata access privileges to get topic schema. Subscribers can be from different teams, and the producers and subscribers should be able to get the topic schema instead of asking tenant admin to do so before publishing and consuming messages.

Fixed the consumption performance regression. PR-15162​

Issue​

This performance regression was introduced in 2.10.0, 2.9.1, and 2.8.3. You may find a significant performance drop with message listeners while using Java Client. The root cause is each message will introduce the thread switching from the external thread pool to the internal thread poll, and then to the external thread pool.

Resolution​

2.10.1 is the first version to have this issue fixed by avoiding the thread switching for each message to improve consumption throughput.

Fixed a deadlock issue of topic creation. PR-15570​

Issue​

This deadlock issue occurred during topic creation by trying to re-acquire the same StampedLock from the same thread when removing it. This will cause the topic to stop service for a long time, and ultimately with a failure in the deduplication or geo-replication check. The workaround is restarting the broker.

Fixed key-shared delivery of messages with interleaved delays. PR-15409​

Issue​

This is a regression issue introduced in 2.10.0. When delayed messages with interleaved delays occurred on a shared/key-shared subscription, many of the messages were not delivered but stayed in the backlog. The reason was that when peeking into getMessagesToReplayNow(), we could not discard the returned set due to untracked message IDs in the delayed message controller.

Optimized the memory usage of brokers.​

Issue​

Pulsar has some internal data structures, such as ConcurrentLongLongPairHashMap, and ConcurrentLongPairHashMap, which can reduce the memory usage rather than using the Boxing type. However, in earlier versions, the data structures were not supported for shrinking even if the data was removed, which wasted a certain amount of memory in certain situations.

Pull requests

Resolution​

Support the shrinking of the internal data structures, such as ConcurrentSortedLongPairSet, ConcurrentOpenHashMap, and so on.

What’s Next?

If you are interested in learning more about Pulsar 2.10.1, you can download and try it out now!

Pulsar Summit San Francisco 2022 will take place on August 18th, 2022. Register now and help us make it an even bigger success by spreading the word on social media!

For more information about the Apache Pulsar project and current progress, visit the Pulsar website, follow the project on Twitter @apache_pulsar, and join Pulsar Slack!