Apache Kafka vs Apache Pulsar

What is Apache Pulsar?

Apache Pulsar, created by Yahoo in 2013, is an open-sourced distributed messaging system. Initially, it was created as a queuing system; however, later, its features were widened based on the needs of the customers. Now it works as an event streaming platform also! For its storage, Pulsar makes use of Apache BookKeeper. All those events that can be done on Apache Kafka can be done efficiently and with high quality on Pulsar. 

Comparison

We will compare different features and structures of both of the open-sourced platforms to discuss their functions.

Throughput

When it comes to throughput, Kafka is the best in the game. It provides writing two times faster than Pulsar, which is based on the popular benchmark—open messaging.

Latency

Kafka provides the lowest latency at high throughput, but it also provides durability and high availability. 

However, it is even faster than Pulsar in all benchmarks in its default structure.

General

We will discuss and compare some other general information of both Kafka and Pulsar in this one.

License

Both of them are open-sourced licensed. However, Kafka has some added security, storage, and more.

Components

Some functions can be found in both the platforms, such as Broker, zookeeper, Multi Data Replication, and service discovery. Although the level of quality varies, in some features, the Pulsar is Better than the Kafka and vice versa. 

Message Consumption Model

Kafka uses a pull-based approach, and Pulsar uses push-based architecture. Kafka provides messages where consumers pull messages from the server with the use of long-polling; it is made sure that new messages are available forthwith. 

Whereas Apache Pulsar uses a push-based approach and an API that facilitates consumer pulls. A pull-based approach is preferable for high throughput because they allow their users to manage the flow themselves. It helps them to fetch what they need! Moreover, Push-based architecture doesn't have such functions; it requires flow control and backpressure so that it can be integrated with the broker.

Architectural comparison

There is not much difference between Kafka and Apache; both follow the same concept for the messaging system. However, Pulsar is somewhat better and more efficient than Kafka. And this basic difference lies in the agricultural approach followed by them. Kafka follows a partition-based monolithic approach, whereas Pulsar has a multi-layered design with a segment-centric approach.

Kafka's design has a drawback since Kafka has a partition-based architectural design. Due to that, the partition has to be stored on a disk, and this leads to a problem; the max size of the partition will be that of the disc. And once the space in the disc is maxed out, you either have to delete past messages or the recent ones to create space for incoming messages, as it will stop incoming messages. In such cases, you are not left with many options. You can recopy the messages, but it is not a great solution as the entire partition is offline; also, recopying is not a fault-tolerant solution.

However, with a segmented approach, the situation is not as difficult as it was with partition-based architecture. Pulsar divides partition into segments which are rolled over onset time or size and distributed evenly to the bookie. So, even when the space is maxed out, you don't have to clear spaces or replicate content. Just add a new bookie, and your work is done! Till then, Pulsar will keep writing messages to the bookie, and once the new bookie is added, the required workload is shifted to the bookie.

[ Related Article: Kafka vs RabbitMQ ]

Pros and cons of using Kafka and Pulsar

• It is very well-documented
• It has been existing for over a decade now, which is why it has a larger community.
• Not very technical because it has fewer components and means fewer complications.
• Offsets from continuous sequence— the user can reach the last messages instantly.

• Do not provide multi-tenancy models
• It is not structured to scale elastically.
• The consumer can not acknowledge messages from different threads.

• It has plenty of decent features such as multi-tenancy, Multi-DC replication, etc.
• It is structured to scale elastically.
• The consumer can acknowledge messages from different threads.
• It consists of flexible clients like API

Cons of using Pulsar

• Due to the heavy tech stack, it is complex to use.
• No transactions
• Readers need to browse to the last message to read the last message.
• It is a new, so lesser community.

Conclusion

As you have seen the detailed comparison of Kafka and Pulsar, now the choice is up to you. Although both of them are quite robust streaming platforms, there are slight differences. The choice should be dependent upon your requirements and expectations from a streaming platform. So choose the best configuration.