ScholarNest Blogs
Unlock Apache Spark Memory Allocation – All you want to know
In this blog post, I will explain the memory allocation for the Spark driver and executor. If you are here, I assume you are already familiar with Apache Spark, its architecture, and why Spark needs memory. However, I will quickly revise a few concepts to bring all readers to the same page. Let’s start with the following question. What is &he...
Continue Reading
Unlock Apache Kafka – All you want to know and focus
What is Apache Kafka? Apache Kafka is a distributed streaming platform that is built on the principles of a messaging system. Apache Kafka’s implementation started as a messaging system to create a robust data pipeline. However, over time, Kafka has evolved into a full-fledged streaming platform that offers all the core capabilities to implem...
Continue Reading