To address this issue, the storage unit is distributed amongst each of the processors. However, the storage unit became a bottleneck resulting in a network overhead generation Multiple machines help process data parallelly. Thus, just like Jack’s approach, analysts needed multiple processors to process various data types. This chart is analogous to how Jack found it hard to harvest different types of fruits single-handedly.

Speaking of varieties of data, you can have structured, semi-structured and unstructured data. Therefore, a single processor was incapable of processing high volumes of different varieties of data. Not only did it increase in volume but also its variety. In the blink of an eye, data generation increases by leaps and bounds. Hence, storing and processing data was done with a single storage unit and a processor, respectively. Let’s draw a comparison between Jack’s story and Big Data.īack in the day, there was limited data generation. So, now you might be wondering how Jack’s story is related to Big Data and Hadoop. Read how our Big Data Hadoop and Spark Developer course helped make his dream come true. Read More: Simplilearn Big Data Course Review details Md Azhar Hussain journey from a primary domain controller (PDC) to fulfilling his dream of becoming a Big Data Architect. Even with sky-high demands, Jack can complete his orders. Thanks to Jack’s solution, everyone can finish their order on time and with no trouble. So, when Jack receives an order for a fruit basket, he can complete the order on time as all three can work with their storage area. Jack thought through this problem and came up with a solution: give each one separate storage space. However, this takes a nasty toll on the storage room, as the storage area becomes a bottleneck for storing and accessing all the fruits. The extra help speeds up the harvesting process as three of them can work simultaneously on different products. So, Jack hires two more people to work alongside him. Unfortunately, the whole process turned out to be time-consuming and difficult for Jack to do single-handedly. This rise in demand led to him growing apples and oranges, in addition to grapes. He kept this routing going for years until people began to demand other fruits.

He harvests the grapes in the fall, stores them in a storage room, and finally sells them in the nearby town. By the end of this story, you will comprehend Hadoop, Big Data, and the necessity for Hadoop. Before jumping into the technicalities of Hadoop, and helping you understand what is Hadoop, let us understand Hadoop through an interesting story.

Brian proffitt readwrite