What is Big Data?
We use smartphones, but have you ever wondered how much data it generates in the form of text, phone calls, emails, photos, videos, searches, and music. Approximately 40 exabytes of data is generated every month by a single smartphone user. Now, imagine this number multiplied by 5 billion smartphone users. That’s a lot to remind even the process, isn’t it?
This amount of data is quite a lot for a traditional computing system to handle and this massive amount of data is what we term as Big Data.
Data with a huge size is referred to as Big data. Big data is a term used to describe a collection of data which is huge in volume and yet growing exponentially with time.
Facebook uncovered some enormous details on big data to a couple of columnists at its HQ, including that its framework measures 2.5 billion bits of content and 500+ terabytes of data every day. It’s pulling in 2.7 billion Like actions and 300 million photos per day, and it scans roughly 105 terabytes of data every half-hour.
Let’s have a look at the data generated per minute on the internet.
That’s a lot!!!!!!
Do you Know?
1021 bytes equal to 1 zettabyte or one billion terabytes forms a zettabyte.
Bit –Byte –Kilobyte –Megabyte –Gigabyte –Terabyte –Petabyte –Exabyte –Zettabyte –Yottabyte
Taking a gander at these figures, one can without much of a stretch comprehend why the name Big Data is given and envision the difficulties engaged with its processing and storage.
Explore the World of Cryptocurrency
How would you characterize any data as big data?
This is possible with the concept of 5 V’s.
Let’s understand the same with the help of an example from the healthcare industry.
Hospitals & clinics around the world generate a massive volume of data that is 2134 Exabytes data is collected annually in the form of patient records and test results. All this data is generated at very high speed with a tribute to the velocity of big data.
Do you know?
Data stored in relational database management systems is one example of ‘structured’ data.
Variety refers to the various data types of structured data,semi-structured data, unstructured data and some of the examples include excel records, log files, and x-ray images.
Accuracy and Trustworthiness of generated data are termed as the veracity.
Analysing all this data will benefit the medical sector. By enabling faster disease detection, better treatment, and reduced cost. This is known as the value of big data.
How do we process and store big data?
To do this job, we have various frameworks such as Cassandra, Hadoop, and Spark.
Let’s see how Hadoop stores and processes big data.
Hadoop uses a distributed file system known as Hadoop Distributed File System (HDFS) to store big data. If you have a huge file, then your file will be broken down into smaller parts and stored in various machines. Not only that, when you break the file you also make copies of it which go to different nodes and this way you store big data. Even if one machine fails, your data is saved on another.
Mapreduce technique is used to process big data
A lengthy Task (A) is broken into smaller Task (X), Task (Y), and Task (Z). Now, instead of one machine, the machine takes up each task and compiles it in a parallel fashion and assembles the result at the end.
The processing became easy and faster. This is known as parallel processing. Now, we have stored and processed big data. We can analyse this data for numerous applications in games like HALO3 and CALL OF DUTY. Designers analyse the user data to understand that at which state a user pauses, restarts or quits. This incident can help them to rework the storyline of the game and improve user experience, which reduces customer churn rate.
Big data also helped in disaster management during the Sandy hurricane in 2012. It was used to get a better understanding of storms that affect the east coast of the US. It could predict the hurricanes landfalls five days in advance which wasn’t possible earlier.
These are clear indications how valuable big data is once it is accurately processed and analysed.
The world is moving towards a more associated future, and big data arrangements will have a major influence in improvement of AI advancements and automation. Organizations like Google are now utilizing Machine Learning measures for more noteworthy accuracy in conveying their administrations. As innovations around the world become more coordinated and interoperable, big data will turn into the center that interfaces them together.