In the digital age, every action we take leaves a digital footprint. From our travel habits to our exercise routines and entertainment choices, many internet-connected devices continuously gather vast amounts of data about us. This phenomenon is known as Big Data.
According to Ernst and Young, “Big Data refers to the dynamic, large, and disparate volumes of data generated by people, tools, and machines. It requires innovative and scalable technology to collect, host, and analyze these vast datasets to provide real-time business insights related to consumers, risk, profit, performance, productivity management, and enhanced shareholder value.”
Although definitions of Big Data vary, they commonly include five key characteristics known as the V’s: velocity, volume, variety, veracity, and value.
- 1. Velocity: This refers to the speed at which data is generated and processed. Modern technology enables near-real-time data processing through local and cloud-based systems.
- Volume: This indicates the sheer amount of data being stored. The volume of data is increasing due to more data sources, higher-resolution sensors, and scalable infrastructure.
- Variety: This encompasses the different types of data, both structured (organized in rows and columns) and unstructured (such as tweets, blog posts, images, and videos). Data comes from various sources, including mobile technologies, social media, wearable devices, and more.
- Veracity: This pertains to the accuracy and reliability of data. It includes attributes like consistency, completeness, integrity, and ambiguity. The cost and need for traceability are critical drivers of data veracity.
- Value: This is about deriving meaningful insights from data. Big Data is not just about profit; it can also provide medical, social, customer, employee, and personal benefits.
Let’s explore some practical examples of these V’s in action:
- Velocity: Every minute, hours of video footage are uploaded to YouTube, generating vast amounts of data.
- Volume: With a global population of around seven billion, most people use digital devices like mobile phones, computers, and wearables, generating approximately 2.5 quintillion bytes of data daily. This volume is equivalent to 10 million Blu-ray DVDs.
- Variety: Data comes in many forms, such as text, images, videos, sound, and health data from wearables. It also includes data from devices connected to the Internet of Things.
- Veracity: Since 80% of data is unstructured, developing methods to produce accurate and reliable insights is essential. This data needs to be categorized, analyzed, and visualized effectively.
- Value: Data scientists extract insights from Big Data to tackle challenges presented by these extensive datasets. Traditional data analysis tools are often inadequate due to the scale of data, necessitating alternative tools that leverage distributed computing power, such as Apache Spark and Hadoop. These tools enable organizations to gain new insights and connect with customers more effectively.
Next time you use a smartwatch, smartphone, or fitness tracker, remember that your data begins a journey that may travel worldwide, through Big Data analysis, ultimately delivering valuable insights back to you.
Disclaimer: This article is referenced from the IBM Data Engineering course on Coursera.
