What is BIG DATA and Why all Tech Giants want to use this ?

 

BIG DATA = BIG + DATA ?

Well let's understand by simple definition.

Get up in morning to go to bed in night in between that durations we use a lot of electronic gadgets in that we use whatsapp, snapchat, facebook, instagram, banking transaction, watching netflix, shopping on Amazon or Flipkart, do you guys know how much data we generate in entire day?

the answer is a million of millions data. Remember its for only one person and one day if you see india's population 1.4 billion and we have 194 country on glob so multiply by that data its lots of data right? lets see how we can use that data and utilize that data.

Big data is a combination of structured, semistructured and unstructured data collected by organizations that can be mined for information and used in machine learning projects, predictive modeling and other applications.

Systems that process and store big data have become a common component of data management an architectures in organisations. Big data is often characterised by the 3Vs: the large volume of data in many environments, the wide variety of data types stored in big data systems and the velocity at which the data is generated, collected and processed. These characteristics were first identified by Doug Laney, then an analyst at Meta Group Inc. in 2001; Gartner further popularised them after it acquired Meta Group in 2005. More recently, several other Vs have been added to different descriptions of big data, including veracity, value and variability.

Although big data doesn’t equate to any specific volume of data, big data deployments often involve terabytes (TB), petabytes (PB) and even exabytes (EB) of data captured over time

Importance of big data

Companies use the big data accumulated in their systems to improve operations, provide better customer service, create personalized marketing campaigns based on specific customer preferences and, ultimately, increase profitability. Businesses that utilize big data hold a potential compective advantage over those that don’t since they’re able to make faster and more informed business decisions, provided they use the data effectively.

For example, big data can provide companies with valuable insights into their customers that can be used to refine marketing campaigns and techniques in order to increase customer engagement and conversion rates.

Examples of big data

Big data comes from myriad different sources, such as business transaction systems, customer databases, medical records, internet clickstream logs, mobile applications, social networks, scientific research repositories, machine-generated data and real-time data sensors used in internet of things (lot) environments. The data may be left in its raw form in big data systems or preprocessed using data mining tools or data preparation software so it’s ready for particular analytics uses.

Using customer data as an example, the different branches of analytics that can be done with the information found in sets of big data include the following:

• Comparative analysis. This includes the examination of user behavior metrics and the observation of real-time customer engagment in order to compare one company’s products, services and brand authority with those of its competition.

   . Social media listening. This is information about what people are saying on social media about a specific business or product that goes beyond what can be delivered in a poll or survey. This data can be used to help identify target audiences for marketing campaigns by observing the activity surrounding specific topics across various sources.

• Marketing analysis. This includes information that can be used to make the promotion of new products, services and initiatives more informed and innovative.
• Customer satisfaction and sentiment analysis. All of the information gathered can reveal how customers are feeling about a company or brand, if any potential issues may arise, how brand loyalty might be preserved and how customer service efforts might be improved.

Volume
It is not surprising that Big Data is large in volume. It is estimated that we create 2.3 trillion gigabytes of data every day. And that will only increase. This increase is of course partly caused by the gigantic mobile telephone network. To give you an idea: six of the seven billion people in the world now have a mobile phone. Text and WhatsApp messages, photos, videos and many apps ensure that the amount of data increases significantly.


Velocity
Velocity, or speed, refers to the enormous speed with which data is generated and processed. Until a few years ago, it took a while to process the right data and to surface the right information. Today, data is available in real time. This is not only a consequence of the speed of the internet, but also of the presence of Big Data itself. Because the more data we create, the more methods are needed to monitor all this data, and the more data is monitored. This creates a vicious circle.


Variety
The high speed and considerable volume are related to the variety of forms of data. After all, smart IT solutions are available today for all sectors, from the medical world to construction and business. Consider, for example, the electronic patient records in healthcare, which contribute to many trillions of gigabytes of data. And that’s not even talking about the videos we watch on Youtube, the posts we share on Facebook and the blog articles we write. When all parts of the world have the internet in the future, the volume and variety will only increase.


Veracity
How truthful Big Data is remains a difficult point. Data quickly becomes outdated and the information shared via the internet and social media does not necessarily have to be correct. Many managers and directors in the business community do not dare to make decisions based on Big Data.

How big data is stored and processed

The need to handle big data velocity imposes unique demands on the underlying compute infrastructure. The computing power required to quickly process huge volumes and varieties of data can overwhelm a single server or server cluster. Organizations must apply adequate processing capacity to big data tasks in order to achieve the required velocity. This can potentially demand hundreds or thousands of servers that can distribute the processing work and operate collaboratively in a clustered architecture, often based on technologies like Hadoop and Apache Spark.

Achieving such velocity in a cost-effective manner is also a challenge. Many enterprise leaders are reticent to invest in an extensive server and storage infrastructure to support big data workloads, particularly ones that don’t run 24/7. As a result, public cloud computing is now a primary vehicle for hosting big data systems. A public cloud provider can store petabytes of data and scale up the required number of servers just long enough to complete a big data analytics project. The business only pays for the storage and compute time actually used, and the cloud instances can be turned off until they’re needed again.

To improve service levels even further, public cloud providers offer big data capabilities through managed services that include the following:

• Amazon EMR(formerly Elastic MapReduce)
• Microsoft Azure HDInsight
• Google Cloud Dataproc

In cloud environments, big data can be stored in the following:

• Hadoop Distributed File System (HDFS);
• lower-cost cloud object storage, such as Amazon Simple Storage Service (S3);
• NoSQL databases; and
• relational database.

Some challanges:

Here I attached PPT which created by me you can download that.

CLICK HERE

Also, I attached drive link of Book about BIG DATA.

DOWNLOAD