Hadoop

We need Hadoop

Nowadays, the application of big data is showing more and more of its advantages, and it impacts more and more areas, E-commerce, O2O, logistics and distribution, etc. Big data is helping enterprises to continuously develop new businesses and innovate operating models. With the concept of big data, the judgment of consumer behavior, the forecast of product sales, the precise marketing scope and the replenishment of inventory have been comprehensively improved and optimized.

"Big data" in the Internet industry is considered a phenomenon: internet companies generate and accumulate user network behavior data in their daily operations. The scale of this data is so massive t that it cannot be measured in G or T.

The great value that "big data" may bring is gradually being recognized by people. It provides people with a new way of looking at the world through the innovation and development of technology, as well as the comprehensive perception, collection, analysis and sharing of data. Making decisions based more on facts and data can predictably drive dramatic changes in some decisions that implemented based on “almost.”

Real experience

Because these massive amounts of data are traces of human activities, we only need to verify that the sources of these data are reliable before they can be used by us. When we can obtain these data, we actually grasp the laws of human activities and the laws of social development, and these experiences will become our strong backing.

We can understand data

Massive data is complex and the original data will dazzle us, so it is difficult for people to understand. This requires data visualization technology, and data is artificially lined up like classification through calculation, filtering, and iteration. When the data is neat, we can understand it. Through data visualization, we can present graphs, percentages, and probabilities, so that the readability of the data becomes simple.

Get data quickly and efficiently

Data is huge. Previously, we could only rely on human resources to obtain data one by one, so the cost became uncontrollable. To achieve the acquisition of big data, robots are indispensable, because these repetitive tasks are more suitable for robots. In addition, robots can obtain millions of data in one second, which is a speed that humans cannot match.

Data is general direction, not precision guidance

There is so much research data that we are no longer obsessed with precision; there was very little data to analyze before, so we must quantify our records as precisely as possible, and the obsession with precision will diminish as we scale ;With big data, we no longer need to get to the bottom of a phenomenon, as long as we grasp the general development direction, properly ignoring the accuracy at the micro level will allow us to have better insight at the macro level.

Data is not causation, it's correlation

We are no longer interested in finding causal relationships. Finding causality is a long-standing habit of human beings. In the era of big data, we no longer need to pay close attention to the causal relationship between things, but should look for the correlation between things; correlation may not tell us exactly why something happened, but it reminds us that it is happening.

What data can do for us

Example 1: When you are still using social platforms such as Weibo as a tool for expressing emotions or discussing, Wall Street's wealth-gainers are mining the "data wealth" of these Internets, and using them to predict market trends one step ahead of others, and made decent gains.
Example 2: Banks infer employment rates based on the number of jobs on job search websites.
Example 3: US President Barack Obama's campaign team analyzes voters' preferences for presidential candidates in real time based on voters' Weibo.

The limitations of data

Without perfect tools, the defects of big data are obvious. There is no absolute truth in the data we have, because human beings always have privacy, there are dark areas in transactions, and even human nature is uncontrollable. What we see is what you can see, and what you can't see is a lot more. Therefore, we can not absolutely trust the data, can only become our information and reference, more need is our rational analysis.

This is Data：

Cost of acquiring data has fallen 85%

Crease chances success 65%

Data acquisition speed is improved 90%

Decision up 35%

Value content of decision 75%