Human generated content is huge, and its metadata is even bigger. Metadata is the information about a file: who might have created it, what type of file it is, what folder it is stored in, who has been reading it and who has access to it. The content and metadata together make up human generated big data.
The problem is that most of us, meaning organisations and governments, are not yet equipped with the tools to exploit human generated big data. The conclusion of a recent survey of over 1000 Internet experts and other Internet users, published by the Pew Research Centre and the Imagining the Internet Center at Elon University, is that the world may not be ready to properly handle and understand Big Data.
These experts have come to the conclusion that the huge quantities of data, which they term "digital exhaust," which will be created by the year 2020 could very well enhance productivity, improve organisational transparency and expand the frontier of the "knowable future." However they are concerned about whose hands this information is in and whether government or corporates will use this information wisely.
The survey found that "…human and machine analysis of big data could improve social, political and economic intelligence by 2020. The rise of what is known as Big Data will facilitate things like real-time forecasting of events; the development of "inferential software" that assesses data patterns to project outcomes; and the creation of algorithms for advanced correlations that enable new understanding of the world."
Of those surveyed, 39% of the Internet experts asked agreed with the counter-argument to Big Data’s benefits, which posited that "Human and machine analysis of Big Data will cause more problems than it solves by 2020. The existence of huge data sets for analysis will engender false confidence in our predictive powers and will lead many to make significant and hurtful mistakes. Moreover, analysis of Big Data will be misused by powerful people and institutions with selfish agendas who manipulate findings to make the case for what they want."
As one of the study’s participants, entrepreneur Bryan Trogdon put it: "Big Data is the new oil," observing that, "…the companies, governments, and organisations that are able to mine this resource will have an enormous advantage over those that don’t. With speed, agility, and innovation determining the winners and losers, Big Data allows us to move from a mindset of ‘measure twice, cut once’ to one of ‘place small bets fast.’"
Jeff Jarvis, professor, and blogger said: "Media and regulators are demonizing Big Data and its supposed threat to privacy. Such moral panics have occurred often thanks to changes in technology. But the moral of the story remains: there is value to be found in this data, value in our newfound ability to share. Google’s founders have urged government regulators not to require them to quickly delete searches because, in their patterns and anomalies, they have found the ability to track the outbreak of the flu before health officials could and they believe that by similarly tracking a pandemic, millions of lives could be saved. Demonizing data, big or small, is demonising knowledge, and that is never wise."
Reading our newsletter every Monday will keep you up-to-date with security news.
Receive a daily digest of the latest security news.