The emerging concept of “big data” concerns the large amount of information that is being processed, analyzed, and put to extraordinary new uses. Big data is already being used by Google and New York City authorities, among others, and is likely to be used more and more in the future.
What’s Inside?
Over the last decade or so, the amount of available information has experienced a quantum jump. The authors describe this larger-than-ever volume of information as “big data.” They explain how big data is currently being used (in both the broader public sphere and the business world) and how it is likely to be used in the future. If its limitations are recognized and understood, it is likely to become an important resource and tool.
How Is This Article Useful to Practitioners?
Investment practitioners need to analyze and process economic and financial data. Because the volume of available information is growing every day, investment practitioners need to be prepared to adapt their strategies to be compatible with big data.
The amount of available information has increased and will continue to increase. Therefore, the authors argue, the way information is used has to change in three profound ways. First, rather than collecting a small sample (as statisticians have done for more than a century), analysts need to use a lot of data. The argument for using sample data is that the cost associated with collecting and analyzing the entire population would be too large. But today, information is readily available and the cost of processing has decreased; it makes more sense to use all of the available information.
Second, analysts need to shed their preference for accuracy and tolerate a bit of inaccuracy because the benefits of using a vast amount of data of variable quality outweigh the costs of using smaller but exact data. Backed up by recent examples of Google’s and New York City authorities’ use of big data, the authors argue that the sheer size of the data is more important than some inaccuracy within it. And third, the examination of correlations rather than causations should be analysts’ focus because big data is excellent for establishing correlations.
But big data does have its limitations. It should be used as a tool and resource to inform, not as a way to explain everything. It can, at times, lead to misunderstanding.
Abstractor’s Viewpoint
The concept of big data is fascinating. As more and more information is at users’ fingertips and as the cost of processing decreases, the way we use data will change along the lines the authors mention. Having said that, we also need to be mindful of the limitations around big data.