Facing the Big Data Tsunami 

An estimated 40 zettabytes (43 trillion gigabytes) of data will be created by 2020 (an increase of 300 times the amount of data in circulation in 2005), at a rate of 2.5 quintillion bytes per day. “The explosion in big data is both good news and bad news,” says James D’Arezzo, CEO of Condusiv Technologies. “It will make possible advances in dozens of fields, but it will also present serious challenges to the economy’s already overburdened IT sector.” D’Arezzo, whose company is a world leader in software-only storage performance solutions for virtual and physical server environments, adds, “Collecting data won’t be much of a problem. The real issue will be storage, accessibility and performance.”

The Internet of Things is, of course, a major contributor to the boom in data collection and analysis; healthcare is one of many fields that is being greatly impacted by IoT advances. But D’Arezzo notes that in the business sector, a major across-the-board function—marketing—has gone from being a largely analog and promotional activity to a heavily digitized means of delivering business growth that has come to rival may soon surpass traditional IT as a center for technology spend.

All told, it is estimated that 86 million U.S. workers now perform jobs that require the regular use of a computer. A recent survey of global IT managers suggests that degraded system performance due to slow processing of database applications costs each of these workers an average of fifteen minutes per day; at the current median salary of $58,000, that represents an annual time loss equivalent to $1,812 per worker, or almost $156 billion across the economy. Given the pressures of the big data boom, that productivity loss may soon increase dramatically. But it will not increase evenly, and those who fail to recoup endangered productivity will soon face a steepening competitive disadvantage.

“To keep up, not just with the overall demands of big data but with their nimblest competitors, organizations will have to get the most out of their IT technology,” says D’Arezzo. “Part of the solution may be buying new hardware, but there are quicker and more cost-effective things IT system managers can do. One is to implement software that reduces input and output, which can improve [application] performance dramatically.”