Companies need data scientists more than ever but there are not enough qualified workers to fill the need.November 17, 2016
In business, "data science" is today what "big data" was just a few years ago: all the rage.
But more than just a new buzzword, the former is largely driven by the biggering of the latter. Since the amount of recorded data has skyrocketed in the smartphone era, so too has the desire to learn, gain, and benefit from said data, which when properly mined can reveal telling insights about human behavior.
Enter "data scientist." Although the term was coined in the ‘70s, Harvard Business Review recently dubbed it "The Sexiest Job of the 21st Century." Depending on who you ask, upwards of 5 million positions are needed right now. To help fill those openings, Harvard, MIT, Carnegie Mellon, Columbia and many other respected universities have already begun offering graduate programs in the field.
Unfortunately for those competing in the digital frontier, creating and capturing new data is overwhelmingly faster than developing and recruiting suitable talent. "The shortage of data scientists is becoming a serious constraint," wrote the Harvard Business Review. Earlier this year, GlassDoor named "data scientist" the most sought after job in America.
This shortage is especially difficult for companies with limited brand appeal. "We knew we were in for a challenge about five years ago as finding scientists with relevant experience became harder to identify," says Eric Haller, vice president of Experian DataLabs. "Companies such as Google, Amazon, Microsoft, Facebook, LinkedIn, and others were quickly hiring everyone who was qualified."
In response, Haller and his team worked with some of the aforementioned universities to fatten the pipeline. "But as fast as the market is working to train those with relevant skills, the imbalance between supply and demand will likely continue for some time," he says.
Which is why Haller and others are forced to train and develop their own talent in the short term. For instance, they're taking PHDs and statisticians from other fields and converting them into analytics wizards, data visualizers, data hygienists, competitive intelligence professionals, and expert SQL, Python, and Java programmers.
These highly sought after data scientists affect and touch many domains, including finance, machine learning, speech recognition, robots, search engines, economics, biotech, social sciences, and even humanities. In doing their work, they often apply calculus, regression models, and algebra to gargantuan amounts of data and are expected to produce visual answers in days rather than the weeks and months that "business intelligence" workers previously did in written form.
Although hard to describe and even harder to understand "since we're still figuring out the rules," Haller says that such work holds the key to unlocking humanity's future potential. "The good of this data is helping people, business, government, and society at large."
Where might the discipline be in five or 10 years? "In all seriousness, I never think that far ahead," he says. "There is so much taking place right now in biometrics, artificial intelligence, and computational scaling, we feel like every year we see a little bit more of a future that is super exciting."