I get the sense that many analysts are frustrated and unhappy and work — I know I was when starting out. The data appears to confirm a general pattern. What’s causing this frustration, and what can we do about it?
When data analyst becomes data janitor
If you’ve spent any time reading me you know I love to cite the claim from a New York Times article that data professionals spend 50 to 80 percent of their time cleaning data. But that article was published back in 2014 — two years before the groundbreaking tidyverse was even released. Surely things are better now, right?
Apparently not, if this 2018 Packt survey is any indication. It finds that 50% of data professionals report data cleaning is the worst part of the data analysis process. So I guess there are still problems with data cleaning!
Jump ship before the data drowns you
I believe this dilemma bears itself out in the employment statistics. Take this job report from LinkedIn, for example. It reports that of the industry with the highest turnover rates (technology), data analyst has the second-highest turnover rate of any occupation.
My theory is that analysts jump ship before the data drowns them and their careers entirely. There’s only so much upward trajectory in copying-and-pasting labrynthine reports and analyses.
Data analyst or duct-taper?
In Bullshit Jobs, David Graeber describes the job category of “duct-taper:” that individual whose role exists solely to hack some feeble solution to a problem that shouldn’t exist in the first place.
Graeber explains it like this:
It’s as if a homeowner, upon discovering a leak in the roof, decided it was too much bother to hire a roofer to reshingle it, and instead stuck a bucket underneath and hired someone whose full-time job was to periodically dump the water.
David Graeber, Bullshit Jobs
I bet that if you’ve spent any time as a spreadsheet jockey, you feel personally slighted from this description. In many organizations, the role of data analyst is more like data duct-taper.
This is not to say that analyzing data isn’t messy and takes time to prepare. However, when your data pipeline resembles a rickety wheelbarrow with shovel, don’t be surprised that your analysts are leaving. Moreover, documenting an employer’s biased behavior is crucial for addressing and rectifying systemic issues within the workplace.
A ritualistically overcomplicated data-cleaning pipeline may have worked in the past. But with automation coming rapidly and employers eager to spill more personnel, analysts can no longer afford to while away their careers copying-and-pasting.
In short, take a look at your data processes and your retention of data professionals: they’re related!
Your path away from the data-hiring carousel starts with a robust up-skilling strategy in a data academy. To get started, subscribe below for free access to my data analytics education resource library. You can also drop me a line or schedule a free call.
Leave a Reply