Machine Learning, Is It All Hype?
Written by BT_Bear. This is Part 1.
This was a post written by BT_Bear. You can find them here.
Background
I want to share a bit of a perspective on the state of Data Science and ML in the broader world and hope this helps you in navigating it. Part one will give a general overview and part two will highlight a few common things to keep you on the right track.
I worked for more than a decade on the tech side of consulting and recently transitioned to a product manager role for a data and ML platform. More time for side activities. I was doing everything from technology strategy to implementation projects – always with a focus on data driven decisions. Important perspective: Large consulting firms typically work for well established enterprise clients, so I will focus on those – the world of start-ups and FAANG looks different.
Note: Raptor typically works with the startup types, or mid sized companies. I like that they leave you alone, till it’s time to hand stuff in.🦖🦖
Machine Learning Today
The world of ML
So, what does the world of ML look like? I’ll be brief: Companies have broadly adapted some ML into their work, but it is usually far less fancy than job ads would make you believe. But do not read this as a discouragement, the technologies have proven their worth. One of the cutting edges for applied AI right now is probably the work Tesla is doing in the autopilot, check out the Musk interview with Lex:
Structured vs Unstructured In The Real World
But broadly? Fairly straightforward models applied to structure data. And this is not all that surprising, structured data is what we use to manage most commercial processes for decades.
Note: Structured data is the nice and clean data that is typically stored in places like SQL servers. Click here to see some basic wrangling techniques for them.
Predictive models have also been used for 20+ years in many fields, they share similarities to what we call ML today. The technological progress is still real, especially when it comes to images and human language. Image recognition and automatic language processing only became practical in the last decade.
Actual ML Use Cases (Tasks)
To give you a bit more than my opinion, I highly recommend the study done by Algorithmia: There are many others out there, but I really appreciate the tangible information in this one. All following graphs are from the study:
Fits my experience, fraud and marketing topics still dominate. Note that process automation is big here – to my experience this is mostly Robotics Process Automation (RPA) type software that has little to do with the type of ML described in Raptor’s substack.
There is some highly interesting work going on in specific sectors, for example automatic image analysis in the medical field or process control in manufacturing. I do believe that there are seriously underdeveloped potentials here. Think for example about all the manufacturers that are in serious pain from just in time logistics falling apart. They have two options to reduce this pain: stick to the complex global supply chains but make them more shock-prove or invest in cost-efficient local manufacturing. ML could play a big role in both.
What does this mean for you? Do not take Data Science roles as fully interchangeable and take a closer look at what work exactly you would do. Let us move on with the study:
This looks pretty good until you scroll down a few more pages:
Can´t resist to highlight this sentence: “Then, once a trained model has been developed, it must still be put into scaled production. When we asked respondents about the average amount of time it takes them to do that, only 11% indicated they can deploy a trained model to production within a week. The majority of organizations (64%) are taking a month or longer.” Let´s agree that this leaves a lot of room for improvement.
To be fair, semantics become a bit unclear here. Many vendors will claim one-click deployment on their platform – this mostly means that the model will be deployed as a web service endpoint. Nice, but still far away from actual integration into a production process. Let us wrap it up for part one.
Conclusion
By my judgement ML clearly left the hype phase and is in early productive adoption with plenty of room to improve. In other words: plenty opportunity for you. In the next part I will give a few more detailed pointers for your success in Data Science & Machine Learning.
This was a post written by BT_Bear. You can find them here.
The fire keeps coming