Building your ML SaaS
Helping you building your first ML SaaS so your business doesn't fall behind. Written by BT_Hal.
Note: BT_Hal is the one who should get full credit for this one. Incase you missed it, this is who he is. He works on deep learning, and neural nets on a constant basis, and is super heavily involved in the machine learning space. Honestly, I’d argue he understands the world of Machine Learning Engineers (MLE) better than I do, so I’ll probably get him to help me out quite a bit, when I start the MLE series.
You can also reach him here, and slide in his DMs.
Why Build a SaaS?
Fellow autists that follow BowTiedBull (BTB) know why it is important to build a WiFi business. Aside from obvious benefits like side income or a startup that you can spin out later, building a Software as a Service (SaaS) is like going to a gym. As you build your SaaS in addition to the tech skills you are picking up, you are learning to sell, to market, to build an audience, iterating on the product, and just putting yourself out there.
Why ML in SaaS
In the big picture, everything is getting digitized. Writing software becomes necessary to solve the problems and create pleasant experiences arising with digitization. Here is a quote from the Senior Director of AI in Tesla Andrej Karpathy that shows you the evolution of software
The “classical stack” of Software 1.0 is what we’re all familiar with — it is written in languages such as Python, C++, etc. It consists of explicit instructions to the computer written by a programmer.
In contrast, Software 2.0 is written in much more abstract, human unfriendly language, such as the weights of a neural network. The source code comprises 1) the dataset that defines the desirable behavior and 2) the neural net architecture that gives the rough skeleton of the code, but with many details (the weights) to be filled in… most of the active “software development” takes the form of curating, growing, massaging and cleaning labeled datasets.
As you can see from the above, software creation is evolving from writing code that maps input X to output Y using the rules written by a programmer. This is a natural evolution as many of the problems in the real world are impossible to capture through rules. Think of an infinite number of if-else statements you would need to write to build a classifier that recognizes breed of dogs. Much simpler to collect a dataset of pictures of dogs with their corresponding breed, train a neural network to do recognition, and get such software.
Now I am going to prepare you for building ML driven SaaS based on the lessons and experience that I have had, so that you do not fall behind.
The Project
You can see the project live here.
I am building a Google Search for NFTs. The goal is to make anything searchable in NFT space with text: market data, get jpegs with certain attributes, learn the latest news happening with the project.
Just like how you use Google Search. This is a bet on natural language interfaces becoming more prevalent in the future. Currently, I am supporting the attribute search with two collections, and about to expand search capabilities very soon.
Lessons from building this project
Data, data, data
No fancy trendiest neural network gonna beat annotation errors and noise your data
Your model at the end of the day is as good as your data.
Revisit your data several times.
It is not uncommon for me to spend hours going over the data to make sure that it is correctly annotated and it does not miss the potential inputs used by the customer
Quickly bootstrap the data. If possible use synthetic data generation.
Data annotation is expensive and time consuming. The deeper you go into a technical domain (for example medicine) the more expensive your annotation process would become. It is worth thinking about a way to quickly bootstrap the synthetically generated data and use a real dataset (with a fraction of synthetic dataset) in future.
Modeling
Don’t reinvent the wheel.
If you are building for tabular data use XgBoost
If you are building for structured data like images, text, audio use neural networks.
Use an off-the-shelf code framework.
Your goal is to get the best model working as soon as possible. Don’t spend time re-implementing popular models from scratch. If you really want to reimplement these models, do it separately on a toy dataset
Use pre-trained models.
Especially if you are building for audio, text, and images. Huggingface hosts tons of high-quality pre-trained models
Training
GPU training on a cloud is expensive. Costs $3 USD/hour to book P3.2xlarge instance on AWS that will give you one V100 GPU.
Use Google Colab instead. They provide free GPUs and give you an option for 50$ to update to premium GPUs. It is worth it!
Do minimal hyperparameter search. Extra 1-2% with better hyperparameters are not noticeable by the customers. Have a POC (proof-of-concept as soon as possible and do not overengineer any part of the system). Hell I am still using pickle files for storing the NFT data myself in smartnftsearch.xyz cause I am pretty comfortable using it.
Deployment and showing your model to the world
You have your trained model ready and set to go. You tested it on many different cases and satisfied with how it is working. Time to show it to the whole world.
Bad news is that no one got time to replicate your environment, download the model and try it out on their own computer. You need to wrap it in the web demo that everyone can easily open and use
Luckly, Gradio, and Streamlit let you quickly bootstrap a web demo of your ML model. These tools are almost no-code and require implementing extra 30-40 lines of code to get it running.
Final step, host your model on Huggingface spaces and share it with the rest of the world!
Congrats anon, you did a very first step in building your own ML SaaS. This wasn’t easy but very much worth it. Your journey is just starting as you will be sharing your demo with potential customers, getting their feedback, and then iterating. Good luck!
Finally, I am opening my substack where I will be going in-depth of such topics as getting your foot in the door deep learning, finding machine learning jobs, and building SaaS for bigger profit with the focus of natural language processing
Subscribe!
Thanks Raptor for posting these!