Releasing modelstore v0.0.1b, two opinions, and next steps
Last weekend, I quietly published a new Python library to pypi: modelstore.
And by “quietly” I mean that I tweeted about it with emojis and no context:
The public stats in the the-psf.pypi BigQuery project tell me that it has been installed 222 times since then — exciting! And I’ve also been told that it’s being piloted at a good friend’s AI company.
If you want to give it a spin, then I’d suggest opening the docs and trying it with:
❯ pip install modelstore
As always, feedback is welcome ♥️
Let’s talk about some of the why
I’ve been intentionally holding back on many opinions that I have about today’s MLOps & ML platform ecosystem. Now that I’ve released the earliest version of modelstore, I thought I’d share two — fully aware that this is an ever-evolving landscape.
Lego blocks (not platforms)
Companies that are creating value with machine learning are tech companies, and tech companies have their own stack. Today, that means that they’ve picked their own permutation of the cloud provider — programming language(s) — architecture combo, and use that to build a very specific product. At Monzo, for example, we picked AWS, Golang, & microservices on kubernetes, and we’re building a bank. 🏦
Many of the systems that I get pitched for machine learning are end-to-end AI platforms. They sell themselves on either the wonderful technologies they use under the hood, or the little code that I would have to write when using them.
Using a separate end-to-end AI platform would mean branching all machine learning systems away from the primary way that the company ships its value. We’d be constrained to doing things the way that platform allows us to do it, rather than being able to mold that ourselves. We’d be constrained to whatever technology du jour was used to build the platform, and not the one that we want to use. We are already the minority in the tech org, and we would be introducing a completely foreign process.
We decided not to do this. Instead, we decided to only introduce one diff: Python microservices, instead of Golang, for the predict functions. That’s ensured that ML gets a shout-out in Monzo QCon talks as one of the few non-Golang use cases in the company’s backend infrastructure 😄.
If machine learning (in 30 years’ time) is due to become what databases were in the 1970s, then it seems that the future of AI platforms is to become simple, modular, and composable building blocks — not end-to-end — letting companies pick their own permutation of tools to build their products. (I’ve nicked this trio of words from the Tao of HashiCorp)
Workflows (not technologies)
The main problem I’ve seen across many companies is that Machine Learning folks spend a lot of their time doing the work that enables their job. It doesn’t help that the systems we are trying to build are inherently complex — we have to think about hardware (GPUs), software, data, and all the layers in between:
This has spurred a bunch of “no code” deployment products alongside the end-to-end platforms. To be blunt, many of these strike me as “struggling with your current workflow? Why not struggle with ours, instead!” (or, worse, replace the little time you spend writing the code you love with writing config for our system 😱). Swapping one black box out for another does not strike me as solving the workflow problem.
A recurring phrase in my team over the last few years has been that complex does not need to imply complicated (with the difference stated brilliantly here). The tools that I’m thinking about will leave the complexity in your hands, while making it less complicated. After all, isn’t the complex stuff the bit you love the most about the work?
You have a model in a store. What’s next?
Back to modelstore: the replies to my nebulous tweets included some ideas of where this could continue developing:
Supporting more model types (e.g., transformers);
More backend types (e.g., Azure);
Ethan suggested adding controls to optionally override the opinionated parts 😄
These are all very cool ideas and the repo is now open source and welcoming pull requests, issues, and feedback if you would like to contribute. Think of it like a little lego block — I would love to hear if you’ve used it.
The part that I am moving on to is the next step of the workflow: writing the production code. Is this something you’d like to chat about? With the pandemic news in the UK not looking so great right now, I’m going to have plenty of time to email or chat about this stuff. Hit reply!