Introducing Studio.ML: an Open Source Framework that Simplifies and Expedites Machine Learning Model Development

By Arshak Navruzyan
VP Distributed Artificial Intelligence Platform

At Sentient, we are continuously building and iterating on machine learning models to advance our products and research. We’re big fans of Keras, TensorFlow, PyTorch etc., but along the way we’ve found some critical missing features when it comes to running a large set of machine learning experiments in a scalable and collaborative fashion. Specifically, there’s nothing out there that can automatically capture machine learning experiments in order to ensure reproducibility and reuse of modeling assets.

Today, we’re excited to introduce, an open source project dedicated to helping machine learning (ML) professionals, academics, businesses and anyone else interested in ML model building, accelerate and simplify their experiments. is an early-stage, ML model management framework written in Python and developed to minimize the overhead involved with scheduling, running, monitoring and managing artifacts of your machine learning experiments.
Most of the features are compatible with any Python ML framework, including Keras, TensorFlow, PyTorch, and scikit-learn (additional features available for Keras and TensorFlow).

So far, using you can:

  • Capture experiment information- Python environment, files, dependencies and logs- without modifying the experiment code
  • Monitor and organize experiments using a web dashboard that integrates with TensorBoard
  • Run experiments locally, remotely, or in the cloud (Google Cloud or Amazon EC2)
  • Manage artifacts
  • Perform hyperparameter search
  • Create customizable Python environments for remote execution
  • Access the model library to reuse models that have already been created

This code is still in the early phases of development, but we encourage you to try it, report back problems, ask questions, provide feedback and contribute. With your help we believe we can take some of the pain out of ML model building.

Finally, if you’re in San Francisco and would like to find out more, you can either join us at the SF Machine Learning Meetup this Wednesday evening or hear us speak on September 16 at Startup.ML.


Q&A –

What is the complete list of tools is compatible with?
Keras, TensorFlow, PyTorch, scikit-learn

Do I need to modify my code to use Studio?
Studio is designed to minimize any invasion of your existing code. Running an experiment with Studio should be as simple as replacing `python` with `studio run` in your command line with a few flags for capturing your workspace or naming your experiments.

I already save all of my log files for experiments; why should I use Studio?
Studio’s web interface offers one centralized location for keeping track of your experiments and its artifacts. Also, you are able to view experiments of other users (see next question) so Studio makes it easier to resurrect old experiments or those launched by others.

Can I keep any of my projects private?
Yes, you can but we encourage sharing where possible as this increases the value of the project for others. There is documentation for setting up your own private database if you wish to keep your experiments private.

How can I track the training of my models?
You can manage any of your experiments- current, old or queued- through the web interface. Simply run `studio ui` to launch the UI to view details of any of your experiments.

What license is this project offered under?
Apache 2.0

How does integrate with Google Cloud or Amazon EC2?
We use standard Python tools like Boto and Google Cloud Python Client to launch GPU instances that are used for model training and de-provision them when the experiment is finished.

Our team shares a remote server. Would Studio be a good way to launch and keep track of different experiments from different users running on the same server?
Yes! If everyone in your team uses Studio’s remote workers to launch and manage experiments, then you can view the experiments of other users through the web interface.

Can I use Studio with Docker containers?
Yes, you can run on bare metal or containerize your studio runner.

What’s in it for Sentient?
Sentient will benefit from a more robust tool with which to service its customers and research partners.

Please add any additional questions to our wiki and we’ll answer them as soon as we can.