Run, tune, and scale generative fashions that energy AI purposes


Luis Ceze, CEO and Co-Founding father of OctoML, gave this presentation on the Generative AI Summit in Boston in 2023.

I’ve spent most of my skilled life constructing environment friendly pc techniques for a bunch of purposes, from life sciences and bioinformatics to simulations. For the previous 10 years, I’ve been making AI techniques extra environment friendly utilizing {hardware} and software program methods.

OctoML spun out of the College of Washington. We constructed a system referred to as Apache TVM, which was one of many early machine studying compilers that enabled AI to be deployed in your telephone, the cloud, and all over the place else. 

OctoML has about 100 folks, and we began 4 years in the past. We began by specializing in mannequin optimization, and over the previous yr or so, after realizing how sizzling GenAI would turn out to be, we began constructing a product that as a substitute of serving machine studying engineers, serves software builders who’re constructing with AI.

We’ll speak about what OctoAI is and the way we assist our clients achieve success, after which we’re going to have a look underneath the hood to see the way it all works. 

What’s OctoAI?

OctoAI is a platform that allows builders to run AI fashions and add AI fashions to purposes they select, comparable to open-source fashions or customized fashions. It means that you can fine-tune the mannequin on the platform, and when you’re placing in deployments, it makes it scale. Meaning it solves {hardware} availability issues.

"OctoAI is a platform that enables developers to run AI models and add AI models to applications they choose, such as open-source models or custom models. It allows you to fine-tune the model on the platform, and once you’re putting in deployments, it makes it scale."

Scaling these fashions includes doing a whole lot of deep optimizations, which is what we do rather well.

It takes about 10 trillion pc operations to generate one high-definition GenAI-generated picture. That is a whole lot of compute. 

The rationale I point out that is that as you scale this to have a worldwide affect, you could have to have the ability to present the compute for that and make it quick sufficient for folks to have the persistence to make use of it and now have the assets to take action.

Primarily based on our experience and mannequin optimization, we constructed a self-optimizing compute service that robotically chooses the correct optimizations in your mannequin to be viable and chooses the correct {hardware} (you do not have to go and choose a particular GPU to do the work) and chooses the system parameters robotically.

After which you’ll be able to construct your software by selecting fashions you are most likely accustomed to, like Steady Diffusion, Llama 2, or ControlNet.

So that is what OctoAI is, and it isn’t simply compute. The best way this works is we provide a set of end-to-end options, like picture technology, gaming, private expression via artwork, advertising and marketing, and so forth. 

We even have options for language, comparable to summarizing textual content, questions and solutions, and many others.

All of that’s underpinned by a set of extremely optimized open-source fashions. We really imagine that open-source fashions are the best way ahead to construct techniques that you may depend on, you perceive how they work, and you’ve got management over the deployment. 

We run these fashions for you. You do not have to provision the compute. That is a part of the platform.

You possibly can tune the fashions on the platform. For picture technology, you’ll be able to carry photos to the system after which fine-tune the picture technology mannequin with these photos and use them in your software within the system, after which we are able to handle these personalized fashions for you. 

Turn into a member to see the remainder

Sure items of content material are solely to AIAI members and also you’ve landed on one in every of them. To entry tonnes of
articles like this and extra, get your self signed up.

Already a member? Superior Check in

Get product advertising and marketing licensed.

PMMC™ unleashes product entrepreneurs’ potential. Lauded by main lights like Fb and HubSpot, it
gives knowledgeable insights, priceless tuition, and superior assets. No subject missed. No web page unturned.

Supply hyperlink


Please enter your comment!
Please enter your name here