dstack

I tested dstack as a tool to do multi-cloud deployment of my own inference software. Works as a charm for any app, even non-AI/ML ones thanks to easy-to-use docker support. Code is simple to read and pretty customizable.

I am running automated paper translations with a custom AI model every day on dstack, and it works perfect. With lots of paper, it helped me not only distribute the workloads on multiple GPU machines, but also easy to provision them with built-in Python API.

great for cost focused training finetuning and production runs

easy

Cool stuff