Challenges to deploy ML models as an API?

Question

So I'm building a low code tool to help users launch ML models via a flowchart like UI. Very early stages, but the main idea behind it is to make AI more accessible. As a ML engineer, I often come around discussions that for many ML engineers, getting their models off their jupyter notebook is a challenge. Where do you guys stand on it?

Nha Hyerin · Accepted Answer

Deploying ML models as an API presents challenges such as ensuring scalability, handling latency, managing data privacy, and maintaining model performance over time. Integration with existing systems and ensuring robust error handling and monitoring are also key concerns.

Wiktor Wysocki · Answer

I think that if you want to deploy your model it has to be very specific. There are multiple big models from big companies which are very general. If you find a niche, it should work.
Other than that, it is a question of hosting and compute availability.

Challenges to deploy ML models as an API?

Replies