Hi guys very curious to know what your net profit % is if you are using llms?
Azlan Tariq
22 replies
Saw an article recently which suggested that people integrating gbt,gemini etc are facing this problem of managing the cost of these llms as they are very expensive
Replies
Uthappa Robb@uthappa_robb
Our experience has been that integrating language models has helped us improve efficiency and accuracy, which indirectly boosts profits.
Share
@uthappa_robb Yeah thats what the trend seems to be!
I’ve noticed the same thing. It seems like striking a balance between leveraging the power of LLMs and keeping costs in check is tricky
@jasper_james What i Feel that Model Compression will be a great way for the future but enterprises are having risk thoughts about open source llms lately
@jasper_james Yeah thats the stage we are in aswell
I’ve heard that optimizing LLM usage, like fine-tuning models for specific tasks, can help reduce costs
@kistine_sheffield Yeah using the old models for freemium or basic versions and the latest for the premium seems to be the go to strategy.
@kistine_sheffield @azlan_tariq Fine-tuning the model on the comprehensive dataset is what is going to be the future of development. We at Future4coding are pushing boundaries in AI in software development Do check us out
We outsource to an agency. They bring fresh ideas and handle all the platforms efficiently.
Since integrating LLMs like GPT and Gemini, we’ve seen our net profit increase by about 15%. However, managing the costs has been a challenge, so we need to continuously monitor and adjust our usage.
@teekaram_teekaramyogi Thats amazing and i agree monitoring of cost is necessary
@teekaram_teekaramyogi Open Souce LLMs are cost-heavy and require lot of computational datasets that's where At Curiolabs our mission is to create a safe and capable artificial intelligence system that can understand complex requirements, write high-quality code autonomously, run comprehensive tests, and seamlessly integrate third-party services. We approach this challenge by advancing AI capabilities and safety measures in tandem, ensuring that our system remains secure and reliable as it becomes increasingly powerful.
i'm experimenting with some techniques to lower the cost for my upcomig product
I read a research paper called Frugal Gpt there are some techniques to lower the cost.
do check it out
@dev_abdulhaseeb Please share would love to check it out!
@azlan_tariq I'm unable to share the link here, just google it you will find their giuthub repo
Absolutely, managing costs while leveraging the power of LLMs is a significant challenge. The balance involves several key factors:
1. Usage Efficiency: Optimizing how often and for what purposes LLMs are used can help control costs. For example, using LLMs for tasks that truly benefit from their capabilities rather than routine or low-value tasks.
2. Model Selection: Choosing the right size and type of model for specific applications can impact costs. Smaller models or fine-tuned versions may be more cost-effective for certain tasks compared to larger, more general models.
3. Infrastructure Costs: The cost of running LLMs, including cloud infrastructure and computational resources, can add up. Effective management of these resources, such as through batch processing or serverless options, can help keep expenses under control.
4. Scalability: Implementing solutions that can scale efficiently with demand without a linear increase in costs is crucial. Techniques like caching responses or using hybrid models can help in this regard.
5. Monitoring and Optimization: Regularly monitoring usage and performance metrics allows for fine-tuning and adjustments to reduce unnecessary expenditure.
Balancing these factors requires ongoing adjustments and strategic planning to maximize the benefits of LLMs while keeping costs in check.
@kennethmarko Yeah thats very insightful and you have made very good points there, using the old models for freemium or basic versions and the latest for the premium seems to be the go to strategy.
@kennethmarko Wow, great! I agree with your points. Balancing usage efficiency, model selection, infrastructure costs, scalability, and ongoing monitoring is essential for managing costs while leveraging LLMs effectively.
I’m exploring it for my business and wondering if it’s worth the investment.
@debra_hetrick Let me know if you do! Im in the same stage
Profit margins heavily depend on use case and architecture. Running LLMs yourself cost effectively is still very challenging. For a low volume app, maybe 20-30% after infra costs. For high volume, likely single digit % or even losing money initially. Using APIs like OpenAI is more predictable, maybe 50%+ depending on pricing. Definitely a balancing act! What's worked for others?