I can recall so many times I have explained people higher/ lower than me that open source does not mean “free”. You have to pay for inferencing and stuff.
I believe OS shines when hyper focused very small models are considered. In the range of ~100M to 500M. Encoder use case only though.
Decoders / text generators in this range are very bad.
Thanks so much for sharing this. 25 years ago I ran my own email server using open source code. It was great. You could customize things. It was feature rich, when a lot of the online email serviced were less than ideal. BUT… it was the most time-consuming thing I’ve ever done in my life. I’m gonna stick to online AI services for now. 😀
Extremely intriguing blog! I'm building the infra platform layer to minimise such overheards for folks looking for open source models while minimising their AI bills.
Would be happy to chat with people struggling with this problem to understand your use case better!
“Open-source LLMs are not free — they just move the bill from licensing to engineering, infrastructure, maintenance, and strategic risk”
Isnt that the case with every OSS?
Databases, message brokers, data processing frameworks etc.
Isnt what you are talking about just self hosting vs managed infra?
No free lunch, and great analysis on the hidden costs.
Damn this went deep.
I can recall so many times I have explained people higher/ lower than me that open source does not mean “free”. You have to pay for inferencing and stuff.
I believe OS shines when hyper focused very small models are considered. In the range of ~100M to 500M. Encoder use case only though.
Decoders / text generators in this range are very bad.
Wow. Such a great article!
Thank you
Thanks so much for sharing this. 25 years ago I ran my own email server using open source code. It was great. You could customize things. It was feature rich, when a lot of the online email serviced were less than ideal. BUT… it was the most time-consuming thing I’ve ever done in my life. I’m gonna stick to online AI services for now. 😀
loved reading this...relates very much to the problem I faced recently
Thank you
Brilliant Deep Dive 🌟 Thanks for sharing 🌞
Extremely intriguing blog! I'm building the infra platform layer to minimise such overheards for folks looking for open source models while minimising their AI bills.
Would be happy to chat with people struggling with this problem to understand your use case better!
Great piece!!!🙌🙌