Subscribe
Sign in
  1. Discussions
  2. Will RAG be eliminated by long-context models?

Will RAG be eliminated by long-context models?

Stain Lu
11 replies
There seems to be much debate on it among researchers, what do you think? My view is that there will always be a “memory” as retrieval or buffer.

Replies

Ted Schaefer
Albert
Albert
definitely think RAG will be here forever in some form or another, since an LLM is lossy by definition. it will be pretty cool when LLMs are able to train on new information in real time though
Share
ᗰᗩ᙭ ᒍ.
moji AI wearable
moji AI wearable
I think there will be use case for both, also at what point is long-context long enough, depending on the use case.
Share
Gurkaran Singh
It's fascinating to see the ongoing debate among researchers on the future of RAG in the era of long-context models. I believe that the concept of "memory" in the form of retrieval or buffer will continue to play a crucial role in information processing.
Share
Aris Nakos
Llanai
Llanai
@thestarkster Solid comments. Follow @swyx on Twitter for great ideas on the matter. He aggregates top knowledge from the field. My premonition is that LLM embeddings become more nuanced and thus long context wouldn't need to be a necessary feature in the future of LLM as much. However, I need to read some more!
Share
Ted Schaefer
Albert
Albert
@thestarkster @stain it'll also be more expensive until someone figures out a way to improve the architecture - it takes a lot of memory to crunch through 100K tokens!
Share
Ivan Spadacenta
@stain I'm incorporating RAG (Retrieval-Augmented Generation) into a project focused on an AI writing assistant. The rationale behind utilizing RAG stems from the fact that conventional AI models often lack the capacity to stay updated with the latest events and their intricate details. Consequently, they may either decline assistance for such tasks or, worst, offer responses that are not grounded in real-time data, potentially yielding inaccurate or fantastical outputs.
Share