Vector RAG, Graph RAG and the hunt for Accuracy

Retrieval Augmented Generation has been a powerful framework for developers working on Generative AI. Having said that over the past 2 years, RAG has taken on various forms and has gotten significantly more complex in the search for reliability and accuracy concerns posed by LLMs. We’ve moved on from Vectors and embeddings to Graphs and Knowledge Graphs. Graph based structures clearly are more deterministic in nature when it comes to retrieval, thereby enabling GenAI applications to utilize their reasoning capabilities to retrieve accurate and well thought out answers. This would lead you to believe that Graphs are the obvious answer to the reliability and accuracy concerns around GenAI. Well, yes and no.  

In some cases, Graph RAG might miss out on information that Vector RAG can retrieve. However, the logic of structuring information and keeping it as nodes in a graph will definitely give you more reliable answers. So, in that sense, it's not comparable. But that does not imply that you don’t need vectors, you do need vectors in some cases. Take for example the likes of Neo4J who have combined vectors inside of their system. Microsoft is also saying the same thing with Cosmos. It’s important to realize that there are tradeoffs here. Hence it would be wrong to claim that the graph is it. Graph RAG doesn't fully solve the problem, but it's definitely a step in the right direction.  

Breaking Down the Concept of Accuracy in GenAI

Accuracy from the perspective of ML, Computational linguistics, statistics is broken into two simpler concepts of Precision and Recall. Both of these go hand-in-hand.  

Precision in a RAG context measures the accuracy of positive predictions. It's calculated as the number of true positives divided by the total number of predicted positives (true positives + false positives). Precision is particularly important in scenarios where false positives are costly or undesirable, such as spam detection or recommendation systems – think healthcare, pharma, life sciences, finance et al.  

Recall, in the context of RAG is about the relevance of retrieved data. Depending upon the retrieval mechanism, every retrieval query would end up with both relevant and less relevant data. It's calculated as the number of true positives divided by the total number of actual positives (true positives + false negatives). Recall also gives a clear insight on the coverage of concepts across your data sources. Let’s say there are 100 concepts in your data sources, and you expect queries across all these 100 concepts. The conversation is now shifting towards recall – i.e. how extensively ideally would want to get all those 100 concepts to ensure 100% coverage for your queries/ information needs.  

High recall indicates that the model identifies most of the relevant instances, minimizing false negatives. However, high recall alone doesn't guarantee overall accuracy, as it doesn't account for false positives. Recall is particularly important in scenarios where missing relevant items is costly, such as in medical diagnoses or fraud detection.

That’s the reality – It’s not a question between choosing one or the other. The debate between Graph-based and Vector-based RAG systems isn't about which method is superior, but rather using them for their strengths. Graph-based retrieval provides a more structured, reliable way to access information, while vector-based methods can capture nuances that graphs might miss. The focus is shifting from just precision to recall, ensuring that all important concepts in the data are covered. Ultimately, the effectiveness of RAG systems hinges more on how well the information is structured and organized, rather than the underlying retrieval mechanism itself.

‍