Some thoughts about RAG

I have been using AnythingLLM and notebookLM for a while, but until today I found a few benefits that notebookLM have. 

Generally gemini has stronger hallucination, so notebookLM is only a way to get a quick glimpse of uploaded sources. It only takes pdf, txt, or very limited google drive content it has access to. 

I converted two tables made with spreadsheets into PDF files, and test how these two LLM tool work. The two PDF files have many names, but I need to find the names that shown in both list. It's hard for human to check that, esp. when the list is not linear (it's brick like stack together). The two tools return with different numbers: 29 and 31. 

 

I made further investigation and found NotebookLM is correct, the names shall be 31. That's a rare case it beats anythingLLM (I chose chatgpt 4o for inference). 

The reasons could still be chunking, I have found anythingLLM performs better with html files, although it also accepts PDF. In some cases, huge tables converted into PDF will have issues in chunking esp. with special layout.  

 If we need to be compatible with all tools and AI, we must not lock ourselves in certain format or tool. Understanding subtle limitation like this can help reduce hallucination and design better process for all.