Introducing BeyondLLM

Introducing BeyondLLM

Apr 17, 2024

Apr 17, 2024

Large Language Models have recently demonstrated remarkable potential, offering a wide range of features for diverse applications. The rapid growth of LLMs in open-source libraries enables the development of various tools such as RAG applications, LLM evaluation, fine-tuning, observability, and more. LLM is way beyond just prompt engineering.

We at AI Planet are excited to introduce BeyondLLM, an open-source framework designed to streamline the development of RAG and LLM applications, complete with evaluations, all in just 5–7 lines of code.

Yes, you read that correctly. Only 5–7 lines of code. Let’s understand what and why one needs BeyondLLM.

Build Evaluate Experiment Observe Repeat

In the AI era, everything begins with data. Our focus at BeyondLLM is to empower users to develop LLM applications like RAG for quick experimentation. Drawing inspiration from open-source repositories like Sklearn, BeyondLLM offers a user-friendly library that prioritizes flexibility for fellow Data Scientists. When building an application integrating RAG, selecting the most effective retriever and efficient chunking is crucial. We provide a source and auto_retriever that facilitates the implementation of advanced retrieval techniques, which are essential for reducing hallucinations — a key challenge in enterprise adoption of LLMs in their products.

BeyondLLM not only simplifies the construction of complex RAG pipelines with minimal coding but also enhances the evaluation process with comprehensive benchmarks like Context Relevance, Answer Relevance, Groundedness, and Ground Truth. These metrics assess everything from the retriever’s ability to fetch relevant information to the LLMs response accuracy and factual truth, all streamlined within the framework which also automates quick experimentation.

Why BeyondLLM?

Easily build a model with 5 lines of code

  • Building a robust RAG (Retrieval-Augmented Generation) system involves integrating various components and managing associated hyperparameters. BeyondLLM offers an optimal framework for quickly experimenting with RAG applications.

  • With components like source and auto_retriever, which support several parameters, most of the integration work is automated, eliminating the need for manual coding.

Evaluation of Embeddings and LLM

  • The evaluation of RAG in the market largely relies on the OpenAI API Key and closed-source LLMs. However, with BeyondLLM, you have the flexibility to select any LLM for evaluating both LLMs and embeddings.

  • BeyondLLM offers support for 2 evaluation metrics for embeddings: Hit rate and MRR (Mean Reciprocal Rank), allowing users to choose the most suitable model based on their specific needs.

  • Additionally, it provides 4 evaluation metrics for assessing Large Language Models across various criteria, in line with current research standards.

Advanced Techniques to reduce LLM Hallucinations

  • Certainly, the primary objective is to minimize or eliminate hallucinations within the framework.

  • To support this goal, we’ve developed the Advanced RAG section, facilitating rapid experimentation for constructing RAG pipelines with reduced hallucination risks.

  • BeyondLLM features, including source and auto_retriever, incorporate functionalities such as Markdown splitter, chunking strategies, Re-ranking (Cross encoders and flag embedding), and Hybrid Search, enhancing the reliability of RAG applications.

  • It’s worth noting Andrej Karpathy’s insight: “Hallucination is a LLM’s greatest feature and not a bug,” underscoring the inherent capabilities of language models.

Use Cases

Chat with your own data

  • BeyondLLM allows you to talk to your own data i.e. you can basically create a chatbot on your data. This data could be in any format: .pdf, .docx, .pptx, and many more!

  • We abstract the loading and chunking steps so that you can begin to chat with your data with only a few lines of code. Based on your query, the most relevant chunks are retrieved and passed to your LLM.

  • This use case can help you summarize and gain important information from a boring PPT or a textbook PDF. This finds uses in the Education, Legal industries, and many others.

Customer Service Q&A Bot

  • BeyondLLM allows you to make a customer care bot in a few lines of code.

  • Just add the description of your product/company in the form of .pdf, .docx, URL, etc.

  • Set the system prompt to set the tone of the bot and voila you are good to go.

  • This particular use case can be adapted to E-commerce, Logistics industries, and so on.

Document Search & Chat

  • BeyondLLM allows you to have conversations with your documents that require searching.

  • Feed the data in any format: .pdf, .docx, .pptx, and many more, and start having conversations.

  • This particular use case can be adapted to Finance, Legal, Healthcare industries, and so on.

Multilingual LLM support

  • Beyond LLM can be further used to build Multilingual RAG that basically allows the model to understand multiple languages.

  • Beyond LLM can extend support to use Open Source embeddings that can understand the multiple languages tokenizer and then retrieve the document accordingly.

  • This particular use case can be adapted to Media, E-learning (Education) industries, and so on.

Call for Community

BeyondLLM is completely open source. Feel free to raise any questions by opening issues on our GitHub repository. We value your feedback and look forward to connecting with you there.

Open your PR here: https://github.com/aiplanethub/beyondllm

In order for you to get started, check out the documentation for more details: https://beyondllm.aiplanet.com/

Quickstart Guide Colab Notebook: https://colab.research.google.com/drive/1S1UL2uCahHkfJsurRA3f7dcR6IHjg-IM?usp=sharing

Don’t forget to ⭐️ and fork the repository.