Location>code7788 >text

RAG+Agent AI Platform: RAGflow implements GraphRA Knowledge Base Q&A for the ultimate multimodal Q&A and AI orchestration flow experience

Popularity：773 ℃/2024-09-20 15:02:53

RAG+Agent Artificial Intelligence Platform: RAGflow implements GraphRA Knowledge Base Q&A to create the ultimate multimodal Q&A and AI orchestration streaming experience

synopsis

  • Last Updated:

    • 2024-09-13 Add knowledge base quiz search mode.
    • 2024-09-09 Add medical inquiry template to Agent.
    • 2024-08-22 Support for conversion from natural language to SQL statements using RAG technology.
    • 2024-08-02 Support GraphRAG Inspired bygraphrag and mind maps.
    • 2024-07-23 Support for parsing audio files.
    • 2024-07-08 Support Agentic RAG: based onGraph of the workflow.
    • 2024-06-27 Q&A The parsing method supports Markdown files and Docx files, and supports extracting pictures from Docx files and tables from Markdown files.
    • 2024-05-23 RealizedRAPTOR Provide better text retrieval.

  • Key Features

    • "Quality in, quality out"

      • Based on deep document understanding, it is able to extract insights from unstructured data in various complex formats.
      • Really fast needle-in-a-haystack testing in infinite context (token) scenarios.

    • Template-based text slicing

      • Not just smart, but also controllable and interpretable.
      • Multiple text templates to choose from

    • Justified, minimized hallucinations (hallucination)

      • Visualization of the text slicing process with support for manual adjustment.
      • Justified: answers provide snapshots of key references and support traceability.

    • Compatible with various heterogeneous data sources

      • Support rich file types, including Word documents, PPT, excel tables, txt files, images, PDF, photocopies, copies, structured data, web pages and more.

    • Automated RAG workflows

      • Fully optimized RAG workflows support ecosystems ranging from personal applications to mega-enterprises.
      • Configurations are supported for the Large Language Model LLM as well as the Vector Model.
      • Based on multiplexed recall, fusion reordering.
      • Provides easy-to-use APIs for easy integration into various enterprise systems.

  • system architecture

2. Quick Start

  • segmentation fault
    • CPU >= 4 cores
    • RAM >= 16 GB
    • Disk >= 50 GB
    • Docker >= 24.0.0 & Docker Compose >= v2.26.1

      If you don't have Docker installed locally (on Windows, Mac, or Linux), you can refer to the documentation forInstall Docker Engine Self-installation.

2.1 Starting the server

vm.max_map_countis an important parameter in the Linux kernel that defines the maximum number of memory-mapped regions a process can have. Memory-mapped areas are usually referred to as memory-mapped files, anonymous memory maps, and so on.

  • performance optimization: By increasing the value of vm.max_map_count, applications can be allowed to create more memory-mapped regions, thus improving performance and efficiency. This optimization is especially effective for applications that require frequent access to a large number of files or data.

  • Stability Assurance: If an application attempts to create more memory mapped regions than the system set limit, it may cause the mapping to fail, which in turn may cause performance issues or directly crash the application. Therefore, a reasonable setting of the vm.max_map_count parameter can help guarantee system stability.

  • Setting method

    • temporary setup: This can be done bysysctlcommand temporarily modifies the value of vm.max_map_count, but this change expires after a system reboot. For example, to set the value of vm.max_map_count to 262144, you can execute thesudo sysctl -w vm.max_map_count=262144Command.
    • Permanent settings: To ensure that the value of vm.max_map_count remains valid after a system reboot, the value needs to be written to the/etc/file. Add or updatevm.max_map_count=262144(or other desired values) to that file and save the changes. Afterwards, you can change the values in the file by executing thesudo sysctl -pcommand to make the changes take effect immediately.

  1. assurevm.max_map_count Not less than 262144:

    For confirmationvm.max_map_count The size of the

    $ sysctl vm.max_map_count

    in the event thatvm.max_map_count value is less than 262144, a reset can be performed:

    # Here we set it to 262144: $ sudo sysctl -w vm.max_map_count=262144
$ sudo sysctl -w vm.max_map_count=262144

    Your changes will be reset on the next system reboot. If you wish to make permanent changes, you'll also need to add a new file to the/etc/ file and put thevm.max_map_count value is updated again accordingly:

    vm.max_map_count=262144

  2. Cloning Warehouse:

    $ git clone /infiniflow/

  3. go intodocker folder, start the server with the pre-compiled Docker image:

    $ cd ragflow/docker
$ chmod +x ./
$ docker compose -f  up -d

    Note that running the above command will automatically download the development version of the RAGFlow docker image. If you want to download and run a specific version of the docker image, find the RAGFLOW_VERSION variable in the docker/.env file and change it to the corresponding version. For example, RAGFLOW_VERSION=v0.11.0, then run the above command.

    The core image file is about 9 GB and may take some time to pull. Please be patient.

    Mirror pulls in the too slow chemistry reference link:Mirror pull to boost

  4. Reconfirm the server status after the server has started successfully:

    $ docker logs -f ragflow-server

    The following screen prompt appears to indicate that the server has started successfully:

        ____                 ______ __
   / __ \ ____ _ ____ _ / ____// /____  _      __
  / /_/ // __ `// __ `// /_   / // __ \| | /| / /
 / _, _// /_/ // /_/ // __/  / // /_/ /| |/ |/ /
/_/ |_| \__,_/ \__, //_/    /_/ \____/ |__/|__/
              /____/

 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:9380
 * Running on :9380
 INFO:werkzeug:Press CTRL+C to quit

    If you skip this system confirmation step and log into RAGFlow, your browser may prompt you tonetwork abnormal maybenetwork anomalyThis is because RAGFlow may not have been fully activated successfully.

  5. Enter the IP address of your server in your browser and log in to RAGFlow.

    In the example above, you only need to enter http://IP_OF_YOUR_MACHINE: there is no need to enter a port if the configuration has not been changed (the default HTTP service port is 80).

  6. existservice_conf.yaml documentationuser_default_llm column to configure the LLM factory andAPI_KEY Fill in the column with the API key that corresponds to the big model you chose.

    for further details, refer tollm_api_key_setup。

Deployment encounter problem solving (🔺)

Insufficient resources problem, ES will take up more resources recommend setting larger

Modify the .env file and set it according to your own memory resources, I'll set it to 70G, es default eat half

#Increase or decrease based on the available host memory (in bytes)

MEM_LIMIT=72864896288

Encountered knowledge base build, index build stuck unable to parse

Problem description: The index building process has been stuck, after troubleshooting found that the system disk space is not enough 95% +, the error is reported as follows

ApiError('search_phase_execution_exception', meta=ApiResponseMeta(status=503, http_version='1.1', headers={'X-elastic-product': 'Elasticsearch', 'content-type': 'application/+json;compatible-with=8', 'content-length': '365'}, duration=0.004369974136352539, node=NodeConfig(scheme='http', host='es01', port=9200, path_prefix='', headers={'user-agent': 'elasticsearch-py/8.12.1 (Python/3.11.0; elastic-transport/8.12.0)'}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=True, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=True, _extras={})), body={'error': {'root_cause': [{'type': 'no_shard_available_action_exception', 'reason': None}], 'type': 'search_phase_execution_exception', 'reason': 'all shards failed', 'phase': 'query', 'grouped': True, 'failed_shards': [{'shard': 0, 'index': 'ragflow_304817a205d211efa4de0242ac160005', 'node': None, 'reason': {'type': 'no_shard_available_action_exception', 'reason': None}}]}, 'status': 503})
  • If you don't have enough space on your system disk, please migrate the docker

Modifying the Docker Default Storage Path Reference

Post-migration problem solving:

I have to say, ragflow's documentation parsing is pretty good!

2.2 System configuration

The system configuration involves the following three documents:

  • .env: Stores some basic system environment variables such asSVR_HTTP_PORT、MYSQL_PASSWORD、MINIO_PASSWORD etc.
  • service_conf.yaml: Configure various backend services.
  • : The system relies on this file to complete startup.

Please be sure to.env The variable settings in the file are the same as those in theservice_conf.yaml The configuration in the file is consistent!

./docker/README file provides detailed information on environment variable settings and service configuration. Pleasemustassure./docker/README The values of the environment variables listed in the file are the same as those in theservice_conf.yaml The system configuration in the file is consistent.

To update the default HTTP service port (80), you can add a new port in the file will configure the80:80 change into<YOUR_SERVING_PORT>:80。

All system configurations need to take effect through a system reboot:

$ docker compose -f  up -d

2.3 Source Compile, Install Docker Image

To install a Docker image from source:

$ git clone /infiniflow/
$ cd ragflow/
$ docker build -t infiniflow/ragflow:v0.11.0 .
$ cd ragflow/docker
$ chmod +x ./
$ docker compose up -d

2.4 Source code to start the service

To start the service from source, refer to the following steps:

  1. clone warehouse
$ git clone /infiniflow/
$ cd ragflow/
  1. Create a virtual environment (make sure you have Anaconda or Miniconda installed)
$ conda create -n ragflow python=3.11.0
$ conda activate ragflow
$ pip install -r

If cuda > 12.0, the following additional commands are required:

$ pip uninstall -y onnxruntime-gpu
$ pip install onnxruntime-gpu --extra-index-url /PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
  1. Copy the entry script and configure the environment variables
$ cp docker/ .
$ vi

Use the following commands to get the python path and the ragflow project path:

$ which python
$ pwd

connect the abovewhich python The output of thePY The value of thepwd The output of thePYTHONPATH The value of the

LD_LIBRARY_PATH If the environment is already configured, you can comment it out.

#The configuration here needs to be adjusted according to the actual situation, the two exports are new configurations
PY=${PY}
export PYTHONPATH=${PYTHONPATH}
#Optional: add Hugging Face image
export HF_ENDPOINT=
  1. Starting Basic Services
$ cd docker
$ docker compose -f  up -d

  1. Checking configuration files
    assuredocker/.envThe configuration in theconf/service_conf.yamlThe configuration is consistent with that in theservice_conf.yamlThe IP address and port of the relevant service in the container should be changed to the local IP address and the port mapped by the container.

  2. Starting services

$ chmod +x ./
$ bash ./
  1. Starting the WebUI service
$ cd web
$ npm install --registry= --force
$ vim .
# Modify to http://127.0.0.1:9380
$ npm run dev
  1. Deploying WebUI services
$ cd web
$ npm install --registry= --force
$ umi build
$ mkdir -p /ragflow/web
$ cp -r dist /ragflow/web
$ apt install nginx -y
$ cp ../docker/nginx/ /etc/nginx
$ cp ../docker/nginx/ /etc/nginx
$ cp ../docker/nginx/ /etc/nginx/
$ systemctl start nginx

3. Rapid practice of cases

3.1 Model Access

  • Business Model Access:

Reference Links:Domestic large model LLM selection and mainstream large model rapid use tutorials

  • ollama access

Reference Links:Ollama Simplifies Processes, OpenLLM Flexible Deployment, LocalAI Local Optimization

  • xinference Access

Xinference Practical Guide

3.0 Knowledge base construction

Template Description File format
General Files are consecutively chunked based on a preset chunk token number. DOCX, EXCEL, PPT, PDF, TXT, JPEG, JPG, PNG, TIF, GIF
Q&A EXCEL, CSV/TXT
Manual PDF
Table EXCEL, CSV/TXT
Paper PDF
Book DOCX, PDF, TXT
Laws DOCX, PDF, TXT
Presentation PDF, PPTX
Picture JPEG, JPG, PNG, TIF, GIF
One The entire document is chunked as one. DOCX, EXCEL, PDF, TXT
Knowledge Graph DOCX、EXCEL、PPT、IMAGE、PDF、TXT、MD、JSON、EML
  • "General" chunking methodology note
    Supported file formats are DOCX, EXCEL, PPT, IMAGE, PDF, TXT, MD, JSON, EML, HTML.

This method applies the simple method to block files:

The system will use a visual detection model to segment continuous text into multiple segments.
Next, these consecutive fragments are merged into chunks whose number of tokens does not exceed the "number of tokens".

  • "Q&A" Chunking Methodology Statement
    This block method supports excel and csv/txt file formats.

If the file is in excel format, it should consist of two columns No headings: one to ask the question, the other for the answer, and a question column before the answer column. Multiple sheets are acceptable as long as the columns are correctly structured.
If the file is in csv/txt format it is used as a delimiter to separate the question from the answer.
Lines of text that do not follow the above rules will be ignored, and each question-answer pair will be considered a unique section.

  • Description of the "Knowledge Graph" chunking methodology
    Supported file formats are DOCX, EXCEL, PPT, IMAGE, PDF, TXT, MD, JSON, EML

After the document is chunked, the chunks are used to extract a knowledge graph and mind map of the entire document. This method applies a simple approach to chunked documents: continuous text will be sliced into chunks of about 512 tokens.

Next, chunks are transferred to the LLM to extract the nodes and relationships of the knowledge graph and mind map.

  • Support for embedding model
    • BAAI/bge-large-zh-v1.5
    • BAAI/bge-base-en-v1.5
    • BAAI/bge-large-en-v1.5
    • BAAI/bge-small-en-v1.5
    • BAAI/bge-small-zh-v1.5
    • jinaai/jina-embeddings-v2-base-en
    • jinaai/jina-embeddings-v2-small-en
    • nomic-ai/nomic-embed-text-v1.5
    • sentence-transformers/all-MiniLM-L6-v2
    • maidalun1020/bce-embedding-base_v1

Smart Q&A & AI Orchestration Streaming

Agent Module-->Template Selection-->HR Recruiting Assistant

  • technical documentation
  • Quickstart
  • User guide
  • References
  • FAQ
<Previous Next>