AI Agentä¸RAGç»åï¼æ建ç¥è¯å¢å¼ºåæºè½ä½å¤§è¯è¨æ¨¡åï¼LLMï¼è½ç¶ç¥è¯å¹¿åï¼ä½å¨é¢å¯¹ç¹å®é¢åé®é¢æ¶å¾å¾åä¸ä»å¿ââè¦ä¹äº§çå¹»è§ï¼è¦ä¹ç¥è¯å·²ç»è¿æ¶ãæ£ç´¢å¢å¼ºçæï¼RAGï¼ææ¯æ°å¥½ä¸º AI Agent è¡¥å äºå®æ¶ç²¾åç¥è¯è¿ä¸å ³é®è½åãæ¬æå°ç³»ç»è®²è§£ RAG åçãåéæ£ç´¢æºå¶ãææ¡£å¤ççç¥ä»¥åä¸ä¸ªå®æ´ç代ç å®ç°ãä¸ãRAG æ ¸å¿åç1.1 ä»ä¹æ¯ RAGRAGï¼Retrieval-Augmented Generationï¼çæ ¸å¿ææ³æ¯ï¼å¨çæåçåï¼å ä»å¤é¨ç¥è¯åºæ£ç´¢ç¸å ³ä¿¡æ¯ï¼å°æ£ç´¢ç»æä½ä¸ºä¸ä¸æè¾å ¥å¤§è¯è¨æ¨¡åï¼å¼å¯¼å ¶çææ´åç¡®çåçãç¸æ¯å¾®è° LLMï¼RAG å ·æä¸ä¸ªä¼å¿ï¼ç¥è¯å®æ¶æ§ï¼æ ééæ°è®ç»å³å¯æ´æ°ç¥è¯åºå¯æº¯æºï¼æ¯ä¸ªåçé½å¯è¿½æº¯å°å ·ä½æ¥æºå¹»è§æå¶ï¼æ£ç´¢å°ççå®ä¿¡æ¯ææ约æ模åè¾åº1.2 æ åæµç¨ä¸ä¸ªå ¸åç RAG ç³»ç»å å«ï¼ææ¡£å è½½ â ææ¡£åå â åéåµå ¥ â åéåå¨ â æ£ç´¢ â éæåº â æ示æ建 â çæåçãäºãææ¡£ååçç¥2.1 ååçç¥å¯¹æ¯| çç¥ | 说æ | éç¨åºæ¯ | |------|------|----------| | åºå®åç¬¦æ° | æåºå®é¿åº¦åå | éç¨åºæ¯ï¼å®ç°ç®å | | éå½å符åå | å æ段è½ï¼åæå¥ååå | ç»æåææ¡£ | | éå åå | ç¸é» chunk ä¿çéå åºå | é¿å è¾¹çä¿¡æ¯ä¸¢å¤± |2.2 ååå®ç°import re from typing import List class DocumentChunker: def __init__(self, chunk_size: int 500, chunk_overlap: int 100): self.chunk_size chunk_size self.chunk_overlap chunk_overlap def split_text(self, text: str) - List[str]: paragraphs [p.strip() for p in text.split(\n\n) if p.strip()] chunks [] for p in paragraphs: if len(p) self.chunk_size: chunks.append(p) else: chunks.extend(self._split_by_sentences(p)) return self._merge_small_chunks(chunks) def _split_by_sentences(self, text: str) - List[str]: sentences re.split(r(?[ãï¼.?!])\s, text) sentences [s.strip() for s in sentences if s.strip()] chunks, current [], for s in sentences: if len(current) len(s) self.chunk_size: current s else: if current: chunks.append(current) overlap current[-self.chunk_overlap:] if self.chunk_overlap 0 else current overlap s if current: chunks.append(current) return chunks def _merge_small_chunks(self, chunks: List[str]) - List[str]: merged, current [], for chunk in chunks: if len(current) len(chunk) self.chunk_size: current chunk else: if current: merged.append(current) current chunk if current: merged.append(current) return mergedä¸ãåéæ£ç´¢ä¸åå¨3.1 åéæ°æ®åºææ¬éè¿åµå ¥æ¨¡å转æ¢ä¸ºé«ç»´åéï¼åå¨å¨åéæ°æ®åºä¸ã常è§çå¼æºæ¹æ¡å æ¬ BGE-M3ãE5-Mistral çãä¸é¢ç¨çº¯ Python å®ç°ä¸ä¸ªå ååéåºï¼ç产ç¯å¢å»ºè®®ä½¿ç¨ Milvus æ Chromaï¼ï¼import numpy as np from typing import List, Tuple, Dict from dataclasses import dataclass dataclass class Document: id: str content: str embedding: np.ndarray metadata: dict class SimpleVectorStore: def __init__(self, embedding_dim: int
