Built in
πΊοΈ Sitemap
Add all web pages from an xml-sitemap use the SitemapLoader
.
Install Sitemap addon
npm install @llm-tools/embedjs-loader-sitemap
Usage
import { RAGApplicationBuilder } from '@llm-tools/embedjs';
import { OpenAiEmbeddings } from '@llm-tools/embedjs-openai';
import { HNSWDb } from '@llm-tools/embedjs-hnswlib';
import { SitemapLoader } from '@llm-tools/embedjs-loader-sitemap';
const app = await new RAGApplicationBuilder()
.setModel(SIMPLE_MODELS.OPENAI_GPT4_O)
.setEmbeddingModel(new OpenAiEmbeddings())
.setVectorDb(new HNSWDb())
.build();
app.addLoader(new SitemapLoader({ url: '...' }))
Was this page helpful?