🗺️ Sitemap

Add all web pages from an xml-sitemap use the SitemapLoader.

Install Sitemap addon

npm install @llm-tools/embedjs-loader-sitemap

Usage

import { RAGApplicationBuilder } from '@llm-tools/embedjs';
import { OpenAiEmbeddings } from '@llm-tools/embedjs-openai';
import { HNSWDb } from '@llm-tools/embedjs-hnswlib';
import { SitemapLoader } from '@llm-tools/embedjs-loader-sitemap';

const app = await new RAGApplicationBuilder()
.setModel(SIMPLE_MODELS.OPENAI_GPT4_O)
.setEmbeddingModel(new OpenAiEmbeddings())
.setVectorDatabase(new HNSWDb())
.build();

app.addLoader(new SitemapLoader({ url: '...' }))