Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ Create a multi-agent research workflow where different AI agents collaborate to
- [Custom Endpoints](./with-custom-endpoints) — Add bespoke REST endpoints alongside agent/workflow routes.
- [Dynamic Parameters](./with-dynamic-parameters) — Validate and inject runtime parameters into agents with Zod.
- [Dynamic Prompts](./with-dynamic-prompts) — Build prompts from templates and live data programmatically.
- [Firecrawl Search](./with-firecrawl-search) — Search the web and read pages as clean Markdown with Firecrawl.
- [Google AI](./with-google-ai) — Use Google Gemini models via the AI SDK provider.
- [Google Drive (MCP)](./with-google-drive-mcp) — Browse and read Drive files through a Google Drive MCP server.
- [Google Vertex AI](./with-google-vertex-ai) — Connect agents to Vertex AI models in your GCP project.
Expand Down
6 changes: 6 additions & 0 deletions examples/with-firecrawl-search/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# OpenAI API Key (required)
OPENAI_API_KEY=your_openai_api_key_here

# Firecrawl API Key (required)
# Get one at https://www.firecrawl.dev/signin?utm_source=voltagent&utm_medium=integration&redirect=%2Fapp%2Fapi-keys
Comment thread
JuampiHernandez marked this conversation as resolved.
FIRECRAWL_API_KEY=your_firecrawl_api_key_here
90 changes: 90 additions & 0 deletions examples/with-firecrawl-search/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# VoltAgent with Firecrawl Search

This example shows how to give a VoltAgent agent web search and page reading with Firecrawl. [Firecrawl search](https://www.firecrawl.dev/search?utm_source=voltagent&utm_medium=integration) returns web results and can include each page's content as Markdown in the same call. A second tool fetches any URL as Markdown.

## Features

- **Web Search**: Find current pages on any topic, with titles, URLs and descriptions
- **Page Content in Search**: Optionally get each result's page content as Markdown in the same call
- **Page Reading**: Fetch any URL as clean Markdown
- **Cited Answers**: The agent searches first, reads the most relevant pages when needed, and cites the URLs it used

## Prerequisites

1. **Firecrawl API Key**: Get your API key from the [Firecrawl dashboard](https://www.firecrawl.dev/signin?utm_source=voltagent&utm_medium=integration&redirect=%2Fapp%2Fapi-keys)
2. **OpenAI API Key**: For the AI model integration

## Setup

1. **Install dependencies**:

```bash
pnpm install
```

2. **Configure environment variables**:
Copy the example environment file and add your API keys:

```bash
cp .env.example .env
```

Then edit the `.env` file with your actual API keys:

```env
OPENAI_API_KEY=your_actual_openai_api_key
FIRECRAWL_API_KEY=your_actual_firecrawl_api_key
```

3. **Run the example**:
```bash
pnpm dev
```

## Usage

The agent will be available at `http://localhost:3141`. Try queries like:

- "What changed in the latest Node.js LTS release?"
- "Find the VoltAgent docs page about tools and summarize how to create one."
- "Read https://voltagent.dev/docs/ and list the main sections."

## Tools Available

### 1. Firecrawl Search Tool

- **Purpose**: Search the web, optionally with page content
- **Endpoint**: `POST https://api.firecrawl.dev/v2/search`
- **Parameters**:
- `query`: Search query string
- `limit`: Number of results to return (default: 5, max: 10)
- `scrapeContent`: Include each result's page content as Markdown (default: false)

### 2. Firecrawl Scrape Tool

- **Purpose**: Fetch a single URL as Markdown
- **Endpoint**: `POST https://api.firecrawl.dev/v2/scrape`
- **Parameters**:
- `url`: URL of the page to fetch
- `onlyMainContent`: Drop navigation, headers and footers (default: true)

Page content is truncated so it fits in the model's context: 5,000 characters per search result and 20,000 characters for a scraped page. You can adjust both limits with `MAX_SEARCH_MARKDOWN_LENGTH` and `MAX_SCRAPE_MARKDOWN_LENGTH` in `src/tools.ts`. Returning page content with search uses more Firecrawl credits per result.

See the [Firecrawl search docs](https://docs.firecrawl.dev/features/search?utm_source=voltagent&utm_medium=integration) for more options, such as `sources`, `location` and `tbs`.

## Configuration

The agent is configured with:

- **Model**: GPT-4o-mini
- **Port**: 3141 (configurable)
- **Logging**: Pino logger with info level
- **Memory**: LibSQL
- **Tools**: Firecrawl search and scrape tools

## Troubleshooting

- **API Key Issues**: Ensure both `OPENAI_API_KEY` and `FIRECRAWL_API_KEY` are set
- **401 errors from Firecrawl**: Check that the key in `.env` is correct and active
- **402 or 429 errors from Firecrawl**: Your account is out of credits or over its rate limit; check your usage in the Firecrawl dashboard
- **Port Conflicts**: Change the port in the server configuration if needed
38 changes: 38 additions & 0 deletions examples/with-firecrawl-search/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"name": "voltagent-example-with-firecrawl-search",
"author": "",
"dependencies": {
"@voltagent/cli": "^0.1.21",
"@voltagent/core": "^2.11.0",
"@voltagent/internal": "^1.0.3",
"@voltagent/libsql": "^2.1.2",
"@voltagent/logger": "^2.0.2",
"@voltagent/server-hono": "^2.0.14",
"ai": "^6.0.0",
"zod": "^3.25.76"
},
"devDependencies": {
"@types/node": "^24.2.1",
"tsx": "^4.21.0",
"typescript": "^5.8.2"
},
"keywords": [
"agent",
"ai",
"voltagent"
],
"license": "MIT",
"private": true,
"repository": {
"type": "git",
"url": "https://github.com/VoltAgent/voltagent.git",
"directory": "examples/with-firecrawl-search"
},
"scripts": {
"build": "tsc",
"dev": "tsx watch --env-file=.env ./src",
"start": "node dist/index.js",
"volt": "volt"
},
"type": "module"
}
40 changes: 40 additions & 0 deletions examples/with-firecrawl-search/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import { Agent, Memory, VoltAgent } from "@voltagent/core";
import { LibSQLMemoryAdapter } from "@voltagent/libsql";
import { createPinoLogger } from "@voltagent/logger";
import { honoServer } from "@voltagent/server-hono";
import { firecrawlScrapeTool, firecrawlSearchTool } from "./tools.js";

// Create logger
const logger = createPinoLogger({
name: "firecrawl-search-agent",
level: "info",
});

// Create Memory instance backed by LibSQL
const memory = new Memory({
storage: new LibSQLMemoryAdapter(),
});

// Create the research agent with Firecrawl tools
const researchAgent = new Agent({
name: "Web Research Agent",
instructions: `You answer questions with current information from the web.

1. Use firecrawlSearch to find relevant pages.
2. If the search descriptions are not enough to answer, use firecrawlScrape on the one or two most relevant URLs to read the full page.
3. Answer from what you found and cite the URLs you used.

If a tool returns an error, tell the user what went wrong.`,
model: "openai/gpt-4o-mini",
tools: [firecrawlSearchTool, firecrawlScrapeTool],
memory,
});

// Initialize the VoltAgent with the research agent and server
new VoltAgent({
agents: {
researchAgent,
},
logger,
server: honoServer(),
});
159 changes: 159 additions & 0 deletions examples/with-firecrawl-search/src/tools.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
import { createTool } from "@voltagent/core";
import { safeStringify } from "@voltagent/internal";
import { z } from "zod";

const FIRECRAWL_API_URL = "https://api.firecrawl.dev/v2";
const REQUEST_TIMEOUT_MS = 120_000;

// Keep page content small enough for the model's context window
const MAX_SEARCH_MARKDOWN_LENGTH = 5_000;
const MAX_SCRAPE_MARKDOWN_LENGTH = 20_000;

const missingApiKeyResult = {
success: false,
error: "Firecrawl API key not configured",
message: "Firecrawl API key is required. Please set FIRECRAWL_API_KEY environment variable.",
};

const truncateMarkdown = (markdown: string, maxLength: number) =>
markdown.length > maxLength ? `${markdown.slice(0, maxLength)}\n\n[Content truncated]` : markdown;

const callFirecrawl = async (path: string, apiKey: string, body: Record<string, unknown>) => {
const response = await fetch(`${FIRECRAWL_API_URL}${path}`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
},
body: safeStringify({ ...body, origin: "voltagent" }),
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
});
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.

// Keep the HTTP status error for failed responses, but surface invalid JSON on success
const data = await response.json().catch((error) => {
if (response.ok) throw error;
return {};
});

if (!response.ok) {
const detail = typeof data.error === "string" ? `: ${data.error}` : "";
throw new Error(`Firecrawl API error ${response.status}${detail}`);
}

return data;
};

export const firecrawlSearchTool = createTool({
name: "firecrawlSearch",
description:
"Search the web with Firecrawl and return titles, URLs and descriptions, optionally with the page content as Markdown.",
parameters: z.object({
query: z.string().describe("Search query (e.g., 'latest TypeScript release notes')"),
limit: z
.number()
.int()
.min(1)
.max(10)
.optional()
.describe("Number of results to return (default: 5, max: 10)"),
scrapeContent: z
.boolean()
.optional()
.describe("Also return each result's page content as Markdown (default: false)"),
}),
execute: async ({ query, limit = 5, scrapeContent = false }) => {
try {
console.log("Firecrawl search:", query);

const apiKey = process.env.FIRECRAWL_API_KEY;
if (!apiKey) {
return missingApiKeyResult;
}

const data = await callFirecrawl("/search", apiKey, {
query,
limit,
...(scrapeContent ? { scrapeOptions: { formats: ["markdown"] } } : {}),
});

const results = (data.data?.web ?? []).map((item: any) => ({
title: item.title || "No Title",
url: item.url || "",
description: item.description || "",
...(item.markdown
? { markdown: truncateMarkdown(item.markdown, MAX_SEARCH_MARKDOWN_LENGTH) }
: {}),
}));

return {
success: true,
query,
results,
totalResults: results.length,
message:
results.length > 0
? `Found ${results.length} web results for "${query}".`
: `No web results found for "${query}". Try a different search query.`,
};
} catch (error) {
console.error("Firecrawl search error:", error);
return {
success: false,
error: error instanceof Error ? error.message : "Firecrawl search failed",
message: `Firecrawl search failed: ${error instanceof Error ? error.message : "Unknown error"}`,
};
}
},
});

export const firecrawlScrapeTool = createTool({
name: "firecrawlScrape",
description: "Fetch a URL with Firecrawl and return the page as clean Markdown.",
parameters: z.object({
url: z.string().describe("URL of the page to fetch"),
onlyMainContent: z
.boolean()
.optional()
.describe("Drop navigation, headers and footers (default: true)"),
}),
execute: async ({ url, onlyMainContent = true }) => {
try {
console.log("Firecrawl scrape:", url);

const apiKey = process.env.FIRECRAWL_API_KEY;
if (!apiKey) {
return missingApiKeyResult;
}

const data = await callFirecrawl("/scrape", apiKey, {
url,
formats: ["markdown"],
onlyMainContent,
});

const markdown = data.data?.markdown;
if (!markdown) {
return {
success: false,
error: "No content returned",
message: `No content could be fetched from ${url}`,
};
}

return {
success: true,
url,
title: data.data.metadata?.title || "No Title",
markdown: truncateMarkdown(markdown, MAX_SCRAPE_MARKDOWN_LENGTH),
message: `Fetched ${url} as Markdown.`,
};
} catch (error) {
console.error("Firecrawl scrape error:", error);
return {
success: false,
error: error instanceof Error ? error.message : "Firecrawl scrape failed",
message: `Firecrawl scrape failed: ${error instanceof Error ? error.message : "Unknown error"}`,
};
}
},
});
17 changes: 17 additions & 0 deletions examples/with-firecrawl-search/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"outDir": "dist",
"esModuleInterop": true,
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
"forceConsistentCasingInFileNames": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"skipLibCheck": true,
"declaration": true,
"declarationMap": true,
"sourceMap": true
},
"include": ["src/**/*"]
}
Loading