geoffsee/open-web-agent-rs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add comprehensive documentation and finalize project license

Browse Source 
This commit introduces detailed documentation files, including installation, configuration, API, agents, and an index. It also updates the README with new sections and registers the MIT license for the project. These changes aim to improve clarity and ease of use for developers and contributors.
This commit is contained in:
geoffsee
2025-05-23 10:59:17 -04:00
parent 6420efc68a
commit 7e0b25da59
8 changed files with 540 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
Cargo.toml
View File

@@ -2,6 +2,7 @@
name = "web-agent-rs" name = "web-agent-rs"
version = "0.1.0" version = "0.1.0"
edition = "2021" edition = "2021"
license = "MIT"
[[bin]] [[bin]]
edition = "2021" edition = "2021"
@@ -30,4 +31,3 @@ tower-http = { version = "0.6.2", features = ["trace"] }
anyhow = "1.0.97" anyhow = "1.0.97"
base64 = "0.22.1" base64 = "0.22.1"
fips204 = "0.4.6" fips204 = "0.4.6"

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Geoff Seemueller
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

105
README.md
View File

@@ -1,5 +1,8 @@
# web-agent-rs # web-agent-rs
Hacky genaiscript host for integration into conversational AI applications. Remote genaiscript host for integration into conversational AI applications.
> This project is actively being developed to suit more use-cases, expect breaking changes.
### Disclaimer ### Disclaimer
This has not undergone a formal security assessment. You should do your own evaluation before using this. This has not undergone a formal security assessment. You should do your own evaluation before using this.
@@ -8,6 +11,102 @@ This has not undergone a formal security assessment. You should do your own eval
1. A chat client specifies the URL to this host in their environment. 1. A chat client specifies the URL to this host in their environment.
2. They send a request with their credentials to create a stream resource 2. They send a request with their credentials to create a stream resource
## Adding New Agents
## Docs This project allows you to create and integrate new agents that can perform various tasks. Here's how to add a new agent:
The [docs](./docs) folder is under construction
### 1. Create a GenAIScript File
Create a new `.genai.mts` file in the `packages/genaiscript/genaisrc/` directory. This file will contain the agent's logic.
Example structure of a GenAIScript file:
```typescript
import {SomeClient} from "@agentic/some-package";
import "./tools/some-tool.genai.mjs"
script({
title: "your_agent_name",
maxTokens: 8192,
cache: false,
tools: ["tool-name"],
});
def("USER_INPUT", env.vars.user_input);
$`You are an assistant that performs a specific task.
- Instruction 1
- Instruction 2
- Instruction 3`
```
### 2. Create a Rust Agent Function
Create a new Rust file in the `src/agents/` directory or add a function to an existing file. This function will be a wrapper that calls the GenAIScript file.
Example agent function:
```rust
use tokio::process::Child;
use tracing;
use crate::utils::utils::run_agent;
pub async fn your_agent_name(stream_id: &str, input: &str) -> Result<Child, String> {
run_agent(stream_id, input, "./packages/genaiscript/genaisrc/your-agent.genai.mts").await
}
```
### 3. Register the Agent in the Module
Add your agent to the `src/agents/mod.rs` file:
```rust
pub mod your_agent_name;
```
### 4. Register the Agent in the Webhook Handler
Add your agent to the match statement in the `handle_webhooks` function in `src/handlers/webhooks.rs`:
```
// In the handle_webhooks function
let cmd = match resource.as_str() {
"web-search" => search_agent(stream_id.as_str(), &*input).await,
"news-search" => news_agent(stream_id.as_str(), &*input).await,
// Add your agent here
"your-resource-name" => your_agent_name(stream_id.as_str(), &*input).await,
_ => {
tracing::error!("Unsupported resource type: {}", resource);
return StatusCode::BAD_REQUEST.into_response();
}
};
```
### 5. Configure Environment Variables
If your agent requires specific API keys or configuration, add them to the `ShimBinding` struct in `src/utils/utils.rs`.
## Existing Agents
The project currently includes the following agents:
- **Web Search**: Performs web searches using SearxNG
- **News Search**: Searches for news articles
- **Image Generator**: Generates images based on text prompts
- **Finance Query**: Provides financial information
- **Web Scrape**: Scrapes content from web pages
## Documentation
Comprehensive documentation is available in the [docs](./docs) directory:
- [Installation Guide](./docs/installation.md) - How to install and set up the project
- [Configuration Guide](./docs/configuration.md) - Environment variables and configuration options
- [API Documentation](./docs/api.md) - API endpoints and usage examples
- [Authentication](./docs/tokens.md) - Authentication system documentation
- [Agents Guide](./docs/agents.md) - How to create and use agents
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

138
docs/agents.md Normal file
View File

@@ -0,0 +1,138 @@
# Agents Documentation
## Overview
Agents are the core components of web-agent-rs that perform specific tasks. Each agent is implemented as a GenAIScript file that defines its behavior and a Rust function that wraps the script.
## Available Agents
The following agents are currently available:
| Agent Type | Description | Resource Name |
|------------|-------------|---------------|
| Web Search | Performs web searches using SearxNG | `web-search` |
| News Search | Searches for news articles | `news-search` |
| Image Generator | Generates images based on text prompts | `image-generator` |
| Finance Query | Provides financial information | `finance-query` |
| Web Scrape | Scrapes content from web pages | `web-scrape` |
## Creating a New Agent
### 1. Create a GenAIScript File
Create a new `.genai.mts` file in the `packages/genaiscript/genaisrc/` directory. This file will contain the agent's logic.
Example structure of a GenAIScript file:
```typescript
import {SomeClient} from "@agentic/some-package";
import "./tools/some-tool.genai.mjs"
script({
title: "your_agent_name",
maxTokens: 8192,
cache: false,
tools: ["tool-name"],
});
def("USER_INPUT", env.vars.user_input);
$`You are an assistant that performs a specific task.
- Instruction 1
- Instruction 2
- Instruction 3`
```
### 2. Create a Rust Agent Function
Create a new Rust file in the `src/agents/` directory or add a function to an existing file. This function will be a wrapper that calls the GenAIScript file.
Example agent function:
```rust
use tokio::process::Child;
use tracing;
use crate::utils::utils::run_agent;
pub async fn your_agent_name(stream_id: &str, input: &str) -> Result<Child, String> {
run_agent(stream_id, input, "./packages/genaiscript/genaisrc/your-agent.genai.mts").await
}
```
### 3. Register the Agent in the Module
Add your agent to the `src/agents/mod.rs` file:
```rust
pub mod your_agent_name;
```
### 4. Register the Agent in the Webhook Handler
Add your agent to the match statement in the `handle_webhooks_post` function in `src/handlers/webhooks.rs`:
```
// In the handle_webhooks_post function
let cmd = match resource.as_str() {
"web-search" => search_agent(stream_id.as_str(), &*input).await,
"news-search" => news_agent(stream_id.as_str(), &*input).await,
// Add your agent here
"your-resource-name" => your_agent_name(stream_id.as_str(), &*input).await,
_ => {
tracing::error!("Unsupported resource type: {}", resource);
return StatusCode::BAD_REQUEST.into_response();
}
};
```
### 5. Configure Environment Variables
If your agent requires specific API keys or configuration, add them to the `ShimBinding` struct in `src/utils/utils.rs`.
## Agent Tools
Agents can use various tools to perform their tasks. These tools are defined in the `packages/genaiscript/genaisrc/tools/` directory.
To use a tool in your agent, import it in your GenAIScript file:
```typescript
import "./tools/some-tool.genai.mjs"
```
And add it to the `tools` array in the `script` function:
```typescript
script({
title: "your_agent_name",
maxTokens: 8192,
cache: false,
tools: ["tool-name"],
});
```
## Testing Agents
You can test your agent by sending a request to the API:
```bash
curl -X POST https://your-server.com/api/webhooks \
-H "Authorization: Bearer <session_token>" \
-H "Content-Type: application/json" \
-d '{"resource": "your-resource-name", "input": "Your test input"}'
```
Then consume the stream to see the agent's response:
```bash
curl https://your-server.com/webhooks/<stream_id> \
-H "Authorization: Bearer <session_token>"
```
## Best Practices
1. Keep your GenAIScript files focused on a single task
2. Use appropriate tools for the task
3. Handle errors gracefully
4. Provide clear instructions in the agent's prompt
5. Test your agent thoroughly before deploying

112
docs/api.md Normal file
View File

@@ -0,0 +1,112 @@
# API Documentation
## Authentication
All API endpoints require authentication using a session token. The token should be included in the `Authorization` header as a Bearer token:
```
Authorization: Bearer <session_token>
```
For more information on authentication and token generation, see the [Authentication](./tokens.md) documentation.
## Endpoints
### Health Check
```
GET /health
```
Returns a simple "ok" response to indicate that the server is running.
**Response:**
```
200 OK
ok
```
### Create Stream Resource
```
POST /api/webhooks
```
Creates a new stream resource for an agent.
**Request Body:**
```json
{
"resource": "web-search",
"input": "What is the capital of France?"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `resource` | string | The type of agent to use (e.g., "web-search", "news-search", "image-generator", "finance-query", "web-scrape") |
| `input` | string | The input query or prompt for the agent |
**Response:**
```json
{
"stream_id": "abc123"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `stream_id` | string | The ID of the created stream resource |
### Consume Stream Resource
```
GET /webhooks/:stream_id
```
Consumes a stream resource, returning the agent's response as a server-sent event stream.
**Path Parameters:**
- `stream_id`: The ID of the stream resource to consume
**Response:**
Server-sent event stream with the agent's response.
## Available Agents
The following agent types are available for use with the `resource` field in the Create Stream Resource endpoint:
| Agent Type | Description |
|------------|-------------|
| `web-search` | Performs web searches using SearxNG |
| `news-search` | Searches for news articles |
| `image-generator` | Generates images based on text prompts |
| `finance-query` | Provides financial information |
| `web-scrape` | Scrapes content from web pages |
## Error Responses
| Status Code | Description |
|-------------|-------------|
| 400 | Bad Request - The request was malformed or missing required fields |
| 401 | Unauthorized - Authentication failed or token is invalid |
| 404 | Not Found - The requested resource was not found |
| 500 | Internal Server Error - An unexpected error occurred |
## Example Usage
### Creating a Stream Resource
```bash
curl -X POST https://your-server.com/api/webhooks \
-H "Authorization: Bearer <session_token>" \
-H "Content-Type: application/json" \
-d '{"resource": "web-search", "input": "What is the capital of France?"}'
```
### Consuming a Stream Resource
```bash
curl https://your-server.com/webhooks/abc123 \
-H "Authorization: Bearer <session_token>"
```

50
docs/configuration.md Normal file
View File

@@ -0,0 +1,50 @@
# Configuration Guide
## Environment Variables
web-agent-rs uses environment variables for configuration. These can be set in a `.env` file in the root directory or directly in your environment.
### Required Environment Variables
| Variable | Description | Example |
|----------|-------------|---------|
| `OPENAI_API_KEY` | API key for OpenAI services | `sk-...` |
| `BING_SEARCH_API_KEY` | API key for Bing Search | `...` |
| `TAVILY_API_KEY` | API key for Tavily | `tvly-...` |
| `GENAISCRIPT_MODEL_LARGE` | Large language model to use with GenAIScript | `gpt-4-turbo` |
| `GENAISCRIPT_MODEL_SMALL` | Small language model to use with GenAIScript | `gpt-3.5-turbo` |
| `SEARXNG_API_BASE_URL` | Base URL for SearxNG API | `https://searxng.example.com` |
### Optional Environment Variables
You may also want to configure these optional variables:
| Variable | Description | Default | Example |
|----------|-------------|---------|---------|
| `PORT` | Port for the server to listen on | `3006` | `8080` |
| `LOG_LEVEL` | Logging level | `info` | `debug`, `info`, `warn`, `error` |
## Docker Configuration
When running with Docker, you can pass environment variables using the `--env-file` flag or by setting them in the `compose.yml` file:
```yaml
services:
web-agent:
build:
context: .
dockerfile: Dockerfile
ports:
- "3006:3006"
environment:
- OPENAI_API_KEY=${OPENAI_API_KEY}
- BING_SEARCH_API_KEY=${BING_SEARCH_API_KEY}
- TAVILY_API_KEY=${TAVILY_API_KEY}
- GENAISCRIPT_MODEL_LARGE=${GENAISCRIPT_MODEL_LARGE}
- GENAISCRIPT_MODEL_SMALL=${GENAISCRIPT_MODEL_SMALL}
- SEARXNG_API_BASE_URL=${SEARXNG_API_BASE_URL}
```
## Authentication Configuration
The application uses FIPS204 signatures for authentication. See the [Authentication](./tokens.md) documentation for more details on configuring and using the authentication system.

45
docs/index.md Normal file
View File

@@ -0,0 +1,45 @@
# web-agent-rs Documentation
Welcome to the documentation for web-agent-rs, a GenAIScript host for integration into conversational AI applications.
## Table of Contents
- [Installation Guide](./installation.md) - How to install and set up the project
- [Configuration Guide](./configuration.md) - Environment variables and configuration options
- [API Documentation](./api.md) - API endpoints and usage examples
- [Authentication](./tokens.md) - Authentication system documentation
- [Agents Guide](./agents.md) - How to create and use agents
## Overview
web-agent-rs is a server that hosts GenAIScript agents for integration into conversational AI applications. It provides a simple API for creating and consuming stream resources that execute various agents to perform tasks like web search, news search, image generation, finance queries, and web scraping.
## Architecture
The application is built with Rust using the Axum web framework. It uses GenAIScript for defining agent behavior and provides a streaming API for consuming agent responses.
### Key Components
1. **Server** - The main application server that handles HTTP requests and responses
2. **Agents** - GenAIScript files that define agent behavior
3. **Handlers** - Rust functions that process HTTP requests
4. **Authentication** - FIPS204 signature-based authentication system
5. **Configuration** - Environment variables for configuring the application
## Getting Started
To get started with web-agent-rs, follow these steps:
1. Read the [Installation Guide](./installation.md) to set up the project
2. Configure the application using the [Configuration Guide](./configuration.md)
3. Learn how to use the API with the [API Documentation](./api.md)
4. Understand the authentication system with the [Authentication](./tokens.md) documentation
5. Create your own agents using the [Agents Guide](./agents.md)
## Security Considerations
Please note that this project has not undergone a formal security assessment. You should do your own evaluation before using it in production environments.
## Contributing
Contributions to web-agent-rs are welcome! Please feel free to submit issues and pull requests to improve the project.

71
docs/installation.md Normal file
View File

@@ -0,0 +1,71 @@
# Installation Guide
## Prerequisites
Before installing web-agent-rs, ensure you have the following prerequisites:
- [Rust](https://www.rust-lang.org/tools/install) (latest stable version)
- [Node.js](https://nodejs.org/) (for GenAIScript)
- [Bun](https://bun.sh/) (for package management)
- [Docker](https://www.docker.com/get-started) (optional, for containerized deployment)
## Environment Setup
1. Clone the repository:
```bash
git clone https://github.com/seemueller-io/open-web-agent-rs.git
cd open-web-agent-rs
```
2. Create a `.env` file in the root directory with the following variables:
```
OPENAI_API_KEY=your_openai_api_key
BING_SEARCH_API_KEY=your_bing_search_api_key
TAVILY_API_KEY=your_tavily_api_key
GENAISCRIPT_MODEL_LARGE=gpt-4-turbo
GENAISCRIPT_MODEL_SMALL=gpt-3.5-turbo
SEARXNG_API_BASE_URL=your_searxng_url
```
## Local Development
1. Install Rust dependencies:
```bash
cargo build
```
2. Install JavaScript dependencies:
```bash
bun install
```
3. Run the server:
```bash
cargo run
```
The server will start on `http://localhost:3006`.
## Docker Deployment
You can also run the application using Docker:
1. Build the Docker image:
```bash
docker build -t web-agent-rs -f Dockerfile .
```
2. Run the container:
```bash
docker run -p 3006:3006 --env-file .env web-agent-rs
```
Alternatively, you can use Docker Compose:
```bash
docker-compose up
```
## Configuration Options
The application can be configured using environment variables. See the [Configuration](./configuration.md) documentation for more details.