geoffsee/predict-otron-9001
1
0
Fork 0
mirror of https://github.com/geoffsee/predict-otron-9001.git synced 2025-09-08 22:46:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

reorg + update docs with new paths

Browse Source
This commit is contained in:
geoffsee
2025-09-04 12:27:13 -04:00
parent 400c70f17d
commit ff55d882c7
43 changed files with 493 additions and 182 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
README.md
View File

@@ -53,14 +53,17 @@ The project uses a 9-crate Rust workspace plus TypeScript components:
crates/
├── predict-otron-9000/ # Main orchestration server (Rust 2024)
├── inference-engine/ # Multi-model inference orchestrator (Rust 2021)
├── embeddings-engine/ # FastEmbed embeddings service (Rust 2024)
└── chat-ui/ # WASM web frontend (Rust 2021)
integration/
├── cli/ # CLI client crate (Rust 2024)
│ └── package/
│ └── cli.ts # TypeScript/Bun CLI client
├── gemma-runner/ # Gemma model inference via Candle (Rust 2021)
├── llama-runner/ # Llama model inference via Candle (Rust 2021)
├── embeddings-engine/ # FastEmbed embeddings service (Rust 2024)
├── chat-ui/ # WASM web frontend (Rust 2021)
├── helm-chart-tool/ # Kubernetes deployment tooling (Rust 2024)
└── cli/ # CLI client crate (Rust 2024)
└── package/
└── cli.ts # TypeScript/Bun CLI client
└── utils/ # Shared utilities (Rust 2021)
```
### Service Architecture
@@ -160,16 +163,16 @@ cd crates/chat-ui
#### TypeScript CLI Client
```bash
# List available models
cd crates/cli/package && bun run cli.ts --list-models
cd integration/cli/package && bun run cli.ts --list-models
# Chat completion
cd crates/cli/package && bun run cli.ts "What is the capital of France?"
cd integration/cli/package && bun run cli.ts "What is the capital of France?"
# With specific model
cd crates/cli/package && bun run cli.ts --model gemma-3-1b-it --prompt "Hello, world!"
cd integration/cli/package && bun run cli.ts --model gemma-3-1b-it --prompt "Hello, world!"
# Show help
cd crates/cli/package && bun run cli.ts --help
cd integration/cli/package && bun run cli.ts --help
```
## API Usage
@@ -464,7 +467,7 @@ curl -s http://localhost:8080/v1/models | jq
**CLI client test:**
```bash
cd crates/cli/package && bun run cli.ts "What is 2+2?"
cd integration/cli/package && bun run cli.ts "What is 2+2?"
```
**Web frontend:**