geoffsee/manifold-workflow-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rename LLM service to Intent service and refactor relevant code

Browse Source 
Refactored the `DummyLlmService` to `DummyIntentService` across the codebase for clarity and consistency. Updated variable names, method calls, and documentation to align with this change. This improves code readability and better reflects the service's purpose.
This commit is contained in:
Geoff Seemueller
2024-11-14 15:26:51 -05:00
parent 94ae9c487a
commit 2712ffecf4
4 changed files with 90 additions and 304 deletions
Download Patch File Download Diff File Expand all files Collapse all files

302
README.md
View File

@@ -1,10 +1,9 @@
# workflow-function-manifold
> A TypeScript/JavaScript library for building dynamic, LLM-driven workflows
> using a region-based execution model
> A TypeScript/JavaScript library for building dynamic, LLM-driven workflows using a region-based execution model.
![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)
![Node Version](https://img.shields.io/badge/node-%3E%3D%2014.0.0-brightgreen)
![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)
![Node Version](https://img.shields.io/badge/Node-%3E%3D14-green.svg)
CLI: `npx workflow-function-manifold`
@@ -27,7 +26,7 @@ CLI: `npx workflow-function-manifold`
`workflow-function-manifold` enables you to create dynamic workflows that:
- Navigate between different execution regions based on LLM-interpreted intents.
- Execute operations within regions based on state and context.
- Execute operations within regions using state and context.
- Maintain workflow state across operations.
- Support flexible region-to-region connections.
@@ -36,7 +35,7 @@ graph TD
MF[Workflow Function Manifold] -->|Initialize| CR[Current Region]
MF -->|Navigate| AR[Adjacent Regions]
MF -->|Execute| OP[Operators]
OP -->|Use| LLM[LLM Service]
OP -->|Use| Intent[Intent Service]
LLM -->|Match| INT[Intent]
INT --> OP
OP -->|Update| ST[Workflow State]
@@ -45,54 +44,21 @@ graph TD
NR -->|If Valid| CR
CR -->|Continue| OP
NR -->|If Invalid| END[End]
style MF fill:#000000,stroke:#FFFFFF,stroke-width:4px,color:#ffffff
style CR fill:#222222,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
style AR fill:#222222,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
style OP fill:#333333,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
style LLM fill:#333333,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
style INT fill:#333333,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
style NR fill:#333333,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
style END fill:#222222,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
style ST fill:#444444,stroke:#FFFFFF,stroke-width:2px,color:#ffffff
```
## Installation
Use as a library:
Install the library via npm:
```bash
npm install workflow-function-manifold
```
For a very basic demonstration, invoke the CLI via npx:
```shell
$ npx workflow-function-manifold
🔄 Executing Workflow...
📍 Step: Nested: Data Validation
Prompt: "validate the input"
✓ Validating data structure
✅ Execution complete
📍 Step: Nested: Data Cleaning
Prompt: "clean the data"
↪ Navigation successful
✓ Cleaning data
✅ Execution complete
📍 Step: Main: Data Analysis
Prompt: "analyze the results"
↪ Navigation successful
✓ Performing data analysis
✅ Execution complete
📍 Step: Main: Data Transformation
Prompt: "transform the output"
↪ Navigation successful
✓ Transforming results
✅ Execution complete
🎉 Workflow Demonstration Complete!
Run a basic demonstration using the CLI:
```bash
npx workflow-function-manifold
```
## Quick Start
@@ -102,147 +68,91 @@ import {
WorkflowFunctionManifold,
ManifoldRegion,
WorkflowOperator,
DummyLlmService,
DummyIntentService,
} from 'workflow-function-manifold';
// Initialize the manifold with an LLM service
const llm = new DummyLlmService();
const llm = new DummyIntentService();
const manifold = new WorkflowFunctionManifold(llm);
// Create an operator
const analysisOperator = new WorkflowOperator('analysis', async state => {
console.log('Analyzing data...');
return { ...state, analyzed: true };
});
const analysisOperator = new WorkflowOperator('analysis', async state => ({
...state,
analyzed: true,
}));
// Create and connect regions
const analysisRegion = new ManifoldRegion('analysis', [analysisOperator]);
manifold.addRegion(analysisRegion);
// Execute workflow
await manifold.navigate('analyze the data'); // This navigates to the 'analysis' region
await manifold.executeWorkflow('analyze the data'); // Executes the operator in the 'analysis' region
console.log(manifold.state); // { analyzed: true }
await manifold.navigate('analyze the data');
await manifold.executeWorkflow('analyze the data');
```
> **Note:** The `DummyLlmService` matches specific keywords in prompts. Ensure
> your prompts contain keywords like `'analyze'`, `'process'`, or `'transform'`
> for the default operators to function.
> **Note:** `DummyIntentService` uses basic keyword matching. Include keywords like `'analyze'`, `'process'`, or `'transform'` for default operators to work.
## Core Components
### WorkflowFunctionManifold
### `WorkflowFunctionManifold`
The main class that orchestrates workflow execution.
The main orchestrator for workflow execution.
```javascript
const manifold = new WorkflowFunctionManifold(llmService);
// Add regions
manifold.addRegion(region);
// Navigate between regions
await manifold.navigate(prompt);
// Execute operations
await manifold.executeWorkflow(prompt);
```
### ManifoldRegion
### `ManifoldRegion`
Represents a workflow region containing operators and connections to other
regions.
Represents a workflow region containing operators and connections to other regions.
```javascript
const region = new ManifoldRegion('regionName', [operator1, operator2]);
// Connect regions
region.connectTo(otherRegion);
// Add operators
region.addOperator(newOperator);
```
### WorkflowOperator
### `WorkflowOperator`
Defines operations that can be executed within regions.
Defines an operation that can be executed within a region.
```javascript
const operator = new WorkflowOperator('operatorName', async state => {
// Modify state
return newState;
});
const operator = new WorkflowOperator('operatorName', async state => newState);
```
### DummyLlmService
### `DummyIntentService`
A simple LLM service simulation for intent matching.
A basic intent-matching service.
```javascript
const llm = new DummyLlmService();
const intent = await llm.query('analyze the data');
// Returns: { confidence: 0.9, action: 'analysis' }
```
## Nested Manifolds
### NestedManifoldRegion
A `NestedManifoldRegion` allows embedding a `WorkflowFunctionManifold` inside a region. This enables creating hierarchical workflows where regions themselves contain sub-workflows.
```javascript
import {
WorkflowFunctionManifold,
NestedManifoldRegion,
ManifoldRegion,
WorkflowOperator,
DummyLlmService,
} from 'workflow-function-manifold';
// Sub-workflow
const nestedLlm = new DummyLlmService();
const nestedManifold = new WorkflowFunctionManifold(nestedLlm);
const nestedOperator = new WorkflowOperator('nestedOperation', async state => {
return { ...state, nested: true };
});
const nestedRegion = new ManifoldRegion('nestedRegion', [nestedOperator]);
nestedManifold.addRegion(nestedRegion);
// Main workflow
const mainLlm = new DummyLlmService();
const mainManifold = new WorkflowFunctionManifold(mainLlm);
const nestedManifoldRegion = new NestedManifoldRegion('nestedManifoldRegion', nestedManifold);
mainManifold.addRegion(nestedManifoldRegion);
await mainManifold.navigate('perform nested operation');
await mainManifold.executeWorkflow('perform nested operation');
const intentService = new DummyIntentService();
const intent = await intentService.query('analyze the data');
```
## Complete Example
Here's a full example demonstrating a three-stage workflow:
Here's a complete workflow demonstration:
```javascript
async function createWorkflow() {
const llm = new DummyLlmService();
const manifold = new WorkflowFunctionManifold(llm);
const intentService = new DummyIntentService();
const manifold = new WorkflowFunctionManifold(intentService);
// Create operators
const analysisOp = new WorkflowOperator('analysis', async state => {
return { ...state, analyzed: true };
});
const processingOp = new WorkflowOperator('processing', async state => {
return { ...state, processed: true };
});
const transformOp = new WorkflowOperator('transformation', async state => {
return { ...state, transformed: true };
});
const analysisOp = new WorkflowOperator('analysis', async state => ({
...state,
analyzed: true,
}));
const processingOp = new WorkflowOperator('processing', async state => ({
...state,
processed: true,
}));
const transformOp = new WorkflowOperator('transformation', async state => ({
...state,
transformed: true,
}));
// Create and connect regions
const analysisRegion = new ManifoldRegion('analysis', [analysisOp]);
const processingRegion = new ManifoldRegion('processing', [processingOp]);
const transformRegion = new ManifoldRegion('transformation', [transformOp]);
@@ -250,7 +160,6 @@ async function createWorkflow() {
analysisRegion.connectTo(processingRegion);
processingRegion.connectTo(transformRegion);
// Add regions to manifold
manifold.addRegion(analysisRegion);
manifold.addRegion(processingRegion);
manifold.addRegion(transformRegion);
@@ -258,37 +167,31 @@ async function createWorkflow() {
return manifold;
}
// Execute workflow
const manifold = await createWorkflow();
const prompts = [
'analyze the data',
'process the results',
'transform the output',
];
const prompts = ['analyze the data', 'process the results', 'transform the output'];
for (const prompt of prompts) {
await manifold.navigate(prompt);
await manifold.executeWorkflow(prompt);
}
console.log(manifold.state); // Final state after all operations
```
## API Reference
### WorkflowFunctionManifold
### `WorkflowFunctionManifold`
#### Constructor
- `constructor(llmService: LLMService)`
- `constructor(intentService: IntentService)`
#### Methods
- `addRegion(region: ManifoldRegion): void`
- `async navigate(prompt: string): Promise<boolean>`
- `async executeWorkflow(prompt: string): Promise<boolean>`
- `addRegion(region: ManifoldRegion | NestedManifoldRegion): void`
- `navigate(prompt: string): Promise<boolean>`
- `executeWorkflow(prompt: string): Promise<boolean>`
### ManifoldRegion
### `ManifoldRegion`
#### Constructor
@@ -298,104 +201,37 @@ console.log(manifold.state); // Final state after all operations
- `addOperator(operator: WorkflowOperator): void`
- `connectTo(region: ManifoldRegion): void`
- `async getValidOperators(state: any): Promise<WorkflowOperator[]>`
### WorkflowOperator
#### Constructor
- `constructor(name: string, operation: (state: any) => Promise<any>)`
#### Methods
- `async execute(state: any): Promise<any>`
### NestedManifoldRegion
- Extends `ManifoldRegion` and embeds a `WorkflowFunctionManifold`.
## State Management
The workflow maintains state across operations. Each operator can access and
modify the state:
Operators access and modify the state persistently:
```javascript
const operator = new WorkflowOperator('example', async state => {
// Access existing state
const { previousValue } = state;
// Return modified state
return {
...state,
newValue: 'updated',
};
});
```
The updated state persists across operators and regions.
## LLM Integration
The system uses LLM services for intent recognition. The default
`DummyLlmService` provides basic intent matching, but you can implement your own
LLM service:
```javascript
class CustomLLMService {
async query(prompt) {
// Implement custom LLM logic
return {
confidence: number,
action: string,
};
}
}
const operator = new WorkflowOperator('example', async state => ({
...state,
newValue: 'updated',
}));
```
## Error Handling
This library includes basic error handling to ensure workflows run smoothly,
even when unexpected issues arise.
### Navigation Errors
- If a prompt doesn't match a valid adjacent region:
- Logs a warning: `No valid region found for prompt: "<prompt>"`.
- Logs warnings for unmatched prompts.
### Operator Execution Errors
- If no matching operator is found:
- Logs a warning: `No matching operator found for intent: <intent>`.
- Logs warnings for unmatched operators.
### LLM Query Errors
- If issues arise during LLM queries:
- Logs an error:
`Error during navigation for prompt "<prompt>": <error message>`.
### Example Error Logging
```javascript
try {
await manifold.navigate('unknown command');
} catch (error) {
console.error('Critical navigation error:', error);
}
try {
await manifold.executeWorkflow('perform unknown action');
} catch (error) {
console.error('Critical execution error:', error);
}
```
---
## Contributing
1. Fork the repository.
2. Create your feature branch: `git checkout -b feature/my-feature`.
3. Commit your changes: `git commit -am 'Add new feature'`.
4. Push to the branch: `git push origin feature/my-feature`.
5. Submit a pull request.
2. Create a new branch: `git checkout -b feature/my-feature`.
3. Commit changes: `git commit -m "Add my feature"`.
4. Push the branch: `git push origin feature/my-feature`.
5. Open a pull request.
## License

41
bin.js
View File

@@ -1,7 +1,7 @@
#!/usr/bin/env node
import {
DummyLlmService,
DummyIntentService,
ManifoldRegion,
WorkflowFunctionManifold,
WorkflowOperator,
@@ -9,45 +9,33 @@ import {
} from './lib.js';
async function demonstrateNestedManifold() {
console.log('\n🚀 Starting Nested Manifold Demonstration\n');
console.log('📦 Creating Secondary Manifold...');
const nestedLlm = new DummyLlmService();
const nestedManifold = new WorkflowFunctionManifold(nestedLlm);
const nestedIntentService = new DummyIntentService();
const nestedManifold = new WorkflowFunctionManifold(nestedIntentService);
const validateOp = new WorkflowOperator('validation', async state => {
console.log(' ✓ Validating data structure');
return { ...state, validated: true };
});
const cleanOp = new WorkflowOperator('cleaning', async state => {
console.log(' ✓ Cleaning data');
return { ...state, cleaned: true };
});
const validateRegion = new ManifoldRegion('validation', [validateOp]);
const cleanRegion = new ManifoldRegion('cleaning', [cleanOp]);
// Set up nested manifold regions
validateRegion.connectTo(cleanRegion);
nestedManifold.addRegion(validateRegion);
nestedManifold.addRegion(cleanRegion);
console.log('📦 Creating Primary Manifold...');
const mainLlm = new DummyLlmService();
const mainManifold = new WorkflowFunctionManifold(mainLlm);
const mainIntentService = new DummyIntentService();
const mainManifold = new WorkflowFunctionManifold(mainIntentService);
const analysisOp = new WorkflowOperator('analysis', async state => {
console.log(' ✓ Performing data analysis');
return { ...state, analyzed: true };
});
const transformOp = new WorkflowOperator('transformation', async state => {
console.log(' ✓ Transforming results');
return { ...state, transformed: true };
});
// Set up main manifold regions
const nestedPreprocessRegion = new NestedManifoldRegion('preprocessing', nestedManifold);
const analysisRegion = new ManifoldRegion('analysis', [analysisOp]);
const transformRegion = new ManifoldRegion('transformation', [transformOp]);
@@ -59,8 +47,6 @@ async function demonstrateNestedManifold() {
mainManifold.addRegion(analysisRegion);
mainManifold.addRegion(transformRegion);
console.log('\n🔄 Executing Workflow...\n');
const prompts = [
{ text: 'validate the input', description: 'Nested: Data Validation' },
{ text: 'clean the data', description: 'Nested: Data Cleaning' },
@@ -69,31 +55,24 @@ async function demonstrateNestedManifold() {
];
for (const { text, description } of prompts) {
console.log(`📍 Step: ${description}\n Prompt: "${text}"`);
try {
// First try to navigate
const navigated = await mainManifold.navigate(text);
if (navigated) {
console.log(' ↪ Navigation successful');
console.log(`📍 Step: ${description}`);
}
// Then execute the workflow
const executed = await mainManifold.executeWorkflow(text);
if (executed) {
console.log(' ✅ Execution complete\n');
console.log(`✅ Execution complete`);
} else {
console.log(' ❌ Execution failed - No matching operator found\n');
console.log(`⚠️ Execution failed`);
}
} catch (error) {
console.error(` ❌ Error: ${error.message}\n`);
console.error(`❌ Error: ${error.message}`);
}
}
console.log('🎉 Workflow Demonstration Complete!\n');
}
demonstrateNestedManifold().catch(error => {
console.error('❌ Fatal Error:', error);
console.error(`❌ Critical Error: ${error.message}`);
process.exit(1);
});

15
lib.d.ts vendored
View File

@@ -2,25 +2,20 @@ export interface QueryResult {
confidence: number;
action: string;
}
export interface LLMService {
export interface IntentService {
query(prompt: string): Promise<QueryResult>;
}
export interface State {
[key: string]: any;
}
export class DummyLlmService implements LLMService {
export class DummyIntentService implements IntentService {
query(prompt: string): Promise<QueryResult>;
}
export class WorkflowOperator {
constructor(name: string, operation: (state: State) => Promise<State>);
name: string;
execute(state: State): Promise<State>;
}
export class ManifoldRegion {
constructor(name: string, operators?: WorkflowOperator[]);
name: string;
@@ -30,17 +25,15 @@ export class ManifoldRegion {
connectTo(region: ManifoldRegion): void;
getValidOperators(state: State): Promise<WorkflowOperator[]>;
}
export class NestedManifoldRegion extends ManifoldRegion {
constructor(name: string, nestedManifold: WorkflowFunctionManifold);
nestedManifold: WorkflowFunctionManifold;
navigate(prompt: string): Promise<boolean>;
executeWorkflow(prompt: string): Promise<boolean>;
}
export class WorkflowFunctionManifold {
constructor(llmService: LLMService);
llmService: LLMService;
constructor(intentService: IntentService);
intentService: IntentService;
regions: Map<string, ManifoldRegion | NestedManifoldRegion>;
currentRegion: ManifoldRegion | NestedManifoldRegion | null;
state: State;

32
lib.js
View File

@@ -1,4 +1,4 @@
export class DummyLlmService {
export class DummyIntentService {
async query(prompt) {
const intents = {
analyze: { confidence: 0.9, action: 'analysis' },
@@ -19,7 +19,6 @@ export class WorkflowOperator {
this.name = name;
this.operation = operation;
}
async execute(state) {
return await this.operation(state);
}
@@ -31,16 +30,13 @@ export class ManifoldRegion {
this.operators = operators;
this.adjacentRegions = new Set();
}
addOperator(operator) {
this.operators.push(operator);
}
connectTo(region) {
this.adjacentRegions.add(region);
region.adjacentRegions.add(this);
}
async getValidOperators(_state) {
return this.operators;
}
@@ -51,18 +47,15 @@ export class NestedManifoldRegion extends ManifoldRegion {
super(name);
this.nestedManifold = nestedManifold;
}
async getValidOperators(state) {
if (!this.nestedManifold.currentRegion) {
return [];
}
return await this.nestedManifold.currentRegion.getValidOperators(state);
}
async navigate(prompt) {
return await this.nestedManifold.navigate(prompt);
}
async executeWorkflow(prompt) {
const result = await this.nestedManifold.executeWorkflow(prompt);
return result;
@@ -70,74 +63,59 @@ export class NestedManifoldRegion extends ManifoldRegion {
}
export class WorkflowFunctionManifold {
constructor(llmService) {
this.llmService = llmService;
constructor(intentService) {
this.intentService = intentService;
this.regions = new Map();
this.currentRegion = null;
this.state = {};
}
addRegion(region) {
this.regions.set(region.name, region);
if (!this.currentRegion) {
this.currentRegion = region;
}
}
async navigate(prompt) {
try {
// If current region is nested, try to navigate within it first
if (this.currentRegion instanceof NestedManifoldRegion) {
const nestedNavigated = await this.currentRegion.navigate(prompt);
if (nestedNavigated) {
return true;
}
}
const intent = await this.llmService.query(prompt);
const intent = await this.intentService.query(prompt);
if (intent.confidence <= 0.5) {
return false;
}
// Try to find matching region
const nextRegion = Array.from(this.currentRegion.adjacentRegions).find(region =>
region.name.toLowerCase().includes(intent.action)
);
if (nextRegion) {
this.currentRegion = nextRegion;
return true;
}
return false;
} catch (error) {
console.error('Navigation error:', error);
return false;
}
}
async executeWorkflow(prompt) {
try {
if (this.currentRegion instanceof NestedManifoldRegion) {
const nestedResult = await this.currentRegion.executeWorkflow(prompt);
return nestedResult;
}
const intent = await this.llmService.query(prompt);
const intent = await this.intentService.query(prompt);
const operators = await this.currentRegion.getValidOperators(this.state);
const matchedOperator = operators.find(op =>
op.name.toLowerCase().includes(intent.action)
);
if (matchedOperator && intent.confidence > 0.5) {
this.state = await matchedOperator.execute(this.state);
return true;
}
return false;
} catch (error) {
console.error('Execution error:', error);
return false;
}
}