Agent Workflow Builder

An open-source, visual workflow builder for creating AI-powered agent workflows, similar to N8N or Zapier, built with React Flow and modern web technologies. Create sophisticated automation workflows with AI models, web scraping, and data processing capabilities.

🚀 Features

Visual Workflow Editor

• Drag & Drop Interface: Intuitive node-based workflow creation
• Real-time Connection: Connect nodes with visual edges
• Node Palette: Easy access to all available node types
• Responsive Design: Works on desktop and tablet devices
• Variable Substitution: Use {{nodeId.output}} or {{nodelabel.output}} to pass data between nodes

AI-Powered Node Types

• 🌐 Web Scraping: Extract data from websites using Firecrawl AI
• 🤖 LLM Task: General language model operations (DeepSeek, OpenAI)
• 🧠 Embedding Generator: Create vector embeddings for semantic search
• 🔍 Similarity Search: Find similar content in vector stores
• 📋 Structured Output: Parse and structure data with JSON schemas
• 📥 Data Input: Workflow entry points (text, JSON, CSV, URL, PDF)
• 📤 Data Output: Workflow exit points with multiple formats

AI Copilot Assistant

• Natural Language Processing: Describe workflows in plain English
• Intelligent Workflow Generation: AI creates workflows from descriptions
• Contextual Suggestions: Get smart recommendations based on your workflow
• Validation & Optimization: Automatic workflow validation and improvement tips
• Mixed Intent Analysis: Handle complex, multi-step workflow requirements

Workflow Management

• Save & Load: Persistent workflow storage in browser
• Import/Export: Share workflows as JSON files
• Version Control: Track workflow changes with timestamps
• Workflow Library: Manage multiple workflows with search and filtering
• Demo Workflows: Pre-built example workflows to get started

Execution & Testing

• Real-time Execution: Step-by-step workflow processing with live updates
• Visual Feedback: Live status indicators for each node (idle, running, success, error)
• Test Interface: Run workflows with sample data and custom inputs
• Execution Logs: Detailed execution history and debugging information
• Error Handling: Clear error messages and recovery suggestions
• Performance Monitoring: Track execution times and resource usage

Advanced Features

• Configurable Parameters: Customize each node's behavior and settings
• Data Flow Visualization: See data passing between nodes in real-time
• Execution Results: View and export workflow outputs in multiple formats
• Modular Architecture: Easy to extend with new node types
• PDF Processing: Upload and process PDF documents with text extraction
• API Integration: Seamless integration with DeepSeek, OpenAI, and Firecrawl APIs

🛠️ Technology Stack

• Frontend: React 18 + TypeScript
• Visual Editor: React Flow v11
• Styling: Tailwind CSS with custom animations
• State Management: Zustand
• Icons: Lucide React
• Notifications: React Hot Toast
• AI Integration:
  • DeepSeek API (primary LLM)
  • OpenAI API (alternative LLM)
  • Firecrawl AI (web scraping)
• PDF Processing: Browser-based PDF.js
• Build Tool: Create React App

📦 Installation

1. Clone the repository

   git clone https://github.com/Nikhil-Doye/workflow-builder.git
   cd workflow-builder
   

2. Install dependencies

   npm install
   

3. Configure API Keys (Optional but recommended)

   • The app will prompt you to configure API keys on first launch
   • You can also configure them later through the settings menu
   • Required APIs:
     • DeepSeek API: For LLM tasks and AI processing
     • Firecrawl API: For web scraping functionality
     • OpenAI API: Alternative LLM provider

4. Start the development server

   npm start
   

5. Open your browser Navigate to http://localhost:3000

🎯 Quick Start

Method 1: AI Copilot (Recommended)

1. Start the Application

   • Open the app in your browser
   • Click "New Workflow" to create your first workflow

2. Use AI Copilot

   • Click the "AI Copilot" button in the toolbar
   • Describe your workflow in natural language (e.g., "scrape a website and analyze the content")
   • Let the AI generate the workflow structure for you
   • Review and modify the generated workflow as needed

3. Test and Execute

   • Use the Testing Panel to provide sample input data
   • Click "Run Test" to execute the workflow
   • View results in real-time as each node processes

Method 2: Manual Workflow Creation

1. Add Nodes

   • Click "Show Node Palette" to reveal available node types
   • Drag nodes from the palette onto the canvas
   • Connect nodes by dragging from output handles to input handles

2. Configure Nodes

   • Click on any node to open its configuration panel
   • Set parameters like URLs, prompts, model settings, etc.
   • Use variable substitution: {{nodeId.output}} or {{nodelabel.output}} to pass data between nodes

3. Test Your Workflow

   • Use the Testing Panel to provide sample input data
   • Click "Run Test" to execute the workflow
   • View results in real-time as each node processes

4. Save and Export

   • Click "Save" to persist your workflow
   • Use "Export" to download your workflow as a JSON file
   • Share workflows by importing JSON files

Example Workflows

Web Scraping + AI Analysis

1. Data Input → Web Scraping → LLM Task → Data Output
2. Configure the Web Scraping node with a URL
3. Set up the LLM Task with a prompt like "Analyze this content: {{scraper.output}}"
4. Test with a sample URL

Document Processing Pipeline

1. Data Input (PDF) → LLM Task → Structured Output → Data Output
2. Upload a PDF document
3. Extract and process the text with AI
4. Structure the output according to a JSON schema

Embedding & Search Workflow

1. Data Input → Embedding Generator → Similarity Search → Data Output
2. Create embeddings from input text
3. Search for similar content in a vector store
4. Return the most relevant results

🏗️ Architecture

Project Structure

src/
├── components/           # React components
│   ├── nodes/           # Node type components
│   │   ├── BaseNode.tsx
│   │   ├── DataInputNode.tsx
│   │   ├── DataOutputNode.tsx
│   │   ├── WebScrapingNode.tsx
│   │   ├── LLMNode.tsx
│   │   ├── EmbeddingNode.tsx
│   │   ├── SimilaritySearchNode.tsx
│   │   └── StructuredOutputNode.tsx
│   ├── WorkflowEditor.tsx
│   ├── WorkflowList.tsx
│   ├── WorkflowToolbar.tsx
│   ├── NodeConfiguration.tsx
│   ├── ExecutionPanel.tsx
│   ├── TestingPanel.tsx
│   ├── CopilotPanel.tsx
│   ├── OnboardingModal.tsx
│   └── OpenAIConfig.tsx
├── services/            # External service integrations
│   ├── openaiService.ts
│   ├── firecrawlService.ts
│   ├── copilotService.ts
│   └── pdfService.ts
├── store/               # State management
│   └── workflowStore.ts
├── types/               # TypeScript definitions
│   └── index.ts
├── utils/               # Utility functions
│   ├── workflowSerialization.ts
│   ├── workflowValidator.ts
│   ├── workflowGenerator.ts
│   ├── variableSubstitution.ts
│   └── patternMatchers.ts
├── examples/            # Example workflows and documentation
└── App.tsx             # Main application

Key Components

• WorkflowEditor: Main visual editor using React Flow with drag-and-drop functionality
• CopilotPanel: AI-powered workflow generation and assistance
• NodeConfiguration: Modal for configuring node parameters and settings
• ExecutionPanel: Real-time execution monitoring with live status updates
• TestingPanel: Test interface with sample data and custom inputs
• WorkflowList: Workflow management with search, filtering, and CRUD operations
• workflowStore: Centralized state management with Zustand for workflow data
• Services: Modular service layer for AI APIs, web scraping, and PDF processing

🔧 Configuration

API Keys Setup

The application requires API keys for full functionality:

1. DeepSeek API Key (Primary LLM)

   • Get your key from DeepSeek Platform
   • Used for LLM tasks and AI processing
   • Format: sk-...

2. Firecrawl API Key (Web Scraping)

   • Get your key from Firecrawl
   • Used for web scraping functionality
   • Format: fc-...

3. OpenAI API Key (Alternative LLM)

   • Get your key from OpenAI Platform
   • Used as fallback for LLM tasks
   • Format: sk-...

Node Types and Parameters

Each node type supports specific configuration options:

Variable Substitution with Node Labels

The workflow builder now supports intuitive variable substitution using node labels:

Traditional Method (Node IDs):

{{node-1234567890.output}}

New Method (Node Labels):

{{Website URL.output}}
{{Web Scraper.output}}
{{Content Analyzer.output}}

Benefits:

• More Intuitive: Use descriptive names instead of cryptic IDs
• Easier to Read: Workflow configurations are self-documenting
• Better UX: No need to remember or look up node IDs
• Backward Compatible: Both methods work together

Requirements:

• Each node must have a unique label within the workflow
• Labels are case-sensitive
• Empty labels are not allowed
• The system validates label uniqueness automatically

Web Scraping Node (Firecrawl AI)

• url: Target website URL (supports variable substitution)
• formats: Output formats (markdown, html, text, summary, links, images)
• onlyMainContent: Extract only main content (boolean)
• includeTags: CSS selectors to include (comma-separated)
• excludeTags: CSS selectors to exclude (comma-separated)
• maxLength: Maximum content length to extract
• waitFor: Wait time for dynamic content (milliseconds)
• timeout: Request timeout (milliseconds)

LLM Task Node

• prompt: The prompt to send to the language model (supports variables)
• model: AI model selection (deepseek-chat, deepseek-reasoner)
• temperature: Response creativity (0-2, default: 0.7)
• maxTokens: Maximum response length (default: 1000)

Data Input Node

• dataType: Input data type (text, json, csv, url, pdf)
• defaultValue: Default input value or sample data

Data Output Node

• format: Output format (json, text, csv)
• filename: Output filename for exports

Embedding Generator

• model: Embedding model selection (text-embedding-ada-002, text-embedding-3-small)
• dimensions: Vector dimensions (default: 1536)

Similarity Search

• vectorStore: Target vector database (pinecone, weaviate, chroma)
• topK: Number of similar results to return (default: 5)
• threshold: Similarity threshold (0-1, default: 0.8)

Structured Output

• schema: JSON schema for structured output
• model: AI model for structured generation

🚀 Deployment

Build for Production

npm run build

The build artifacts will be stored in the build/ directory.

Deploy to Vercel

npm install -g vercel
vercel --prod

Deploy to Netlify

npm run build
# Upload the 'build' folder to Netlify

Deploy to GitHub Pages

npm run build
# Push the build folder to a gh-pages branch

Environment Variables

For production deployment, you may want to set up environment variables:

• REACT_APP_DEEPSEEK_API_KEY: DeepSeek API key
• REACT_APP_FIRECRAWL_API_KEY: Firecrawl API key
• REACT_APP_OPENAI_API_KEY: OpenAI API key

Note: API keys are currently stored in browser localStorage for security.

🤝 Contributing

We welcome contributions! Here's how you can help:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Commit your changes: git commit -m 'Add amazing feature'
4. Push to the branch: git push origin feature/amazing-feature
5. Open a Pull Request

Development Guidelines

• Follow TypeScript best practices
• Use Tailwind CSS for styling
• Write meaningful commit messages
• Add tests for new features
• Update documentation as needed

🐛 Troubleshooting

Common Issues

API Key Issues

• Ensure API keys are properly configured in the settings
• Check that API keys have the correct format (sk-... for AI, fc-... for Firecrawl)
• Verify API keys have sufficient credits/permissions

Nodes not connecting?

• Ensure you're dragging from output handles (bottom) to input handles (top)
• Check that nodes are properly positioned and not overlapping
• Verify the workflow has a clear data flow path

Workflow not executing?

• Verify all required node parameters are configured
• Check the execution panel for detailed error messages
• Ensure all nodes are properly connected in sequence
• Check that variable substitutions are correct (e.g., {{nodeId.output}} or {{nodelabel.output}})

Web Scraping Issues

• Verify the target URL is accessible and not blocked
• Check Firecrawl API key and credits
• Try different output formats or content selectors
• Ensure the website allows scraping (check robots.txt)

LLM Task Issues

• Verify AI API key is valid and has credits
• Check that prompts are properly formatted
• Ensure variable substitutions in prompts are correct
• Try different models or adjust temperature settings

Import/Export not working?

• Ensure you're using valid JSON format
• Check browser console for error messages
• Verify the workflow structure is complete

PDF Processing Issues

• Ensure the PDF file is not password-protected
• Check file size limits (large files may timeout)
• Verify the PDF contains extractable text (not just images)

Getting Help

• Check the Execution Panel for detailed error messages and logs
• Use the Testing Panel to debug with sample data
• Review the AI Copilot suggestions for workflow improvements
• Check browser console for technical error details

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• React Flow for the visual editor
• Tailwind CSS for styling
• Lucide React for icons
• Zustand for state management
• DeepSeek for AI language models
• Firecrawl for web scraping capabilities
• OpenAI for alternative AI models

📚 Additional Resources

• Getting Started Guide - Detailed setup and first workflow tutorial
• Variable Substitution Examples - Learn how to pass data between nodes
• Demo Workflows - Pre-built example workflows

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Check the Getting Started guide and examples

🔮 Roadmap

• More AI model integrations (Claude, Gemini)
• Advanced workflow templates
• Team collaboration features
• Workflow scheduling and automation
• Custom node development SDK
• Workflow analytics and monitoring
• Database integrations
• API endpoint creation

---

Built with ❤️ for the AI community

Empowering developers to create sophisticated AI workflows without the complexity.