Workflow Designer Documentation

Overview (Non-Technical)

The Workflow Designer is a visual automation builder that enables users to create sophisticated business processes without coding. Users define triggers (what starts the workflow), steps (what actions happen), conditions (branching logic), and outcomes (completion states). The system executes these workflows automatically, logging every action and handling errors gracefully.

Key Purpose: Enable non-technical users to design, test, and deploy automated processes that execute business logic reliably and auditably.

---

Standards

Workflow Architecture

Workflow Definition Components:

┌─ Trigger (What starts the workflow)
│  ├─ Manual trigger (user clicks button)
│  ├─ Schedule trigger (daily at X time)
│  ├─ Webhook trigger (external system)
│  └─ Event trigger (record created/updated)
│
├─ Steps (What the workflow does)
│  ├─ Action (call API, update record, send email)
│  ├─ Condition (if X then Y else Z)
│  ├─ Loop (repeat step for each item)
│  ├─ Transform (reshape data)
│  └─ Wait (pause until date/event)
│
└─ Output (What happens at the end)
   ├─ Record created/updated
   ├─ Data exported
   ├─ Notification sent
   └─ Status returned to caller
```text

### Configuration Schema

**Workflow Definition Structure:**
```json
{
  "workflowId": "string",
  "workflowName": "string",
  "workflowDescription": "string",
  "version": "1.0",
  "status": "draft|published|archived",
  "triggers": [
    {
      "triggerId": "trigger-1",
      "triggerType": "manual|schedule|webhook|event",
      "triggerName": "Start Approval Process",
      "configuration": {
        "eventType": "record.created",
        "entity": "requests",
        "conditions": [
          {
            "field": "amount",
            "operator": "greaterThan",
            "value": 1000
          }
        ]
      }
    }
  ],
  "steps": [
    {
      "stepId": "step-1",
      "stepType": "action|condition|loop|transform|wait|parallel",
      "stepName": "Send Notification",
      "description": "Notify approver of pending request",
      "configuration": {
        "actionType": "send-email",
        "template": "approval-request",
        "recipients": "${requestor.manager.email}",
        "variables": {
          "requestAmount": "${input.amount}",
          "requestId": "${input.id}"
        }
      },
      "errorHandling": {
        "onError": "retry|skip|halt",
        "retryCount": 3,
        "retryDelay": 5000
      },
      "successCondition": "statusCode = 200",
      "nextSteps": ["step-2", "step-3-on-error"]
    },
    {
      "stepId": "step-2",
      "stepType": "condition",
      "stepName": "Check Approval",
      "configuration": {
        "conditions": [
          {
            "name": "Approved",
            "expression": "${approval.status} === 'approved'",
            "nextStep": "step-4"
          },
          {
            "name": "Rejected",
            "expression": "${approval.status} === 'rejected'",
            "nextStep": "step-5"
          }
        ]
      }
    }
  ],
  "outputs": {
    "status": "success|failure|timeout",
    "data": {},
    "logs": []
  },
  "metadata": {
    "createdBy": "string",
    "createdDate": "ISO8601",
    "modifiedBy": "string",
    "modifiedDate": "ISO8601",
    "lastExecuted": "ISO8601"
  }
}
```text

### Workflow Types

**1. Sequential Workflow:**
```text
Start → Step1 → Step2 → Step3 → End
(Each step runs after previous completes)
```text

**2. Conditional Workflow:**
```text
Start → Check Condition
        ├─ if True → StepA → End
        └─ if False → StepB → End
```text

**3. Parallel Workflow:**
```text
Start → [StepA, StepB, StepC] in parallel → Merge → End
(Multiple steps run simultaneously)
```text

**4. Loop Workflow:**
```text
Start → For each item in collection
        ├─ StepA
        ├─ StepB
        └─ Loop end → End
```text

---

## Do's and Don'ts

### ✅ DO

- **Start with a clear business process:** Document steps before building
- **Use meaningful step names:** "Send Approval Email" not "Step 1"
- **Add error handling to every action:** Retry or skip failures gracefully
- **Log important decisions and values:** For debugging and audit trails
- **Test with sample data first:** Before publishing to production
- **Version workflows:** Track changes and enable rollback
- **Use timeouts on external API calls:** Prevent hanging workflows
- **Validate input data at start:** Catch errors early
- **Implement approval steps for critical actions:** Add human gates
- **Monitor execution metrics:** Track success rate, duration, failures
- **Document variable dependencies:** Show what data flows between steps
- **Use parallel steps for independent operations:** Improve performance

### ❌ DON'T

- **Create infinite loops:** Always include loop exit conditions
- **Store secrets in workflow definitions:** Use secure vault integration
- **Hardcode IDs and values:** Use data binding and variables
- **Skip error handling:** Always specify error recovery behavior
- **Create workflows that are too complex:** Break into sub-workflows
- **Ignore timeout settings:** Long-running operations need explicit waits
- **Mix multiple concerns in one workflow:** Keep workflows focused
- **Forget to test on staging first:** Workflow bugs can cascade
- **Publish without version number:** Track every change
- **Use deprecated API endpoints:** Keep integrations current
- **Create circular dependencies:** Workflow A triggers B, B triggers A
- **Assume data is always valid:** Validate at every step boundary

---

## Technical Details

### Step Types & Implementations

**Action Steps:**
```json
{
  "stepType": "action",
  "actionType": "send-email|http-call|record-update|ui-action|custom",
  "configuration": {
    "sendEmail": {
      "to": "${notification.email}",
      "subject": "Request #${input.id} Pending Approval",
      "template": "approval-notification",
      "attachments": ["document.pdf"]
    },
    "httpCall": {
      "method": "POST",
      "url": "https://api.example.com/webhooks/approval",
      "headers": { "Authorization": "Bearer ${config.apiToken}" },
      "body": "${input | json}"
    },
    "recordUpdate": {
      "entity": "requests",
      "recordId": "${input.id}",
      "updates": {
        "status": "pending-approval",
        "approvedBy": "${user.id}",
        "approvedDate": "now"
      }
    }
  }
}
```text

**Condition Steps:**
```json
{
  "stepType": "condition",
  "configuration": {
    "conditions": [
      {
        "name": "High Value Request",
        "expression": "${input.amount} > 10000",
        "nextStep": "step-vp-approval"
      },
      {
        "name": "Medium Value",
        "expression": "${input.amount} > 1000",
        "nextStep": "step-manager-approval"
      },
      {
        "name": "Default",
        "expression": "true",
        "nextStep": "step-auto-approve"
      }
    ]
  }
}
```text

**Loop Steps:**
```json
{
  "stepType": "loop",
  "configuration": {
    "iterableVariable": "${input.items}",
    "itemVariable": "currentItem",
    "steps": [
      {
        "stepId": "loop-step-1",
        "configuration": {
          "actionType": "record-update",
          "recordId": "${currentItem.id}",
          "updates": { "processed": true }
        }
      }
    ],
    "exitCondition": "all items processed or maxIterations reached"
  }
}
```text

**Wait Steps:**
```json
{
  "stepType": "wait",
  "configuration": {
    "type": "delay|untilDate|untilEvent",
    "delay": 3600000,  // 1 hour in ms
    "untilDate": "2024-12-31T17:00:00Z",
    "untilEvent": "approval.submitted",
    "onTimeout": "proceed|fail",
    "timeoutDuration": 86400000  // 24 hours max
  }
}
```text

### Variable System

**Variable Scopes:**
```text
${input}            // Workflow input data (immutable)
${workflow}         // Workflow execution context
${step}             // Current step output
${previousStep}     // Last completed step output
${user}             // Current user context
${time}             // Current timestamp
${config}           // System configuration values
${cache}            // Workflow execution cache
```text

**Variable Examples:**
```text
${input.customerId}           // Direct field access
${input.address.city}         // Nested object access
${input.items[0].price}       // Array index access
${input.items | sum(price)}   // Array aggregation
${step.output.status}         // Step output
${user.id | concat('-', input.id)}  // String functions
${time | formatDate('YYYY-MM-DD')}  // Date functions
```text

### Data Transformation

**Transform Steps:**
```json
{
  "stepType": "transform",
  "configuration": {
    "transformType": "json|sql|javascript",
    "input": "${previousStep.output}",
    "transformation": {
      "customerId": "${.customerId}",
      "fullName": "${.firstName + ' ' + .lastName}",
      "orderTotal": "${.items | sum(.price)}",
      "isPremium": "${.customerType == 'premium'}"
    },
    "outputVariable": "transformedData"
  }
}
```text

### Error Handling Strategies

**Error Handling Configuration:**
```json
{
  "errorHandling": {
    "onError": "retry|skip|alternate|halt",
    "retryCount": 3,
    "retryDelay": 5000,
    "retryBackoff": "exponential",
    "skipCondition": "${error.code} == 'NOT_FOUND'",
    "alternateStep": "step-fallback",
    "onMaxRetriesExceeded": "halt|skip|log",
    "logError": true,
    "notifyOnError": ["admin@company.com"]
  }
}
```text

**Error Recovery Pattern:**
```javascript
// Middleware: services/workflow.service.js
async executeStep(step, context) {
  let retries = 0;
  const maxRetries = step.errorHandling.retryCount || 0;
  
  while (retries <= maxRetries) {
    try {
      return await this.runAction(step, context);
    } catch (error) {
      if (retries < maxRetries) {
        const delay = step.errorHandling.retryDelay * (
          step.errorHandling.retryBackoff === 'exponential' 
            ? Math.pow(2, retries) 
            : 1
        );
        await sleep(delay);
        retries++;
      } else {
        // Handle final failure
        const onError = step.errorHandling.onError;
        if (onError === 'skip') return null;
        if (onError === 'alternate') return this.executeStep(alternateStep, context);
        if (onError === 'halt') throw error;
      }
    }
  }
}
```text

### Execution Engine

**Workflow Execution States:**
```text
Draft → Running → [Paused ←→ Resumed] → Completed
                ↓
            Failed → Logged
                ↓
            Cancelled
```text

**Execution Flow (Middleware):**
```javascript
// WorkflowService
async startWorkflow(workflowId, { inputData, version, userId, tenantId }) {
  // 1. Validate workflow exists & published
  const workflow = await getWorkflow(workflowId, version);
  
  // 2. Create instance record
  const instance = await createInstance(workflow, inputData, userId);
  
  // 3. Validate input against schema
  validateInput(inputData, workflow.inputSchema);
  
  // 4. Queue first steps
  for (const trigger of workflow.triggers) {
    if (matchesTriggerConditions(inputData, trigger)) {
      await queueSteps(instance, trigger.nextSteps);
    }
  }
  
  // 5. Trigger background worker
  emitEvent('workflow.started', { instanceId: instance.id });
  
  return instance.id;
}

// Worker: processes queued steps
async processQueuedJob(job) {
  const { stepId, instanceId, stepInput } = job;
  const instance = await getInstance(instanceId);
  const step = instance.workflow.steps.find(s => s.stepId === stepId);
  
  try {
    const output = await executeStep(step, stepInput);
    
    // Determine next steps based on conditions
    const next = step.nextSteps
      .filter(ns => matchesCondition(ns.condition, output))
      .map(ns => ns.stepId);
    
    // Queue next steps
    for (const nextStepId of next) {
      await queueStep(instance, nextStepId, output);
    }
    
    // Log execution
    await logExecution(instanceId, stepId, 'success', output);
  } catch (error) {
    // Handle error per step configuration
    await handleStepError(step, error, instanceId);
  }
}
```text

### Monitoring & Logging

**Execution Logs Table (Tenant DB):**
```sql
WFT_LOGS
├─ LogId (PK)
├─ InstanceId (FK to WFT_INSTANCES)
├─ WorkflowId
├─ StepId
├─ LogType (Info, Warning, Error)
├─ Message
├─ Data (JSON)
│  ├─ StepInput
│  ├─ StepOutput
│  ├─ ExecutionTime (ms)
│  └─ Variables snapshot
├─ Timestamp (UTC)
└─ TenantId
```text

**Monitoring Dashboard Metrics:**
```javascript
// Track these KPIs
- Workflow Success Rate (%)
- Average Execution Time (ms)
- Failed Workflows (count, trending)
- Most Failed Steps (by count)
- Longest Running Steps (avg, p95)
- Error Frequency (by error type)
- Throughput (workflows/hour)
- Queue Depth (pending jobs)
```text

---

## Key Files & Imports

| Purpose | File | Notes |
|---------|------|-------|
| Workflow execution | `services/workflow.service.js` | Core engine, 973 lines |
| Step execution | `services/workflow.service.js` | executeStep() method |
| Error handling | `services/workflow.service.js` | Retry/recovery logic |
| Logging | `utils/logger.util.js` | Execution logs with context |
| Background jobs | `jobs/`, `workers/` | Queue processing |
| Workflow routes | `routes/workflows.js` | API endpoints |
| Service layer | `services/` | Domain services (email, http, etc.) |

---

## Common Issues & Solutions

| Issue | Cause | Solution |
|-------|-------|----------|
| Workflow hangs | Step waiting forever | Add timeout, check wait condition |
| Variable undefined in step | Wrong variable scope/path | Check syntax: ${input.field} |
| Loop infinite | No exit condition | Add maxIterations or proper exit logic |
| Condition never matches | Wrong expression syntax | Test expression with sample data |
| Retry not happening | maxRetries = 0 or onError not set | Configure errorHandling section |
| Workflow not starting | Trigger condition not met | Verify trigger conditions match input |
| Data not flowing to next step | previousStep var not set | Check step output variable name |
| External API call fails silently | Error not logged | Enable logError: true, check stderr |

---

## Workflow Creator Workflow

1. **Define Process** → Outline steps on paper
2. **Create Workflow** → Set name, description
3. **Add Triggers** → Select trigger type, configure conditions
4. **Design Steps** → Add actions, conditions, loops
5. **Configure Variables** → Map data between steps
6. **Set Error Handling** → Define recovery behavior
7. **Test** → Run with test data, verify results
8. **Debug** → Check logs, adjust logic
9. **Publish** → Create versionversion, set status to published
10. **Monitor** → Track executions, collect metrics

---

## Related Documentation
- [Page Designer Documentation](./PAGE.md) - Form interface
- [DataView Documentation](./DATAVIEW.md) - Display workflow data
- [Renderers Documentation](../../technical/components/renderers.md) - Component execution