Flows Engine

What are Flows?

Flows are automated pipelines that process your uploaded files. When a user uploads an image, a flow can automatically resize it, generate thumbnails, optimize it for the web, and save all versions to your storage - all without any manual intervention.

Think of flows like a recipe: you define the steps once, and Uploadista executes them every time a file is uploaded.

How Flows Work

Trigger Flow → Create Job → Input Nodes (upload files) → Processing Nodes → Output Nodes → Done

A flow is made up of connected nodes:

• Input nodes - Initiate uploads using the Upload Engine and receive files
• Processing nodes - Transform the file (resize, compress, convert, etc.)
• Output nodes - Save results to storage

Nodes are connected with edges that define how data flows between them.

Flow Execution and Uploads

When a flow is triggered:

1. A job is created to track the execution
2. Input nodes initiate one or more uploads via the Upload Engine
3. The flow pauses while waiting for uploads to complete
4. Once all input uploads finish, the flow resumes processing
5. Processing and output nodes execute in sequence

Common Use Cases

Use Case	Flow Description
Profile photos	Resize to standard sizes, crop to square, save as WebP
Product images	Generate thumbnail + full-size, optimize for web
Document upload	Validate file type, scan for viruses, store securely
Video thumbnails	Extract frame at 5 seconds, resize, save as JPEG
Batch processing	Apply same transformation to multiple files

Quick Start

Using the Visual Flow Builder

The easiest way to create flows is with the visual builder in the Uploadista dashboard:

1. Go to Flows in your dashboard
2. Click Create Flow
3. Drag nodes from the sidebar onto the canvas
4. Connect nodes by dragging from output to input handles
5. Configure each node by clicking on it
6. Save and activate your flow

Example: Image Processing Flow

Here’s a typical image processing flow:

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Upload    │───►│   Resize    │───►│  Save to    │
│   (Input)   │    │  (800x600)  │    │     S3      │
└─────────────┘    └─────────────┘    └─────────────┘
                          │
                          ▼
                   ┌─────────────┐    ┌─────────────┐
                   │  Thumbnail  │───►│  Save to    │
                   │  (200x150)  │    │     S3      │
                   └─────────────┘    └─────────────┘

This flow:

1. Receives an uploaded image
2. Resizes it to 800x600 for the main image
3. Creates a 200x150 thumbnail
4. Saves both versions to S3

Available Nodes

Input Nodes

Node	Description
File Input	Receives uploaded files
URL Input	Fetches files from a URL

Image Processing Nodes

Node	Description
Resize	Change image dimensions
Crop	Crop to specific area or aspect ratio
Rotate	Rotate by degrees
Format Convert	Convert between JPEG, PNG, WebP, AVIF
Optimize	Compress while maintaining quality
Watermark	Add text or image watermark
Blur	Apply blur effect
Grayscale	Convert to black and white

AI Image Nodes (with Replicate plugin)

Node	Description
Background Remove	Remove image background
Upscale	AI-powered image upscaling
Face Restore	Enhance and restore faces

Document Nodes

Node	Description
PDF to Image	Convert PDF pages to images
Extract Text	OCR text extraction

Utility Nodes

Node	Description
Conditional	Route files based on conditions
Merge	Combine multiple files
ZIP	Create archive from files

Output Nodes

Node	Description
Save to Storage	Save to your configured data store
Webhook	Send result to external URL

Configuring Flows

Triggering Flows

Flows are triggered explicitly - the user or application decides when to start a flow:

Via API:

POST /api/flows/image-processing/execute

Via Client SDK:

// The client triggers the flow, then uploads file(s)
const job = await uploadista.executeFlow("image-processing");
// The flow's input nodes handle the upload process

From the Dashboard:

1. Go to Flows in your dashboard
2. Select a flow
3. Click Execute to start a new job

When a flow is triggered, it creates a job and the input nodes initiate uploads. The flow automatically pauses until all uploads complete, then resumes processing.

Flow Configuration

Each flow has configuration options:

{
  name: "image-processing",
  description: "Process uploaded images",

  // Input validation (applied to input nodes)
  input: {
    allowedMimeTypes: ["image/jpeg", "image/png", "image/webp"],
    maxFileSize: 10 * 1024 * 1024, // 10MB
  },

  // Processing timeout
  timeout: 60000, // 1 minute

  // Retry on failure
  retries: 3,
}

Flow Results

When a flow completes, you receive the outputs:

{
  "flowId": "image-processing",
  "jobId": "job-abc123",
  "status": "completed",
  "outputs": {
    "main": {
      "url": "https://cdn.example.com/uploads/abc123/main.webp",
      "size": 245760,
      "mimeType": "image/webp"
    },
    "thumbnail": {
      "url": "https://cdn.example.com/uploads/abc123/thumb.webp",
      "size": 12288,
      "mimeType": "image/webp"
    }
  },
  "duration": 2340
}

Accessing Flow Results

Via Webhook:

// Configure webhook output node to send results
app.post("/webhooks/flow-complete", (req, res) => {
  const { flowId, outputs } = req.body;
  // Store URLs in your database
  await db.update("uploads", {
    mainUrl: outputs.main.url,
    thumbnailUrl: outputs.thumbnail.url,
  });
});

Via WebSocket events:

ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  if (message.type === "flow_complete") {
    console.log("Flow outputs:", message.outputs);
  }
};

Via API:

GET /api/flows/jobs/job-abc123

# Response
{
  "id": "job-abc123",
  "status": "completed",
  "outputs": { ... }
}

Common Patterns

Multi-Size Image Generation

Create multiple image sizes from one upload:

Upload ──► Resize 1920px ──► Save (fullsize)
       │
       ├─► Resize 800px ──► Save (medium)
       │
       └─► Resize 200px ──► Save (thumbnail)

Conditional Processing

Route files based on their properties:

Upload ──► Check Size ──► Large (>5MB) ──► Compress ──► Save
                     │
                     └─► Small (<5MB) ──► Save directly

Preserving Original + Processed

Keep the original file while also creating processed versions:

Upload ──┬─► Save (original)
         │
         └─► Process ──► Save (processed)

Preserving Intermediate Outputs

By default, only the final outputs (nodes with no outgoing connections) are saved. Intermediate processing results are automatically cleaned up to save storage.

When You Need Intermediate Outputs

Sometimes you need to keep intermediate results:

• Invoice processing: Keep both the original PDF and the extracted text
• Multi-format images: Keep the resized version before converting to WebP and JPEG
• Audit trails: Preserve each processing step

Using “Keep Output”

In the flow builder, select any node and enable “Keep output as result”. This preserves that node’s output even if it has outgoing connections.

Upload (keep) ──► OCR ──► Analysis
   │
   └─► Original PDF is preserved alongside the OCR text and analysis

Nodes with “Keep output” enabled show an amber badge in the flow builder.

Error Handling

What Happens When a Flow Fails

1. Automatic retry - Uploadista retries failed nodes (configurable)
2. Error event - A flow_error event is sent via WebSocket
3. Job status - The job is marked as failed with error details

Checking Flow Status

GET /api/flows/jobs/job-abc123

# Response for failed flow
{
  "id": "job-abc123",
  "status": "failed",
  "error": {
    "nodeId": "resize-node",
    "message": "Image format not supported",
    "code": "UNSUPPORTED_FORMAT"
  }
}

Common Errors

Error	Cause	Solution
UNSUPPORTED_FORMAT	File type not supported by node	Check file mime type before flow
FILE_TOO_LARGE	File exceeds node limits	Increase limits or compress first
TIMEOUT	Processing took too long	Increase timeout or simplify flow
STORAGE_ERROR	Failed to save output	Check storage configuration

Performance Tips

For Large Files

• Use larger chunk sizes in your data store configuration
• Increase flow timeout for processing-heavy operations
• Consider streaming nodes that process files in chunks

For High Volume

• Use Redis-backed KV and event stores (not Memory)
• Deploy multiple server instances
• Consider dedicated worker processes for flow execution

For Complex Flows

• Keep flows focused - one purpose per flow
• Use conditional nodes to skip unnecessary processing
• Set appropriate timeouts per node

Related Concepts

• Plugins - Add processing capabilities to flows
• Upload Engine - Provides files to flows
• Uploadista Server - Configure and trigger flows
• Event System - Receive flow progress updates
• Data Stores - Where flow outputs are saved