Flow Nodes Overview

Flow nodes are the building blocks of Uploadista processing pipelines. Each node performs a specific operation - from file routing to image optimization to virus scanning. Connect nodes together to create powerful file processing workflows.

Node Categories

Category	Purpose	Node Package	Plugin Packages
Utility	Data flow control	`@uploadista/flow-utility-nodes`	`flow-utility-zipjs`
Images	Image processing	`@uploadista/flow-images-nodes`	`flow-images-sharp`, `flow-images-photon`, `flow-images-replicate`
Videos	Video processing	`@uploadista/flow-videos-nodes`	`flow-videos-av-node`
Documents	PDF and document processing	`@uploadista/flow-documents-nodes`	`flow-documents-pdflib`, `flow-documents-unpdf`, `flow-documents-replicate`
Security	Malware scanning	`@uploadista/flow-security-nodes`	`flow-security-clamscan`

Common Node Options

All transform nodes support these common options:

Mode

Controls how file data is processed for memory efficiency.

const node = yield* createResizeNode("resize", {
  width: 800,
  height: 600,
  fit: "cover",
}, {
  mode: "auto", // "auto" | "buffered" | "streaming"
  streamingConfig: {
    fileSizeThreshold: 1_048_576, // 1MB - switch to streaming above this
    chunkSize: 65_536, // 64KB chunks
  },
});

Mode	Description	When to Use
`auto`	Automatically selects streaming for large files	Default, recommended
`buffered`	Loads entire file into memory	Small files, predictable memory
`streaming`	Processes file as chunks	Large files, memory-constrained

Default Thresholds:

Images: 1MB (streaming for files > 1MB)
Videos: 10MB (streaming for files > 10MB)

Keep Output

Controls whether processed output is included in flow results.

const node = yield* createOptimizeNode("optimize", {
  quality: 80,
  format: "webp",
}, {
  keepOutput: true, // Include output bytes in flow results
});

Value	Description
`false` (default)	Output is stored but not returned in results (saves memory)
`true`	Output bytes included in flow results (useful for chaining or inspection)

Naming

Controls how output files are named after processing.

// Auto mode - appends node-specific suffix automatically
const autoNaming = yield* createResizeNode("resize", {
  width: 800,
  height: 600,
  fit: "cover",
}, {
  naming: { mode: "auto" }, // "photo.jpg" → "photo-800x600.jpg"
});

// Custom mode - use a template pattern
const customNaming = yield* createOptimizeNode("optimize", {
  quality: 80,
  format: "webp",
}, {
  naming: {
    mode: "custom",
    pattern: "{{baseName}}-optimized.{{extension}}",
  },
});

Mode	Description	Example Output
`auto`	Appends operation-specific suffix	`photo-800x600.jpg`, `video-webm.webm`
`custom`	Use template pattern with placeholders	`photo-optimized.webp`

Template Placeholders:

{{baseName}} - Original filename without extension
{{extension}} - File extension
{{nodeId}} - Node identifier
{{nodeType}} - Type of operation (resize, optimize, etc.)
{{flowId}} - Flow identifier
{{jobId}} - Job identifier

Node Comparison Table

Node	Speed	Cost	Backend	Streaming
Resize (Sharp)	50-100ms	Free	Node.js	Yes
Resize (Photon)	5-10ms	Free	Edge	No
Optimize	100-200ms	Free	Sharp/Photon	Yes
Remove BG	5-15s	$0.001	AI/Replicate	No
Describe Image	10-20ms	$0.001	AI/Replicate	No
Transcode Video	1-60s	Free	AV Node	Yes
Resize Video	1-60s	Free	AV Node	Yes
Trim Video	1-30s	Free	AV Node	Yes
Describe Video	100-500ms	Free	AV Node	No
OCR	5-15s	$0.005	AI/Replicate	No
Extract Text	Less than 1s	Free	unpdf	No
Split PDF	Less than 2s	Free	pdf-lib	No
Merge PDF	Less than 2s	Free	pdf-lib	No
Scan Virus	1-30s	Free	ClamAV	No
Merge	Instant	Free	Memory	No
Conditional	Instant	Free	Memory	No
Multiplex	Instant	Free	Memory	No
Zip	100-200ms	Free	CPU	No

Architecture Patterns

Pattern 1: Image Pipeline

Multi-stage image processing with resize, optimize, and describe:

Input → Resize → Optimize → Describe Image

import { createFlow, createInputNode } from "@uploadista/core";
import {
  createResizeNode,
  createOptimizeNode,
  createDescribeImageNode,
} from "@uploadista/flow-images-nodes";

const imagePipelineFlow = createFlow({
  flowId: "image-pipeline-flow",
  name: "Image Pipeline Flow",
  nodes: {
    input: createInputNode("input"),
    resize: createResizeNode("resize", {
      width: 1200,
      height: 900,
      fit: "cover",
    }),
    optimize: createOptimizeNode("optimize", {
      quality: 85,
      format: "webp",
    }),
    "describe-image": createDescribeImageNode("describe-image"),
  },
  edges: [
    { source: "input", target: "resize" },
    { source: "resize", target: "optimize" },
    { source: "optimize", target: "describe-image" },
  ],
});

Pattern 2: Conditional Routing

Route files based on properties like size:

Input → Conditional (size > 2MB?)
        ├─ YES: Resize → Optimize Large
        └─ NO: Optimize Small

import { createFlow, createInputNode } from "@uploadista/core";
import { createResizeNode, createOptimizeNode } from "@uploadista/flow-images-nodes";
import { createConditionalNode } from "@uploadista/flow-utility-nodes";

const conditionalImageFlow = createFlow({
  flowId: "conditional-image-flow",
  name: "Conditional Image Flow",
  nodes: {
    input: createInputNode("input"),
    conditional: createConditionalNode("conditional", {
      field: "size",
      operator: "greaterThan",
      value: 2000000, // 2MB
    }),
    resize: createResizeNode("resize", {
      width: 1600,
      height: 1200,
      fit: "contain",
    }),
    "optimize-large": createOptimizeNode("optimize-large", {
      quality: 80,
      format: "webp",
    }),
    "optimize-small": createOptimizeNode("optimize-small", {
      quality: 85,
      format: "webp",
    }),
  },
  edges: [
    { source: "input", target: "conditional" },
    { source: "conditional", target: "resize" },
    { source: "resize", target: "optimize-large" },
    { source: "conditional", target: "optimize-small" },
  ],
});

Pattern 3: Multi-Format Output

Generate multiple format versions and archive them:

Input → Multiplex (3 outputs)
        ├─ Optimize WebP
        ├─ Optimize JPEG
        └─ Optimize PNG
        → Zip

import { createFlow, createInputNode } from "@uploadista/core";
import { createOptimizeNode } from "@uploadista/flow-images-nodes";
import { createMultiplexNode, createZipNode } from "@uploadista/flow-utility-nodes";

const multiFormatFlow = createFlow({
  flowId: "multi-format-flow",
  name: "Multi-Format Flow",
  nodes: {
    input: createInputNode("input"),
    multiplex: createMultiplexNode("multiplex", {
      outputCount: 3,
      strategy: "copy",
    }),
    "optimize-webp": createOptimizeNode("optimize-webp", {
      quality: 80,
      format: "webp",
    }),
    "optimize-jpeg": createOptimizeNode("optimize-jpeg", {
      quality: 85,
      format: "jpeg",
    }),
    "optimize-png": createOptimizeNode("optimize-png", {
      quality: 90,
      format: "png",
    }),
    zip: createZipNode("zip", {
      zipName: "multi-format.zip",
      includeMetadata: false,
      inputCount: 3,
    }),
  },
  edges: [
    { source: "input", target: "multiplex" },
    { source: "multiplex", target: "optimize-webp" },
    { source: "multiplex", target: "optimize-jpeg" },
    { source: "multiplex", target: "optimize-png" },
    { source: "optimize-webp", target: "zip" },
    { source: "optimize-jpeg", target: "zip" },
    { source: "optimize-png", target: "zip" },
  ],
});

Pattern 4: Security Pipeline

Scan files before processing:

Input → Scan Virus → Conditional (clean?)
        ├─ YES: Process → Store
        └─ NO: Quarantine

Pattern 5: Video Pipeline

Multi-stage video processing:

Input → Trim → Transcode → Thumbnail

import { createFlow, createInputNode } from "@uploadista/core";
import {
  createTrimVideoNode,
  createTranscodeVideoNode,
  createVideoThumbnailNode,
} from "@uploadista/flow-videos-nodes";

const videoPipelineFlow = createFlow({
  flowId: "video-pipeline-flow",
  name: "Video Pipeline Flow",
  nodes: {
    input: createInputNode("input"),
    trim: createTrimVideoNode("trim", {
      startTime: 0,
      endTime: 60,
    }),
    transcode: createTranscodeVideoNode("transcode", {
      format: "webm",
      codec: "vp9",
      videoBitrate: "1000k",
    }),
    thumbnail: createVideoThumbnailNode("thumbnail", {
      timestamp: 5,
      format: "jpeg",
      quality: 85,
    }),
  },
  edges: [
    { source: "input", target: "trim" },
    { source: "trim", target: "transcode" },
    { source: "transcode", target: "thumbnail" },
  ],
});

Selection Guide

Choose Sharp (Node.js) When:

Running on Node.js servers
Speed critical (sub-100ms)
Cost sensitive (free)
Simple operations
Self-hosted preference

Choose Photon (Edge) When:

Using Cloudflare Workers
Global users
Need instant response
Value simplicity

Choose Replicate (AI) When:

Need AI capabilities (background removal, upscaling)
Value accuracy over speed
Budget allows per-request cost
Advanced operations
Batch processing acceptable

Flows Engine - How flows work
Plugins - Plugin system and available backends
Data Stores - Storage backends for processed files