first commit
Security: Sync from Public / sync-from-public (push) Has been cancelled
Test: Benchmark Nightly / build (push) Has been cancelled
Test: Benchmark Nightly / Notify Cats on failure (push) Has been cancelled
CI: Python / Checks (push) Has been cancelled
Test: Evals Python / Workflow Comparison Python (push) Has been cancelled
Util: Check Docs URLs / check-docs-urls (push) Has been cancelled
Test: Visual Storybook / Cloudflare Pages (push) Has been cancelled
Test: E2E Performance / build-and-test-performance (push) Has been cancelled
Test: Workflows Nightly / Run Workflow Tests (push) Has been cancelled
Util: Cleanup CI Docker Images / Delete stale CI images (push) Has been cancelled
Test: Benchmark Destroy Env / build (push) Has been cancelled
Util: Update Node Popularity / update-popularity (push) Has been cancelled
Test: E2E Coverage Weekly / Coverage Tests (push) Has been cancelled

This commit is contained in:
2026-03-17 16:22:57 +03:30
commit 3d5eaf9445
15349 changed files with 2847338 additions and 0 deletions
@@ -0,0 +1,426 @@
# Expression Runtime Architecture
This package provides a secure, isolated expression evaluation runtime that works across multiple execution environments (isolated-vm, Web Workers, and task runners).
## Design Goals
1. **Environment Agnostic**: Single codebase that works in Node.js (isolated-vm), browsers (Web Workers), and task runner processes
2. **Security**: Expressions run in isolated contexts with memory limits and timeouts
3. **Performance**: Lazy data loading, code caching, and efficient data transfer
4. **Observability**: Built-in metrics, traces, and logs
5. **Maintainability**: Clear separation of concerns with well-defined interfaces
## Three-Layer Architecture
The architecture is split into three distinct layers:
```
┌─────────────────────────────────────────────────────────┐
│ Host Process │
│ │
│ ┌────────────────────────────────────────────────┐ │
│ │ ExpressionEvaluator (Layer 3) │ │
│ │ - Public API │ │
│ │ - Tournament integration │ │
│ │ - Code caching │ │
│ │ - Observability │ │
│ └────────────────┬───────────────────────────────┘ │
│ │ │
│ ┌────────────────▼───────────────────────────────┐ │
│ │ Bridge (Layer 2) │ │
│ │ - IsolatedVmBridge (Phase 1.1) │ │
│ │ - WebWorkerBridge (Phase 2+) │ │
│ │ - Task Runner Integration (TBD) │ │
│ └────────────────┬───────────────────────────────┘ │
│ │ IPC/Message Passing │
└───────────────────┼─────────────────────────────────────┘
┌───────────────────▼─────────────────────────────────────┐
│ Isolated Context │
│ │
│ ┌────────────────────────────────────────────────┐ │
│ │ Runtime (Layer 1) │ │
│ │ - Runs inside isolation │ │
│ │ - No Node.js dependencies │ │
│ │ - Lazy loading proxies │ │
│ │ - Helper functions ($json, $item, etc.) │ │
│ │ - lodash, Luxon │ │
│ └────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────┘
```
### Layer 1: Runtime (Isolated Context)
**Location**: Runs inside the isolated context (isolate, worker, subprocess)
**Purpose**: Provides the JavaScript execution environment for expressions
**Key Components**:
- **Lazy Loading Proxies**: Fetch data fields on-demand from host to avoid memory limits
- **Helper Functions**: `$json`, `$item`, `$input`, `$`, etc.
- **Libraries**: lodash, Luxon (bundled)
- **No Node.js APIs**: Pure JavaScript only
**Bundle**: IIFE format for isolated-vm, ESM for Web Workers
### Layer 2: Bridge (Host Process)
**Location**: Runs in the host process
**Purpose**: Manages communication between host and isolated context
**Key Components**:
- **RuntimeBridge Interface**: Abstract interface for all bridge implementations
- **IsolatedVmBridge**: Uses isolated-vm API for Node.js backend (Phase 1.1)
- **WebWorkerBridge**: Uses postMessage API for browser (Phase 2+)
- **Task Runner Integration**: TBD - May use IsolatedVmBridge locally or direct evaluation (Phase 2+)
**Responsibilities**:
- Initialize isolated context
- Transfer code to context
- Handle data requests from runtime (lazy loading)
- Enforce memory limits and timeouts
- Dispose of context when needed
### Layer 3: Evaluator (Host Process)
**Location**: Runs in the host process
**Purpose**: Public API for expression evaluation
**Key Components**:
- **ExpressionEvaluator**: Main class used by workflow package
- **Tournament Integration**: AST transformation and security validation
- **Code Cache**: Cache transformed code (not evaluation results)
- **Observability**: Emit metrics, traces, and logs
**Responsibilities**:
- Accept expression strings and workflow data
- Transform expressions with Tournament
- Cache transformed code to avoid re-transformation
- Convert WorkflowData to WorkflowDataProxy for lazy loading
- Use bridge to evaluate in isolated context
- Handle errors gracefully
- Emit observability data
## Data Flow
### Expression Evaluation Flow
```mermaid
sequenceDiagram
participant WF as Workflow
participant Eval as ExpressionEvaluator
participant Bridge as IsolatedVmBridge
participant Runtime as Runtime (Isolated)
WF->>Eval: evaluate(expr, data)
Eval->>Eval: Transform with Tournament (cached)
Eval->>Bridge: execute(transformedCode, data)
Bridge->>Bridge: registerCallbacks(data) — creates ivm.Reference callbacks
Bridge->>Runtime: resetDataProxies() — initialise $json, $input, etc. as lazy proxies
Bridge->>Runtime: run wrapped code (this === __data)
Runtime->>Runtime: Access $json.field
Runtime->>Bridge: __getValueAtPath(['$json','field']) via ivm.Reference
Bridge->>Bridge: Navigate data object
Bridge-->>Runtime: Metadata or primitive
Runtime-->>Bridge: Expression result
Bridge-->>Eval: Result (copied from isolate)
Eval-->>WF: Result
```
### Lazy Data Loading
Data access from inside the isolate goes through `ivm.Reference` callbacks
registered by the bridge — not through a method on `RuntimeBridge` itself.
```mermaid
sequenceDiagram
participant Runtime as Runtime (Isolated)
participant Proxy as Lazy Proxy
participant Bridge as IsolatedVmBridge (host)
Runtime->>Proxy: $json.user.email
Proxy->>Bridge: __getValueAtPath(['$json','user','email']) via ivm.Reference
Bridge->>Bridge: Navigate data object registered via registerCallbacks()
Bridge-->>Proxy: "test@example.com" (primitive copied into isolate)
Proxy-->>Runtime: "test@example.com"
```
## Environment-Specific Implementations
### IsolatedVmBridge (Node.js Backend)
Uses [isolated-vm](https://github.com/laverdet/isolated-vm) for V8 isolate-based isolation:
```typescript
class IsolatedVmBridge implements RuntimeBridge {
private isolate: ivm.Isolate;
private context: ivm.Context;
async initialize(): Promise<void> {
this.isolate = new ivm.Isolate({
memoryLimit: 128
});
this.context = await this.isolate.createContext();
// Load runtime code
await this.context.eval(runtimeCode);
}
async execute(code: string, dataId: string): Promise<unknown> {
// Implementation...
}
}
```
### WebWorkerBridge (Browser Frontend)
Uses Web Workers for browser-based isolation:
```typescript
class WebWorkerBridge implements RuntimeBridge {
private worker: Worker;
async initialize(): Promise<void> {
this.worker = new Worker('/runtime.worker.js');
// Setup message handlers
}
async execute(code: string, dataId: string): Promise<unknown> {
// Implementation...
}
}
```
### Task Runner Integration (TBD - Phase 2+)
Task runners already provide process-level isolation. When code nodes call `evaluateExpression()`, evaluation happens **inside the task runner** (not via IPC to worker).
**Architecture decision pending - two options**:
**Option A**: Task runner uses `IsolatedVmBridge` locally
```typescript
// Inside task runner process
const evaluator = new ExpressionEvaluator({
bridge: new IsolatedVmBridge(config), // Evaluates locally
});
// Code node calls evaluateExpression()
const result = await evaluator.evaluate(expression, workflowData);
// ^ All happens inside task runner, no IPC, no lazy loading needed
```
**Option B**: Task runner evaluates directly (no extra sandbox)
```typescript
// Task runner already isolated at process level
// No need for isolated-vm sandbox on top
const result = evaluateExpressionDirectly(expression, workflowData);
```
**Key point**: Task runner already has all workflow data, so no lazy loading or IPC communication is needed for data access.
## Package Structure
```
packages/@n8n/expression-runtime/
├── ARCHITECTURE.md # This file
├── README.md
├── package.json
├── tsconfig.json
├── tsconfig.build.json
├── vitest.config.ts
├── esbuild.config.js # Bundles src/runtime/index.ts → dist/bundle/runtime.iife.js
├── src/
│ ├── index.ts # Public API exports
│ │
│ ├── types/ # TypeScript interfaces (no implementations)
│ │ ├── index.ts
│ │ ├── bridge.ts # RuntimeBridge, BridgeConfig
│ │ ├── evaluator.ts # IExpressionEvaluator, EvaluatorConfig, error classes
│ │ └── runtime.ts # RuntimeHostInterface, RuntimeGlobals, RuntimeError
│ │
│ ├── runtime/ # Layer 1: runs inside the V8 isolate
│ │ └── index.ts # Proxy system, resetDataProxies, __sanitize,
│ │ # SafeObject, SafeError, Lodash/Luxon wiring,
│ │ # all extension functions
│ │
│ ├── bridge/ # Layer 2: host-process isolate management
│ │ └── isolated-vm-bridge.ts # IsolatedVmBridge (ivm.Isolate, callbacks, script cache)
│ │
│ ├── evaluator/ # Layer 3: public-facing API
│ │ └── expression-evaluator.ts # Tournament integration, expression code cache
│ │
│ ├── extensions/ # Expression extension functions (bundled into runtime)
│ │ ├── array-extensions.ts
│ │ ├── boolean-extensions.ts
│ │ ├── date-extensions.ts
│ │ ├── number-extensions.ts
│ │ ├── object-extensions.ts
│ │ ├── string-extensions.ts
│ │ ├── extend.ts
│ │ ├── extensions.ts
│ │ ├── expression-extension-error.ts
│ │ └── utils.ts
│ │
│ └── __tests__/
│ └── integration.test.ts
└── dist/
├── *.js / *.d.ts # Compiled TypeScript (tsc output)
└── bundle/
└── runtime.iife.js # Self-contained IIFE loaded into isolated-vm
```
## Key Design Decisions
### 1. Why Three Layers?
**Separation of Concerns**: Each layer has a single responsibility:
- Runtime: Execute expressions in isolation
- Bridge: Handle environment-specific communication
- Evaluator: Provide clean API with observability
**Environment Agnostic**: The Runtime and Evaluator layers are identical across all environments. Only the Bridge changes.
### 2. Why Lazy Loading?
**Memory Efficiency**: Large workflow data (100MB+) cannot fit in isolate memory limits (128MB). Lazy loading fetches only the fields that expressions actually access.
**Performance**: Transferring only accessed fields is faster than transferring entire objects.
**Limitation**: Lazy loading requires **synchronous** callbacks from runtime to host. This works for:
-**isolated-vm**: Uses `ivm.Reference` for true synchronous callbacks
-**Node.js vm**: Direct synchronous function calls
-**Web Workers**: postMessage is always async (see Known Limitations below)
### 3. Why Bundle the Runtime?
**No Node.js Dependencies**: Runtime must work in environments without Node.js (browser, isolated-vm). Bundling produces a self-contained IIFE/ESM module.
**Immutability**: Bundled runtime is immutable and can be cached.
### 4. Why Abstract Bridge?
**Future-Proofing**: Frontend will use Web Workers. Backend uses isolated-vm. Abstract bridge allows adding new environments without changing other layers.
**Testing**: Integration tests use `IsolatedVmBridge` directly (see `src/__tests__/integration.test.ts`).
## Known Limitations
### Lazy Loading with Async Boundaries
JavaScript Proxy trap handlers are **synchronous**, which creates a fundamental limitation:
```javascript
const proxy = new Proxy({}, {
get(target, prop) {
// This handler MUST be synchronous
// Cannot use await or return Promise
return someValue;
}
});
```
**Impact by Environment**:
1. **isolated-vm**
- Uses `ivm.Reference` for true synchronous callbacks from isolate to host
- Full lazy loading support
2. **Node.js vm**
- Direct synchronous function calls
- Full lazy loading support (used for testing)
3. **Web Workers**
- `postMessage` is always async
- **Phase 1 Limitation**: No lazy loading, must pre-fetch all data before evaluation
- **Future Enhancement (Phase 2+)**: Explore `SharedArrayBuffer` + `Atomics` for synchronous data access
### Web Worker Support Roadmap
**Phase 1** (Initial implementation):
- WebWorkerBridge will pre-fetch all workflow data
- Transfer complete data object to worker before evaluation
- Works for small/medium datasets (< 50MB)
- No lazy loading benefit
**Phase 2+** (Future enhancement):
- Investigate `SharedArrayBuffer` + `Atomics` for sync access
- Or accept pre-fetching as the Web Worker approach
- Decision based on real-world usage patterns
### Security Boundaries
The runtime has **no access** to:
- ❌ Node.js APIs (fs, net, child_process, etc.)
- ❌ Host process memory
- ❌ Other isolates/workers
- ❌ Cookies
The runtime **can only**:
- ✅ Call back to host via `ivm.Reference` callbacks to fetch workflow data
- ✅ Access lodash and Luxon libraries
- ✅ Execute pure JavaScript code
## Testing Strategy
**Integration Tests** (vitest):
- Use `IsolatedVmBridge` with real `isolated-vm`
- Test lazy loading, helpers, error handling, security wrappers
**Bridge Tests** (vitest):
- Test each bridge implementation
- Mock environment-specific APIs
- Test memory limits, timeouts, disposal
**Evaluator Tests** (vitest):
- Test Tournament integration (transformation and validation)
- Test code caching (transformed code, not results)
- Test WorkflowData to WorkflowDataProxy conversion
- Test observability emission
- Test error handling
**Integration Tests** (jest in workflow package):
- Test full stack with real isolated-vm
- Test concurrent evaluations
- Test with real workflow data
## Observability
All layers emit metrics, traces, and logs:
**Metrics**:
- `expression.evaluation.count`
- `expression.evaluation.duration_ms`
- `expression.code_cache.hit` (transformed code cache)
- `expression.code_cache.miss`
- `expression.isolate.memory_mb`
**Traces**:
- `expression.evaluate` span wraps entire evaluation
- `expression.tournament` span for AST transformation
- `expression.isolate.execute` span for isolated execution
**Logs**:
- Errors at all levels
- Warnings for memory pressure
- Debug logs for development
See observability package documentation for details.
## Next Steps
1. Implement TypeScript interfaces (Phase 0.1)
2. Implement observability infrastructure (Phase 0.2)
3. Create comprehensive benchmarks (Phase 0.3)
4. Implement runtime package (Phase 1.1)
5. Implement isolate pooling (Phase 1.2)
## References
- [isolated-vm GitHub](https://github.com/laverdet/isolated-vm)
- [Web Workers MDN](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API)
- [n8n workflow package](../workflow/)
+305
View File
@@ -0,0 +1,305 @@
# @n8n/expression-runtime
Secure, isolated expression evaluation runtime for n8n workflows.
## Status
**In progress — landing as a series of incremental PRs.**
Implemented so far:
- ✅ TypeScript interfaces and architecture design (PR 1)
- ✅ Core architecture documentation (PR 1)
- ✅ Runtime bundle: extension functions, deep lazy proxy system (PR 2)
-`IsolatedVmBridge`: V8 isolate management via `isolated-vm` (PR 3)
-`ExpressionEvaluator`: tournament integration, expression code caching (PR 4)
- ✅ Integration tests (PR 4)
Coming in later PRs:
- 🚧 Workflow integration behind `N8N_EXPRESSION_ENGINE=vm` flag (PR 5)
- 🚧 Web Worker support (Phase 2+)
- 🚧 Performance optimizations (Phase 3)
## Overview
This package provides a secure runtime for evaluating expressions in isolated contexts.
Currently supports:
- **Node.js Backend**: Uses `isolated-vm` for V8 isolate-based isolation with lazy data loading
Future support (Phase 2+):
- **Browser Frontend**: Will use Web Workers for browser-based isolation
- **Task Runners**: Will use IPC for separate process isolation
## Features
- 🔒 **Secure**: Expressions run in isolated V8 contexts with memory limits (128MB) and timeouts (5s)
- 🚀 **Performant**: Lazy data loading via proxies, script compilation caching, and expression code caching
- 📊 **Observable**: Built-in metrics, traces, and logs support (interfaces defined; providers coming later)
- 🌐 **Universal**: Works in Node.js backend (browsers and task runners in Phase 2+)
- 🛡️ **AST Security**: Tournament AST hooks (`ThisSanitizer`, `PrototypeSanitizer`, `DollarSignValidator`) validate expressions before execution
## Architecture
The runtime uses a three-layer architecture:
1. **Runtime** (Layer 1): Runs inside isolated context, provides expression execution environment
2. **Bridge** (Layer 2): Manages communication between host and isolated context
3. **Evaluator** (Layer 3): Public API with Tournament integration and observability
See [ARCHITECTURE.md](./ARCHITECTURE.md) for detailed design documentation.
## Installation
```bash
pnpm add @n8n/expression-runtime
```
## Usage
### Basic Example
```typescript
import { ExpressionEvaluator, IsolatedVmBridge } from '@n8n/expression-runtime';
// Create bridge
const bridge = new IsolatedVmBridge({
memoryLimit: 128,
timeout: 5000,
});
// Create evaluator
const evaluator = new ExpressionEvaluator({
bridge,
});
// Initialize
await evaluator.initialize();
// Evaluate expression using {{ }} template syntax
const result = evaluator.evaluate(
'{{ $json.user.email }}',
{
$json: {
user: { email: 'test@example.com' }
}
}
);
console.log(result); // "test@example.com"
// Clean up
await evaluator.dispose();
```
### With Security Hooks (Production)
Pass AST security hooks from `expression-sandboxing.ts` to enable full security validation. This is the pattern used by the workflow package:
```typescript
import { ExpressionEvaluator, IsolatedVmBridge } from '@n8n/expression-runtime';
import {
ThisSanitizer,
PrototypeSanitizer,
DollarSignValidator,
} from 'n8n-workflow/expression-sandboxing';
const bridge = new IsolatedVmBridge({ timeout: 5000 });
const evaluator = new ExpressionEvaluator({
bridge,
hooks: {
before: [ThisSanitizer],
after: [PrototypeSanitizer, DollarSignValidator],
},
});
await evaluator.initialize();
```
When `hooks` is omitted the evaluator still runs tournament transformation (template parsing, `this` binding) but without AST security validation — suitable for development and testing.
### With Observability (Not Yet Implemented)
```typescript
import { OpenTelemetryProvider } from '@n8n/expression-runtime/observability';
const observability = new OpenTelemetryProvider({
serviceName: 'n8n-expressions',
});
const evaluator = new ExpressionEvaluator({
bridge,
observability,
});
```
**Note**: Observability providers are not yet implemented. The `ObservabilityProvider` interface exists but no implementations are available yet.
## API
### ExpressionEvaluator
Main class for expression evaluation.
```typescript
class ExpressionEvaluator {
constructor(config: EvaluatorConfig);
initialize(): Promise<void>;
evaluate(expression: string, data: WorkflowData, options?: EvaluateOptions): unknown;
dispose(): Promise<void>;
isDisposed(): boolean;
}
```
### RuntimeBridge
Abstract interface for bridge implementations.
```typescript
interface RuntimeBridge {
initialize(): Promise<void>;
execute(code: string, data: Record<string, unknown>): unknown;
dispose(): Promise<void>;
isDisposed(): boolean;
}
```
### Bridge Implementations
- **IsolatedVmBridge**: ✅ For Node.js backend (isolated-vm with V8 isolates)
- Memory isolation with hard 128MB limit
- Timeout enforcement (5s default)
- Deep lazy proxy system for workflow data
- Synchronous callbacks via ivm.Reference
- Security wrappers (SafeObject, SafeError)
- `E()` error handler for tournament-generated try-catch code
- **WebWorkerBridge**: 🚧 For browser frontend (Web Workers) - Phase 2+
- **Task Runner Integration**: 🚧 TBD - May use IsolatedVmBridge locally or direct evaluation - Phase 2+
## Configuration
```typescript
interface EvaluatorConfig {
bridge: RuntimeBridge; // required
observability?: ObservabilityProvider; // optional - interfaces defined, providers not yet implemented
hooks?: TournamentHooks; // optional - AST security hooks for tournament
}
interface BridgeConfig {
memoryLimit?: number; // Default: 128 MB
timeout?: number; // Default: 5000 ms
debug?: boolean; // Default: false
}
```
## Environment Variables (Not Yet Implemented)
```bash
# Bridge configuration (not yet implemented)
N8N_EXPRESSION_MEMORY_LIMIT_MB=128
N8N_EXPRESSION_TIMEOUT_MS=5000
N8N_EXPRESSION_DEBUG=false
# Code cache (not yet implemented - caches transformed code, not results)
N8N_EXPRESSION_CODE_CACHE_ENABLED=true
N8N_EXPRESSION_CODE_CACHE_MAX_SIZE=1000
# Observability (not yet implemented)
N8N_EXPRESSION_OBSERVABILITY_ENABLED=true
N8N_EXPRESSION_METRICS_ENABLED=true
N8N_EXPRESSION_TRACES_ENABLED=true
N8N_EXPRESSION_TRACE_SAMPLE_RATE=0.01
```
**Note**: Currently, configuration is passed via constructor options. Environment variable support will be added in future phases.
## Development
```bash
# Install dependencies
pnpm install
# Build package
pnpm build
# Run tests
pnpm test
# Run tests in watch mode
pnpm test:watch
# Type check
pnpm typecheck
# Lint
pnpm lint
```
## Testing
The package uses vitest for fast, isolated testing:
```typescript
import { ExpressionEvaluator, IsolatedVmBridge } from '@n8n/expression-runtime';
describe('ExpressionEvaluator', () => {
it('evaluates simple expression', async () => {
const bridge = new IsolatedVmBridge({ timeout: 5000 });
const evaluator = new ExpressionEvaluator({ bridge });
await evaluator.initialize();
const result = evaluator.evaluate('{{ $json.value }}', { $json: { value: 42 } });
expect(result).toBe(42);
await evaluator.dispose();
});
});
```
Run tests:
```bash
pnpm test # Run all tests
pnpm test integration # Run integration tests only
```
## Performance
The runtime uses several optimizations (implemented in PRs 24):
- **Lazy Loading**: Only fetch data fields that expressions actually access via proxy traps
- **Script Compilation Caching**: Compiled scripts are cached to avoid recompilation
- **Metadata-Driven**: Only structure (keys, lengths) transferred across isolate boundary, not full data
- **Expression Code Caching**: Tournament-transformed code is cached per evaluator instance (same expressions repeat within a workflow, so cache hit rate is high in practice)
Performance characteristics:
- Arrays: Always lazy-loaded — only length transferred, elements fetched on demand
- Objects: Always lazy-loaded — only keys transferred, values fetched on demand
## Security
The runtime enforces strict security at multiple layers (implemented in PRs 24):
- **Memory limits**: Hard 128MB limit via isolated-vm (configurable)
- **Execution timeouts**: 5s default timeout (configurable)
- **Complete isolation**: No access to Node.js APIs (require, fs, process, etc.)
- **Security wrappers**: SafeObject and SafeError prevent dangerous method access
- **Native function blocking**: Prevents access to native code
- **AST transforms**: `ThisSanitizer` rewrites `$json``this.$json`; `PrototypeSanitizer` wraps computed property access in `this.__sanitize(key)` to block prototype chain attacks; `DollarSignValidator` enforces correct `$`-variable usage
- **Runtime sanitizer**: `__sanitize()` inside the isolate blocks access to `__proto__`, `constructor`, `prototype`, and other dangerous properties at runtime
Future security features (Phase 2+):
- 🚧 Additional sandboxing for browser environments
## Contributing
See the main n8n repository for contribution guidelines.
## License
See [LICENSE.md](../../LICENSE.md) in the n8n repository root.
## Related
- [n8n workflow package](../workflow/)
- [isolated-vm](https://github.com/laverdet/isolated-vm)
- [@n8n/tournament](https://github.com/n8n-io/tournament)
@@ -0,0 +1,91 @@
%% Expression Runtime Architecture
%% Three-layer design for environment-agnostic expression evaluation
graph TB
subgraph "Host Process"
WF[Workflow Package]
subgraph "Layer 3: Evaluator"
EVAL[ExpressionEvaluator]
TOUR[Tournament]
CACHE[Code Cache]
OBS[Observability]
end
subgraph "Layer 2: Bridge"
BRIDGE_IF[RuntimeBridge Interface]
ISOVM[IsolatedVmBridge]
WEBW[WebWorkerBridge]
TASKR[TaskRunnerBridge]
end
DATASTORE[(Data Store)]
end
subgraph "Isolated Context (isolate/worker/subprocess)"
subgraph "Layer 1: Runtime"
RUNTIME[Runtime Entry]
PROXY[Lazy Proxy]
HELPERS[Helper Functions]
LODASH[lodash]
LUXON[Luxon]
end
end
WF -->|evaluate| EVAL
EVAL --> TOUR
EVAL --> CACHE
EVAL --> OBS
EVAL -->|execute| BRIDGE_IF
BRIDGE_IF -.->|implements| ISOVM
BRIDGE_IF -.->|implements| WEBW
BRIDGE_IF -.->|implements| TASKR
ISOVM -->|IPC/Reference| RUNTIME
WEBW -->|postMessage| RUNTIME
TASKR -->|IPC| RUNTIME
RUNTIME --> PROXY
RUNTIME --> HELPERS
RUNTIME --> LODASH
RUNTIME --> LUXON
PROXY -.->|getData request| ISOVM
ISOVM --> DATASTORE
DATASTORE -.->|value| ISOVM
ISOVM -.->|value| PROXY
style EVAL fill:#e1f5ff
style BRIDGE_IF fill:#fff4e1
style RUNTIME fill:#f0ffe1
style ISOVM fill:#fff4e1,stroke:#ff9800
style WEBW fill:#fff4e1,stroke:#9e9e9e,stroke-dasharray: 5 5
style TASKR fill:#fff4e1,stroke:#9e9e9e,stroke-dasharray: 5 5
%% Data Flow Sequence
sequenceDiagram
participant WF as Workflow
participant Eval as ExpressionEvaluator
participant Bridge as IsolatedVmBridge
participant Runtime as Runtime (Isolated)
WF->>Eval: evaluate(expr, data)
Eval->>Eval: Transform with Tournament
Eval->>Eval: Check code cache
Eval->>Bridge: execute(code, data)
Bridge->>Bridge: Register ivm.Reference callbacks with data
Bridge->>Runtime: evalSync("resetDataProxies()")
Runtime->>Runtime: Create lazy proxies for $json, $input, etc.
Bridge->>Runtime: Run compiled script
Runtime->>Runtime: Access $json.email
Runtime->>Bridge: __getValueAtPath(['$json','email']) [ivm.Reference]
Bridge->>Bridge: Navigate path in data
Bridge-->>Runtime: Value
Runtime-->>Bridge: Expression result
Bridge-->>Eval: Result
Eval-->>WF: Result
@@ -0,0 +1,235 @@
# Deep Lazy Proxy
## Overview
The Deep Lazy Proxy is a memory-efficient mechanism for providing workflow data to expression evaluation contexts. Instead of copying entire data structures upfront, it loads data on-demand as properties are accessed.
## Key Features
- **On-Demand Loading**: Only fetches data when accessed
- **Metadata-Driven**: Returns object structure (keys, length) without values
- **Caching**: Values are cached after first access to avoid redundant lookups
- **Type Support**: Handles objects, arrays, functions, and primitives correctly
- **Memory Efficient**: Large arrays and objects don't cause memory overhead
## Architecture
The deep lazy proxy is implemented in `src/runtime/lazy-proxy.ts`, which is bundled
together with the other runtime modules into `dist/bundle/runtime.iife.js` and injected
into the V8 isolate at startup.
Key functions exposed on `globalThis` inside the isolate:
- `createDeepLazyProxy(basePath)` — creates recursive object/array proxies
- `resetDataProxies()` — called before each evaluation to reinitialise `$json`,
`$input`, `$node`, etc. as fresh lazy proxies backed by the three host callbacks
- `__sanitize(key)` — runtime property-access guard that blocks `__proto__`,
`constructor`, `prototype`, etc.
Host-side callbacks registered by `IsolatedVmBridge` as `ivm.Reference` objects
(synchronous cross-isolate calls):
- `__getValueAtPath(path[])` — returns a primitive, array metadata, or object metadata
- `__getArrayElement(path[], index)` — returns a single array element (or its metadata)
- `__callFunctionAtPath(path[], ...args)` — invokes a host-side function and returns the result
## Usage
The proxy system runs **inside the V8 isolate** and is not directly importable from
host code. The host sets up the data context by calling `bridge.execute(code, data)`,
which internally:
1. Registers three `ivm.Reference` callbacks with the current `data` object
2. Calls `resetDataProxies()` in the isolate to create fresh lazy proxies for
`$json`, `$binary`, `$input`, `$node`, `$parameter`, `$workflow`, `$prevNode`
3. Runs the tournament-transformed expression code with `this === __data`
From the expression's perspective it just sees normal objects:
```typescript
// Inside an expression (runs in isolate):
$json.user.email // triggers getValueAtPath(['$json','user','email'])
$json.items[150].id // triggers getArrayElement(['$json','items'], 150)
$items() // triggers callFunctionAtPath(['$items'])
```
### Array metadata
Arrays are **never transferred in full** — only their length is returned. Elements
are loaded individually on demand. Length can be determined from the host object
in O(1), but serialization cost is proportional to the total byte size of all
elements, which cannot be bounded from length alone.
```typescript
// __getValueAtPath returns:
{ __isArray: true, __length: 1000 } // always metadata only
{ __isObject: true, __keys: ['name','email'] } // object — lazy
42 // primitive
```
## How It Works
### Metadata Pattern
Instead of transferring entire objects/arrays, the proxy uses metadata:
**Arrays** (all sizes):
```typescript
{
__isArray: true,
__length: 1000 // Only length; elements loaded on demand via __getArrayElement
}
```
**Objects**:
```typescript
{
__isObject: true,
__keys: ['name', 'email', 'age'] // Only keys, not values
}
```
### Caching
Once a property is accessed, it's cached in the proxy's target object:
```typescript
proxy.$json.user.name // First access: fetches via callback
proxy.$json.user.name // Second access: returns cached value
```
### Recursive Proxies
When accessing nested objects or arrays, new proxies are created:
```typescript
proxy.$json.user // Creates proxy for user object
proxy.$json.items[50] // Creates proxy for object at index 50
```
## Security
### Function Handling
- **Custom Functions**: Allowed and passed directly
- **Native Functions**: Blocked for security (e.g., `Object.keys`)
```typescript
const customFn = (x: number) => x * 2; // Allowed
const nativeFn = Object.keys; // Blocked (returns undefined)
```
Detection is done by checking if `fn.toString()` contains `'[native code]'`.
### Symbol Properties
Symbol properties return `undefined` to prevent security issues.
## Performance
### Memory Efficiency
- **Arrays**: Always lazy-loaded — only length transferred, elements fetched on demand
- **Objects**: Always lazy-loaded — only keys transferred, values fetched on demand
### Access Patterns
Best performance when:
- Accessing few properties from large objects
- Accessing specific array elements (not iterating entire array)
- Accessing the same properties multiple times (caching means only the first access pays)
Suboptimal performance when:
- Iterating entire arrays (`.map()`, `.filter()`) — each element triggers a separate callback
- Accessing most properties of large objects
- No property reuse (no benefit from caching)
## Known Limitations
1. **Array Methods**: Methods like `.map()`, `.filter()` iterate all elements.
Each element triggers a separate `__getArrayElement` callback call, which is slow
for large arrays.
- **Workaround**: Avoid iterating large arrays in expressions; access specific indices instead
2. **Circular References**: May cause infinite loops in the proxy handler.
- **Current**: No cycle detection; circular structures should be avoided in expression data
## Testing
### Integration Tests
```bash
cd packages/@n8n/expression-runtime
pnpm test
```
Test coverage:
- ✅ Basic property access
- ✅ Nested properties
- ✅ Array element access (lazy-loaded via `__getArrayElement`)
- ✅ Object proxies
- ✅ Function handling
- ✅ Caching behavior
- ✅ Edge cases (circular refs, symbols, "in" operator)
## API Reference (inside the isolate bundle)
These functions are available on `globalThis` within the V8 isolate after the
runtime bundle (`dist/bundle/runtime.iife.js`) is loaded.
### `resetDataProxies()`
Called by the bridge before each expression evaluation. Reads `$json`, `$binary`,
`$input`, `$node`, `$parameter`, `$workflow`, `$prevNode`, `$runIndex`, `$itemIndex`,
and `$items` from `__data` (populated via host callbacks) and exposes them on both
`globalThis` and `__data` so tournament-transformed code can access them via
`this.$json`, `this.$input`, etc.
### `createDeepLazyProxy(basePath)`
Creates a recursive Proxy for a given property path. Intercepts property access and
calls back to the host via `__getValueAtPath` to fetch structure metadata, then
creates nested proxies for objects or arrays as needed.
**Parameter:**
- `basePath: string[]` — path from the root data object to the node this proxy represents
## Examples
### Accessing nested data (expression syntax)
```
{{ $json.order.customer.name }} // lazy-loads order.customer.name
{{ $json.order.items[1].product }} // lazy-loads array element at index 1
{{ $json.items[0] }} // fetches only the first element
```
### Array iteration is slow for large arrays
```
{{ $json.items.reduce((sum, x) => sum + x, 0) }}
// items has 10 000 elements → length transferred, then 10 000 callback
// calls to fetch each element. Prefer accessing specific indices.
```
Note: lodash (`_`) is not available in expressions — it is bundled internally for
use by extension functions but not exposed on `globalThis`.
## Contributing
When modifying the proxy implementation:
1. **Run tests**: `pnpm test proxy`
2. **Type check**: `pnpm typecheck`
3. **Build**: `pnpm build`
4. **Add tests** for new features
5. **Update this documentation**
## Related Files
- Proxy implementation: `packages/@n8n/expression-runtime/src/runtime/lazy-proxy.ts``createDeepLazyProxy`
- Reset: `packages/@n8n/expression-runtime/src/runtime/reset.ts``resetDataProxies`
- Security globals: `packages/@n8n/expression-runtime/src/runtime/safe-globals.ts``SafeObject`, `SafeError`, `__sanitize`
- Runtime entry: `packages/@n8n/expression-runtime/src/runtime/index.ts` — wires all modules to `globalThis`
- Bridge: `packages/@n8n/expression-runtime/src/bridge/isolated-vm-bridge.ts` — registers `ivm.Reference` callbacks, loads bundle, calls `resetDataProxies`
- Build: `packages/@n8n/expression-runtime/esbuild.config.js` — bundles runtime to `dist/bundle/runtime.iife.js`
@@ -0,0 +1,155 @@
# Implementation Phases
This document maps interfaces to implementation phases to help developers focus on what's needed when.
## Phase 1.1: Core Runtime Package (MVP)
**Goal**: Basic expression evaluation working in CLI/backend
**Interfaces Needed**:
- `RuntimeBridge` - Main bridge interface
- `BridgeConfig` (without `debug` field)
- `RuntimeHostInterface` - Runtime-to-host communication
- `RuntimeGlobals` - Globals injected into runtime
- `WorkflowDataProxy` - Data access helper
- `IExpressionEvaluator` - Public API
- `EvaluatorConfig` (without observability)
- `WorkflowData` - Input data format
- `EvaluateOptions` (basic)
**Implementations Required**:
- `IsolatedVmBridge` - For CLI/backend
- `ExpressionEvaluator` - Main evaluator class
- Runtime code (runs inside isolate, bundled via esbuild)
- Lazy loading proxies
- Expression code cache (per-evaluator, caches tournament-transformed code)
**Can Skip**:
- Observability (use `NoOpProvider` stub)
- Debug mode
- Specific error types (use generic `Error`)
- Web Workers
- Task runners
## Phase 0.2: Observability Infrastructure (PARALLEL)
**Goal**: Add metrics, traces, and logs
**Interfaces Needed**:
- `ObservabilityProvider`
- `MetricsAPI`
- `TracesAPI`
- `LogsAPI`
- `Span`
**Implementations Required**:
- `NoOpProvider` (zero overhead when disabled)
- `OpenTelemetryProvider`
- `PostHogProvider` (optional)
- `CompositeProvider` (use multiple providers)
**Integration**:
- Add to `EvaluatorConfig.observability`
- Emit metrics/traces from evaluator and bridge
- Smart sampling implementation
## Phase 1.2: Isolate Pooling
**Goal**: Handle concurrent evaluations
**New Interfaces**: None (uses existing `RuntimeBridge`)
**Implementations Required**:
- `IsolatePool` class
- Pool configuration
- Acquire/release mechanism
- Disposal detection and replacement
## Phase 1.3: Extension Framework
**Goal**: 100% test compatibility
**New Interfaces**: None
**Implementation**: Extension functions in runtime
## Phase 1.4: Error Handling
**Goal**: Graceful error handling with clear messages
**Interfaces Needed**:
- `ExpressionError`
- `MemoryLimitError`
- `TimeoutError`
- `SecurityViolationError`
- `SyntaxError`
**Implementation**: Error handling in all code paths
## Phase 2+: Future Enhancements
### Web Worker Support
**Interfaces**: Already defined (same `RuntimeBridge`)
**Implementations**:
- `WebWorkerBridge`
- Runtime bundled as ESM
- Note: No lazy loading initially (pre-fetch data)
### Chrome DevTools Debugging
**Config**: `BridgeConfig.debug` field
**Implementation**:
- Inspector protocol integration
- Debug mode in IsolatedVmBridge
### Task Runner Integration (Architecture TBD)
Task runners already have process-level isolation. Expression evaluation happens **inside the task runner** (no IPC to worker needed).
**Option A**: Use `IsolatedVmBridge` locally within task runner
- Adds another sandbox layer for extra security
- Task runner creates local evaluator instance
- No lazy loading needed (task runner has all data)
**Option B**: Evaluate directly without extra sandbox
- Reuse task runner's existing process isolation
- Simpler, potentially faster
- May be sufficient given process-level isolation
**Decision pending** - will be made during Phase 2+ implementation.
---
## Quick Start Guides
### For Frontend Developers (Web Worker Integration)
**Phase 1**: Skip - Web Workers are Phase 2+
**Phase 2**: Focus on:
1. `RuntimeBridge` interface - Your bridge must implement this
2. `BridgeConfig` - Configuration options
3. `WorkflowDataProxy` - How to structure data
4. Ignore: Observability interfaces (optional)
**Key Difference**: Web Workers can't do lazy loading initially, so you'll need to pre-fetch all data before calling `execute()`.
### For CLI/Backend Developers
**Phase 1.1**: Focus on:
1. `IsolatedVmBridge` implementation
2. `ExpressionEvaluator` class
3. Runtime code (runs inside isolate)
4. Code cache implementation
**Use**: `NoOpProvider` for observability initially
**Phase 0.2**: Add real observability providers
### For Testing
Integration tests use `IsolatedVmBridge` directly (see `src/__tests__/integration.test.ts`).
All interfaces in `src/types/` are stable enough to write against before the
bridge implementation lands.
@@ -0,0 +1,41 @@
// esbuild configuration for bundling runtime code
// Runtime code runs inside isolated context (isolate/worker/subprocess)
// and must be bundled as a self-contained IIFE or ESM module
const esbuild = require('esbuild');
const path = require('path');
async function build() {
const sharedOptions = {
entryPoints: [path.join(__dirname, 'src/runtime/index.ts')],
bundle: true,
minify: true,
sourcemap: true,
target: 'es2020',
platform: 'neutral', // Works in both Node.js and browser
mainFields: ['module', 'main'], // Needed for neutral platform to resolve packages
external: [], // Bundle everything (lodash, luxon)
};
// IIFE bundle for isolated-vm
await esbuild.build({
...sharedOptions,
format: 'iife',
globalName: '__n8nRuntime',
outfile: path.join(__dirname, 'dist/bundle/runtime.iife.js'),
});
// ESM bundle for Web Workers
await esbuild.build({
...sharedOptions,
format: 'esm',
outfile: path.join(__dirname, 'dist/bundle/runtime.esm.js'),
});
console.log('✅ Runtime bundles created successfully');
}
build().catch((error) => {
console.error('❌ Build failed:', error);
process.exit(1);
});
@@ -0,0 +1,46 @@
{
"name": "@n8n/expression-runtime",
"version": "0.3.0",
"description": "Secure, isolated expression evaluation runtime for n8n",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"scripts": {
"build": "tsc -p tsconfig.build.json && pnpm build:runtime",
"build:runtime": "node esbuild.config.js",
"test": "vitest run",
"test:dev": "vitest --watch --silent false",
"typecheck": "tsc --noEmit"
},
"keywords": [
"n8n",
"expression",
"evaluation",
"isolated-vm",
"web-worker",
"security"
],
"license": "SEE LICENSE IN LICENSE.md",
"dependencies": {
"@n8n/tournament": "1.0.6",
"isolated-vm": "^6.0.2",
"js-base64": "catalog:",
"jssha": "3.3.1",
"lodash": "catalog:",
"luxon": "catalog:",
"md5": "2.3.0",
"title-case": "3.0.3",
"transliteration": "2.3.5"
},
"devDependencies": {
"@types/lodash": "catalog:",
"@types/luxon": "3.2.0",
"@types/md5": "^2.3.5",
"typescript": "catalog:",
"vitest": "catalog:"
},
"files": [
"dist",
"ARCHITECTURE.md",
"LICENSE.md"
]
}
@@ -0,0 +1,205 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { ExpressionEvaluator } from '../evaluator/expression-evaluator';
import { IsolatedVmBridge } from '../bridge/isolated-vm-bridge';
import { TimeoutError, MemoryLimitError } from '../types';
describe('Integration: ExpressionEvaluator + IsolatedVmBridge', () => {
let evaluator: ExpressionEvaluator;
beforeAll(async () => {
const bridge = new IsolatedVmBridge({ timeout: 5000 });
evaluator = new ExpressionEvaluator({ bridge });
await evaluator.initialize();
});
afterAll(async () => {
await evaluator.dispose();
});
it('should evaluate simple property access', async () => {
const data = {
$json: { email: 'test@example.com' },
};
const result = evaluator.evaluate('{{ $json.email }}', data);
expect(result).toBe('test@example.com');
});
it('should evaluate nested property access', async () => {
const data = {
$json: {
user: {
profile: {
name: 'John Doe',
},
},
},
};
const result = evaluator.evaluate('{{ $json.user.profile.name }}', data);
expect(result).toBe('John Doe');
});
it('should evaluate array access', async () => {
const data = {
$json: {
items: [{ id: 1 }, { id: 2 }, { id: 3 }],
},
};
const result = evaluator.evaluate('{{ $json.items[1].id }}', data);
expect(result).toBe(2);
});
it('should evaluate math operations', async () => {
const data = {
$json: {
price: 100,
quantity: 3,
},
};
const result = evaluator.evaluate('{{ $json.price * $json.quantity }}', data);
expect(result).toBe(300);
});
it('should use luxon DateTime', async () => {
const data = {
$json: {
date: '2024-01-15',
},
};
const result = evaluator.evaluate(
'{{ DateTime.fromISO($json.date).toFormat("MMMM dd, yyyy") }}',
data,
{},
);
expect(result).toBe('January 15, 2024');
});
it('should invoke functions from workflow data', async () => {
const data = {
$items: function () {
return 'items-result';
},
};
const result = evaluator.evaluate('{{ $items() }}', data);
expect(result).toBe('items-result');
});
it('should evaluate zero values', async () => {
const data = {
$json: { zero: 0 },
};
const result = evaluator.evaluate('{{ $json.zero }}', data);
expect(result).toBe(0);
});
it('should evaluate empty string values', async () => {
const data = {
$json: { empty: '' },
};
const result = evaluator.evaluate('{{ $json.empty }}', data);
expect(result).toBe('');
});
it('should evaluate array index 0 (falsy index)', async () => {
const data = {
$json: { items: ['first', 'second'] },
};
const result = evaluator.evaluate('{{ $json.items[0] }}', data);
expect(result).toBe('first');
});
it('should evaluate primitive array elements', async () => {
const data = {
$json: { numbers: [42, 99] },
};
const result = evaluator.evaluate('{{ $json.numbers[0] }}', data);
expect(result).toBe(42);
});
it('should evaluate array .length', async () => {
const data = {
$json: { items: [1, 2, 3] },
};
const result = evaluator.evaluate('{{ $json.items.length }}', data);
expect(result).toBe(3);
});
it('should evaluate null values', async () => {
const data = {
$json: { field: null },
};
const result = evaluator.evaluate('{{ $json.field }}', data);
expect(result).toBeNull();
});
it('should evaluate boolean values', async () => {
const data = {
$json: { active: true },
};
const result = evaluator.evaluate('{{ $json.active }}', data);
expect(result).toBe(true);
});
it('should handle large arrays with lazy loading', async () => {
const data = {
$json: {
// Create array with 200 items to exercise lazy loading
items: Array.from({ length: 200 }, (_, i) => ({ id: i })),
},
};
// Access element deep in the array via lazy proxy
const result = evaluator.evaluate('{{ $json.items[150].id }}', data);
expect(result).toBe(150);
});
});
describe('Integration: IsolatedVmBridge error handling', () => {
it('should throw TimeoutError when expression exceeds timeout', async () => {
const bridge = new IsolatedVmBridge({ timeout: 100 });
await bridge.initialize();
try {
expect(() => bridge.execute('while(true){}', {})).toThrow(TimeoutError);
} finally {
await bridge.dispose();
}
});
it('should throw MemoryLimitError when expression exceeds memory limit', async () => {
const bridge = new IsolatedVmBridge({ memoryLimit: 8 });
await bridge.initialize();
try {
expect(() =>
bridge.execute('let a=[]; while(true){a.push(new Array(1000000).fill(1))}', {}),
).toThrow(MemoryLimitError);
} finally {
await bridge.dispose();
}
});
});
@@ -0,0 +1,505 @@
import ivm from 'isolated-vm';
import { readFile } from 'node:fs/promises';
import * as path from 'node:path';
import { fileURLToPath } from 'node:url';
import type { RuntimeBridge, BridgeConfig } from '../types';
import { DEFAULT_BRIDGE_CONFIG, TimeoutError, MemoryLimitError } from '../types';
// Get __dirname equivalent for ES modules
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
/**
* IsolatedVmBridge - Runtime bridge using isolated-vm for secure expression evaluation.
*
* This bridge creates a V8 isolate with:
* - Hard memory limit (128MB default)
* - No access to Node.js APIs
* - Timeout enforcement
* - Complete isolation from host process
*
* Context reuse pattern: Create isolate/context once, reset state between evaluations.
*/
export class IsolatedVmBridge implements RuntimeBridge {
private isolate: ivm.Isolate;
private context?: ivm.Context;
private initialized = false;
private disposed = false;
private config: Required<BridgeConfig>;
// Script compilation cache for performance
// Maps expression code -> compiled ivm.Script
private scriptCache = new Map<string, ivm.Script>();
// Active ivm.Reference callbacks — released before each re-registration
// to prevent reference accumulation across execute() calls
private valueAtPathRef?: ivm.Reference;
private arrayElementRef?: ivm.Reference;
private callFunctionRef?: ivm.Reference;
constructor(config: BridgeConfig = {}) {
this.config = {
...DEFAULT_BRIDGE_CONFIG,
...config,
};
// Create isolate with memory limit
// Note: memoryLimit is in MB
this.isolate = new ivm.Isolate({ memoryLimit: this.config.memoryLimit });
}
/**
* Initialize the isolate and create execution context.
*
* Steps:
* 1. Create context
* 2. Set up basic globals (global reference)
* 3. Load runtime bundle (DateTime, extend, proxy system)
* 4. Verify proxy system
*
* Must be called before execute().
*/
async initialize(): Promise<void> {
if (this.initialized) {
return;
}
// Create context in the isolate
this.context = await this.isolate.createContext();
// Set up basic globals
// jail is a reference to the context's global object
const jail = this.context.global;
// Set 'global' to reference itself (pattern from POC)
// This allows code in isolate to access 'global.something'
await jail.set('global', jail.derefInto());
// Load runtime bundle (DateTime, extend, SafeObject, proxy system)
await this.loadVendorLibraries();
// Verify proxy system loaded correctly
await this.verifyProxySystem();
// Inject E() error handler needed by tournament-generated try-catch code
await this.injectErrorHandler();
this.initialized = true;
if (this.config.debug) {
console.log('[IsolatedVmBridge] Initialized successfully');
}
}
/**
* Load runtime bundle into the isolate.
*
* The runtime bundle includes:
* - DateTime, extend, extendOptional (expression engine globals)
* - SafeObject and SafeError wrappers
* - createDeepLazyProxy function
* - __data object initialization
*
* @private
* @throws {Error} If context not initialized or bundle loading fails
*/
private async loadVendorLibraries(): Promise<void> {
if (!this.context) {
throw new Error('Context not initialized');
}
try {
// Load runtime bundle (includes vendor libraries + proxy system)
// Path: dist/bundle/runtime.iife.js
const runtimeBundlePath = path.join(__dirname, '../../dist/bundle/runtime.iife.js');
const runtimeBundle = await readFile(runtimeBundlePath, 'utf-8');
// Evaluate bundle in isolate context
// This makes all exported globals available (DateTime, extend, extendOptional, SafeObject, SafeError, createDeepLazyProxy, resetDataProxies, __data)
await this.context.eval(runtimeBundle);
if (this.config.debug) {
console.log('[IsolatedVmBridge] Runtime bundle loaded from:', runtimeBundlePath);
}
// Verify vendor libraries loaded correctly
const hasDateTime = await this.context.eval('typeof DateTime !== "undefined"');
const hasExtend = await this.context.eval('typeof extend !== "undefined"');
if (!hasDateTime || !hasExtend) {
throw new Error(
`Library verification failed: DateTime=${hasDateTime}, extend=${hasExtend}`,
);
}
if (this.config.debug) {
console.log('[IsolatedVmBridge] Vendor libraries verified successfully');
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
throw new Error(`Failed to load runtime bundle: ${errorMessage}`);
}
}
/**
* Verify the proxy system loaded correctly.
*
* The proxy system is loaded as part of the runtime bundle in loadVendorLibraries().
* This method verifies all required components are available.
*
* @private
* @throws {Error} If context not initialized or proxy system verification fails
*/
private async verifyProxySystem(): Promise<void> {
if (!this.context) {
throw new Error('Context not initialized');
}
try {
// Verify proxy system components loaded correctly
const hasProxyCreator = await this.context.eval('typeof createDeepLazyProxy !== "undefined"');
const hasData = await this.context.eval('typeof __data !== "undefined"');
const hasSafeObject = await this.context.eval('typeof SafeObject !== "undefined"');
const hasSafeError = await this.context.eval('typeof SafeError !== "undefined"');
const hasResetFunction = await this.context.eval('typeof resetDataProxies !== "undefined"');
if (!hasProxyCreator || !hasData || !hasSafeObject || !hasSafeError || !hasResetFunction) {
throw new Error(
`Proxy system verification failed: ` +
`createDeepLazyProxy=${hasProxyCreator}, __data=${hasData}, ` +
`SafeObject=${hasSafeObject}, SafeError=${hasSafeError}, ` +
`resetDataProxies=${hasResetFunction}`,
);
}
if (this.config.debug) {
console.log('[IsolatedVmBridge] Proxy system verified successfully');
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
throw new Error(`Failed to verify proxy system: ${errorMessage}`);
}
}
/**
* Inject the E() error handler into the isolate context.
*
* Tournament wraps expressions with try-catch that calls E(error, this).
* This handler:
* - Re-throws security violations from __sanitize
* - Swallows TypeErrors (failed attack attempts return undefined)
* - Re-throws all other errors
*
* @private
* @throws {Error} If context not initialized
*/
private async injectErrorHandler(): Promise<void> {
if (!this.context) {
throw new Error('Context not initialized');
}
await this.context.eval(`
if (typeof E === 'undefined') {
globalThis.E = function(error, _context) {
// Re-throw security violations from __sanitize
if (error && error.message && error.message.includes('due to security concerns')) {
throw error;
}
// Swallow TypeErrors (failed attack attempts return undefined)
if (error instanceof TypeError) {
return undefined;
}
throw error;
};
}
`);
if (this.config.debug) {
console.log('[IsolatedVmBridge] Error handler injected successfully');
}
}
/**
* Reset data proxies in the isolate context.
*
* This method should be called before each execute() to:
* 1. Clear proxy caches from previous evaluations
* 2. Initialize fresh workflow data references
* 3. Expose workflow properties to globalThis
*
* The reset function runs in the isolate and calls back to the host
* via ivm.Reference callbacks to fetch workflow data.
*
* @private
* @throws {Error} If context not initialized or reset fails
*/
private resetDataProxies(): void {
if (!this.context) {
throw new Error('Context not initialized');
}
try {
// Call the resetDataProxies function in the isolate
// This function is loaded as part of the runtime bundle
this.context.evalSync('resetDataProxies()');
if (this.config.debug) {
console.log('[IsolatedVmBridge] Data proxies reset successfully');
}
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
throw new Error(`Failed to reset data proxies: ${errorMessage}`);
}
}
/**
* Register callback functions for cross-isolate communication.
*
* Creates three ivm.Reference callbacks that the runtime bundle uses
* to fetch data from the host process:
*
* - __getValueAtPath: Returns metadata or primitive for a property path
* - __getArrayElement: Returns individual array elements
* - __callFunctionAtPath: Executes functions in host context
*
* These callbacks are called synchronously from isolate proxy traps.
*
* @param data - Current workflow data to use for callback responses
* @private
*/
private registerCallbacks(data: Record<string, unknown>): void {
if (!this.context) {
throw new Error('Context not initialized');
}
// Callback 1: Get value/metadata at path
// Used by createDeepLazyProxy when accessing properties
const getValueAtPath = new ivm.Reference((path: string[]) => {
// Navigate to value
let value: unknown = data;
for (const key of path) {
value = (value as Record<string, unknown>)?.[key];
if (value === undefined || value === null) {
return value;
}
}
// Handle functions - return metadata marker
if (typeof value === 'function') {
const fnString = value.toString();
// Block native functions for security
if (fnString.includes('[native code]')) {
return undefined;
}
return { __isFunction: true, __name: path[path.length - 1] };
}
// Handle arrays - always lazy, only transfer length
if (Array.isArray(value)) {
return {
__isArray: true,
__length: value.length,
__data: null,
};
}
// Handle objects - return metadata with keys
if (value !== null && typeof value === 'object') {
return {
__isObject: true,
__keys: Object.keys(value),
};
}
// Primitive value
return value;
});
// Callback 2: Get array element at index
// Used by array proxy when accessing numeric indices
const getArrayElement = new ivm.Reference((path: string[], index: number) => {
// Navigate to array
let arr: unknown = data;
for (const key of path) {
arr = (arr as Record<string, unknown>)?.[key];
if (arr === undefined || arr === null) {
return undefined;
}
}
if (!Array.isArray(arr)) {
return undefined;
}
const element = arr[index];
// If element is object/array, return metadata
if (element !== null && typeof element === 'object') {
if (Array.isArray(element)) {
return {
__isArray: true,
__length: element.length,
__data: null,
};
}
return {
__isObject: true,
__keys: Object.keys(element),
};
}
// Primitive element
return element;
});
// Callback 3: Call function at path with arguments
// Used when expressions invoke functions from workflow data
const callFunctionAtPath = new ivm.Reference((path: string[], ...args: unknown[]) => {
// Navigate to function, tracking parent to preserve `this` context
let fn: unknown = data;
let parent: unknown = undefined;
for (const key of path) {
parent = fn;
fn = (fn as Record<string, unknown>)?.[key];
}
if (typeof fn !== 'function') {
throw new Error(`${path.join('.')} is not a function`);
}
// Block native functions for security (same check as getValueAtPath)
if (fn.toString().includes('[native code]')) {
throw new Error(`${path.join('.')} is a native function and cannot be called`);
}
// Execute function with parent as `this` to preserve method context
return (fn as (...fnArgs: unknown[]) => unknown).call(parent, ...args);
});
// Release previous references before replacing to avoid accumulation
this.valueAtPathRef?.release();
this.arrayElementRef?.release();
this.callFunctionRef?.release();
// Store references so they can be released on the next call or on dispose()
this.valueAtPathRef = getValueAtPath;
this.arrayElementRef = getArrayElement;
this.callFunctionRef = callFunctionAtPath;
// Register all callbacks in isolate global context
this.context.global.setSync('__getValueAtPath', getValueAtPath);
this.context.global.setSync('__getArrayElement', getArrayElement);
this.context.global.setSync('__callFunctionAtPath', callFunctionAtPath);
if (this.config.debug) {
console.log('[IsolatedVmBridge] Callbacks registered successfully');
}
}
/**
* Execute JavaScript code in the isolated context.
*
* Flow:
* 1. Register callbacks as ivm.Reference for cross-isolate communication
* 2. Call resetDataProxies() to initialize workflow data proxies
* 3. Compile script (with caching for performance)
* 4. Execute with timeout enforcement
* 5. Return result (copied from isolate)
*
* @param code - JavaScript expression to evaluate
* @param data - Workflow data (e.g., { $json: {...}, $runIndex: 0 })
* @returns Result of the expression
* @throws {Error} If bridge not initialized or execution fails
*/
execute(code: string, data: Record<string, unknown>): unknown {
if (!this.initialized || !this.context) {
throw new Error('Bridge not initialized. Call initialize() first.');
}
try {
// Step 1: Register callbacks with current data context
this.registerCallbacks(data);
// Step 2: Reset proxies for this evaluation
// This initializes $json, $binary, etc. as lazy proxies
this.resetDataProxies();
// Step 3: Wrap transformed code so 'this' === __data in the isolate.
// Tournament generates: this.$json.email, this.$items(), etc.
// __data has $json, $items, etc. as lazy proxies (set in resetDataProxies).
const wrappedCode = `(function() {\n${code}\n}).call(__data)`;
let script = this.scriptCache.get(code);
if (!script) {
script = this.isolate.compileScriptSync(wrappedCode);
this.scriptCache.set(code, script);
if (this.config.debug) {
console.log('[IsolatedVmBridge] Script compiled and cached');
}
}
// Step 4: Execute with timeout and copy result back
const result = script.runSync(this.context, {
timeout: this.config.timeout,
copy: true,
});
if (this.config.debug) {
console.log('[IsolatedVmBridge] Expression executed successfully');
}
return result;
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
if (errorMessage.includes('Script execution timed out')) {
throw new TimeoutError(`Expression timed out after ${this.config.timeout}ms`, {});
}
if (errorMessage.includes('memory limit')) {
throw new MemoryLimitError(
`Expression exceeded memory limit of ${this.config.memoryLimit}MB`,
{},
);
}
throw new Error(`Expression evaluation failed: ${errorMessage}`);
}
}
/**
* Dispose of the isolate and free resources.
*
* After disposal, the bridge cannot be used again.
*/
async dispose(): Promise<void> {
if (this.disposed) {
return;
}
// Dispose isolate (this also disposes all contexts, references, etc.)
if (!this.isolate.isDisposed) {
this.isolate.dispose();
}
// Release callback references
this.valueAtPathRef?.release();
this.arrayElementRef?.release();
this.callFunctionRef?.release();
this.disposed = true;
this.initialized = false;
this.scriptCache.clear();
if (this.config.debug) {
console.log('[IsolatedVmBridge] Disposed');
}
}
/**
* Check if the bridge has been disposed.
*
* @returns true if disposed, false otherwise
*/
isDisposed(): boolean {
return this.disposed;
}
}
@@ -0,0 +1,92 @@
import { Tournament } from '@n8n/tournament';
import type {
IExpressionEvaluator,
EvaluatorConfig,
WorkflowData,
EvaluateOptions,
} from '../types';
export class ExpressionEvaluator implements IExpressionEvaluator {
private config: EvaluatorConfig;
private disposed = false;
// Lazy-initialized tournament instance (expensive to create, reused across evaluations)
private tournament?: Tournament;
// Cache: template expression → tournament-transformed JavaScript code
// Cache hit rate in production: ~99.9% (same expressions repeat within a workflow)
private codeCache = new Map<string, string>();
constructor(config: EvaluatorConfig) {
this.config = config;
}
async initialize(): Promise<void> {
await this.config.bridge.initialize();
}
evaluate(expression: string, data: WorkflowData, _options?: EvaluateOptions): unknown {
if (this.disposed) throw new Error('Evaluator disposed');
// Transform template expression → sanitized JavaScript (cached)
const transformedCode = this.getTransformedCode(expression);
try {
const result = this.config.bridge.execute(transformedCode, data);
if (this.config.observability) {
this.config.observability.metrics.counter('expression.evaluation.success', 1);
}
return result;
} catch (error) {
if (this.config.observability) {
this.config.observability.metrics.counter('expression.evaluation.error', 1);
}
throw error;
}
}
/**
* Transform a template expression to executable JavaScript via tournament.
*
* Input: "{{ $json.email }}"
* Output: JavaScript string with tournament security transforms applied
* ($json → this.$json, computed access wrapped in this.__sanitize(), etc.)
*
* Result is cached by expression string (tournament AST parsing is expensive).
*/
private getTransformedCode(expression: string): string {
const cached = this.codeCache.get(expression);
if (cached !== undefined) {
return cached;
}
if (!this.tournament) {
// Tournament requires an errorHandler but we only use getExpressionCode()
// for AST transformation — we never call tournament.execute(), so this
// handler is never invoked. Runtime errors are handled by the bridge's
// own E() injection in injectErrorHandler().
const errorHandler = () => {};
this.tournament = new Tournament(errorHandler, undefined, undefined, {
before: this.config.hooks?.before ?? [],
after: this.config.hooks?.after ?? [],
});
}
const [transformedCode] = this.tournament.getExpressionCode(expression);
this.codeCache.set(expression, transformedCode);
return transformedCode;
}
async dispose(): Promise<void> {
this.disposed = true;
this.codeCache.clear();
await this.config.bridge.dispose();
}
isDisposed(): boolean {
return this.disposed;
}
}
@@ -0,0 +1,705 @@
import isEqual from 'lodash/isEqual';
import uniqWith from 'lodash/uniqWith';
import type { Extension, ExtensionMap } from './extensions';
import { ExpressionExtensionError } from './expression-extension-error';
import { compact as oCompact } from './object-extensions';
function randomInt(max: number): number {
return crypto.getRandomValues(new Uint32Array(1))[0] % max;
}
function first(value: unknown[]): unknown {
return value[0];
}
function isEmpty(value: unknown[]): boolean {
return value.length === 0;
}
function isNotEmpty(value: unknown[]): boolean {
return value.length > 0;
}
function last(value: unknown[]): unknown {
return value[value.length - 1];
}
function pluck(value: unknown[], extraArgs: unknown[]): unknown[] {
if (!Array.isArray(extraArgs)) {
throw new ExpressionExtensionError('arguments must be passed to pluck');
}
if (!extraArgs || extraArgs.length === 0) {
return value;
}
const plucked = value.reduce<unknown[]>((pluckedFromObject, current) => {
if (current && typeof current === 'object') {
const p: unknown[] = [];
Object.keys(current).forEach((k) => {
(extraArgs as string[]).forEach((field) => {
if (current && field === k) {
p.push((current as { [key: string]: unknown })[k]);
}
});
});
if (p.length > 0) {
pluckedFromObject.push(p.length === 1 ? p[0] : p);
}
}
return pluckedFromObject;
}, new Array<unknown>());
return plucked;
}
function randomItem(value: unknown[]): unknown {
const len = value === undefined ? 0 : value.length;
return len ? value[randomInt(len)] : undefined;
}
function unique(value: unknown[], extraArgs: string[]): unknown[] {
const mapForEqualityCheck = (item: unknown): unknown => {
if (extraArgs.length > 0 && item && typeof item === 'object') {
return extraArgs.reduce<Record<string, unknown>>((acc, key) => {
acc[key] = (item as Record<string, unknown>)[key];
return acc;
}, {});
}
return item;
};
return uniqWith(value, (a, b) => isEqual(mapForEqualityCheck(a), mapForEqualityCheck(b)));
}
const ensureNumberArray = (arr: unknown[], { fnName }: { fnName: string }) => {
if (arr.some((i) => typeof i !== 'number')) {
throw new ExpressionExtensionError(`${fnName}(): all array elements must be numbers`);
}
};
function sum(value: unknown[]): number {
ensureNumberArray(value, { fnName: 'sum' });
return value.reduce((p: number, c: unknown) => {
if (typeof c === 'string') {
return p + parseFloat(c);
}
if (typeof c !== 'number') {
return NaN;
}
return p + c;
}, 0);
}
function min(value: unknown[]): number {
ensureNumberArray(value, { fnName: 'min' });
return Math.min(
...value.map((v) => {
if (typeof v === 'string') {
return parseFloat(v);
}
if (typeof v !== 'number') {
return NaN;
}
return v;
}),
);
}
function max(value: unknown[]): number {
ensureNumberArray(value, { fnName: 'max' });
return Math.max(
...value.map((v) => {
if (typeof v === 'string') {
return parseFloat(v);
}
if (typeof v !== 'number') {
return NaN;
}
return v;
}),
);
}
export function average(value: unknown[]) {
ensureNumberArray(value, { fnName: 'average' });
// This would usually be NaN but I don't think users
// will expect that
if (value.length === 0) {
return 0;
}
return sum(value) / value.length;
}
function compact(value: unknown[]): unknown[] {
return value
.filter((v) => {
if (v && typeof v === 'object' && Object.keys(v).length === 0) return false;
return v !== null && v !== undefined && v !== 'nil' && v !== '';
})
.map((v) => {
if (typeof v === 'object' && v !== null) {
return oCompact(v);
}
return v;
});
}
function smartJoin(value: unknown[], extraArgs: string[]): object {
const [keyField, valueField] = extraArgs;
if (!keyField || !valueField || typeof keyField !== 'string' || typeof valueField !== 'string') {
throw new ExpressionExtensionError(
'smartJoin(): expected two string args, e.g. .smartJoin("name", "value")',
);
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any, @typescript-eslint/no-unsafe-return
return value.reduce<any>((o, v) => {
if (typeof v === 'object' && v !== null && keyField in v && valueField in v) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-explicit-any
o[(v as any)[keyField]] = (v as any)[valueField];
}
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
return o;
}, {});
}
function chunk(value: unknown[], extraArgs: number[]) {
const [chunkSize] = extraArgs;
if (typeof chunkSize !== 'number' || chunkSize === 0) {
throw new ExpressionExtensionError('chunk(): expected non-zero numeric arg, e.g. .chunk(5)');
}
const chunks: unknown[][] = [];
for (let i = 0; i < value.length; i += chunkSize) {
chunks.push(value.slice(i, i + chunkSize));
}
return chunks;
}
function renameKeys(value: unknown[], extraArgs: string[]): unknown[] {
if (extraArgs.length === 0 || extraArgs.length % 2 !== 0) {
throw new ExpressionExtensionError(
'renameKeys(): expected an even amount of args: from1, to1 [, from2, to2, ...]. e.g. .renameKeys("name", "title")',
);
}
return value.map((v) => {
if (typeof v !== 'object' || v === null) {
return v;
}
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-explicit-any
const newObj = { ...(v as any) };
const chunkedArgs = chunk(extraArgs, [2]) as string[][];
chunkedArgs.forEach(([from, to]) => {
if (from in newObj) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-unsafe-member-access
newObj[to] = newObj[from];
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
delete newObj[from];
}
});
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
return newObj;
});
}
function mergeObjects(value: Record<string, unknown>, extraArgs: unknown[]): unknown {
const [other] = extraArgs;
if (!other) {
return value;
}
if (typeof other !== 'object') {
throw new ExpressionExtensionError('merge(): expected object arg');
}
const newObject = { ...value };
for (const [key, val] of Object.entries(other)) {
if (!(key in newObject)) {
newObject[key] = val;
}
}
return newObject;
}
function merge(value: unknown[], extraArgs: unknown[][]): unknown {
const [others] = extraArgs;
if (others === undefined) {
// If there are no arguments passed, merge all objects within the array
const merged = value.reduce((combined, current) => {
if (current !== null && typeof current === 'object' && !Array.isArray(current)) {
combined = mergeObjects(combined as Record<string, unknown>, [current]);
}
return combined;
}, {});
return merged;
}
if (!Array.isArray(others)) {
throw new ExpressionExtensionError(
'merge(): expected array arg, e.g. .merge([{ id: 1, otherValue: 3 }])',
);
}
const listLength = value.length > others.length ? value.length : others.length;
let merged = {};
for (let i = 0; i < listLength; i++) {
if (value[i] !== undefined) {
if (typeof value[i] === 'object' && typeof others[i] === 'object') {
merged = Object.assign(
merged,
mergeObjects(value[i] as Record<string, unknown>, [others[i]]),
);
}
}
}
return merged;
}
function union(value: unknown[], extraArgs: unknown[][]): unknown[] {
const [others] = extraArgs;
if (!Array.isArray(others)) {
throw new ExpressionExtensionError('union(): expected array arg, e.g. .union([1, 2, 3, 4])');
}
const newArr: unknown[] = Array.from(value);
for (const v of others) {
if (newArr.findIndex((w) => isEqual(w, v)) === -1) {
newArr.push(v);
}
}
return unique(newArr, []);
}
function difference(value: unknown[], extraArgs: unknown[][]): unknown[] {
const [others] = extraArgs;
if (!Array.isArray(others)) {
throw new ExpressionExtensionError(
'difference(): expected array arg, e.g. .difference([1, 2, 3, 4])',
);
}
const newArr: unknown[] = [];
for (const v of value) {
if (others.findIndex((w) => isEqual(w, v)) === -1) {
newArr.push(v);
}
}
return unique(newArr, []);
}
function intersection(value: unknown[], extraArgs: unknown[][]): unknown[] {
const [others] = extraArgs;
if (!Array.isArray(others)) {
throw new ExpressionExtensionError(
'intersection(): expected array arg, e.g. .intersection([1, 2, 3, 4])',
);
}
const newArr: unknown[] = [];
for (const v of value) {
if (others.findIndex((w) => isEqual(w, v)) !== -1) {
newArr.push(v);
}
}
for (const v of others) {
if (value.findIndex((w) => isEqual(w, v)) !== -1) {
newArr.push(v);
}
}
return unique(newArr, []);
}
function append(value: unknown[], extraArgs: unknown[][]): unknown[] {
return value.concat(extraArgs);
}
export function toJsonString(value: unknown[]) {
return JSON.stringify(value);
}
export function toInt() {
return undefined;
}
export function toFloat() {
return undefined;
}
export function toBoolean() {
return undefined;
}
export function toDateTime() {
return undefined;
}
average.doc = {
name: 'average',
aliases: ['mean'],
description:
'Returns the average of the numbers in the array. Throws an error if there are any non-numbers.',
examples: [{ example: '[12, 1, 5].average()', evaluated: '6' }],
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-average',
};
compact.doc = {
name: 'compact',
aliases: ['removeEmpty'],
description:
'Removes any empty values from the array. <code>null</code>, <code>""</code> and <code>undefined</code> count as empty.',
examples: [{ example: '[2, null, 1, ""].compact()', evaluated: '[2, 1]' }],
returnType: 'Array',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-compact',
};
isEmpty.doc = {
name: 'isEmpty',
description: 'Returns <code>true</code> if the array has no elements or is <code>null</code>',
examples: [
{ example: '[].isEmpty()', evaluated: 'true' },
{ example: "['quick', 'brown', 'fox'].isEmpty()", evaluated: 'false' },
],
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-isEmpty',
};
isNotEmpty.doc = {
name: 'isNotEmpty',
description: 'Returns <code>true</code> if the array has at least one element',
examples: [
{ example: "['quick', 'brown', 'fox'].isNotEmpty()", evaluated: 'true' },
{ example: '[].isNotEmpty()', evaluated: 'false' },
],
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-isNotEmpty',
};
first.doc = {
name: 'first',
aliases: ['head'],
description: 'Returns the first element of the array',
examples: [{ example: "['quick', 'brown', 'fox'].first()", evaluated: "'quick'" }],
returnType: 'any',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-first',
};
last.doc = {
name: 'last',
aliases: ['tail'],
description: 'Returns the last element of the array',
examples: [{ example: "['quick', 'brown', 'fox'].last()", evaluated: "'fox'" }],
returnType: 'any',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-last',
};
max.doc = {
name: 'max',
description:
'Returns the largest number in the array. Throws an error if there are any non-numbers.',
examples: [{ example: '[1, 12, 5].max()', evaluated: '12' }],
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-max',
};
min.doc = {
name: 'min',
description:
'Returns the smallest number in the array. Throws an error if there are any non-numbers.',
examples: [{ example: '[12, 1, 5].min()', evaluated: '1' }],
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-min',
};
randomItem.doc = {
name: 'randomItem',
description: 'Returns a randomly-chosen element from the array',
examples: [
{ example: "['quick', 'brown', 'fox'].randomItem()", evaluated: "'brown'" },
{ example: "['quick', 'brown', 'fox'].randomItem()", evaluated: "'quick'" },
],
returnType: 'any',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-randomItem',
};
sum.doc = {
name: 'sum',
description:
'Returns the total of all the numbers in the array. Throws an error if there are any non-numbers.',
examples: [{ example: '[12, 1, 5].sum()', evaluated: '18' }],
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-sum',
};
chunk.doc = {
name: 'chunk',
description: 'Splits the array into an array of sub-arrays, each with the given length',
examples: [{ example: '[1, 2, 3, 4, 5, 6].chunk(2)', evaluated: '[[1,2],[3,4],[5,6]]' }],
returnType: 'Array',
args: [
{
name: 'length',
optional: false,
description: 'The number of elements in each chunk',
type: 'number',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-chunk',
};
difference.doc = {
name: 'difference',
description:
"Compares two arrays. Returns all elements in the base array that aren't present\nin <code>otherArray</code>.",
examples: [{ example: '[1, 2, 3].difference([2, 3])', evaluated: '[1]' }],
returnType: 'Array',
args: [
{
name: 'otherArray',
optional: false,
description: 'The array to compare to the base array',
type: 'Array',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-difference',
};
intersection.doc = {
name: 'intersection',
description:
'Compares two arrays. Returns all elements in the base array that are also present in the other array.',
examples: [{ example: '[1, 2].intersection([2, 3])', evaluated: '[2]' }],
returnType: 'Array',
args: [
{
name: 'otherArray',
optional: false,
description: 'The array to compare to the base array',
type: 'Array',
},
],
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-intersection',
};
merge.doc = {
name: 'merge',
description:
'Merges two Object-arrays into one object by merging the key-value pairs of each element.',
examples: [
{
example:
"[{ name: 'Nathan' }, { age: 42 }].merge([{ city: 'Berlin' }, { country: 'Germany' }])",
evaluated: "{ name: 'Nathan', age: 42, city: 'Berlin', country: 'Germany' }",
},
],
returnType: 'Object',
args: [
{
name: 'otherArray',
optional: false,
description: 'The array to merge into the base array',
type: 'Array',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-merge',
};
pluck.doc = {
name: 'pluck',
description:
"Returns an array containing the values of the given field(s) in each Object of the array. Ignores any array elements that aren't Objects or don't have a key matching the field name(s) provided.",
examples: [
{
example: "[{ name: 'Nathan', age: 42 },{ name: 'Jan', city: 'Berlin' }].pluck('name')",
evaluated: '["Nathan", "Jan"]',
},
{
example: "[{ name: 'Nathan', age: 42 },{ name: 'Jan', city: 'Berlin' }].pluck('age')",
evaluated: '[42]',
},
],
returnType: 'Array',
args: [
{
name: 'fieldNames',
optional: false,
variadic: true,
description: 'The keys to retrieve the value of',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-pluck',
};
renameKeys.doc = {
name: 'renameKeys',
description:
'Changes all matching keys (field names) of any Objects in the array. Rename more than one key by\nadding extra arguments, i.e. <code>from1, to1, from2, to2, ...</code>.',
examples: [
{
example: "[{ name: 'bob' }, { name: 'meg' }].renameKeys('name', 'x')",
evaluated: "[{ x: 'bob' }, { x: 'meg' }]",
},
],
returnType: 'Array',
args: [
{
name: 'from',
optional: false,
description: 'The key to rename',
type: 'string',
},
{ name: 'to', optional: false, description: 'The new key name', type: 'string' },
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-renameKeys',
};
smartJoin.doc = {
name: 'smartJoin',
description:
'Creates a single Object from an array of Objects. Each Object in the array provides one field for the returned Object. Each Object in the array must contain a field with the key name and a field with the value.',
examples: [
{
example:
"[{ field: 'age', value: 2 }, { field: 'city', value: 'Berlin' }].smartJoin('field', 'value')",
evaluated: "{ age: 2, city: 'Berlin' }",
},
],
returnType: 'Object',
args: [
{
name: 'keyField',
optional: false,
description: 'The field in each Object containing the key name',
type: 'string',
},
{
name: 'nameField',
optional: false,
description: 'The field in each Object containing the value',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-smartJoin',
};
union.doc = {
name: 'union',
description: 'Concatenates two arrays and then removes any duplicates',
examples: [{ example: '[1, 2].union([2, 3])', evaluated: '[1, 2, 3]' }],
returnType: 'Array',
args: [
{
name: 'otherArray',
optional: false,
description: 'The array to union with the base array',
type: 'Array',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-union',
};
unique.doc = {
name: 'unique',
description: 'Removes any duplicate elements from the array',
examples: [
{ example: "['quick', 'brown', 'quick'].unique()", evaluated: "['quick', 'brown']" },
{
example: "[{ name: 'Nathan', age: 42 }, { name: 'Nathan', age: 22 }].unique()",
evaluated: "[{ name: 'Nathan', age: 42 }, { name: 'Nathan', age: 22 }]",
},
{
example: "[{ name: 'Nathan', age: 42 }, { name: 'Nathan', age: 22 }].unique('name')",
evaluated: "[{ name: 'Nathan', age: 42 }]",
},
],
returnType: 'any',
aliases: ['removeDuplicates'],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-unique',
args: [
{
name: 'fieldNames',
optional: false,
variadic: true,
description: 'The object keys to check for equality',
type: 'any',
},
],
};
toJsonString.doc = {
name: 'toJsonString',
description:
"Converts the array to a JSON string. The same as JavaScript's <code>JSON.stringify()</code>.",
examples: [
{
example: "['quick', 'brown', 'fox'].toJsonString()",
evaluated: '\'["quick","brown","fox"]\'',
},
],
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-toJsonString',
returnType: 'string',
};
append.doc = {
name: 'append',
aliases: ['push'],
description:
'Adds new elements to the end of the array. Similar to <code>push()</code>, but returns the modified array. Consider using spread syntax instead (see examples).',
examples: [
{ example: "['forget', 'me'].append('not')", evaluated: "['forget', 'me', 'not']" },
{ example: '[9, 0, 2].append(1, 0)', evaluated: '[9, 0, 2, 1, 0]' },
{
example: '[...[9, 0, 2], 1, 0]',
evaluated: '[9, 0, 2, 1, 0]',
description: 'Consider using spread syntax instead',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-append',
returnType: 'Array',
args: [
{
name: 'elements',
optional: false,
variadic: true,
description: 'The elements to append, in order',
type: 'any',
},
],
};
const removeDuplicates: Extension = unique.bind({});
removeDuplicates.doc = { ...unique.doc, hidden: true };
export const arrayExtensions: ExtensionMap = {
typeName: 'Array',
functions: {
removeDuplicates,
unique,
first,
last,
pluck,
randomItem,
sum,
min,
max,
average,
isNotEmpty,
isEmpty,
compact,
smartJoin,
chunk,
renameKeys,
merge,
union,
difference,
intersection,
append,
toJsonString,
toInt,
toFloat,
toBoolean,
toDateTime,
},
};
@@ -0,0 +1,41 @@
import type { Extension, ExtensionMap } from './extensions';
export function toBoolean(value: boolean) {
return value;
}
export function toInt(value: boolean) {
return value ? 1 : 0;
}
export function toDateTime() {
return undefined;
}
const toFloat = toInt;
const toNumber: Extension = toInt.bind({});
toNumber.doc = {
name: 'toNumber',
description:
'Converts <code>true</code> to <code>1</code> and <code>false</code> to <code>0</code>.',
examples: [
{ example: 'true.toNumber()', evaluated: '1' },
{ example: 'false.toNumber()', evaluated: '0' },
],
section: 'cast',
returnType: 'number',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/booleans/#boolean-toNumber',
};
export const booleanExtensions: ExtensionMap = {
typeName: 'Boolean',
functions: {
toBoolean,
toInt,
toFloat,
toNumber,
toDateTime,
},
};
@@ -0,0 +1,623 @@
import { DateTime } from 'luxon';
import type {
DateTimeUnit,
DurationLike,
DurationObjectUnits,
LocaleOptions,
WeekdayNumbers,
} from 'luxon';
import type { ExtensionMap } from './extensions';
import { ExpressionExtensionError } from './expression-extension-error';
import { toDateTime as stringToDateTime } from './string-extensions';
import { convertToDateTime } from './utils';
const durationUnits = [
'milliseconds',
'seconds',
'minutes',
'hours',
'days',
'weeks',
'months',
'quarters',
'years',
] as const;
type DurationUnit = (typeof durationUnits)[number];
const dateParts = [
'day',
'week',
'month',
'year',
'hour',
'minute',
'second',
'millisecond',
'weekNumber',
'yearDayNumber',
'weekday',
] as const;
type DatePart = (typeof dateParts)[number];
const DURATION_MAP: Record<string, DurationUnit> = {
day: 'days',
month: 'months',
year: 'years',
week: 'weeks',
hour: 'hours',
minute: 'minutes',
second: 'seconds',
millisecond: 'milliseconds',
ms: 'milliseconds',
sec: 'seconds',
secs: 'seconds',
hr: 'hours',
hrs: 'hours',
min: 'minutes',
mins: 'minutes',
};
const DATETIMEUNIT_MAP: Record<string, DateTimeUnit> = {
days: 'day',
months: 'month',
years: 'year',
hours: 'hour',
minutes: 'minute',
seconds: 'second',
milliseconds: 'millisecond',
hrs: 'hour',
hr: 'hour',
mins: 'minute',
min: 'minute',
secs: 'second',
sec: 'second',
ms: 'millisecond',
};
function isDateTime(date: unknown): date is DateTime {
return date ? DateTime.isDateTime(date) : false;
}
function toDateTime(date: string | Date | DateTime): DateTime {
if (isDateTime(date)) return date;
if (typeof date === 'string') {
return stringToDateTime(date);
}
return DateTime.fromJSDate(date);
}
function generateDurationObject(durationValue: number, unit: DurationUnit): DurationObjectUnits {
const convertedUnit = DURATION_MAP[unit] || unit;
return { [`${convertedUnit}`]: durationValue };
}
function beginningOf(date: Date | DateTime, extraArgs: DurationUnit[]): Date | DateTime {
const [rawUnit = 'week'] = extraArgs;
const unit = DATETIMEUNIT_MAP[rawUnit] || rawUnit;
if (isDateTime(date)) return date.startOf(unit);
return DateTime.fromJSDate(date).startOf(unit).toJSDate();
}
function endOfMonth(date: Date | DateTime): Date | DateTime {
if (isDateTime(date)) return date.endOf('month');
return DateTime.fromJSDate(date).endOf('month').toJSDate();
}
function extract(date: Date | DateTime, args: DatePart[]): number {
let [part = 'week'] = args;
if (part === 'yearDayNumber') {
date = isDateTime(date) ? date.toJSDate() : date;
const firstDayOfTheYear = new Date(date.getFullYear(), 0, 0);
const diff =
date.getTime() -
firstDayOfTheYear.getTime() +
(firstDayOfTheYear.getTimezoneOffset() - date.getTimezoneOffset()) * 60 * 1000;
return Math.floor(diff / (1000 * 60 * 60 * 24));
}
if (part === 'week') part = 'weekNumber';
const unit = (DATETIMEUNIT_MAP[part] as keyof DateTime) || part;
if (isDateTime(date)) return date.get(unit);
return DateTime.fromJSDate(date).get(unit);
}
function format(date: Date | DateTime, extraArgs: unknown[]): string {
const [dateFormat, localeOpts = {}] = extraArgs as [string, LocaleOptions];
if (isDateTime(date)) {
return date.toFormat(dateFormat, { ...localeOpts });
}
return DateTime.fromJSDate(date).toFormat(dateFormat, { ...localeOpts });
}
function isBetween(
date: Date | DateTime,
extraArgs: Array<string | Date | DateTime>,
): boolean | undefined {
if (extraArgs.length !== 2) {
throw new ExpressionExtensionError('isBetween(): expected exactly two args');
}
const [first, second] = extraArgs;
const firstDate = convertToDateTime(first);
const secondDate = convertToDateTime(second);
if (!firstDate || !secondDate) {
return;
}
if (firstDate > secondDate) {
return secondDate < date && date < firstDate;
}
return secondDate > date && date > firstDate;
}
function isDst(date: Date | DateTime): boolean {
if (isDateTime(date)) {
return date.isInDST;
}
return DateTime.fromJSDate(date).isInDST;
}
function isInLast(date: Date | DateTime, extraArgs: unknown[]): boolean {
const [durationValue = 0, unit = 'minutes'] = extraArgs as [number, DurationUnit];
const dateInThePast = DateTime.now().minus(generateDurationObject(durationValue, unit));
let thisDate = date;
if (!isDateTime(thisDate)) {
thisDate = DateTime.fromJSDate(thisDate);
}
return dateInThePast <= thisDate && thisDate <= DateTime.now();
}
const WEEKEND_DAYS: WeekdayNumbers[] = [6, 7];
function isWeekend(date: Date | DateTime): boolean {
const { weekday } = isDateTime(date) ? date : DateTime.fromJSDate(date);
return WEEKEND_DAYS.includes(weekday);
}
function minus(
date: Date | DateTime,
args: [DurationLike] | [number, DurationUnit],
): Date | DateTime {
if (args.length === 1) {
const [arg] = args;
if (isDateTime(date)) return date.minus(arg);
return DateTime.fromJSDate(date).minus(arg).toJSDate();
}
const [durationValue = 0, unit = 'minutes'] = args;
const duration = generateDurationObject(durationValue, unit);
if (isDateTime(date)) return date.minus(duration);
return DateTime.fromJSDate(date).minus(duration).toJSDate();
}
function plus(
date: Date | DateTime,
args: [DurationLike] | [number, DurationUnit],
): Date | DateTime {
if (args.length === 1) {
const [arg] = args;
if (isDateTime(date)) return date.plus(arg);
return DateTime.fromJSDate(date).plus(arg).toJSDate();
}
const [durationValue = 0, unit = 'minutes'] = args;
const duration = generateDurationObject(durationValue, unit);
if (isDateTime(date)) return date.plus(duration);
return DateTime.fromJSDate(date).plus(duration).toJSDate();
}
function diffTo(date: DateTime, args: [string | Date | DateTime, DurationUnit | DurationUnit[]]) {
const [otherDate, unit = 'days'] = args;
let units = Array.isArray(unit) ? unit : [unit];
if (units.length === 0) {
units = ['days'];
}
const allowedUnitSet = new Set([...dateParts, ...durationUnits]);
const errorUnit = units.find((u) => !allowedUnitSet.has(u));
if (errorUnit) {
throw new ExpressionExtensionError(
`Unsupported unit '${String(errorUnit)}'. Supported: ${durationUnits
.map((u) => `'${u}'`)
.join(', ')}.`,
);
}
const diffResult = date.diff(toDateTime(otherDate), units);
if (units.length > 1) {
return diffResult.toObject();
}
return diffResult.as(units[0]);
}
function diffToNow(date: DateTime, args: [DurationUnit | DurationUnit[]]) {
const [unit] = args;
return diffTo(date, [DateTime.now(), unit]);
}
function toInt(date: Date | DateTime): number {
if (isDateTime(date)) {
return date.toMillis();
}
return date.getTime();
}
const toFloat = toInt;
function toBoolean() {
return undefined;
}
// Only null/undefined return true, this is handled in ExpressionExtension.ts
function isEmpty(): boolean {
return false;
}
function isNotEmpty(): boolean {
return true;
}
endOfMonth.doc = {
name: 'endOfMonth',
returnType: 'DateTime',
hidden: true,
description: 'Transforms a date to the last possible moment that lies within the month.',
section: 'edit',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-endOfMonth',
};
isDst.doc = {
name: 'isDst',
returnType: 'boolean',
hidden: true,
description: 'Checks if a Date is within Daylight Savings Time.',
section: 'query',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-isDst',
};
isWeekend.doc = {
name: 'isWeekend',
returnType: 'boolean',
hidden: true,
description: 'Checks if the Date falls on a Saturday or Sunday.',
section: 'query',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-isWeekend',
};
beginningOf.doc = {
name: 'beginningOf',
description: 'Transform a Date to the start of the given time period. Default unit is `week`.',
section: 'edit',
hidden: true,
returnType: 'DateTime',
args: [{ name: 'unit?', type: 'DurationUnit' }],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-beginningOf',
};
extract.doc = {
name: 'extract',
description:
'Extracts a part of the date or time, e.g. the month, as a number. To extract textual names instead, see <code>format()</code>.',
examples: [
{ example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.extract('month')", evaluated: '3' },
{ example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.extract('hour')", evaluated: '18' },
],
section: 'query',
returnType: 'number',
args: [
{
name: 'unit',
optional: true,
description:
'The part of the date or time to return. One of: <code>year</code>, <code>month</code>, <code>week</code>, <code>day</code>, <code>hour</code>, <code>minute</code>, <code>second</code>',
default: '"week"',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-extract',
};
format.doc = {
name: 'format',
description:
'Converts the DateTime to a string, using the format specified. <a target="_blank" href="https://moment.github.io/luxon/#/formatting?id=table-of-tokens">Formatting guide</a>. For common formats, <code>toLocaleString()</code> may be easier.',
examples: [
{
example: "dt = '2024-04-30T18:49'.toDateTime()\ndt.format('dd/LL/yyyy')",
evaluated: "'30/04/2024'",
},
{
example: "dt = '2024-04-30T18:49'.toDateTime()\ndt.format('dd LLL yy')",
evaluated: "'30 Apr 24'",
},
{
example: "dt = '2024-04-30T18:49'.toDateTime()\ndt.setLocale('fr').format('dd LLL yyyy')",
evaluated: "'30 avr. 2024'",
},
{
example: "dt = '2024-04-30T18:49'.toDateTime()\ndt.format(\"HH 'hours and' mm 'minutes'\")",
evaluated: "'18 hours and 49 minutes'",
},
],
returnType: 'string',
section: 'format',
args: [
{
name: 'fmt',
description:
'The <a target="_blank" href="https://moment.github.io/luxon/#/formatting?id=table-of-tokens">format</a> of the string to return ',
default: "'yyyy-MM-dd'",
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-format',
};
isBetween.doc = {
name: 'isBetween',
description: 'Returns <code>true</code> if the DateTime lies between the two moments specified',
examples: [
{
example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.isBetween('2020-06-01', '2025-06-01')",
evaluated: 'true',
},
{
example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.isBetween('2020', '2025')",
evaluated: 'true',
},
],
section: 'compare',
returnType: 'boolean',
args: [
{
name: 'date1',
description:
'The moment that the base DateTime must be after. Can be an ISO date string or a Luxon DateTime.',
type: 'string | DateTime',
},
{
name: 'date2',
description:
'The moment that the base DateTime must be before. Can be an ISO date string or a Luxon DateTime.',
type: 'string | DateTime',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-isBetween',
};
isInLast.doc = {
name: 'isInLast',
hidden: true,
description: 'Checks if a Date is within a given time period. Default unit is `minute`.',
section: 'query',
returnType: 'boolean',
args: [
{ name: 'n', type: 'number' },
{ name: 'unit?', type: 'DurationUnit' },
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-isInLast',
};
toDateTime.doc = {
name: 'toDateTime',
description:
'Converts a JavaScript Date to a Luxon DateTime. The DateTime contains the same information, but is easier to manipulate.',
examples: [
{
example: "jsDate = new Date('2024-03-30T18:49')\njsDate.toDateTime().plus(5, 'days')",
evaluated: '[DateTime: 2024-05-05T18:49:00.000Z]',
},
],
returnType: 'DateTime',
hidden: true,
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-toDateTime',
};
minus.doc = {
name: 'minus',
description: 'Subtracts a given period of time from the DateTime',
examples: [
{
example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.minus(7, 'days')",
evaluated: '[DateTime: 2024-04-23T18:49:00.000Z]',
},
{
example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.minus(4, 'years')",
evaluated: '[DateTime: 2020-04-30T18:49:00.000Z]',
},
],
section: 'edit',
returnType: 'DateTime',
args: [
{
name: 'n',
description:
'The number of units to subtract. Or use a Luxon <a target="_blank" href="https://moment.github.io/luxon/api-docs/index.html#duration">Duration</a> object to subtract multiple units at once.',
type: 'number | object',
},
{
name: 'unit',
optional: true,
description:
'The units of the number. One of: <code>years</code>, <code>months</code>, <code>weeks</code>, <code>days</code>, <code>hours</code>, <code>minutes</code>, <code>seconds</code>, <code>milliseconds</code>',
default: '"milliseconds"',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-minus',
};
plus.doc = {
name: 'plus',
description: 'Adds a given period of time to the DateTime',
examples: [
{
example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.plus(7, 'days')",
evaluated: '[DateTime: 2024-04-07T18:49:00.000Z]',
},
{
example: "dt = '2024-03-30T18:49'.toDateTime()\ndt.plus(4, 'years')",
evaluated: '[DateTime: 2028-03-30T18:49:00.000Z]',
},
],
section: 'edit',
returnType: 'DateTime',
args: [
{
name: 'n',
description:
'The number of units to add. Or use a Luxon <a target="_blank" href="https://moment.github.io/luxon/api-docs/index.html#duration">Duration</a> object to add multiple units at once.',
type: 'number | object',
},
{
name: 'unit',
optional: true,
description:
'The units of the number. One of: <code>years</code>, <code>months</code>, <code>weeks</code>, <code>days</code>, <code>hours</code>, <code>minutes</code>, <code>seconds</code>, <code>milliseconds</code>',
default: '"milliseconds"',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-plus',
};
diffTo.doc = {
name: 'diffTo',
description: 'Returns the difference between two DateTimes, in the given unit(s)',
examples: [
{
example: "dt = '2025-01-01'.toDateTime()\ndt.diffTo('2024-03-30T18:49:07.234', 'days')",
evaluated: '276.21',
},
{
example:
"dt1 = '2025-01-01T00:00:00.000'.toDateTime();\ndt2 = '2024-03-30T18:49:07.234'.toDateTime();\ndt1.diffTo(dt2, ['months', 'days'])",
evaluated: '{ months: 9, days: 1.21 }',
},
],
section: 'compare',
returnType: 'number | Record<DurationUnit, number>',
args: [
{
name: 'otherDateTime',
default: '$now',
description:
'The moment to subtract the base DateTime from. Can be an ISO date string or a Luxon DateTime.',
type: 'string | DateTime',
},
{
name: 'unit',
default: "'days'",
description:
'The unit, or array of units, to return the result in. Possible values: <code>years</code>, <code>months</code>, <code>weeks</code>, <code>days</code>, <code>hours</code>, <code>minutes</code>, <code>seconds</code>, <code>milliseconds</code>.',
type: 'string | string[]',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-diffTo',
};
diffToNow.doc = {
name: 'diffToNow',
description:
'Returns the difference between the current moment and the DateTime, in the given unit(s). For a textual representation, use <code>toRelative()</code> instead.',
examples: [
{
example: "dt = '2023-03-30T18:49:07.234'.toDateTime()\ndt.diffToNow('days')",
evaluated: '371.9',
},
{
example: "dt = '2023-03-30T18:49:07.234'.toDateTime()\ndt.diffToNow(['months', 'days'])",
evaluated: '{ months: 12, days: 5.9 }',
},
],
section: 'compare',
returnType: 'number | Record<DurationUnit, number>',
args: [
{
name: 'unit',
description:
'The unit, or array of units, to return the result in. Possible values: <code>years</code>, <code>months</code>, <code>weeks</code>, <code>days</code>, <code>hours</code>, <code>minutes</code>, <code>seconds</code>, <code>milliseconds</code>.',
default: "'days'",
type: 'string | string[]',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/dates/#date-diffToNow',
};
isEmpty.doc = {
name: 'isEmpty',
description:
'Returns <code>false</code> for all DateTimes. Returns <code>true</code> for <code>null</code>.',
examples: [
{ example: "dt = '2023-03-30T18:49:07.234'.toDateTime()\ndt.isEmpty()", evaluated: 'false' },
{ example: 'dt = null\ndt.isEmpty()', evaluated: 'true' },
],
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-isEmpty',
};
isNotEmpty.doc = {
name: 'isNotEmpty',
description:
'Returns <code>true</code> for all DateTimes. Returns <code>false</code> for <code>null</code>.',
examples: [
{ example: "dt = '2023-03-30T18:49:07.234'.toDateTime()\ndt.isNotEmpty()", evaluated: 'true' },
{ example: 'dt = null\ndt.isNotEmpty()', evaluated: 'false' },
],
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/arrays/#array-isNotEmpty',
};
export const dateExtensions: ExtensionMap = {
typeName: 'Date',
functions: {
beginningOf,
endOfMonth,
extract,
isBetween,
isDst,
isInLast,
isWeekend,
minus,
plus,
format,
toDateTime,
diffTo,
diffToNow,
toInt,
toFloat,
toBoolean,
isEmpty,
isNotEmpty,
},
};
@@ -0,0 +1,11 @@
export class ExpressionExtensionError extends Error {
description?: string;
constructor(message: string, options?: { description?: string }) {
super(message);
this.name = 'ExpressionExtensionError';
if (options?.description !== undefined) {
this.description = options.description;
}
}
}
@@ -0,0 +1,161 @@
import { DateTime } from 'luxon';
import { arrayExtensions } from './array-extensions';
import { booleanExtensions } from './boolean-extensions';
import { dateExtensions } from './date-extensions';
import type { ExtensionMap } from './extensions';
import { ExpressionExtensionError } from './expression-extension-error';
import { numberExtensions } from './number-extensions';
import { objectExtensions } from './object-extensions';
import { stringExtensions } from './string-extensions';
import { checkIfValueDefinedOrThrow } from './utils';
function isEmpty(value: unknown) {
return value === null || value === undefined || !value;
}
function isNotEmpty(value: unknown) {
return !isEmpty(value);
}
export const EXTENSION_OBJECTS: ExtensionMap[] = [
arrayExtensions,
dateExtensions,
numberExtensions,
objectExtensions,
stringExtensions,
booleanExtensions,
];
// eslint-disable-next-line @typescript-eslint/no-restricted-types
const genericExtensions: Record<string, Function> = {
isEmpty,
isNotEmpty,
};
function isDate(input: unknown): boolean {
if (typeof input !== 'string' || !input.length) {
return false;
}
if (!/\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}Z/.test(input)) {
return false;
}
const d = new Date(input);
return d instanceof Date && !isNaN(d.valueOf()) && d.toISOString() === input;
}
interface FoundFunction {
type: 'native' | 'extended';
// eslint-disable-next-line @typescript-eslint/no-restricted-types
function: Function;
}
function findExtendedFunction(input: unknown, functionName: string): FoundFunction | undefined {
// eslint-disable-next-line @typescript-eslint/no-restricted-types
let foundFunction: Function | undefined;
if (Array.isArray(input)) {
foundFunction = arrayExtensions.functions[functionName];
} else if (isDate(input) && functionName !== 'toDate' && functionName !== 'toDateTime') {
// If it's a string date (from $json), convert it to a Date object,
// unless that function is `toDate`, since `toDate` does something
// very different on date objects
input = new Date(input as string);
foundFunction = dateExtensions.functions[functionName];
} else if (typeof input === 'string') {
foundFunction = stringExtensions.functions[functionName];
} else if (typeof input === 'number') {
foundFunction = numberExtensions.functions[functionName];
} else if (input && (DateTime.isDateTime(input) || input instanceof Date)) {
foundFunction = dateExtensions.functions[functionName];
} else if (input !== null && typeof input === 'object') {
foundFunction = objectExtensions.functions[functionName];
} else if (typeof input === 'boolean') {
foundFunction = booleanExtensions.functions[functionName];
}
// Look for generic or builtin
if (!foundFunction) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const inputAny: any = input;
// This is likely a builtin we're implementing for another type
// (e.g. toLocaleString). We'll return that instead
if (inputAny && functionName && typeof inputAny[functionName] === 'function') {
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
return { type: 'native', function: inputAny[functionName] };
}
// Use a generic version if available
foundFunction = genericExtensions[functionName];
}
if (!foundFunction) {
return undefined;
}
return { type: 'extended', function: foundFunction };
}
/**
* Extender function injected by expression extension plugin to allow calls to extensions.
*
* ```ts
* extend(input, "functionName", [...args]);
* ```
*/
export function extend(input: unknown, functionName: string, args: unknown[]) {
const foundFunction = findExtendedFunction(input, functionName);
// No type specific or generic function found. Check to see if
// any types have a function with that name. Then throw an error
// letting the user know the available types.
if (!foundFunction) {
checkIfValueDefinedOrThrow(input, functionName);
const haveFunction = EXTENSION_OBJECTS.filter((v) => functionName in v.functions);
if (!haveFunction.length) {
// This shouldn't really be possible but we should cover it anyway
throw new ExpressionExtensionError(`Unknown expression function: ${functionName}`);
}
if (haveFunction.length > 1) {
const lastType = `"${haveFunction.pop()!.typeName}"`;
const typeNames = `${haveFunction.map((v) => `"${v.typeName}"`).join(', ')}, and ${lastType}`;
throw new ExpressionExtensionError(
`${functionName}() is only callable on types ${typeNames}`,
);
} else {
throw new ExpressionExtensionError(
`${functionName}() is only callable on type "${haveFunction[0].typeName}"`,
);
}
}
if (foundFunction.type === 'native') {
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
return foundFunction.function.apply(input, args);
}
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
return foundFunction.function(input, args);
}
export function extendOptional(
input: unknown,
functionName: string,
// eslint-disable-next-line @typescript-eslint/no-restricted-types
): Function | undefined {
const foundFunction = findExtendedFunction(input, functionName);
if (!foundFunction) {
return undefined;
}
if (foundFunction.type === 'native') {
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
return foundFunction.function.bind(input);
}
return (...args: unknown[]) => {
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
return foundFunction.function(input, args);
};
}
@@ -0,0 +1,42 @@
export interface ExtensionMap {
typeName: string;
functions: Record<string, Extension>;
}
// eslint-disable-next-line @typescript-eslint/no-restricted-types
export type Extension = Function & { doc?: DocMetadata };
export type NativeDoc = {
typeName: string;
properties?: Record<string, { doc?: DocMetadata }>;
functions: Record<string, { doc?: DocMetadata }>;
};
export type DocMetadataArgument = {
name: string;
type?: string;
optional?: boolean;
variadic?: boolean;
description?: string;
default?: string;
// Function arguments have nested arguments
args?: DocMetadataArgument[];
};
export type DocMetadataExample = {
example: string;
evaluated?: string;
description?: string;
};
export type DocMetadata = {
name: string;
returnType: string;
description?: string;
section?: string;
hidden?: boolean;
aliases?: string[];
aliasMode?: 'prefix' | 'exact';
args?: DocMetadataArgument[];
examples?: DocMetadataExample[];
docURL?: string;
};
@@ -0,0 +1,267 @@
import { DateTime } from 'luxon';
import type { ExtensionMap } from './extensions';
import { ExpressionExtensionError } from './expression-extension-error';
function format(value: number, extraArgs: unknown[]): string {
const [locales = 'en-US', config = {}] = extraArgs as [
string | string[],
Intl.NumberFormatOptions,
];
return new Intl.NumberFormat(locales, config).format(value);
}
function isEven(value: number) {
if (!Number.isInteger(value)) {
throw new ExpressionExtensionError('isEven() is only callable on integers');
}
return value % 2 === 0;
}
function isOdd(value: number) {
if (!Number.isInteger(value)) {
throw new ExpressionExtensionError('isOdd() is only callable on integers');
}
return Math.abs(value) % 2 === 1;
}
function floor(value: number) {
return Math.floor(value);
}
function ceil(value: number) {
return Math.ceil(value);
}
function abs(value: number) {
return Math.abs(value);
}
function isInteger(value: number) {
return Number.isInteger(value);
}
function round(value: number, extraArgs: number[]) {
const [decimalPlaces = 0] = extraArgs;
return +value.toFixed(decimalPlaces);
}
function toBoolean(value: number) {
return value !== 0;
}
function toInt(value: number) {
return round(value, []);
}
function toFloat(value: number) {
return value;
}
type DateTimeFormat = 'ms' | 's' | 'us' | 'excel';
export function toDateTime(value: number, extraArgs: [DateTimeFormat]) {
const [valueFormat = 'ms'] = extraArgs;
if (!['ms', 's', 'us', 'excel'].includes(valueFormat)) {
throw new ExpressionExtensionError(
`Unsupported format '${String(valueFormat)}'. toDateTime() supports 'ms', 's', 'us' and 'excel'.`,
);
}
switch (valueFormat) {
// Excel format is days since 1900
// There is a bug where 1900 is incorrectly treated as a leap year
case 'excel': {
const DAYS_BETWEEN_1900_1970 = 25567;
const DAYS_LEAP_YEAR_BUG_ADJUST = 2;
const SECONDS_IN_DAY = 86_400;
return DateTime.fromSeconds(
(value - (DAYS_BETWEEN_1900_1970 + DAYS_LEAP_YEAR_BUG_ADJUST)) * SECONDS_IN_DAY,
);
}
case 's':
return DateTime.fromSeconds(value);
case 'us':
return DateTime.fromMillis(value / 1000);
case 'ms':
default:
return DateTime.fromMillis(value);
}
}
ceil.doc = {
name: 'ceil',
description: 'Rounds the number up to the next whole number',
examples: [{ example: '(1.234).ceil()', evaluated: '2' }],
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-ceil',
};
floor.doc = {
name: 'floor',
description: 'Rounds the number down to the nearest whole number',
examples: [{ example: '(1.234).floor()', evaluated: '1' }],
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-floor',
};
isEven.doc = {
name: 'isEven',
description:
"Returns <code>true</code> if the number is even or <code>false</code> if not. Throws an error if the number isn't a whole number.",
examples: [
{ example: '(33).isEven()', evaluated: 'false' },
{ example: '(42).isEven()', evaluated: 'true' },
],
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-isEven',
};
isOdd.doc = {
name: 'isOdd',
description:
"Returns <code>true</code> if the number is odd or <code>false</code> if not. Throws an error if the number isn't a whole number.",
examples: [
{ example: '(33).isOdd()', evaluated: 'true' },
{ example: '(42).isOdd()', evaluated: 'false' },
],
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-isOdd',
};
format.doc = {
name: 'format',
description:
'Returns a formatted string representing the number. Useful for formatting for a specific language or currency. The same as <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat/NumberFormat"><code>Intl.NumberFormat()</code></a>.',
examples: [
{ example: "(123456.789).format('de-DE')", evaluated: '123.456,789' },
{
example: "(123456.789).format('de-DE', {'style': 'currency', 'currency': 'EUR'})",
evaluated: '123.456,79 €',
},
],
returnType: 'string',
args: [
{
name: 'locale',
optional: true,
description:
'A <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl#locales_argument">locale tag</a> for formatting the number, e.g. <code>fr-FR</code>, <code>en-GB</code>, <code>pr-BR</code>',
default: '"en-US"',
type: 'string',
},
{
name: 'options',
optional: true,
description:
'Configuration options for number formatting. <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat/NumberFormat" target="_blank">More info</a>',
type: 'object',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-format',
};
round.doc = {
name: 'round',
description: 'Rounds the number to the nearest integer (or decimal place)',
examples: [
{ example: '(1.256).round()', evaluated: '1' },
{ example: '(1.256).round(1)', evaluated: '1.3' },
{ example: '(1.256).round(2)', evaluated: '1.26' },
],
returnType: 'number',
args: [
{
name: 'decimalPlaces',
optional: true,
description: 'The number of decimal places to round to',
default: '0',
type: 'number',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-round',
};
toBoolean.doc = {
name: 'toBoolean',
description:
'Returns <code>false</code> for <code>0</code> and <code>true</code> for any other number (including negative numbers).',
examples: [
{ example: '(12).toBoolean()', evaluated: 'true' },
{ example: '(0).toBoolean()', evaluated: 'false' },
{ example: '(-1.3).toBoolean()', evaluated: 'true' },
],
section: 'cast',
returnType: 'boolean',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-toBoolean',
};
toDateTime.doc = {
name: 'toDateTime',
description:
'Converts a numerical timestamp into a <a target="_blank" href="https://moment.github.io/luxon/api-docs/">Luxon</a> DateTime. The format of the timestamp must be specified if it\'s not in milliseconds. Uses the timezone specified in workflow settings if available; otherwise, it defaults to the timezone set for the instance.',
examples: [
{ example: "(1708695471).toDateTime('s')", evaluated: '2024-02-23T14:37:51.000+01:00' },
{ example: "(1708695471000).toDateTime('ms')", evaluated: '2024-02-23T14:37:51.000+01:00' },
{ example: "(1708695471000000).toDateTime('us')", evaluated: '2024-02-23T14:37:51.000+01:00' },
{ example: "(45345).toDateTime('excel')", evaluated: '2024-02-23T01:00:00.000+01:00' },
],
section: 'cast',
returnType: 'DateTime',
args: [
{
name: 'format',
optional: true,
description:
'The type of timestamp to convert. Options are <code>ms</code> (for Unix timestamp in milliseconds), <code>s</code> (for Unix timestamp in seconds), <code>us</code> (for Unix timestamp in microseconds) or <code>excel</code> (for days since 1900).',
default: '"ms"',
type: 'string',
},
],
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-toDateTime',
};
abs.doc = {
name: 'abs',
description: "Returns the number's absolute value, i.e. removes any minus sign",
examples: [
{ example: '(-1.7).abs()', evaluated: '1.7' },
{ example: '(1.7).abs()', evaluated: '1.7' },
],
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-abs',
};
isInteger.doc = {
name: 'isInteger',
description: 'Returns <code>true</code> if the number is a whole number',
examples: [
{ example: '(4).isInteger()', evaluated: 'true' },
{ example: '(4.12).isInteger()', evaluated: 'false' },
{ example: '(-4).isInteger()', evaluated: 'true' },
],
returnType: 'boolean',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/numbers/#number-isInteger',
};
export const numberExtensions: ExtensionMap = {
typeName: 'Number',
functions: {
ceil,
floor,
format,
round,
abs,
isInteger,
isEven,
isOdd,
toBoolean,
toInt,
toFloat,
toDateTime,
},
};
@@ -0,0 +1,319 @@
import type { ExtensionMap } from './extensions';
import { ExpressionExtensionError } from './expression-extension-error';
function isEmpty(value: object): boolean {
return Object.keys(value).length === 0;
}
function isNotEmpty(value: object): boolean {
return !isEmpty(value);
}
function keys(value: object): string[] {
return Object.keys(value);
}
function values(value: object): unknown[] {
return Object.values(value);
}
function hasField(value: object, extraArgs: string[]): boolean {
const [name] = extraArgs;
return name in value;
}
function removeField(value: object, extraArgs: string[]): object {
const [name] = extraArgs;
if (name in value) {
const newObject = { ...value };
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-explicit-any
delete (newObject as any)[name];
return newObject;
}
return value;
}
function removeFieldsContaining(value: object, extraArgs: string[]): object {
const [match] = extraArgs;
if (typeof match !== 'string' || match === '') {
throw new ExpressionExtensionError('removeFieldsContaining(): expected non-empty string arg');
}
const newObject = { ...value };
for (const [key, val] of Object.entries(value)) {
if (typeof val === 'string' && val.includes(match)) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-explicit-any
delete (newObject as any)[key];
}
}
return newObject;
}
function keepFieldsContaining(value: object, extraArgs: string[]): object {
const [match] = extraArgs;
if (typeof match !== 'string' || match === '') {
throw new ExpressionExtensionError(
'argument of keepFieldsContaining must be a non-empty string',
);
}
const newObject = { ...value };
for (const [key, val] of Object.entries(value)) {
if (typeof val !== 'string' || (typeof val === 'string' && !val.includes(match))) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-explicit-any
delete (newObject as any)[key];
}
}
return newObject;
}
export function compact(value: object): object {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const newObj: any = {};
for (const [key, val] of Object.entries(value)) {
if (val !== null && val !== undefined && val !== 'nil' && val !== '') {
if (typeof val === 'object') {
if (Object.keys(val as object).length === 0) continue;
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-unsafe-argument
newObj[key] = compact(val);
} else {
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-unsafe-member-access
newObj[key] = val;
}
}
}
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
return newObj;
}
export function urlEncode(value: object) {
return new URLSearchParams(value as Record<string, string>).toString();
}
export function toJsonString(value: object) {
return JSON.stringify(value);
}
export function toInt() {
return undefined;
}
export function toFloat() {
return undefined;
}
export function toBoolean() {
return undefined;
}
export function toDateTime() {
return undefined;
}
isEmpty.doc = {
name: 'isEmpty',
description:
'Returns <code>true</code> if the Object has no keys (fields) set or is <code>null</code>',
examples: [
{ example: "({'name': 'Nathan'}).isEmpty()", evaluated: 'false' },
{ example: '({}).isEmpty()', evaluated: 'true' },
],
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-isEmpty',
};
isNotEmpty.doc = {
name: 'isNotEmpty',
description: 'Returns <code>true</code> if the Object has at least one key (field) set',
examples: [
{ example: "({'name': 'Nathan'}).isNotEmpty()", evaluated: 'true' },
{ example: '({}).isNotEmpty()', evaluated: 'false' },
],
returnType: 'boolean',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-isNotEmpty',
};
compact.doc = {
name: 'compact',
description:
'Removes all fields that have empty values, i.e. are <code>null</code>, <code>undefined</code>, <code>"nil"</code> or <code>""</code>',
examples: [{ example: "({ x: null, y: 2, z: '' }).compact()", evaluated: '{ y: 2 }' }],
returnType: 'Object',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-compact',
};
urlEncode.doc = {
name: 'urlEncode',
description:
"Generates a URL parameter string from the Object's keys and values. Only top-level keys are supported.",
examples: [
{
example: "({ name: 'Mr Nathan', city: 'hanoi' }).urlEncode()",
evaluated: "'name=Mr+Nathan&city=hanoi'",
},
],
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-urlEncode',
};
hasField.doc = {
name: 'hasField',
description:
'Returns <code>true</code> if there is a field called <code>name</code>. Only checks top-level keys. Comparison is case-sensitive.',
examples: [
{ example: "({ name: 'Nathan', age: 42 }).hasField('name')", evaluated: 'true' },
{ example: "({ name: 'Nathan', age: 42 }).hasField('Name')", evaluated: 'false' },
{ example: "({ name: 'Nathan', age: 42 }).hasField('inventedField')", evaluated: 'false' },
],
returnType: 'boolean',
args: [
{
name: 'name',
optional: false,
description: 'The name of the key to search for',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-hasField',
};
removeField.doc = {
name: 'removeField',
aliases: ['delete'],
description: "Removes a field from the Object. The same as JavaScript's <code>delete</code>.",
examples: [
{
example: "({ name: 'Nathan', city: 'hanoi' }).removeField('name')",
evaluated: "{ city: 'hanoi' }",
},
],
returnType: 'Object',
args: [
{
name: 'key',
optional: false,
description: 'The name of the field to remove',
type: 'string',
},
],
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-removeField',
};
removeFieldsContaining.doc = {
name: 'removeFieldsContaining',
description:
"Removes keys (fields) whose values at least partly match the given <code>value</code>. Comparison is case-sensitive. Fields that aren't strings are always kept.",
examples: [
{
example: "({ name: 'Mr Nathan', city: 'hanoi', age: 42 }).removeFieldsContaining('Nathan')",
evaluated: "{ city: 'hanoi', age: 42 }",
},
{
example: "({ name: 'Mr Nathan', city: 'hanoi', age: 42 }).removeFieldsContaining('Han')",
evaluated: '{ age: 42 }',
},
{
example: "({ name: 'Mr Nathan', city: 'hanoi', age: 42 }).removeFieldsContaining('nathan')",
evaluated: "{ name: 'Mr Nathan', city: 'hanoi', age: 42 }",
},
],
returnType: 'Object',
args: [
{
name: 'value',
optional: false,
description: 'The text that a value must contain in order to be removed',
type: 'string',
},
],
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-removeFieldsContaining',
};
keepFieldsContaining.doc = {
name: 'keepFieldsContaining',
description:
"Removes any fields whose values don't at least partly match the given <code>value</code>. Comparison is case-sensitive. Fields that aren't strings will always be removed.",
examples: [
{
example: "({ name: 'Mr Nathan', city: 'hanoi', age: 42 }).keepFieldsContaining('Nathan')",
evaluated: "{ name: 'Mr Nathan' }",
},
{
example: "({ name: 'Mr Nathan', city: 'hanoi', age: 42 }).keepFieldsContaining('nathan')",
evaluated: '{}',
},
{
example: "({ name: 'Mr Nathan', city: 'hanoi', age: 42 }).keepFieldsContaining('han')",
evaluated: "{ name: 'Mr Nathan', city: 'hanoi' }",
},
],
returnType: 'Object',
args: [
{
name: 'value',
optional: false,
description: 'The text that a value must contain in order to be kept',
type: 'string',
},
],
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-keepFieldsContaining',
};
keys.doc = {
name: 'keys',
description:
"Returns an array with all the field names (keys) the Object contains. The same as JavaScript's <code>Object.keys(obj)</code>.",
examples: [{ example: "({ name: 'Mr Nathan', age: 42 }).keys()", evaluated: "['name', 'age']" }],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-keys',
returnType: 'Array',
};
values.doc = {
name: 'values',
description:
"Returns an array with all the values of the fields the Object contains. The same as JavaScript's <code>Object.values(obj)</code>.",
examples: [
{ example: "({ name: 'Mr Nathan', age: 42 }).values()", evaluated: "['Mr Nathan', 42]" },
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-values',
returnType: 'Array',
};
toJsonString.doc = {
name: 'toJsonString',
description:
"Converts the Object to a JSON string. Similar to JavaScript's <code>JSON.stringify()</code>.",
examples: [
{
example: "({ name: 'Mr Nathan', age: 42 }).toJsonString()",
evaluated: '\'{"name":"Nathan","age":42}\'',
},
],
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/objects/#object-toJsonString',
returnType: 'string',
};
export const objectExtensions: ExtensionMap = {
typeName: 'Object',
functions: {
isEmpty,
isNotEmpty,
hasField,
removeField,
removeFieldsContaining,
keepFieldsContaining,
compact,
urlEncode,
keys,
values,
toJsonString,
toInt,
toFloat,
toBoolean,
toDateTime,
},
};
@@ -0,0 +1,881 @@
import { toBase64, fromBase64 } from 'js-base64';
import SHA from 'jssha';
import { DateTime } from 'luxon';
import MD5 from 'md5';
import { titleCase } from 'title-case';
import { transliterate } from 'transliteration';
import type { Extension, ExtensionMap } from './extensions';
import { ExpressionExtensionError } from './expression-extension-error';
import { toDateTime as numberToDateTime } from './number-extensions';
export const SupportedHashAlgorithms = [
'md5',
'sha1',
'sha224',
'sha256',
'sha384',
'sha512',
'sha3',
] as const;
// All symbols from https://www.xe.com/symbols/ as for 2022/11/09
const CURRENCY_REGEXP =
/(\u004c\u0065\u006b|\u060b|\u0024|\u0192|\u20bc|\u0042\u0072|\u0042\u005a\u0024|\u0024\u0062|\u004b\u004d|\u0050|\u043b\u0432|\u0052\u0062|\u17db|\u00a5|\u20a1|\u006b\u006e|\u20b1|\u004b\u010d|\u006b\u0072|\u0052\u0044\u0024|\u00a3|\u20ac|\u00a2|\u0051|\u004c|\u0046\u0074|\u20b9|\u0052\u0070|\ufdfc|\u20aa|\u004a\u0024|\u20a9|\u20ad|\u0434\u0435\u043d|\u0052\u004d|\u20a8|\u20ae|\u004d\u0054|\u0043\u0024|\u20a6|\u0042\u002f\u002e|\u0047\u0073|\u0053\u002f\u002e|\u007a\u0142|\u006c\u0065\u0069|\u20bd|\u0414\u0438\u043d\u002e|\u0053|\u0052|\u0043\u0048\u0046|\u004e\u0054\u0024|\u0e3f|\u0054\u0054\u0024|\u20ba|\u20b4|\u0024\u0055|\u0042\u0073|\u20ab|\u005a\u0024)/gu;
const DOMAIN_EXTRACT_REGEXP =
/^(?:(?:https?|ftp):\/\/)?(?:mailto:)?(?:\/\/)?((?:www\.)?(?:(?:[-\w]+\.)+(?:[a-zA-Z]{2,}|xn--[a-zA-Z0-9]+)|localhost|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))(?::\d+)?(?:\/[^\s?]*)?(?:\?[^\s#]*)?(?:#[^\s]*)?$/i;
const DOMAIN_REGEXP =
/^(?:www\.)?((?:(?:[-\w]+\.)+(?:[a-zA-Z]{2,}|xn--[a-zA-Z0-9]+)|localhost|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))(?::\d+)?(?:\/[^\s?]*)?(?:\?[^\s#]*)?(?:#[^\s]*)?$/i;
const EMAIL_REGEXP =
/(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@(?<domain>(\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))/;
const URL_REGEXP_EXACT =
/^(?:(?:https?|ftp):\/\/)(?:www\.)?((?:(?:[-\w]+\.)+(?:[a-zA-Z]{2,}|xn--[a-zA-Z0-9]+)|localhost|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))(?::\d+)?(?:\/[^\s?#]*)?(?:\?[^\s#]*)?(?=([^\s]+#.*)?)#?[^\s]*$/i;
const URL_REGEXP =
/(?:(?:https?|ftp):\/\/)(?:www\.)?((?:(?:[-\w]+\.)+(?:[a-zA-Z]{2,}|xn--[a-zA-Z0-9]+)|localhost|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))(?::\d+)?(?:\/[^\s?#]*)?(?:\?[^\s#]*)?(?=([^\s]+#.*)?)#?[^\s]*/i;
const CHAR_TEST_REGEXP = /\p{L}/u;
const PUNC_TEST_REGEXP = /[!?.]/;
/**
* Inline version of tryToParseDateTime from n8n-workflow/type-validation.ts
* Avoids circular dependency.
*/
function tryToParseDateTime(value: unknown, defaultZone?: string): DateTime {
if (DateTime.isDateTime(value) && value.isValid) {
return value;
}
if (value instanceof Date) {
const fromJSDate = DateTime.fromJSDate(value, { zone: defaultZone });
if (fromJSDate.isValid) {
return fromJSDate;
}
}
const dateString = String(value).trim();
const isoDate = DateTime.fromISO(dateString, { zone: defaultZone, setZone: true });
if (isoDate.isValid) {
return isoDate;
}
const httpDate = DateTime.fromHTTP(dateString, { zone: defaultZone, setZone: true });
if (httpDate.isValid) {
return httpDate;
}
const rfc2822Date = DateTime.fromRFC2822(dateString, { zone: defaultZone, setZone: true });
if (rfc2822Date.isValid) {
return rfc2822Date;
}
const sqlDate = DateTime.fromSQL(dateString, { zone: defaultZone, setZone: true });
if (sqlDate.isValid) {
return sqlDate;
}
const parsedDateTime = DateTime.fromMillis(Date.parse(dateString), { zone: defaultZone });
if (parsedDateTime.isValid) {
return parsedDateTime;
}
throw new ExpressionExtensionError('Value is not a valid date');
}
function hash(value: string, extraArgs: string[]): string {
const algorithm = extraArgs[0]?.toLowerCase() ?? 'md5';
switch (algorithm) {
case 'base64':
return toBase64(value);
case 'md5':
return MD5(value);
case 'sha1':
case 'sha224':
case 'sha256':
case 'sha384':
case 'sha512':
case 'sha3':
const variant = (
{
sha1: 'SHA-1',
sha224: 'SHA-224',
sha256: 'SHA-256',
sha384: 'SHA-384',
sha512: 'SHA-512',
sha3: 'SHA3-512',
} as const
)[algorithm];
return new SHA(variant, 'TEXT').update(value).getHash('HEX');
default:
throw new ExpressionExtensionError(
`Unknown algorithm ${algorithm}. Available algorithms are: ${SupportedHashAlgorithms.join()}, and Base64.`,
);
}
}
function isEmpty(value: string): boolean {
return value === '';
}
function isNotEmpty(value: string): boolean {
return !isEmpty(value);
}
function length(value: string): number {
return value.length;
}
export function toJsonString(value: string): string {
return JSON.stringify(value);
}
function removeMarkdown(value: string): string {
let output = value;
try {
output = output.replace(/^([\s\t]*)([*\-+]|\d\.)\s+/gm, '$1');
output = output
// Header
.replace(/\n={2,}/g, '\n')
// Strikethrough
.replace(/~~/g, '')
// Fenced codeblocks
.replace(/`{3}.*\n/g, '');
output = output
// Remove HTML tags
.replace(/<[\w|\s|=|'|"|:|(|)|,|;|/|0-9|.|-]+[>|\\>]/g, '')
// Remove setext-style headers
.replace(/^[=-]{2,}\s*$/g, '')
// Remove footnotes?
.replace(/\[\^.+?\](: .*?$)?/g, '')
.replace(/\s{0,2}\[.*?\]: .*?$/g, '')
// Remove images
.replace(/!\[.*?\][[(].*?[\])]/g, '')
// Remove inline links
.replace(/\[(.*?)\][[(].*?[\])]/g, '$1')
// Remove Blockquotes
.replace(/>/g, '')
// Remove reference-style links?
.replace(/^\s{1,2}\[(.*?)\]: (\S+)( ".*?")?\s*$/g, '')
// Remove atx-style headers
.replace(/^#{1,6}\s*([^#]*)\s*(#{1,6})?/gm, '$1')
.replace(/([*_]{1,3})(\S.*?\S)\1/g, '$2')
.replace(/(`{3,})(.*?)\1/gm, '$2')
.replace(/^-{3,}\s*$/g, '')
.replace(/`(.+?)`/g, '$1')
.replace(/\n{2,}/g, '\n\n');
} catch (e) {
return value;
}
return output;
}
function removeTags(value: string): string {
return value.replace(/<[^>]*>?/gm, '');
}
function toDate(value: string): Date {
const date = new Date(Date.parse(value));
if (date.toString() === 'Invalid Date') {
throw new ExpressionExtensionError('cannot convert to date');
}
// If time component is not specified, force 00:00h
if (!/:/.test(value)) {
date.setHours(0, 0, 0);
}
return date;
}
export function toDateTime(value: string, extraArgs: [string] = ['']): DateTime {
try {
const [valueFormat] = extraArgs;
if (valueFormat) {
if (
valueFormat === 'ms' ||
valueFormat === 's' ||
valueFormat === 'us' ||
valueFormat === 'excel'
) {
return numberToDateTime(Number(value), [valueFormat]);
}
return DateTime.fromFormat(value, valueFormat);
}
return tryToParseDateTime(value);
} catch (error) {
throw new ExpressionExtensionError('cannot convert to Luxon DateTime');
}
}
function urlDecode(value: string, extraArgs: boolean[]): string {
const [entireString = false] = extraArgs;
if (entireString) {
return decodeURI(value.toString());
}
return decodeURIComponent(value.toString());
}
function urlEncode(value: string, extraArgs: boolean[]): string {
const [entireString = false] = extraArgs;
if (entireString) {
return encodeURI(value.toString());
}
return encodeURIComponent(value.toString());
}
function toInt(value: string, extraArgs: Array<number | undefined>) {
const [radix] = extraArgs;
const int = parseInt(value.replace(CURRENCY_REGEXP, ''), radix);
if (isNaN(int)) {
throw new ExpressionExtensionError('cannot convert to integer');
}
return int;
}
function toFloat(value: string) {
if (value.includes(',')) {
throw new ExpressionExtensionError('cannot convert to float, expected . as decimal separator');
}
const float = parseFloat(value.replace(CURRENCY_REGEXP, ''));
if (isNaN(float)) {
throw new ExpressionExtensionError('cannot convert to float');
}
return float;
}
function toNumber(value: string) {
const num = Number(value.replace(CURRENCY_REGEXP, ''));
if (isNaN(num)) {
throw new ExpressionExtensionError('cannot convert to number');
}
return num;
}
function quote(value: string, extraArgs: string[]) {
const [quoteChar = '"'] = extraArgs;
return `${quoteChar}${value
.replace(/\\/g, '\\\\')
.replace(new RegExp(`\\${quoteChar}`, 'g'), `\\${quoteChar}`)}${quoteChar}`;
}
function isNumeric(value: string) {
if (value.includes(' ')) return false;
return !isNaN(value as unknown as number) && !isNaN(parseFloat(value));
}
function isUrl(value: string) {
return URL_REGEXP_EXACT.test(value);
}
function isDomain(value: string) {
return DOMAIN_REGEXP.test(value);
}
function isEmail(value: string) {
const result = EMAIL_REGEXP.test(value);
// email regex is loose so check manually for now
if (result && value.includes(' ')) {
return false;
}
return result;
}
function toTitleCase(value: string) {
return titleCase(value);
}
function replaceSpecialChars(value: string) {
return transliterate(value, { unknown: '?' });
}
function toSentenceCase(value: string) {
let current = value.slice();
let buffer = '';
while (CHAR_TEST_REGEXP.test(current)) {
const charIndex = current.search(CHAR_TEST_REGEXP);
current =
current.slice(0, charIndex) +
current[charIndex].toLocaleUpperCase() +
current.slice(charIndex + 1).toLocaleLowerCase();
const puncIndex = current.search(PUNC_TEST_REGEXP);
if (puncIndex === -1) {
buffer += current;
current = '';
break;
}
buffer += current.slice(0, puncIndex + 1);
current = current.slice(puncIndex + 1);
}
return buffer;
}
function toSnakeCase(value: string) {
return value
.toLocaleLowerCase()
.replace(/[ \-]/g, '_')
.replace(/[\u2000-\u206F\u2E00-\u2E7F\\'!"#$%&()*+,.\/:;<=>?@\[\]^`{|}~]/g, '');
}
function extractEmail(value: string) {
const matched = EMAIL_REGEXP.exec(value);
if (!matched) {
return undefined;
}
return matched[0];
}
function extractDomain(value: string) {
if (isEmail(value)) {
const matched = EMAIL_REGEXP.exec(value);
// This shouldn't happen
if (!matched) {
return undefined;
}
return matched.groups?.domain;
}
const domainMatch = value.match(DOMAIN_EXTRACT_REGEXP);
if (domainMatch) {
return domainMatch[1];
}
return undefined;
}
function extractUrl(value: string) {
const matched = URL_REGEXP.exec(value);
if (!matched) {
return undefined;
}
return matched[0];
}
function extractUrlPath(value: string) {
try {
const url = new URL(value);
return url.pathname;
} catch (error) {
return undefined;
}
}
function parseJson(value: string): unknown {
try {
return JSON.parse(value);
} catch (error) {
if (value.includes("'")) {
throw new ExpressionExtensionError("Parsing failed. Check you're using double quotes");
}
throw new ExpressionExtensionError('Parsing failed');
}
}
function toBoolean(value: string): boolean {
const normalized = value.toLowerCase();
const FALSY = new Set(['false', 'no', '0']);
return normalized.length > 0 && !FALSY.has(normalized);
}
function base64Encode(value: string): string {
return toBase64(value);
}
function base64Decode(value: string): string {
return fromBase64(value);
}
removeMarkdown.doc = {
name: 'removeMarkdown',
description: 'Removes any Markdown formatting from the string. Also removes HTML tags.',
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-removeMarkdown',
examples: [{ example: '"*bold*, [link]()".removeMarkdown()', evaluated: '"bold, link"' }],
};
removeTags.doc = {
name: 'removeTags',
description: 'Removes tags, such as HTML or XML, from the string.',
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-removeTags',
examples: [{ example: '"<b>bold</b>, <a>link</a>".removeTags()', evaluated: '"bold, link"' }],
};
toDate.doc = {
name: 'toDate',
description: 'Converts a string to a date.',
section: 'cast',
returnType: 'Date',
hidden: true,
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toDate',
};
toDateTime.doc = {
name: 'toDateTime',
description:
'Converts the string to a <a target="_blank" href="https://moment.github.io/luxon/api-docs/">Luxon</a> DateTime. Useful for further transformation. Supported formats for the string are ISO 8601, HTTP, RFC2822, SQL and Unix timestamp in milliseconds. To parse other formats, use <a target="_blank" href="https://moment.github.io/luxon/api-docs/index.html#datetimefromformat"> <code>DateTime.fromFormat()</code></a>.',
section: 'cast',
returnType: 'DateTime',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toDateTime',
examples: [
{ example: '"2024-03-29T18:06:31.798+01:00".toDateTime()' },
{ example: '"Fri, 29 Mar 2024 18:08:01 +0100".toDateTime()' },
{ example: '"20240329".toDateTime()' },
{ example: '"1711732132990".toDateTime("ms")' },
{ example: '"31-01-2024".toDateTime("dd-MM-yyyy")' },
],
args: [
{
name: 'format',
optional: true,
description:
'The format of the date string. Options are <code>ms</code> (for Unix timestamp in milliseconds), <code>s</code> (for Unix timestamp in seconds), <code>us</code> (for Unix timestamp in microseconds) or <code>excel</code> (for days since 1900). Custom formats can be specified using <a href="https://moment.github.io/luxon/#/formatting?id=table-of-tokens">Luxon tokens</a>.',
type: 'string',
},
],
};
toBoolean.doc = {
name: 'toBoolean',
description:
'Converts the string to a boolean value. <code>0</code>, <code>false</code> and <code>no</code> resolve to <code>false</code>, everything else to <code>true</code>. Case-insensitive.',
section: 'cast',
returnType: 'boolean',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toBoolean',
examples: [
{ example: '"true".toBoolean()', evaluated: 'true' },
{ example: '"false".toBoolean()', evaluated: 'false' },
{ example: '"0".toBoolean()', evaluated: 'false' },
{ example: '"hello".toBoolean()', evaluated: 'true' },
],
};
toFloat.doc = {
name: 'toFloat',
description: 'Converts a string to a decimal number.',
section: 'cast',
returnType: 'number',
aliases: ['toDecimalNumber'],
hidden: true,
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toDecimalNumber',
};
toInt.doc = {
name: 'toInt',
description: 'Converts a string to an integer.',
section: 'cast',
returnType: 'number',
args: [{ name: 'radix?', type: 'number' }],
aliases: ['toWholeNumber'],
hidden: true,
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toInt',
};
toSentenceCase.doc = {
name: 'toSentenceCase',
description:
'Changes the capitalization of the string to sentence case. The first letter of each sentence is capitalized and all others are lowercased.',
examples: [{ example: '"quick! brown FOX".toSentenceCase()', evaluated: '"Quick! Brown fox"' }],
section: 'case',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toSentenceCase',
};
toSnakeCase.doc = {
name: 'toSnakeCase',
description:
'Changes the format of the string to snake case. Spaces and dashes are replaced by <code>_</code>, symbols are removed and all letters are lowercased.',
examples: [{ example: '"quick brown $FOX".toSnakeCase()', evaluated: '"quick_brown_fox"' }],
section: 'case',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toSnakeCase',
};
toTitleCase.doc = {
name: 'toTitleCase',
description:
"Changes the capitalization of the string to title case. The first letter of each word is capitalized and the others left unchanged. Short prepositions and conjunctions aren't capitalized (e.g. 'a', 'the').",
examples: [{ example: '"quick a brown FOX".toTitleCase()', evaluated: '"Quick a Brown Fox"' }],
section: 'case',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toTitleCase',
};
urlEncode.doc = {
name: 'urlEncode',
description:
'Encodes the string so that it can be used in a URL. Spaces and special characters are replaced with codes of the form <code>%XX</code>.',
section: 'edit',
args: [
{
name: 'allChars',
optional: true,
description:
'Whether to encode characters that are part of the URI syntax (e.g. <code>=</code>, <code>?</code>)',
default: 'false',
type: 'boolean',
},
],
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-urlEncode',
examples: [
{ example: '"name=Nathan Automat".urlEncode()', evaluated: '"name%3DNathan%20Automat"' },
{ example: '"name=Nathan Automat".urlEncode(true)', evaluated: '"name=Nathan%20Automat"' },
],
};
urlDecode.doc = {
name: 'urlDecode',
description:
'Decodes a URL-encoded string. Replaces any character codes in the form of <code>%XX</code> with their corresponding characters.',
args: [
{
name: 'allChars',
optional: true,
description:
'Whether to decode characters that are part of the URI syntax (e.g. <code>=</code>, <code>?</code>)',
default: 'false',
type: 'boolean',
},
],
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-urlDecode',
examples: [
{ example: '"name%3DNathan%20Automat".urlDecode()', evaluated: '"name=Nathan Automat"' },
{ example: '"name%3DNathan%20Automat".urlDecode(true)', evaluated: '"name%3DNathan Automat"' },
],
};
replaceSpecialChars.doc = {
name: 'replaceSpecialChars',
description: 'Replaces special characters in the string with the closest ASCII character',
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-replaceSpecialChars',
examples: [{ example: '"déjà".replaceSpecialChars()', evaluated: '"deja"' }],
};
length.doc = {
name: 'length',
section: 'query',
hidden: true,
description: 'Returns the character count of a string.',
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings',
};
isDomain.doc = {
name: 'isDomain',
description: 'Returns <code>true</code> if a string is a domain.',
section: 'validation',
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-isDomain',
examples: [
{ example: '"n8n.io".isDomain()', evaluated: 'true' },
{ example: '"http://n8n.io".isDomain()', evaluated: 'false' },
{ example: '"hello".isDomain()', evaluated: 'false' },
],
};
isEmail.doc = {
name: 'isEmail',
description: 'Returns <code>true</code> if the string is an email.',
section: 'validation',
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-isEmail',
examples: [
{ example: '"me@example.com".isEmail()', evaluated: 'true' },
{ example: '"It\'s me@example.com".isEmail()', evaluated: 'false' },
{ example: '"hello".isEmail()', evaluated: 'false' },
],
};
isNumeric.doc = {
name: 'isNumeric',
description: 'Returns <code>true</code> if the string represents a number.',
section: 'validation',
returnType: 'boolean',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-isNumeric',
examples: [
{ example: '"1.2234".isNumeric()', evaluated: 'true' },
{ example: '"hello".isNumeric()', evaluated: 'false' },
{ example: '"123E23".isNumeric()', evaluated: 'true' },
],
};
isUrl.doc = {
name: 'isUrl',
description: 'Returns <code>true</code> if a string is a valid URL',
section: 'validation',
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-isUrl',
examples: [
{ example: '"https://n8n.io".isUrl()', evaluated: 'true' },
{ example: '"n8n.io".isUrl()', evaluated: 'false' },
{ example: '"hello".isUrl()', evaluated: 'false' },
],
};
isEmpty.doc = {
name: 'isEmpty',
description: 'Returns <code>true</code> if the string has no characters or is <code>null</code>',
section: 'validation',
returnType: 'boolean',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-isEmpty',
examples: [
{ example: '"".isEmpty()', evaluated: 'true' },
{ example: '"hello".isEmpty()', evaluated: 'false' },
],
};
isNotEmpty.doc = {
name: 'isNotEmpty',
description: 'Returns <code>true</code> if the string has at least one character.',
section: 'validation',
returnType: 'boolean',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-isNotEmpty',
examples: [
{ example: '"hello".isNotEmpty()', evaluated: 'true' },
{ example: '"".isNotEmpty()', evaluated: 'false' },
],
};
toJsonString.doc = {
name: 'toJsonString',
description:
"Prepares the string to be inserted into a JSON object. Escapes any quotes and special characters (e.g. new lines), and wraps the string in quotes.The same as JavaScript's JSON.stringify().",
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toJsonString',
examples: [
{
example: 'The "best" colours: red\nbrown.toJsonString()',
evaluated: '"The \\"best\\" colours: red\\nbrown"',
},
{ example: 'foo.toJsonString()', evaluated: '"foo"' },
],
};
extractEmail.doc = {
name: 'extractEmail',
description:
'Extracts the first email found in the string. Returns <code>undefined</code> if none is found.',
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-extractEmail',
examples: [
{ example: '"My email is me@example.com".extractEmail()', evaluated: "'me@example.com'" },
],
};
extractDomain.doc = {
name: 'extractDomain',
description:
'If the string is an email address or URL, returns its domain (or <code>undefined</code> if nothing found). If the string also contains other content, try using <code>extractEmail()</code> or <code>extractUrl()</code> first.',
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-extractDomain',
examples: [
{ example: '"me@example.com".extractDomain()', evaluated: "'example.com'" },
{ example: '"http://n8n.io/workflows".extractDomain()', evaluated: "'n8n.io'" },
{
example: '"It\'s me@example.com".extractEmail().extractDomain()',
evaluated: "'example.com'",
},
],
};
extractUrl.doc = {
name: 'extractUrl',
description:
'Extracts the first URL found in the string. Returns <code>undefined</code> if none is found. Only recognizes full URLs, e.g. those starting with <code>http</code>.',
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-extractUrl',
examples: [{ example: '"Check out http://n8n.io".extractUrl()', evaluated: "'http://n8n.io'" }],
};
extractUrlPath.doc = {
name: 'extractUrlPath',
description:
'Returns the part of a URL after the domain, or <code>undefined</code> if no URL found. If the string also contains other content, try using <code>extractUrl()</code> first.',
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-extractUrlPath',
examples: [
{ example: '"http://n8n.io/workflows".extractUrlPath()', evaluated: "'/workflows'" },
{
example: '"Check out http://n8n.io/workflows".extractUrl().extractUrlPath()',
evaluated: "'/workflows'",
},
],
};
hash.doc = {
name: 'hash',
description:
'Returns the string hashed with the given algorithm. Defaults to md5 if not specified.',
section: 'edit',
returnType: 'string',
args: [
{
name: 'algo',
optional: true,
description:
'The hashing algorithm to use. One of <code>md5</code>, <code>base64</code>, <code>sha1</code>, <code>sha224</code>, <code>sha256</code>, <code>sha384</code>, <code>sha512</code>, <code>sha3</code>, <code>ripemd160</code>\n ',
default: '"md5"',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-hash',
examples: [{ example: '"hello".hash()', evaluated: "'5d41402abc4b2a76b9719d911017c592'" }],
};
quote.doc = {
name: 'quote',
description:
'Wraps a string in quotation marks, and escapes any quotation marks already in the string. Useful when constructing JSON, SQL, etc.',
section: 'edit',
returnType: 'string',
args: [
{
name: 'mark',
optional: true,
description: 'The type of quotation mark to use',
default: '"',
type: 'string',
},
],
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-quote',
examples: [{ example: '\'Nathan says "hi"\'.quote()', evaluated: '\'"Nathan says \\"hi\\""\'' }],
};
parseJson.doc = {
name: 'parseJson',
aliases: ['fromJson'],
description:
"Returns the JavaScript value or object represented by the string, or <code>undefined</code> if the string isn't valid JSON. Single-quoted JSON is not supported.",
section: 'cast',
returnType: 'any',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-parseJson',
examples: [
{ example: '\'{"name":"Nathan"}\'.parseJson()', evaluated: '\'{"name":"Nathan"}\'' },
{ example: "\"{'name':'Nathan'}\".parseJson()", evaluated: 'undefined' },
{ example: "'hello'.parseJson()", evaluated: 'undefined' },
],
};
base64Encode.doc = {
name: 'base64Encode',
aliases: ['toBase64'],
description: 'Converts plain text to a base64-encoded string',
examples: [{ example: '"hello".base64Encode()', evaluated: '"aGVsbG8="' }],
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-base64Encode',
};
base64Decode.doc = {
name: 'base64Decode',
aliases: ['fromBase64'],
description: 'Converts a base64-encoded string to plain text',
examples: [{ example: '"aGVsbG8=".base64Decode()', evaluated: '"hello"' }],
section: 'edit',
returnType: 'string',
docURL:
'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-base64Decode',
};
toNumber.doc = {
name: 'toNumber',
description:
"Converts a string representing a number to a number. Errors if the string doesn't start with a valid number.",
section: 'cast',
returnType: 'number',
docURL: 'https://docs.n8n.io/code/builtin/data-transformation-functions/strings/#string-toNumber',
examples: [
{ example: '"123".toNumber()', evaluated: '123' },
{ example: '"1.23E10".toNumber()', evaluated: '12300000000' },
],
};
const toDecimalNumber: Extension = toFloat.bind({});
const toWholeNumber: Extension = toInt.bind({});
export const stringExtensions: ExtensionMap = {
typeName: 'String',
functions: {
hash,
removeMarkdown,
removeTags,
toDate,
toDateTime,
toBoolean,
toDecimalNumber,
toNumber,
toFloat,
toInt,
toWholeNumber,
toSentenceCase,
toSnakeCase,
toTitleCase,
urlDecode,
urlEncode,
quote,
replaceSpecialChars,
length,
isDomain,
isEmail,
isNumeric,
isUrl,
isEmpty,
isNotEmpty,
toJsonString,
extractEmail,
extractDomain,
extractUrl,
extractUrlPath,
parseJson,
base64Encode,
base64Decode,
},
};
@@ -0,0 +1,29 @@
import { DateTime } from 'luxon';
import { ExpressionExtensionError } from './expression-extension-error';
// Utility functions and type guards for expression extensions
export const convertToDateTime = (value: string | Date | DateTime): DateTime | undefined => {
let converted: DateTime | undefined;
if (typeof value === 'string') {
converted = DateTime.fromJSDate(new Date(value));
if (converted.invalidReason !== null) {
return;
}
} else if (value instanceof Date) {
converted = DateTime.fromJSDate(value);
} else if (DateTime.isDateTime(value)) {
converted = value;
}
return converted;
};
export function checkIfValueDefinedOrThrow<T>(value: T, functionName: string): void {
if (value === undefined || value === null) {
throw new ExpressionExtensionError(`${functionName} can't be used on ${String(value)} value`, {
description: `To ignore this error, add a ? to the variable before this function, e.g. my_var?.${functionName}`,
});
}
}
@@ -0,0 +1,33 @@
// Main exports
export { ExpressionEvaluator } from './evaluator/expression-evaluator';
// Bridge exports
export { IsolatedVmBridge } from './bridge/isolated-vm-bridge';
// Types
export type {
IExpressionEvaluator,
EvaluatorConfig,
WorkflowData,
EvaluateOptions,
RuntimeBridge,
BridgeConfig,
ObservabilityProvider,
MetricsAPI,
TracesAPI,
Span,
LogsAPI,
} from './types';
// Error types
export {
ExpressionError,
MemoryLimitError,
TimeoutError,
SecurityViolationError,
SyntaxError,
} from './types';
// Extension runtime exports
export { extend, extendOptional, EXTENSION_OBJECTS } from './extensions/extend';
export { ExpressionExtensionError } from './extensions/expression-extension-error';
@@ -0,0 +1,428 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { createDeepLazyProxy, isLazyProxy, getProxyPath } from '../lazy-proxy';
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
const ivmCallOpts = { arguments: { copy: true }, result: { copy: true } };
function mockApplySync(returnValue: unknown = undefined) {
return vi.fn().mockReturnValue(returnValue);
}
/** Install the three globalThis callbacks that createDeepLazyProxy relies on. */
function installGlobals(
overrides: {
getValueAtPath?: ReturnType<typeof vi.fn>;
callFunctionAtPath?: ReturnType<typeof vi.fn>;
getArrayElement?: ReturnType<typeof vi.fn>;
} = {},
) {
const getValueAtPath = overrides.getValueAtPath ?? mockApplySync();
const callFunctionAtPath = overrides.callFunctionAtPath ?? mockApplySync();
const getArrayElement = overrides.getArrayElement ?? mockApplySync();
(globalThis as any).__getValueAtPath = { applySync: getValueAtPath };
(globalThis as any).__callFunctionAtPath = { applySync: callFunctionAtPath };
(globalThis as any).__getArrayElement = { applySync: getArrayElement };
return { getValueAtPath, callFunctionAtPath, getArrayElement };
}
function cleanupGlobals() {
delete (globalThis as any).__getValueAtPath;
delete (globalThis as any).__callFunctionAtPath;
delete (globalThis as any).__getArrayElement;
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('createDeepLazyProxy', () => {
let mocks: ReturnType<typeof installGlobals>;
beforeEach(() => {
mocks = installGlobals();
});
afterEach(() => {
cleanupGlobals();
});
// -----------------------------------------------------------------------
// 1. Special properties
// -----------------------------------------------------------------------
describe('special properties', () => {
it('returns undefined for Symbol.toStringTag', () => {
const proxy = createDeepLazyProxy();
expect(proxy[Symbol.toStringTag]).toBeUndefined();
expect(mocks.getValueAtPath).not.toHaveBeenCalled();
});
it('returns undefined for Symbol.toPrimitive', () => {
const proxy = createDeepLazyProxy();
expect(proxy[Symbol.toPrimitive]).toBeUndefined();
expect(mocks.getValueAtPath).not.toHaveBeenCalled();
});
it('toString() returns "[object Object]"', () => {
const proxy = createDeepLazyProxy();
expect(proxy.toString()).toBe('[object Object]');
expect(mocks.getValueAtPath).not.toHaveBeenCalled();
});
it('valueOf() returns the proxy target', () => {
const proxy = createDeepLazyProxy();
const val = proxy.valueOf();
expect(typeof val).toBe('object');
expect(val).not.toBeNull();
expect(mocks.getValueAtPath).not.toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// 1b. Proxy identity helpers (isLazyProxy / getProxyPath)
// -----------------------------------------------------------------------
describe('proxy identity helpers', () => {
it('isLazyProxy() returns true for a proxy', () => {
const proxy = createDeepLazyProxy();
expect(isLazyProxy(proxy)).toBe(true);
});
it('isLazyProxy() returns false for plain objects', () => {
expect(isLazyProxy({})).toBe(false);
expect(isLazyProxy(null)).toBe(false);
expect(isLazyProxy('string')).toBe(false);
});
it('getProxyPath() returns [] when no basePath is provided', () => {
const proxy = createDeepLazyProxy();
expect(getProxyPath(proxy)).toEqual([]);
});
it('getProxyPath() returns the provided basePath', () => {
const proxy = createDeepLazyProxy(['$json', 'user']);
expect(getProxyPath(proxy)).toEqual(['$json', 'user']);
});
it('getProxyPath() returns undefined for non-proxies', () => {
expect(getProxyPath({})).toBeUndefined();
});
});
// -----------------------------------------------------------------------
// 2. Primitive values
// -----------------------------------------------------------------------
describe('primitive values', () => {
it('fetches and returns undefined', () => {
mocks.getValueAtPath.mockReturnValue(undefined);
const proxy = createDeepLazyProxy();
expect(proxy.missing).toBeUndefined();
});
});
// -----------------------------------------------------------------------
// 3. Caching
// -----------------------------------------------------------------------
describe('caching', () => {
it('does not re-fetch on second access', () => {
mocks.getValueAtPath.mockReturnValue('cached');
const proxy = createDeepLazyProxy();
expect(proxy.x).toBe('cached');
expect(proxy.x).toBe('cached');
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
});
it('caches null values', () => {
mocks.getValueAtPath.mockReturnValue(null);
const proxy = createDeepLazyProxy();
proxy.n;
proxy.n;
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
});
it('caches undefined values', () => {
mocks.getValueAtPath.mockReturnValue(undefined);
const proxy = createDeepLazyProxy();
proxy.u;
proxy.u;
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
});
});
// -----------------------------------------------------------------------
// 4. Function metadata
// -----------------------------------------------------------------------
describe('function metadata', () => {
it('creates a callable wrapper for function metadata', () => {
mocks.getValueAtPath.mockReturnValue({ __isFunction: true, __name: 'myFn' });
const proxy = createDeepLazyProxy();
expect(typeof proxy.myFn).toBe('function');
});
it('invokes __callFunctionAtPath with correct args when called', () => {
mocks.getValueAtPath.mockReturnValue({ __isFunction: true, __name: 'myFn' });
mocks.callFunctionAtPath.mockReturnValue('result');
const proxy = createDeepLazyProxy();
proxy.myFn('a', 1);
expect(mocks.callFunctionAtPath).toHaveBeenCalledWith(null, [['myFn'], 'a', 1], ivmCallOpts);
});
it('caches the function wrapper', () => {
mocks.getValueAtPath.mockReturnValue({ __isFunction: true, __name: 'myFn' });
const proxy = createDeepLazyProxy();
const first = proxy.myFn;
const second = proxy.myFn;
expect(first).toBe(second);
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
});
});
// -----------------------------------------------------------------------
// 5. Array metadata (always lazy-loaded via array proxy)
// -----------------------------------------------------------------------
describe('array metadata', () => {
it('creates an array proxy', () => {
mocks.getValueAtPath.mockReturnValue({ __isArray: true, __length: 100 });
const proxy = createDeepLazyProxy();
expect(proxy.items).toBeDefined();
});
it('caches the array proxy', () => {
mocks.getValueAtPath.mockReturnValue({ __isArray: true, __length: 3 });
const proxy = createDeepLazyProxy();
const first = proxy.arr;
const second = proxy.arr;
expect(first).toBe(second);
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
});
});
// -----------------------------------------------------------------------
// 7. Array proxy — element access
// -----------------------------------------------------------------------
describe('array proxy element access', () => {
function proxyWithLargeArray(length = 10) {
mocks.getValueAtPath.mockReturnValue({ __isArray: true, __length: length });
return createDeepLazyProxy();
}
it('creates a nested proxy for object elements', () => {
const proxy = proxyWithLargeArray();
mocks.getArrayElement.mockReturnValue({ __isObject: true, __keys: ['a'] });
const element = proxy.items[0];
expect(isLazyProxy(element)).toBe(true);
expect(getProxyPath(element)).toEqual(['items', '0']);
});
it('creates a nested proxy for array elements that are arrays', () => {
const proxy = proxyWithLargeArray();
mocks.getArrayElement.mockReturnValue({ __isArray: true, __length: 5 });
const element = proxy.items[0];
expect(isLazyProxy(element)).toBe(true);
expect(getProxyPath(element)).toEqual(['items', '0']);
});
it('caches elements after first access', () => {
const proxy = proxyWithLargeArray();
mocks.getArrayElement.mockReturnValue('val');
proxy.items[0];
proxy.items[0];
expect(mocks.getArrayElement).toHaveBeenCalledTimes(1);
});
it('passes correct path for nested arrays', () => {
mocks.getValueAtPath.mockReturnValue({ __isArray: true, __length: 5 });
const proxy = createDeepLazyProxy(['data']);
mocks.getArrayElement.mockReturnValue('val');
proxy.list[3];
expect(mocks.getArrayElement).toHaveBeenCalledWith(null, [['data', 'list'], 3], ivmCallOpts);
});
it('returns undefined for non-numeric non-length properties', () => {
const proxy = proxyWithLargeArray();
expect(proxy.items.foo).toBeUndefined();
expect(mocks.getArrayElement).not.toHaveBeenCalled();
});
it('does not intercept negative indices', () => {
const proxy = proxyWithLargeArray();
// -1 is NaN? No, Number('-1') === -1 which is not NaN, but -1 >= 0 is false
proxy.items[-1];
expect(mocks.getArrayElement).not.toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// 8. Object metadata
// -----------------------------------------------------------------------
describe('object metadata', () => {
it('creates a nested proxy for object metadata', () => {
mocks.getValueAtPath.mockReturnValue({ __isObject: true, __keys: ['a', 'b'] });
const proxy = createDeepLazyProxy();
expect(isLazyProxy(proxy.obj)).toBe(true);
});
it('nested proxy has the correct path', () => {
mocks.getValueAtPath.mockReturnValue({ __isObject: true, __keys: ['a'] });
const proxy = createDeepLazyProxy();
expect(getProxyPath(proxy.obj)).toEqual(['obj']);
});
it('deep nesting builds correct paths', () => {
mocks.getValueAtPath.mockReturnValue({ __isObject: true, __keys: ['x'] });
const proxy = createDeepLazyProxy();
// Each level triggers __getValueAtPath and creates a nested proxy
// a -> returns object metadata
const a = proxy.a;
expect(mocks.getValueAtPath).toHaveBeenLastCalledWith(null, [['a']], ivmCallOpts);
// a.b -> returns object metadata
const b = a.b;
expect(mocks.getValueAtPath).toHaveBeenLastCalledWith(null, [['a', 'b']], ivmCallOpts);
// a.b.c -> returns object metadata
const c = b.c;
expect(mocks.getValueAtPath).toHaveBeenLastCalledWith(null, [['a', 'b', 'c']], ivmCallOpts);
expect(getProxyPath(c)).toEqual(['a', 'b', 'c']);
});
it('caches the nested proxy', () => {
mocks.getValueAtPath.mockReturnValue({ __isObject: true, __keys: ['a'] });
const proxy = createDeepLazyProxy();
const first = proxy.obj;
const second = proxy.obj;
expect(first).toBe(second);
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
});
});
// -----------------------------------------------------------------------
// 9. basePath propagation
// -----------------------------------------------------------------------
describe('basePath propagation', () => {
it('prepends basePath to property paths', () => {
mocks.getValueAtPath.mockReturnValue('val');
const proxy = createDeepLazyProxy(['$json']);
proxy.user;
expect(mocks.getValueAtPath).toHaveBeenCalledWith(null, [['$json', 'user']], ivmCallOpts);
});
it('nested proxies inherit full path', () => {
mocks.getValueAtPath.mockReturnValue({ __isObject: true, __keys: ['name'] });
const proxy = createDeepLazyProxy(['$json']);
const user = proxy.user;
expect(getProxyPath(user)).toEqual(['$json', 'user']);
// Accessing a property on the nested proxy should build the full path
mocks.getValueAtPath.mockReturnValue('Alice');
user.name;
expect(mocks.getValueAtPath).toHaveBeenLastCalledWith(
null,
[['$json', 'user', 'name']],
ivmCallOpts,
);
});
});
// -----------------------------------------------------------------------
// 10. has trap
// -----------------------------------------------------------------------
describe('has trap ("in" operator)', () => {
it('returns false for symbols', () => {
const proxy = createDeepLazyProxy();
expect(Symbol.toStringTag in proxy).toBe(false);
expect(mocks.getValueAtPath).not.toHaveBeenCalled();
});
it('returns true for cached properties without re-fetching', () => {
mocks.getValueAtPath.mockReturnValue('value');
const proxy = createDeepLazyProxy();
// Access to populate cache
proxy.x;
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
// 'in' check should use cache
expect('x' in proxy).toBe(true);
expect(mocks.getValueAtPath).toHaveBeenCalledTimes(1);
});
it('returns true for existing (non-undefined) properties', () => {
mocks.getValueAtPath.mockReturnValue('value');
const proxy = createDeepLazyProxy();
expect('prop' in proxy).toBe(true);
});
it('returns true for null properties (exists but null)', () => {
mocks.getValueAtPath.mockReturnValue(null);
const proxy = createDeepLazyProxy();
expect('prop' in proxy).toBe(true);
});
it('returns false for undefined (non-existent) properties', () => {
mocks.getValueAtPath.mockReturnValue(undefined);
const proxy = createDeepLazyProxy();
expect('prop' in proxy).toBe(false);
});
it('passes the correct path including basePath', () => {
mocks.getValueAtPath.mockReturnValue('val');
const proxy = createDeepLazyProxy(['$json']);
'foo' in proxy;
expect(mocks.getValueAtPath).toHaveBeenCalledWith(null, [['$json', 'foo']], ivmCallOpts);
});
});
// -----------------------------------------------------------------------
// 11. Edge cases
// -----------------------------------------------------------------------
describe('edge cases', () => {
it('plain object with __isFunction=false is treated as primitive', () => {
mocks.getValueAtPath.mockReturnValue({ __isFunction: false, other: 1 });
const proxy = createDeepLazyProxy();
const val = proxy.prop;
// Not a function — falls through to "primitive" caching
expect(typeof val).toBe('object');
expect(val.__isFunction).toBe(false);
expect(val.other).toBe(1);
});
it('plain object with __isArray=false is treated as primitive', () => {
mocks.getValueAtPath.mockReturnValue({ __isArray: false, data: 'x' });
const proxy = createDeepLazyProxy();
const val = proxy.prop;
expect(typeof val).toBe('object');
expect(val.data).toBe('x');
});
it('array proxy does not intercept negative indices', () => {
mocks.getValueAtPath.mockReturnValue({ __isArray: true, __length: 3 });
const proxy = createDeepLazyProxy();
proxy.arr[-1];
expect(mocks.getArrayElement).not.toHaveBeenCalled();
});
it('function wrapper on a basePath proxy passes full path', () => {
mocks.getValueAtPath.mockReturnValue({ __isFunction: true, __name: '$items' });
mocks.callFunctionAtPath.mockReturnValue([]);
const proxy = createDeepLazyProxy(['$root']);
proxy.fn();
expect(mocks.callFunctionAtPath).toHaveBeenCalledWith(null, [['$root', 'fn']], ivmCallOpts);
});
});
});
@@ -0,0 +1,58 @@
import { DateTime } from 'luxon';
import { extend, extendOptional } from '../extensions/extend';
import { SafeObject, SafeError, ExpressionError } from './safe-globals';
import { createDeepLazyProxy } from './lazy-proxy';
import { resetDataProxies } from './reset';
// Augment globalThis with runtime properties
declare global {
namespace globalThis {
// Callbacks from bridge (ivm.Reference)
var __getValueAtPath: any;
var __getArrayElement: any;
var __callFunctionAtPath: any;
// Data container
var __data: Record<string, unknown>;
// Proxy creator function
var createDeepLazyProxy: (basePath?: string[]) => any;
// Reset function (Step 3)
var resetDataProxies: () => void;
// Safe wrappers
var SafeObject: typeof Object;
var SafeError: typeof Error;
// Expression engine globals
var DateTime: typeof import('luxon').DateTime;
var extend: typeof import('../extensions/extend').extend;
var extendOptional: typeof import('../extensions/extend').extendOptional;
}
}
// ============================================================================
// Library Setup
// ============================================================================
// Expose globals required by tournament-transformed expressions
globalThis.extend = extend;
globalThis.extendOptional = extendOptional;
globalThis.DateTime = DateTime;
// ============================================================================
// Expose security globals and runtime functions
// ============================================================================
globalThis.SafeObject = SafeObject;
globalThis.SafeError = SafeError;
(globalThis as any).ExpressionError = ExpressionError;
globalThis.createDeepLazyProxy = createDeepLazyProxy;
globalThis.resetDataProxies = resetDataProxies;
// Initialize empty __data object (populated by resetDataProxies before each evaluation)
globalThis.__data = {};
@@ -0,0 +1,183 @@
// ============================================================================
// Deep Lazy Proxy System
// ============================================================================
// For more information about Proxies see
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy
// ---------------------------------------------------------------------------
// Proxy registry — used only for testing/introspection, not accessible from
// expression code. Avoids shadowing user data keys like __isProxy / __path.
// ---------------------------------------------------------------------------
const proxyPaths = new WeakMap<object, string[]>();
/** Returns true if `obj` is a deep lazy proxy created by createDeepLazyProxy. */
export function isLazyProxy(obj: unknown): boolean {
return typeof obj === 'object' && obj !== null && proxyPaths.has(obj as object);
}
/** Returns the basePath the proxy was created with, or undefined if not a proxy. */
export function getProxyPath(obj: object): string[] | undefined {
return proxyPaths.get(obj);
}
/**
* Creates a deep lazy-loading proxy for workflow data.
*
* This proxy system enables on-demand loading of nested properties across
* the isolate boundary using metadata-driven callbacks.
*
* Pattern:
* 1. When property accessed: Call __getValueAtPath([path]) to get metadata
* 2. Metadata indicates type: primitive, object, array, or function
* 3. For objects/arrays: Create nested proxy for lazy loading
* 4. For functions: Create wrapper that calls __callFunctionAtPath
* 5. Cache all fetched values in target to avoid repeated callbacks
*
* @param basePath - Current path in object tree (e.g., ['$json', 'user'])
* @returns Proxy object with lazy loading behavior
*/
export function createDeepLazyProxy(basePath: string[] = []): any {
const proxy = new Proxy({} as Record<string, unknown>, {
get(target: any, prop: string | symbol): unknown {
// Handle Symbol properties - return undefined
// Symbols like Symbol.toStringTag are accessed internally
// We can't transfer Symbols via isolated-vm
if (typeof prop === 'symbol') {
return undefined;
}
// Handle common Object.prototype methods within isolate
// Don't fetch from parent to avoid native function transfer issues
if (prop === 'toString') {
return function () {
return '[object Object]';
};
}
if (prop === 'valueOf') {
return function () {
return target;
};
}
// Check cache - if already fetched, return cached value
if (prop in target) {
return target[prop];
}
// Build path for this property
const path = [...basePath, prop as string];
// Call back to parent to get metadata/value
// Note: __getValueAtPath is an ivm.Reference set by bridge
const value = globalThis.__getValueAtPath.applySync(null, [path], {
arguments: { copy: true },
result: { copy: true },
});
// Handle undefined/null - cache and return
if (value === undefined || value === null) {
target[prop] = value;
return value;
}
// Handle functions - metadata: { __isFunction: true, __name: string }
if (value && typeof value === 'object' && value.__isFunction) {
// Create function wrapper that calls back to parent
target[prop] = function (...args: any[]) {
return globalThis.__callFunctionAtPath.applySync(null, [path, ...args], {
arguments: { copy: true },
result: { copy: true },
});
};
return target[prop];
}
// Handle arrays - metadata: { __isArray: true, __length: number }
if (value && typeof value === 'object' && value.__isArray) {
const arrayProxy = new Proxy([] as any[], {
get(arrTarget: any, arrProp: string | symbol): unknown {
// Symbols can't be transferred via isolated-vm; return undefined
if (typeof arrProp === 'symbol') {
return undefined;
}
// Handle array length
if (arrProp === 'length') {
return value.__length;
}
// Handle numeric index
const index = Number(arrProp);
if (!isNaN(index) && index >= 0) {
// Check cache
if (!(arrProp in arrTarget)) {
// Fetch element from parent
const element = globalThis.__getArrayElement.applySync(null, [path, index], {
arguments: { copy: true },
result: { copy: true },
});
// Handle element metadata (arrays and objects need proxies)
if (element && typeof element === 'object' && element.__isArray) {
const elementPath = [...path, String(index)];
arrTarget[arrProp] = createDeepLazyProxy(elementPath);
} else if (element && typeof element === 'object' && element.__isObject) {
// Object metadata: create nested proxy
const elementPath = [...path, String(index)];
arrTarget[arrProp] = createDeepLazyProxy(elementPath);
} else {
// Primitive element
arrTarget[arrProp] = element;
}
}
return arrTarget[arrProp];
}
// Array methods (map, filter, etc.)
// Note: These require full array - limitation for now
return arrTarget[arrProp];
},
});
target[prop] = arrayProxy;
return target[prop];
}
// Handle objects - metadata: { __isObject: true, __keys: string[] }
if (value && typeof value === 'object' && value.__isObject) {
// Create nested proxy for recursive lazy loading
target[prop] = createDeepLazyProxy(path);
return target[prop];
}
// Primitive value - cache and return
target[prop] = value;
return value;
},
has(target: any, prop: string | symbol): boolean {
// Implement 'in' operator support
// Example: '$json' in data
if (typeof prop === 'symbol') return false;
// Check cache first
if (prop in target) {
return true;
}
// Build path and check existence via callback
const path = [...basePath, prop as string];
const value = globalThis.__getValueAtPath.applySync(null, [path], {
arguments: { copy: true },
result: { copy: true },
});
// Property exists if value is not undefined
// Note: null values mean property exists but is null
return value !== undefined;
},
});
proxyPaths.set(proxy, basePath);
return proxy;
}
@@ -0,0 +1,138 @@
import { extend, extendOptional } from '../extensions/extend';
import { __sanitize } from './safe-globals';
import { createDeepLazyProxy } from './lazy-proxy';
// ============================================================================
// Reset Function for Data Proxies
// ============================================================================
/**
* Reset workflow data proxies before each evaluation.
*
* This function is called from the bridge before executing each expression
* to clear proxy caches and initialize fresh workflow data references.
*
* Pattern:
* 1. Create lazy proxies for complex properties ($json, $binary, etc.)
* 2. Fetch primitives directly ($runIndex, $itemIndex)
* 3. Create function wrappers for callable properties ($items, etc.)
* 4. Expose all properties to globalThis for expression access
*
* Called from bridge: context.evalSync('resetDataProxies()')
*/
export function resetDataProxies(): void {
// Clear existing __data object
globalThis.__data = {};
// __sanitize must be on __data because PrototypeSanitizer generates:
// obj[this.__sanitize(expr)] where 'this' is __data (via .call(__data) wrapping)
(globalThis.__data as any).__sanitize = __sanitize;
// Verify callbacks are available
// Note: ivm.Reference may not be typeof 'function', check for existence
if (!globalThis.__getValueAtPath) {
throw new Error('__getValueAtPath callback not registered');
}
// -------------------------------------------------------------------------
// Create lazy proxies for complex workflow properties
// -------------------------------------------------------------------------
globalThis.__data.$json = createDeepLazyProxy(['$json']);
globalThis.__data.$binary = createDeepLazyProxy(['$binary']);
globalThis.__data.$input = createDeepLazyProxy(['$input']);
globalThis.__data.$node = createDeepLazyProxy(['$node']);
globalThis.__data.$parameter = createDeepLazyProxy(['$parameter']);
globalThis.__data.$workflow = createDeepLazyProxy(['$workflow']);
globalThis.__data.$prevNode = createDeepLazyProxy(['$prevNode']);
globalThis.__data.$data = createDeepLazyProxy(['$data']);
globalThis.__data.$env = createDeepLazyProxy(['$env']);
// -------------------------------------------------------------------------
// Fetch primitives directly (no lazy loading needed for simple values)
// -------------------------------------------------------------------------
try {
globalThis.__data.$runIndex = globalThis.__getValueAtPath.applySync(null, [['$runIndex']], {
arguments: { copy: true },
result: { copy: true },
});
} catch (error) {
// Property doesn't exist - set to undefined
globalThis.__data.$runIndex = undefined;
}
try {
globalThis.__data.$itemIndex = globalThis.__getValueAtPath.applySync(null, [['$itemIndex']], {
arguments: { copy: true },
result: { copy: true },
});
} catch (error) {
// Property doesn't exist - set to undefined
globalThis.__data.$itemIndex = undefined;
}
// -------------------------------------------------------------------------
// Expose workflow data to globalThis for expression access
// -------------------------------------------------------------------------
(globalThis as any).$json = globalThis.__data.$json;
(globalThis as any).$binary = globalThis.__data.$binary;
(globalThis as any).$input = globalThis.__data.$input;
(globalThis as any).$node = globalThis.__data.$node;
(globalThis as any).$parameter = globalThis.__data.$parameter;
(globalThis as any).$workflow = globalThis.__data.$workflow;
(globalThis as any).$prevNode = globalThis.__data.$prevNode;
(globalThis as any).$runIndex = globalThis.__data.$runIndex;
(globalThis as any).$itemIndex = globalThis.__data.$itemIndex;
(globalThis as any).$data = globalThis.__data.$data;
(globalThis as any).$env = globalThis.__data.$env;
// -------------------------------------------------------------------------
// Handle function properties (check if value is function metadata)
// -------------------------------------------------------------------------
// Check if $items exists and is a function
if (globalThis.__callFunctionAtPath) {
try {
const itemsValue = globalThis.__getValueAtPath.applySync(null, [['$items']], {
arguments: { copy: true },
result: { copy: true },
});
// If it's function metadata, create wrapper
if (itemsValue && typeof itemsValue === 'object' && itemsValue.__isFunction) {
(globalThis as any).$items = function (...args: any[]) {
return globalThis.__callFunctionAtPath.applySync(null, [['$items'], ...args], {
arguments: { copy: true },
result: { copy: true },
});
};
globalThis.__data.$items = (globalThis as any).$items;
} else {
// Not a function - set to undefined or the value itself
(globalThis as any).$items = itemsValue;
globalThis.__data.$items = itemsValue;
}
} catch (error) {
// Property doesn't exist
(globalThis as any).$items = undefined;
globalThis.__data.$items = undefined;
}
}
// -------------------------------------------------------------------------
// Expose globals on __data so tournament's "x in this ? this.x : global.x"
// pattern resolves them correctly (tournament checks __data before global)
// -------------------------------------------------------------------------
(globalThis.__data as any).DateTime = globalThis.DateTime;
// Expose extend/extendOptional on __data so tournament's "x in this ? this.x : global.x"
// pattern resolves them correctly when the VM checks __data first
(globalThis.__data as any).extend = extend;
(globalThis.__data as any).extendOptional = extendOptional;
// TODO: Add other function properties as needed ($item, $vars, etc.)
}
@@ -0,0 +1,121 @@
// ============================================================================
// Safe Wrappers for Security-Sensitive Globals
// ============================================================================
/**
* SafeObject - Blocks dangerous Object methods that could lead to RCE
*
* Blocked methods:
* - defineProperty, setPrototypeOf: Prevent prototype pollution
* - getOwnPropertyDescriptor: Prevent property descriptor manipulation
* - __defineGetter__, __defineSetter__: Legacy descriptor manipulation
*/
export const SafeObject = new Proxy(Object, {
get(target, prop) {
// Block dangerous methods (return undefined)
const blockedMethods = [
'defineProperty',
'defineProperties',
'setPrototypeOf',
'getOwnPropertyDescriptor',
'getOwnPropertyDescriptors',
'__defineGetter__',
'__defineSetter__',
'__lookupGetter__',
'__lookupSetter__',
];
if (blockedMethods.includes(prop as string)) {
return undefined;
}
// Block getPrototypeOf by throwing (more secure than returning undefined)
if (prop === 'getPrototypeOf') {
throw new Error('Object.getPrototypeOf is not allowed');
}
// Allow other Object methods
const value = (target as any)[prop];
if (typeof value === 'function') {
// Use arrow function wrapper to preserve 'this' binding
return (...args: any[]) => value.apply(target, args);
}
return value;
},
});
/**
* SafeError - Blocks stack manipulation methods
*
* Blocked properties:
* - stackTraceLimit, captureStackTrace, prepareStackTrace: Prevent stack manipulation attacks
*/
export const SafeError = new Proxy(Error, {
get(target, prop) {
// Block stack manipulation (return undefined)
const blockedProps = ['stackTraceLimit', 'captureStackTrace', 'prepareStackTrace'];
if (blockedProps.includes(prop as string)) {
return undefined;
}
// Block dangerous methods
const blockedMethods = ['__defineGetter__', '__defineSetter__'];
if (blockedMethods.includes(prop as string)) {
return undefined;
}
const value = (target as any)[prop];
if (typeof value === 'function') {
return (...args: any[]) => value.apply(target, args);
}
return value;
},
set(target, prop, value) {
// Block setting prepareStackTrace
if (prop === 'prepareStackTrace') {
return false;
}
(target as any)[prop] = value;
return true;
},
});
// ============================================================================
// ExpressionError - used by tournament-generated error handlers
// ============================================================================
export class ExpressionError extends Error {
constructor(message: string) {
super(message);
this.name = 'ExpressionError';
}
}
// ============================================================================
// Runtime sanitizer for dynamic property access
// Generated by PrototypeSanitizer: obj[expr] → obj[this.__sanitize(expr)]
// Must match the blocklist in packages/workflow/src/expression-sandboxing.ts
// ============================================================================
const unsafeObjectProperties = new Set([
'__proto__',
'prototype',
'constructor',
'__defineGetter__',
'__defineSetter__',
'__lookupGetter__',
'__lookupSetter__',
'toString',
'valueOf',
'toLocaleString',
'hasOwnProperty',
'isPrototypeOf',
'propertyIsEnumerable',
]);
export function __sanitize(value: unknown): unknown {
if (typeof value === 'string' && unsafeObjectProperties.has(value)) {
throw new ExpressionError(`Cannot access "${value}" due to security concerns`);
}
return value;
}
@@ -0,0 +1,80 @@
// ============================================================================
// Phase 1.1: Bridge Interface (CORE - IMPLEMENT FIRST)
//
// This is the main interface all environments must implement.
// Start here for CLI/backend (IsolatedVmBridge) or frontend (WebWorkerBridge).
// ============================================================================
/**
* Abstract interface for runtime bridges.
*
* A bridge manages communication between the host process and the isolated context.
* Different bridge implementations support different isolation mechanisms:
* - IsolatedVmBridge: Uses isolated-vm for Node.js backend (secure isolation with memory limits)
* - WebWorkerBridge: Uses Web Workers for browser frontend (Phase 2+)
* - Task Runner: TBD - May use IsolatedVmBridge locally or direct evaluation (Phase 2+)
*/
export interface RuntimeBridge {
/**
* Initialize the isolated context and load runtime code.
* Must be called before any execute() calls.
*/
initialize(): Promise<void>;
/**
* Execute JavaScript code in the isolated context.
*
* @param code - Transformed JavaScript code to execute
* @param data - Workflow data proxy from WorkflowDataProxy.getDataProxy()
* @returns Result of the expression evaluation.
* Must be JSON-serializable (no functions, symbols, etc.)
*
* Note: Synchronous for Node.js vm module (Slice 1).
* Will be async for isolated-vm (Slice 2).
*/
execute(code: string, data: Record<string, unknown>): unknown;
/**
* Dispose of the isolated context and free resources.
* After disposal, the bridge cannot be used again.
*/
dispose(): Promise<void>;
/**
* Check if the bridge has been disposed.
* Disposed bridges cannot execute code.
*/
isDisposed(): boolean;
}
/**
* Configuration for runtime bridges.
*/
export interface BridgeConfig {
/**
* Memory limit in MB for isolated context.
* Default: 128MB
*/
memoryLimit?: number;
/**
* Timeout in milliseconds for expression execution.
* Default: 5000ms
*/
timeout?: number;
/**
* Enable debug mode (inspector protocol).
* Default: false
*
* Phase 2+: Chrome DevTools debugging support
*/
debug?: boolean;
}
/** Default values for BridgeConfig. Bridge implementations should use this as their baseline. */
export const DEFAULT_BRIDGE_CONFIG: Required<BridgeConfig> = {
memoryLimit: 128,
timeout: 5000,
debug: false,
};
@@ -0,0 +1,238 @@
import type { TournamentHooks } from '@n8n/tournament';
import type { RuntimeBridge } from './bridge';
// ============================================================================
// Phase 1.1: Core Evaluation Interfaces (MVP)
// These are the minimal interfaces needed to evaluate expressions.
// ============================================================================
/**
* Configuration for ExpressionEvaluator.
*
* Note: Slice 1 keeps this minimal. Tournament integration and code caching
* will be added in later slices.
*/
export interface EvaluatorConfig {
/**
* Runtime bridge implementation.
*/
bridge: RuntimeBridge;
/**
* Observability provider for metrics, traces, and logs.
*/
observability?: ObservabilityProvider;
/**
* AST security hooks for tournament expression transformation.
* Provided by the caller (e.g., workflow package's expression-sandboxing.ts).
* If omitted, expressions are transformed with no security hooks (dev/testing use).
*/
hooks?: TournamentHooks;
}
/**
* Expression evaluator - main public API.
*
* This is the primary interface used by the workflow package.
*/
export interface IExpressionEvaluator {
/**
* Initialize the evaluator and bridge.
* Must be called before evaluate().
*/
initialize(): Promise<void>;
/**
* Evaluate an expression string against workflow data.
*
* @param expression - Expression string (e.g., "{{ $json.email }}")
* @param data - Workflow data context
* @param options - Evaluation options
* @returns Result of the expression
*
* Note: Synchronous for Slice 1 (Node.js vm module).
* Will be async for Slice 2 (isolated-vm).
*/
evaluate(expression: string, data: WorkflowData, options?: EvaluateOptions): unknown;
/**
* Dispose of the evaluator and free resources.
*/
dispose(): Promise<void>;
/**
* Check if the evaluator has been disposed.
*/
isDisposed(): boolean;
}
/**
* Workflow data proxy from WorkflowDataProxy.getDataProxy().
*
* For Slice 1: We pass this directly via VM context (simple pass-through).
* Later: Will implement deep lazy proxy for field-level data fetching.
*/
export type WorkflowData = Record<string, unknown>;
/**
* Options for evaluate().
*/
/**
* Options for evaluate().
*
* Note: Slice 1 is minimal. Tournament options will be added later.
*/
export interface EvaluateOptions {
/**
* Custom timeout for this evaluation (in milliseconds).
* Overrides the bridge's default timeout.
*/
timeout?: number;
}
// ============================================================================
// Phase 0.2 / Phase 1+: Observability Interfaces (OPTIONAL FOR MVP)
//
// These can be stubbed with NoOpProvider initially.
// Full implementation comes in Phase 0.2 (observability infrastructure).
//
// Frontend developers: You can ignore this section for Phase 1.
// CLI/Backend developers: Use NoOpProvider initially, real providers later.
// ============================================================================
/**
* Observability provider interface.
*
* Implementations: NoOpProvider, OpenTelemetryProvider, PostHogProvider
*/
export interface ObservabilityProvider {
/**
* Metrics API.
*/
metrics: MetricsAPI;
/**
* Traces API.
*/
traces: TracesAPI;
/**
* Logs API.
*/
logs: LogsAPI;
}
/**
* Metrics API.
*/
export interface MetricsAPI {
/**
* Increment a counter.
*/
counter(name: string, value: number, tags?: Record<string, string>): void;
/**
* Set a gauge value.
*/
gauge(name: string, value: number, tags?: Record<string, string>): void;
/**
* Record a histogram value.
*/
histogram(name: string, value: number, tags?: Record<string, string>): void;
}
/**
* Traces API.
*/
export interface TracesAPI {
/**
* Start a new span.
*/
startSpan(name: string, attributes?: Record<string, unknown>): Span;
}
/**
* Span interface.
*/
export interface Span {
/**
* Set span status.
*/
setStatus(status: 'ok' | 'error'): void;
/**
* Set span attribute.
*/
setAttribute(key: string, value: unknown): void;
/**
* Record an exception.
*/
recordException(error: Error): void;
/**
* End the span.
*/
end(): void;
}
/**
* Logs API.
*/
export interface LogsAPI {
/**
* Log an error.
*/
error(message: string, error?: Error, context?: Record<string, unknown>): void;
/**
* Log a warning.
*/
warn(message: string, context?: Record<string, unknown>): void;
/**
* Log info.
*/
info(message: string, context?: Record<string, unknown>): void;
/**
* Log debug.
*/
debug(message: string, context?: Record<string, unknown>): void;
}
// ============================================================================
// Phase 1.4: Error Handling (IMPLEMENT WITH EVALUATOR)
//
// These error types provide structured error information.
// Start with basic Error, add these types in Phase 1.4.
// ============================================================================
/**
* Expression evaluation error.
*/
export class ExpressionError extends Error {
constructor(
message: string,
public context: {
expression?: string;
workflowId?: string;
nodeId?: string;
[key: string]: unknown;
},
) {
super(message);
this.name = 'ExpressionError';
}
}
/**
* Specific error types.
*/
export class MemoryLimitError extends ExpressionError {}
export class TimeoutError extends ExpressionError {}
export class SecurityViolationError extends ExpressionError {}
export class SyntaxError extends ExpressionError {}
@@ -0,0 +1,33 @@
/**
* Expression Runtime Types
*
* This module exports all TypeScript interfaces and types for the expression runtime.
*/
// Bridge types
export type { RuntimeBridge, BridgeConfig } from './bridge';
export { DEFAULT_BRIDGE_CONFIG } from './bridge';
// Runtime types
export { RuntimeError } from './runtime';
// Evaluator types
export type {
EvaluatorConfig,
IExpressionEvaluator,
WorkflowData,
EvaluateOptions,
ObservabilityProvider,
MetricsAPI,
TracesAPI,
Span,
LogsAPI,
} from './evaluator';
export {
ExpressionError,
MemoryLimitError,
TimeoutError,
SecurityViolationError,
SyntaxError,
} from './evaluator';
@@ -0,0 +1,24 @@
/**
* Runtime error thrown inside isolated context.
*
* These errors are thrown by the runtime code when something goes wrong during
* expression evaluation. The bridge must catch these and translate them to the
* appropriate ExpressionError subclass (see evaluator.ts).
*
* Translation mapping:
* - code: 'MEMORY_LIMIT' → MemoryLimitError
* - code: 'TIMEOUT' → TimeoutError
* - code: 'SECURITY_VIOLATION' → SecurityViolationError
* - code: 'SYNTAX_ERROR' → SyntaxError
* - other → ExpressionError
*/
export class RuntimeError extends Error {
constructor(
message: string,
public code: string,
public details?: Record<string, unknown>,
) {
super(message);
this.name = 'RuntimeError';
}
}
@@ -0,0 +1,10 @@
{
"extends": ["./tsconfig.json"],
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"tsBuildInfoFile": "dist/build.tsbuildinfo"
},
"include": ["src/**/*.ts"],
"exclude": ["node_modules", "dist", "src/**/__tests__/**"]
}
@@ -0,0 +1,10 @@
{
"extends": "../typescript-config/modern/tsconfig.json",
"compilerOptions": {
"rootDir": ".",
"noUncheckedIndexedAccess": false,
"types": ["node", "vitest/globals"]
},
"include": ["src/**/*.ts"],
"exclude": ["node_modules", "dist"]
}
@@ -0,0 +1,13 @@
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
globals: true,
environment: 'node',
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html'],
exclude: ['dist/**', 'bundle/**', '**/*.test.ts', '**/*.config.ts'],
},
},
});