Worker Architecture
The Worker is a standalone, headless NestJS application that runs background tasks, scheduled jobs, and queue-based processing independently of the main API server. It shares the same database, plugin ecosystem, and core packages as the API but is designed to run as a separate process β with no HTTP server and no inbound API traffic.
Why a Separate Worker?β
The Ever Gauzy platform follows a process separation pattern for background work:
| Concern | API Server | Worker |
|---|---|---|
| Purpose | Serve HTTP requests, REST/GraphQL APIs | Execute scheduled jobs, process queues |
| Inbound | HTTP traffic from clients | None β headless, no HTTP listener |
| Scaling | Horizontal (multiple replicas) | Single instance per environment |
| Lifecycle | Request-driven | Timer-driven (cron/interval) + queue-driven |
Running background work outside the API process provides several advantages:
- Isolation β long-running or CPU-intensive jobs do not block API request handling
- Independent scaling β the worker runs as a single replica while the API scales horizontally
- Fault tolerance β a crashing worker does not take down the API, and vice-versa
- Queue reliability β BullMQ + Redis provide at-least-once delivery guarantees with automatic retries
High-Level Architectureβ
The Worker connects to the same PostgreSQL database as the API, and both processes share a Redis instance that serves as the transport layer for BullMQ job queues.
Application Bootstrapβ
The worker boots through apps/worker/src/main.ts and is fundamentally different from the API:
// apps/worker/src/main.ts
import { NestFactory } from "@nestjs/core";
async function bootstrap() {
loadEnv();
const { registerPluginConfig } = await import("@gauzy/core");
await registerPluginConfig({});
const { AppModule } = await import("./app/app.module");
// NOTE: createApplicationContext, NOT create()
// This boots NestJS WITHOUT an HTTP server
const app = await NestFactory.createApplicationContext(AppModule, {
logger: ["log", "error", "warn", "debug", "verbose"],
});
Logger.log("Worker application started.", "WorkerBootstrap");
}
Key differences from the API bootstrap:
| Aspect | API Server | Worker |
|---|---|---|
| NestJS factory method | NestFactory.create() | NestFactory.createApplicationContext() |
| HTTP listener | Yes (Express, port 3000) | No β headless process |
| Routes/Controllers | Full REST + GraphQL endpoints | None β no HTTP routing |
| Swagger documentation | Available at /swg | N/A |
| Signal handling | Framework-managed | Manual SIGINT/SIGTERM handlers |
The worker uses createApplicationContext() which instantiates the full NestJS dependency injection container, executes all lifecycle hooks (OnModuleInit, OnApplicationBootstrap), and starts all scheduled jobs and queue workers β but without binding to any network port.
Environment Loadingβ
Environment variables are loaded from .env and .env.local files using dotenv before the NestJS modules are initialized:
// apps/worker/src/load-env.ts
export function loadEnv(): void {
const cwd = process.cwd();
loadEnvFile(path.resolve(cwd, ".env"));
loadEnvFile(path.resolve(cwd, ".env.local"), { override: true });
}
Variables from .env.local take precedence over .env, which is the standard dotenv convention.
Module Structureβ
The worker's AppModule is intentionally minimal, importing only what is needed for background task execution:
// apps/worker/src/app/app.module.ts
@Module({
imports: [
DatabaseModule, // Shared database connection (same DB as API)
TokenModule.forRoot({
enableScheduler: WORKER_SCHEDULER_ENABLED,
}),
SchedulerModule.forRoot({
// Core scheduling + queueing infrastructure
enabled: WORKER_SCHEDULER_ENABLED,
enableQueueing: WORKER_QUEUE_ENABLED,
defaultQueueName: WORKER_DEFAULT_QUEUE,
defaultTimezone: process.env.WORKER_TIMEZONE,
defaultJobOptions: {
preventOverlap: true, // No concurrent runs of the same job
retries: 1, // Retry once on failure
retryDelayMs: 5000, // Wait 5 seconds before retrying
},
}),
WorkerJobsModule, // Feature module containing job definitions
],
})
export class AppModule {}
This means the worker has:
- β Full database access (reads and writes to the same PostgreSQL/SQLite DB as the API)
- β
All registered plugins (inherited from
registerPluginConfig()) - β Token management and refresh scheduling
- β BullMQ queue processing (when Redis is available)
- β Cron and interval-based job scheduling
- β No HTTP server, no REST/GraphQL routes, no Swagger
The Scheduler Package (@gauzy/scheduler)β
The @gauzy/scheduler package is a custom NestJS module that provides a unified API for:
- Cron-based scheduling (via
@nestjs/scheduleand thecronlibrary) - Interval-based scheduling (via
setInterval) - BullMQ job queues (via
@nestjs/bullmqandbullmq) - Automatic job discovery (via NestJS
DiscoveryServiceand custom decorators)
Package Exportsβ
The @gauzy/scheduler package exports the following public API:
// Module
export { SchedulerModule } from "./scheduler.module";
// Decorators
export { ScheduledJob } from "./decorators/scheduled-job.decorator";
export { QueueWorker } from "./decorators/queue-worker.decorator";
export { QueueJobHandler } from "./decorators/queue-job-handler.decorator";
// Interfaces
export { ScheduledJobOptions } from "./interfaces/scheduled-job-options.interface";
export { SchedulerModuleOptions } from "./interfaces/scheduler-module-options.interface";
export { SchedulerFeatureOptions } from "./interfaces/scheduler-feature-options.interface";
export { DiscoveredScheduledJob } from "./interfaces/discovered-scheduled-job.interface";
export { SchedulerJobDescriptor } from "./interfaces/scheduler-job-descriptor.interface";
export { SchedulerQueueJobInput } from "./interfaces/scheduler-queue-job.interface";
// Base classes
export { QueueWorkerHost } from "./hosts/queue-worker.host";
// Services
export { SchedulerService } from "./services/scheduler.service";
export { SchedulerQueueService } from "./services/scheduler-queue.service";
Module Initializationβ
The SchedulerModule follows NestJS conventions with forRoot() and forFeature() patterns:
SchedulerModule.forRoot(options)β
Called once in the root AppModule to set up the global scheduling infrastructure:
SchedulerModule.forRoot({
enabled: true, // Master switch for the entire scheduler
enableQueueing: true, // Enable BullMQ integration
defaultQueueName: "worker-default", // Default queue for jobs without explicit queueName
defaultTimezone: "UTC", // Timezone for cron expressions
logRegisteredJobs: true, // Log each discovered job on startup
defaultJobOptions: {
enabled: true, // Jobs are enabled by default
preventOverlap: true, // Skip if previous run still in progress
retries: 1, // Number of retries on failure
retryDelayMs: 5000, // Delay between retries
timeoutMs: undefined, // No timeout by default
maxRandomDelayMs: 0, // No jitter by default
},
});
When enableQueueing is true, this registers:
ScheduleModule.forRoot()from@nestjs/schedule(cron engine)BullModule.forRoot()from@nestjs/bullmq(Redis connection for queues)- All internal scheduler services (discovery, registry, runner, queue service)
SchedulerModule.forFeature(options)β
Called in feature modules to register job providers and additional queues:
@Module({
imports: [
SchedulerModule.forFeature({
jobProviders: [WorkerLifecycleJob, WorkerLifecycleProcessor],
queues: ["worker-default"],
}),
],
})
export class WorkerJobsModule {}
Internal Service Architectureβ
The scheduler package has a layered internal architecture:
Job Systemβ
Decoratorsβ
The scheduler exposes three custom decorators for defining jobs and processors:
@ScheduledJob(options) β Define a Scheduled Jobβ
Applied to a method to mark it as a scheduled job. The discovery service automatically finds and registers methods decorated with @ScheduledJob:
@Injectable()
export class MyJobProvider {
@ScheduledJob({
name: 'my-custom-job', // Unique job identifier (optional)
description: 'Does something', // Human-readable description (optional)
enabled: true, // Can disable individual jobs
cron: CronExpression.EVERY_HOUR, // Cron expression (mutually exclusive with intervalMs)
// intervalMs: 60000, // OR interval in milliseconds
runOnStart: false, // Execute immediately on application boot
preventOverlap: true, // Skip if previous run is still active
retries: 2, // Retry count on failure
retryDelayMs: 3000, // Delay between retries (ms)
timeoutMs: 30000, // Kill the job after 30 seconds
maxRandomDelayMs: 5000, // Add random jitter up to 5 seconds
queueName: 'worker-default', // Target BullMQ queue (optional)
queueJobName: 'my-job', // Job name used in the queue
queueJobOptions: { ... } // BullMQ-specific options (priority, delay, etc.)
})
async doWork(): Promise<any> {
// Return value is used as the queue job payload if queueName is set
return { result: 'done' };
}
}
Schedule types:
| Type | Configuration | Behavior |
|---|---|---|
| Cron | cron: '*/5 * * * *' | Runs on a cron schedule (timezone-aware) |
| Interval | intervalMs: 30000 | Runs every N milliseconds |
| Manual | Neither cron nor intervalMs | Only runs via runOnStart or triggerNow() |
Execution target:
| Configuration | Behavior |
|---|---|
No queueName | Inline β method executes directly |
queueName set + enableQueueing: true | Queued β method return value is enqueued |
When a job targets a queue:
- The cron/interval trigger fires the method
- The method's return value becomes the job payload
- The payload is added to the BullMQ queue
- A separate
@QueueWorkerprocessor picks it up and processes it
@QueueWorker(queueName) β Define a Queue Processorβ
Applied to a class to bind it to a specific BullMQ queue. This is an alias for @nestjs/bullmq's @Processor decorator:
@Injectable()
@QueueWorker("worker-default")
export class MyProcessor extends QueueWorkerHost {
// This class processes jobs from the 'worker-default' queue
}