One queued task can call a summary model, make several embedding requests, and call a relation model. Starting one task per second does not establish one model request per second. This article follows ChatCrystal’s queue, retries, and state tracking, then verifies the cancellation boundary with an offline exercise.
The limit applies to queued task starts
The pinned queue/index.ts uses p-queue, resolved to 9.3.3 in the lockfile, with the following core configuration. This excerpt depends on existing imports and is not a standalone program.
new PQueue({
concurrency: 1,
intervalCap: 1,
interval: 1000,
carryoverIntervalCount: true,
});concurrency limits simultaneously running queued tasks; intervalCap and interval limit task starts within time windows. carryoverIntervalCount counts unfinished tasks from the previous window in the next one. It does not accumulate or discard unused quota. The current configuration does not enable the sliding-window strict mode either.
A task can make multiple requests internally, and both the AI SDK and application layer have retries. This configuration does not establish compliance with a provider’s global quota or constrain another process, an unqueued entry point, or another client using the same API key. List actual request paths before changing concurrency. A value of 1 also does not eliminate writes from other processes or storage races.
Outer retries wait inside the slot
enqueueWithRetry defaults maxRetries to 3. Its loop includes the initial attempt, so the task function can run up to four times. If the error message contains '429' and attempts remain, it waits 1000, 2000, then 4000 milliseconds before continuing. That wait occurs inside the task, which retains the execution slot.
This matches error text rather than a structured HTTP status. Unrelated error text may contain 429; a wrapped rate-limit error may omit it. The code reads no Retry-After header and adds no jitter. These are limitations of the current policy rather than evidence that text matching is the most reliable classification.
Four outer task attempts do not mean at most four HTTP requests. SDK calls inside a task can retry, and multiple embedding chunks produce multiple requests. Check retry behavior for format, business, and persistence failures at each entry point; see structured-prompt failure categories.
Clearing waiting work does not interrupt active work
Install p-queue@9.3.3 in a new temporary directory, save this as lesson.mjs, and run with Node 24+. It is a teaching instance that checks slots and clear, without the application’s global queue, database, or models. It does not test the actual one-second rate limit.
npm init -y
npm install --ignore-scripts --no-audit --no-fund p-queue@9.3.3
node lesson.mjsimport assert from 'node:assert/strict';
import { setTimeout as delay } from 'node:timers/promises';
import PQueue from 'p-queue';
const queue = new PQueue({ concurrency: 1, autoStart: false });
let release;
let entered;
let executions = 0;
const gate = new Promise((resolve) => {
release = resolve;
});
const started = new Promise((resolve) => {
entered = resolve;
});
const active = queue.add(async () => {
executions++;
entered();
await gate;
return 'finished';
});
const waiting = queue.add(async () => {
executions++;
return 'waiting';
});
queue.start();
await started;
assert.equal(queue.pending, 1);
assert.equal(queue.size, 1);
queue.clear();
assert.equal(queue.size, 0);
assert.equal(queue.pending, 1);
release();
assert.equal(await active, 'finished');
await queue.onIdle();
assert.equal(executions, 1);
const waitingState = await Promise.race([
waiting.then(
() => 'resolved',
() => 'rejected'
),
delay(20).then(() => 'unsettled'),
]);
assert.equal(waitingState, 'unsettled');
console.log('clear removes waiting work; active finishes; removed promise stays pending');Change the executions expectation from 1 to 2 and the program fails; restore it and it passes. pending counts running tasks and size counts waiting tasks. onIdle can finish after the queue is cleared and active work ends, while the removed task’s add Promise remains unsettled. The 20 milliseconds only observe this constructed instance; they are no performance measurement. The lasting lack of settlement follows this version’s clear implementation.
If cancellation must settle returned Promises, design a separate AbortSignal path and make running work respond to it. That is not a capability already implemented by cancelQueue. Awaiting a Promise removed by clear cannot establish that all cancellations finished.
State tracking and execution are separate records
tracker.ts records queued, processing, completed, and failed states, plus start and finish times. cancelQueue clears waiting work, then marks tracked queued entries failed with “已取消”; processing work continues.
The tracker’s Map ignores duplicate add calls by ID, but that does not automatically prevent p-queue from accepting duplicate tasks. Check active-ID guards at each entry point separately. Once no active tasks remain, the tracker schedules record clearing after five seconds. Its snapshot is therefore not a durable task log, and a restart cannot restore waiting work. Check execution and tracking consistency together.
SSE carries snapshots, not durable history
The /api/queue/stream handler in notes.ts sends a status event every second with totals, tasks, and duration for finished tasks. When total is positive and no queued or processing task remains, it sends done and closes the connection. Request disconnection also clears its timer.
There are no event IDs or persistent replay on this path. Disconnections, records cleared after five seconds, and service restarts need acceptance cases separate from normal completion. A completion snapshot helps the interface display state; it does not establish that every external request was recorded.
An in-memory queue reduces infrastructure for the current single-process scheduler while leaving explicit loss, cancellation, and shutdown boundaries. See the official p-queue repository for usage and APIs. This exercise establishes clear behavior only; rate limiting and real model pipelines still need separate verification.