How to Queue AI Music API Jobs Without Hitting Rate Limits

Put a local admission gate in front of every generation request. Read the account's current requests-per-minute and concurrency limits, admit work only when both budgets have room, and keep each concurrency lease until the stream ends or the async task reaches a terminal state.

Put a local admission gate in front of every generation request. Read the account's current requests-per-minute and concurrency limits, admit work only when both budgets have room, and keep each concurrency lease until the stream ends or the async task reaches a terminal state. Reschedule a request after a definite `429`; treat a transport timeout as an ambiguous submission instead of blindly sending another POST.

By Sonilo Team · Facts verified August 17, 2026

Disclosure: Sonilo publishes this guide and operates the API used as the concrete example. The `generation-admission-gate-v1` implementation was tested with a deterministic clock and synthetic fixtures. It did not send production generation requests, measure provider latency, or prove multi-worker coordination.

Control two budgets before you submit

Generation APIs commonly enforce both a request-rate budget and a work-in-progress budget. They solve different overload problems, so one counter cannot safely represent both.

The current Sonilo API reference says generation limits are shared across the account. The List Services endpoint returns the account's live `rpm_limit` and `concurrency_limit` values. Read those values at worker startup and refresh them on a schedule; do not copy an example limit into application code.

A rolling local window is intentionally conservative. Sonilo's current full agent reference documents a fixed 60-second provider window and says rejected requests count against it. Limiting starts in every rolling 60-second period may wait longer near a fixed-window boundary, but it avoids depending on an unknown server-window origin.

Concurrency is a lifecycle measure, not just an HTTP-response measure. A streaming request keeps its local slot until the terminal `complete` or `error` event. An async request keeps its slot after the `202` response and releases it only when the related task reaches `succeeded` or `failed`.

Fetch limits from the account endpoint

Keep the provider key on the server. The broader Next.js server-boundary guide explains that trust boundary. A worker can load the limits like this:

1. `async function getGenerationLimits(apiKey) {`
2. ` const response = await fetch(`
3. ` "https://api.sonilo.com/v1/account/services",`
4. ` { headers: { Authorization: "Bearer " + apiKey } },`
5. ` );`
6. ` if (!response.ok) {`
7. ` throw new Error("limits_read_failed:" + response.status);`
8. ` }`
9. ` const data = await response.json();`
10. ` if (!Number.isSafeInteger(data.rpm_limit) || data.rpm_limit < 1) {`
11. ` throw new Error("invalid_rpm_limit");`
12. ` }`
13. ` if (!Number.isSafeInteger(data.concurrency_limit) || data.concurrency_limit < 1) {`
14. ` throw new Error("invalid_concurrency_limit");`
15. ` }`
16. ` return {`
17. ` rpmLimit: data.rpm_limit,`
18. ` concurrencyLimit: data.concurrency_limit,`
19. ` };`
20. `}`

Fail closed when the limit read is missing or malformed. Starting an unbounded worker because configuration failed turns a control-plane error into a burst of rejected generation requests.

Refresh limits outside the hot request path. If a refresh fails, retain the last verified values for a bounded period or pause new admissions according to your operating policy. Never log the bearer value; the safe API logging guide shows a fixed allowlist for job, limit, and status fields.

Use one admission decision for both limits

The gate should return an explicit decision before the provider call:

Persist the application job and request key before admission. The gate controls capacity; it does not make a generation POST idempotent. Use the separate duplicate-job prevention contract so a queue redelivery cannot create a second generation for the same intent.

A tested Node.js admission gate

`generation-admission-gate-v1` uses only JavaScript built-ins. It tracks admitted starts in a rolling window and active lifecycle leases in a `Set`.

Save the core policy as `generation-admission-gate.mjs`:

1. `export function createGenerationAdmissionGate({rpmLimit, concurrencyLimit, windowMs = 60_000}) {`
2. ` for (const [name, value] of Object.entries({rpmLimit, concurrencyLimit, windowMs})) {`
3. ` if (!Number.isSafeInteger(value) || value < 1) {`
4. ` throw new TypeError(name + " must be a positive integer");`
5. ` }`
6. ` }`
7. ` const starts = [];`
8. ` const leases = new Set();`
9. ` let sequence = 0;`
10. ` const prune = (nowMs) => {`
11. ` while (starts.length && nowMs - starts[0] >= windowMs) starts.shift();`
12. ` };`
13. ` return {`
14. ` admit(nowMs = Date.now()) {`
15. ` prune(nowMs);`
16. ` if (leases.size >= concurrencyLimit) {`
17. ` return {ok: false, reason: "concurrency", retryAtMs: null};`
18. ` }`
19. ` if (starts.length >= rpmLimit) {`
20. ` return {ok: false, reason: "rpm", retryAtMs: starts[0] + windowMs};`
21. ` }`
22. ` const leaseId = "generation-" + ++sequence;`
23. ` starts.push(nowMs);`
24. ` leases.add(leaseId);`
25. ` return {ok: true, leaseId};`
26. ` },`
27. ` finish(leaseId) {`
28. ` return leases.delete(leaseId);`
29. ` },`
30. ` };`
31. `}`

The complete tested module also validates timestamps, exposes a safe numeric snapshot, parses both standard `Retry-After` forms, and classifies the two current Sonilo 429 messages.

This in-memory gate is suitable for one process. Multiple workers need one shared authority for starts and active leases. Use an atomic database or queue operation, attach a lease expiry for crashed workers, and test recovery before treating a distributed counter as a limit guarantee.

Hold async slots until terminal task state

Do not call `finish(leaseId)` when an async submission returns `202`. That response means the work was accepted, not completed. Store the provider task ID next to the lease and release the slot only after the current Retrieve Task contract reports `succeeded` or `failed`.

The bounded polling workflow covers status cadence, `Retry-After`, and reconciliation in detail. Keep the two schedulers separate:

• The submission gate decides whether a new generation may start.
• The polling scheduler decides when an accepted task's status may be read again.
• The lifecycle record joins the application job, provider task, and admission lease.

If the worker stops polling, the provider task can still run. Recover active leases from durable job state after a restart; do not assume that a missing process means the provider freed the slot. For planned deploys, use the graceful shutdown handoff to close intake and transfer those durable leases and task IDs deliberately.

Handle 429 by cause

RFC 6585 defines `429 Too Many Requests` as a rate-limiting response and allows it to include `Retry-After`. RFC 9110 defines `Retry-After` as either delay-seconds or an HTTP date. Parse both forms and treat the value as a minimum wait.

The current Sonilo error guide and full agent reference document two 429 causes. A requests-per-minute rejection clears after the minute window. A concurrency rejection clears when a running generation finishes. The response message distinguishes the two current conditions.

Only reschedule automatically after receiving a definite provider `429` response. A connection reset, client timeout, or lost response does not prove the POST was rejected. Freeze that request for reconciliation under the application's idempotency contract. This boundary matters because a retry that solves rate limiting can still create duplicate billable work when the first outcome is unknown. Use the separate circuit breaker for AI music submissions to pause unrelated new work after repeated dependency failures, not after a definite capacity response.

Use bounded retries. A queue that retries forever can keep stale customer work alive, hide a persistent account configuration problem, and starve newer jobs. Record attempt count, next eligible time, last definite response, and a dead-letter reason without storing credentials or customer media URLs.

The ten-test contract

The gate ran under Node.js v22.23.2's test runner with no external packages. Ten deterministic tests passed on August 17, 2026:

1. Admit work only up to the configured concurrency limit.
2. Release a slot only for a known lease.
3. Count finished work against the rolling RPM window.
4. Reopen the RPM gate exactly at the window boundary.
5. Report a numeric snapshot without job payloads or credentials.
6. Parse `Retry-After` delay-seconds.
7. Parse an HTTP-date and clamp past dates to zero.
8. Return `null` for an invalid `Retry-After` value.
9. Classify the two documented Sonilo 429 message shapes.
10. Reject invalid limits before admitting work.

The fixtures prove local state transitions, not live service behavior or production throughput. The test did not generate media, verify billing, induce real 429 responses, or coordinate multiple processes. Re-run it after changing the queue, clock, storage, or provider contract. Use Node.js timers only to wake a worker; always re-check durable eligibility after wake-up rather than trusting an old in-memory delay.

When this gate is not enough

Do not use this module as a distributed semaphore without shared atomic storage. Do not infer completed work from an expired local lease. Do not release a provider slot because your HTTP client disconnected. Do not retry an ambiguous POST just because capacity is now available.

Admission control also does not validate the video input. Run the video preflight workflow before a job enters the expensive queue, and mint remote source access late enough for the presigned-URL lifetime gate. After success, move result media through the temporary-output durability workflow.

If the account regularly runs at its limit, measure queue age, admission wait, generation duration, 429 reason, and terminal outcome. Those metrics can justify a capacity change, but they do not prove the provider's quality or reliability. Keep service-level claims tied to observed production data and a defined measurement period.

Ship the queue contract before the burst

Read live limits, persist the application request, admit once, and hold the lease through the real generation lifecycle. On a definite 429, wait for the provider's signal and re-enter the queue. On an ambiguous outcome, stop automatic replay.

Start with the broader Sonilo Video-to-Music API guide, then implement against the current OpenAPI document. The practical rule is simple: queue capacity before the POST, and release it after the work—not after acceptance.