Video-to-Audio API: Choose the Right Output and Pipeline

The current combined-audio reference accepts multipart/form-data, exactly one of video or video_url, and independent optional music_prompt and sfx_prompt directions. Timed SFX segments can provide finer control. The request is asynchronous and returns HTTP 202 with task_id and a processing status.

curl -X POST https://api.sonilo.com/v1/video-to-sound   -H "Authorization: Bearer $SONILO_API_KEY"   -F "video_url=https://media.example/authorized-clip.mp4"   -F "music_prompt=restrained warm pulse, leave room for narration"   -F "sfx_prompt=natural product handling, soft clicks, no voice"
# Expected submission shape, not a completed asset:
{"task_id":"task_example","status":"processing"}

Do not send a JSON body, do not set a bare multipart boundary manually in most HTTP clients, and do not send both a file and URL. The application should validate the URL or upload before creating a billable upstream job. A public or signed URL must resolve to the media itself and must not become a server-side request-forgery path to private networks.

Prompt only where it adds product intent

The video is the timing and scene input. Prompts should constrain style, material, density, dialogue space or exclusions; they should not narrate every visible frame. Keep music and SFX directions separate. If a sequence needs timed SFX instructions, validate segment ordering, boundaries and duration against the actual probe result before submission. Reject impossible or overlapping product state locally instead of paying for an upstream 422.

Store the task ID immediately with a server-generated job ID, authenticated owner, endpoint, normalized source fingerprint and creation time. Poll GET https://api.sonilo.com/v1/tasks/{task_id} every few seconds. The documented states include processing, succeeded and failed. A successful combined task can expose output_url, output_type, output_bytes, content type, generated music and SFX stems, and an outputs array for multiple variants. Code to the endpoint-specific shape rather than assuming every task returns audio.url.

const taskUrl = "https://api.sonilo.com/v1/tasks/" + taskId;
for (let attempt = 0; attempt < 120; attempt += 1) {
  const response = await fetch(taskUrl, {
    headers: { Authorization: "Bearer " + process.env.SONILO_API_KEY },
    cache: "no-store",
  });
  if (response.status === 429) {
    await wait(readRetryAfter(response) ?? boundedBackoff(attempt));
    continue;
  }
  if (!response.ok) throw classifySoniloError(response);
  const task = await response.json();
  if (task.status === "failed") throw new Error(task.error?.message ?? "generation failed");
  if (task.status === "succeeded") return validateCombinedOutput(task);
  await wait(3000);
}
throw new Error("local polling deadline exceeded; task was not canceled upstream");

Before publishing a success event, require an allowed HTTPS download host/path, expected output type, plausible content type and positive byte count. Download to controlled temporary storage, enforce a maximum response size, compute a hash, verify that media probing succeeds and persist to your own storage if the product needs the file after the presigned URL expires. Never create a placeholder WAV or mark a URL string as the audio asset.

A signed URL is not automatically safe. Resolve and validate redirects; reject loopback, link-local, private and metadata-service addresses; limit schemes; and revalidate at fetch time to reduce DNS rebinding risk. Do not accept arbitrary request headers or upstream URLs from a client. If users can upload third-party, biometric, child, confidential or regulated footage, obtain specialized privacy and legal review before launch.

A timeout does not prove that submission failed. Create an application idempotency gate before calling Sonilo: hash a normalized operation definition such as owner, endpoint, source fingerprint, prompts, segments and output options; store a pending job transactionally; and return the existing job for duplicate requests within the chosen window. Do not blindly retry a POST after a dropped response because a second billable job may already be running.

Call account services to observe the current enabled services and limits. Queue locally when concurrency is full; apply per-user quotas and cost ceilings; record duration and variants because they affect consumption. A user cancel can stop your polling or local wait, but current documentation says there is no dedicated cancellation endpoint and the upstream task can continue and remain billable.

HTTP success only proves that an asset was returned. It does not prove that the audio fits the video. Build a representative evaluation set from footage you are authorized to use: dialogue product demo, fast cuts, low-motion interview, UI capture, physical action, long ambience and intentionally silent moments. Keep the set versioned and do not optimize exclusively to one attractive demo.

Use human reviewers with written rubrics, not only waveform or embedding metrics. Run the combined output in the actual player and editing workflow, then inspect separate music and SFX stems when available. Gate release on both technical validity and creative acceptance.

Before upload, establish who owns or controls the video, faces, voices, performances, logos, locations and embedded audio. At generation retain account/workspace, plan or commercial basis, endpoint, task ID, timestamp, source asset ID/hash, prompt and segments, output IDs/hashes, content type, reviewer, project/client, intended territories/channels, edits and release decision. At publication recheck current Sonilo licensing and platform/client requirements.

Do not claim “copyright-free,” exclusive ownership, legal clearance or safety solely because an API returned audio. Conduct similarity and brand review, preserve generation records and route high-risk paid media, client delivery, entertainment distribution, sublicensing or product embedding to qualified counsel. Privacy, input authority and output rights are separate questions.

1. Output contract names audio-only versus muxed video, combined mix versus stems and expected content types.
2. Runtime calls use api.sonilo.com/v1; docs URLs are never called as the API.
3. Key is server-side, rotated and absent from client bundles, logs, URLs and analytics.
4. Caller, workspace, media ownership, URL safety, bytes, duration and format are validated before submission.
5. Duplicate gate, queue, per-user quota, balance handling, timeouts, backoff and concurrency limits are tested.
6. Task state is stored; polling is bounded; 429 respects Retry-After; stopping locally is not reported as upstream cancellation.
7. Success requires real downloaded bytes, media probe, expected type/duration/channels, hash and durable storage policy.
8. Evaluation covers timing, silence, mix, dialogue, artifacts, stems, devices and representative edge cases.
9. Licensing, privacy, retention, deletion, similarity, human approval and incident procedures are documented.
10. Dashboard tracks submit success, terminal success, latency, retries, duplicates prevented, failures, cost, quality rejects and deletions without leaking media.