Product
Video-to-Music API: Generate Soundtracks from Video
- Written by
- Sonilo Team
- Published
The video-to-music API analyzes a video's pacing, motion, and scene changes and returns a soundtrack that matches its exact duration, no prompt or manual scoring required.
Quick answer
POST /v1/video-to-music takes a video (uploaded file or public/signed URL) and returns a music track that matches the video's exact duration. Sonilo analyzes pacing, motion, and scene changes directly from the footage, so a text prompt is optional, not required. Call it in stream mode for a live NDJSON event stream as audio is generated, or in async mode to get a task_id back immediately and poll for the result. Output is licensed via Shutterstock, so it's cleared for commercial use out of the box.
Request parameters
The request body is multipart/form-data, not JSON, so every field below is sent as a form field.
| Field | Type | Notes |
|---|---|---|
| video | file | The source video file. Provide this or video_url, not both. |
| video_url | string (URL) | Public or signed HTTPS URL to the source video. Provide this or video, not both. |
| prompt | string (optional) | Creative direction, e.g. genre, mood, or instrumentation. Not required; Sonilo scores the video's pacing and motion automatically. |
| segments | string, JSON-encoded (optional) | Per-segment prompts for finer control over how different parts of the video are scored. |
| mode | string: "stream" | "async" | Defaults to "stream". stream returns a live NDJSON event stream; async returns a task_id to poll. |
| preserve_speech | boolean (optional) | Keeps existing dialogue/speech audible under the generated music. Requires mode=async. |
| isolate_vocals | boolean (optional) | Separates vocal stems from the source audio before scoring. Requires mode=async. |
| ducking | boolean (optional) | Automatically lowers music volume under dialogue or key sound moments. Requires mode=async. |
| output_format | string: "m4a" | "wav" | Defaults to "m4a". "wav" requires mode=async. |
Example request
curl -X POST https://api.sonilo.com/v1/video-to-music -H "Authorization: Bearer $SONILO_API_KEY" -F video_url=https://example.com/clip.mp4 -F prompt="Upbeat cinematic score with driving percussion" -F mode=async -F output_format=wav -F ducking=true
Because this request uses mode=async, Sonilo responds immediately with a 202 and a body like {"task_id": "...", "status": "processing"}. Poll GET /v1/tasks/{task_id} until status is "succeeded", at which point the response includes a music object with url, content_type, and file_size, or "failed", at which point the response includes an error object.
FAQ
Do I need to write a prompt?
No. The model analyzes the video's pacing, motion, and scene changes directly, so an empty prompt still produces a soundtrack matched to the footage. Use the optional prompt field when you want to steer genre, mood, or instrumentation, or use segments to give different parts of the video different creative direction.
What's the difference between stream and async mode?
mode=stream is the default and returns a 200 NDJSON event stream over a held-open connection, so audio arrives as it's generated. mode=async returns a 202 with a task_id right away and requires polling GET /v1/tasks/{task_id} for the result. preserve_speech, isolate_vocals, ducking, and wav output are only available in async mode.
What audio formats are supported?
output_format defaults to m4a. Set output_format=wav if you need uncompressed audio for further mixing, but note wav requires mode=async.
Is the output licensed for commercial use?
Yes. Every track generated through the video-to-music API is fully licensed via Shutterstock, so the output is commercial-safe for ads, product demos, films, and other client-facing work.
How does Sonilo's music quality compare to other AI music generators?
In an independent benchmark run with the open-source audio-eval toolkit (https://github.com/shwj114114/audio_eval), Sonilo Music 1.1 beat Suno v5.5 on 10 of 13 MusicCaps metrics.
Related Sonilo pages
- Full API reference: https://platform.sonilo.com/docs/api/video-to-music
- Task polling reference: https://platform.sonilo.com/docs/api/get-task
- Pricing: https://sonilo.com/pricing
