Product
Video-to-Sound-Effects API: Frame-Accurate SFX from Video
- Written by
- Sonilo Team
- Published
video-to-sfx generates realistic, frame-accurate sound effects and Foley synced to what happens on screen — not background music. Here's the full request contract, an example call, and how async processing works.
Quick answer
POST /v1/video-to-sfx takes a video and generates realistic, frame-accurate sound effects and Foley — cuts, motion, object hits, UI interactions, product reveals, game moments, transitions — synced to what's happening on screen. It is not a background-music endpoint; if you need a score, use the separate video-to-music endpoint. Every video-to-sfx request is processed asynchronously: there is no sync or streaming mode. Output is fully licensed via Shutterstock for commercial use. In independent benchmarking with the open-source audio-eval toolkit (https://github.com/shwj114114/audio_eval), Sonilo Sound Effects 1.0 beat ElevenLabs SFX v2 on 16 of 24 text-to-sound-effects metrics.
| Field | Type | Notes |
|---|---|---|
| video | file upload | Provide this or video_url, not both. |
| video_url | string (URL) | Public or signed HTTPS URL; provide this or video, not both. |
| prompt | string, optional | Creative direction for the sound design. Max 2000 characters. |
| segments | string (JSON-encoded), optional | JSON-encoded string specifying particular segments or time ranges to target. |
| audio_format | string, optional | One of "wav", "mp3", "aac", "flac". No default — omit to use the service default. |
Example request
The request body is multipart/form-data, not JSON. Supply either video (a file upload) or video_url — never both — plus any optional fields.
- curl -X POST https://api.sonilo.com/v1/video-to-sfx -H "Authorization: Bearer $SONILO_API_KEY" -F video_url=https://example.com/gameplay-clip.mp4 -F prompt="Punchy UI clicks, footstep foley, and a whoosh on the camera pan" -F audio_format=wav
This always returns 202 immediately with a task_id and status: "processing" — video-to-sfx has no sync or streaming mode, unlike video-to-music. Poll GET /v1/tasks/{task_id} until status is "succeeded" (the result includes an sfx object with url, content_type, and file_size) or "failed".
- curl https://api.sonilo.com/v1/tasks/sfx_8f3a1c -H "Authorization: Bearer $SONILO_API_KEY"
Official SDKs handle the multipart upload and polling loop for you: pip install sonilo (Python) or npm install sonilo (JavaScript/TypeScript).
FAQ
Is video-to-sfx synchronous or asynchronous?
Always asynchronous. Unlike video-to-music, this endpoint has no sync or streaming mode — every call returns 202 with a task_id right away, and you poll GET /v1/tasks/{task_id} until the job succeeds or fails.
What audio formats are supported?
wav, mp3, aac, and flac, set via the optional audio_format field. There is no default format, so omit the field to get the service default output.
Does this work on gameplay footage, or only cinematic video?
Both. video-to-sfx analyzes cuts, motion, objects, and on-screen actions generally, so it works on gameplay recordings, UI screen captures, product demos, vlogs, and cinematic footage alike — anywhere there are visual moments to score.
How is video-to-sfx different from video-to-music?
video-to-sfx generates isolated sound effects and Foley tied to specific visual events — a footstep, a UI tap, a transition whoosh. video-to-music generates a background score or soundtrack for the whole clip. Use both endpoints together for a full audio mix, or either alone depending on what you need.
Is the output cleared for commercial use?
Yes. All generated sound effects are fully licensed via Shutterstock, so output is commercial-safe out of the box.
Related Sonilo pages
- Full API reference: https://platform.sonilo.com/docs/api/video-to-sfx
- Pricing: https://sonilo.com/pricing
