How to Analyze a Video for Music Direction with an API

Analyze the footage first, validate the returned timeline, and only then decide whether to generate music. Sonilo’s POST /v1/video-analysis endpoint returns an asynchronous task for a music-direction brief: time-aligned sections plus one or more music-generation prompts. It does not generate audio or video, so treat its result as structured creative input, not a finished soundtrack.

Analyze the footage first, validate the returned timeline, and only then decide whether to generate music. Sonilo’s `POST /v1/video-analysis` endpoint returns an asynchronous task for a music-direction brief: time-aligned sections plus one or more music-generation prompts. It does not generate audio or video, so treat its result as structured creative input, not a finished soundtrack.

By Sonilo Team · Facts verified August 22, 2026

Disclosure: Sonilo publishes this guide and operates the API used as the concrete example. The `video-music-brief-v1` client and validator were tested with deterministic fixtures. The tests did not upload customer footage, call a production account, evaluate the creative quality of a brief, generate music, or measure provider latency.

Use analysis when the next decision is still open

Video analysis and music generation answer different questions. Analysis describes a possible music plan for the footage. Generation produces media.

The current Sonilo Video Analysis reference defines the analysis output as a time-aligned section plan and one or more music-generation prompts derived from the footage. The current OpenAPI contract says the endpoint does not generate audio or video and is intended to prepare a brief for music endpoints.

This separation is useful when an editor, customer, or automated policy needs to approve direction before a billable media workflow continues. It also gives an application a structured place to reject malformed timeline data instead of passing it deeper into generation.

For the broader product flow, start with the Video-to-Music API guide. This page owns the narrower pre-generation task: submit one video for analysis, recover its task ID, poll safely, and validate the returned brief.

Send exactly one video source

`POST https://api.sonilo.com/v1/video-analysis` accepts `multipart/form-data`. Provide either a `video` upload or a `video_url`, not both. The current endpoint reference lists a default 300 MB upload limit, a 600-second video limit, an optional prompt of up to 2,000 characters, and `variants_num` from 1 through 5.

Account and endpoint rules can change, so recheck the live reference before deploying. Validate local uploads through the video-file preflight workflow before the POST. For a remote object, apply the presigned source-URL lifetime gate and remember that checking an HTTPS URL locally is not a complete server-side request-forgery defense.

The tested sample uses a remote URL because it keeps the runnable example dependency-free. Its input guard requires an absolute HTTPS URL without embedded username or password data. It cannot prove that the remote host is public, stable, safe, or still reachable when the provider fetches it.

Keep the bearer key on the server. The Sonilo API introduction requires `Authorization: Bearer <SONILO_API_KEY>`, while the Next.js server-boundary guide shows how to keep a provider credential out of browser JavaScript.

Submit once, then preserve the task ID

The analysis endpoint returns `202 Accepted` with `task_id` and `status: processing`. Save that identifier before a worker acknowledges its queue item or a web route tells the browser that the request is recoverable.

The submission part of `video-music-brief-v1` is intentionally one attempt:

1. `const form = new FormData();`
2. `form.set('video_url', input.videoUrl);`
3. `if (input.prompt) form.set('prompt', input.prompt);`
4. `form.set('variants_num', String(input.variants));`
5. ``
6. `const response = await fetch(`
7. ` 'https://api.sonilo.com/v1/video-analysis',`
8. ` {`
9. ` method: 'POST',`
10. ` headers: {Authorization: \`Bearer ${apiKey}\`},`
11. ` body: form,`
12. ` signal: AbortSignal.timeout(30_000),`
13. ` },`
14. `);`

Node.js v22 provides stable global `fetch` and `FormData` implementations, and its global API documentation defines `AbortSignal.timeout()` as a signal that aborts after the configured delay. That local timeout bounds the wait for a response. It does not prove whether a remote POST was accepted.

RFC 9110 says clients should not automatically retry a non-idempotent request unless they know the semantics are idempotent or know the original request was not applied. The sample converts a transport failure around the analysis POST into `submission_outcome_unknown` and makes no automatic replay.

Use the separate duplicate-job prevention contract if users, queues, or application retries can repeat one intent. An application request key can stop your own service sending obvious duplicates, but it cannot manufacture a provider-side recovery identifier when the submission response is lost.

Poll the canonical task path

After a usable `202`, poll `GET https://api.sonilo.com/v1/tasks/{task_id}`. The current Retrieve Task reference identifies `segments` and `variations` as video-analysis result fields and tells clients to stop on a terminal task state.

Use the exact task path. Do not guess `/v1/task_id`, omit `/v1`, or add a `/status` suffix. The full bounded polling workflow covers durable leases, retry windows, and recovery after worker restarts.

The sample waits three seconds between ordinary status reads. It retries only a failed task GET, not the analysis POST. A `429` read honors `Retry-After`; transient `502`, `503`, and `504` reads use a capped delay; and the function stops after three read attempts.

The current Sonilo API reference says the service does not send webhooks or callbacks. Polling is therefore the documented completion channel for this endpoint today. Treat that as a dated interface fact, not a promise that the API will never add event delivery.

If many jobs share the account, put status reads behind the generation admission and rate-limit workflow. A new submission consumes different risk and capacity than a status read, but both still belong in an account-wide request budget.

Validate the brief before using it

A `succeeded` status is necessary but not sufficient for an application-owned contract. The current OpenAPI schema defines each analysis segment with numeric `start` and `end` values in seconds plus `label` and `prompt` strings. Each variation contains a full music-generation prompt.

`video-music-brief-v1` applies six local checks:

1. The returned task ID matches the accepted task.
2. The task status is `succeeded`.
3. At least one segment exists.
4. Every segment has finite bounds with `start >= 0` and `end > start`.
5. Segments do not overlap when read in returned order.
6. The variation count matches the number requested, and all labels and prompts are non-empty.

The no-overlap rule is an application policy added by this article, not a statement that the provider guarantees gap-free editorial structure. The validator records gaps instead of rejecting them because a pause or intentionally unscored interval can be legitimate.

The normalized result looks like this:

1. `{`
2. ` "version": "video-music-brief-v1",`
3. ` "taskId": "task-1",`
4. ` "segments": [`
5. ` {"start": 0, "end": 8, "label": "Opening", "prompt": "restrained pulse"},`
6. ` {"start": 8, "end": 20, "label": "Build", "prompt": "add momentum"}`
7. ` ],`
8. ` "variations": [`
9. ` {"prompt": "minimal electronic underscore"}`
10. ` ],`
11. ` "coverage": {"firstStart": 0, "finalEnd": 20, "gaps": []}`
12. `}`

Do not silently sort overlapping or reversed segments. A repair can change creative meaning. Reject the brief, preserve the provider payload in an access-controlled diagnostic record, and investigate the contract or source.

Review direction before generation

Validation proves structural usability, not creative quality. Show an editor or policy layer the timeline, section labels, section prompts, and requested variations before the next paid step.

A practical review records:

• the application request ID and provider task ID;
• the source asset identity, not an expiring signed query string;
• analysis prompt and variation count;
• returned section boundaries and prompts;
• selected variation or rejection reason;
• reviewer, review time, and the later generation request ID.

The analysis prompt should guide without pretending to guarantee a particular musical result. Keep requests specific enough to be reviewable, such as “restrained opening, then more motion after the product reveal,” while letting the footage remain the source of timeline evidence.

If a customer approves one direction, carry that decision into the appropriate current music endpoint and keep the accepted analysis task beside the later generation record. The video-to-music overview explains the creator workflow, while the temporary-output durability guide owns the later task of copying and verifying generated media.

Before opening that later request, apply the pre-submit cost estimator. The analysis reference says `variants_num` scales linearly for analysis; do not assume one approved brief has already paid for future audio generation.

Run the complete Node.js example

Save the tested module as `video-music-brief.mjs` in a trusted server environment running Node.js v22. Then:

1. Set `SONILO_API_KEY` and `SONILO_VIDEO_URL` in the server process environment.
2. Run `node video-music-brief.mjs`.

Optional inputs are `SONILO_ANALYSIS_PROMPT` and `SONILO_VARIANTS`. The script prints only the normalized brief. It does not print the key or the original response headers.

The runnable module includes request normalization, one-shot submission, bounded task reads, `Retry-After` parsing, a wall-clock deadline, task failure handling, and brief validation. Keep it server-side and adapt its storage and authorization boundary to your application.

Do not put raw source URLs, prompts, or provider responses into general logs. Use the safe AI API logging contract for allowlisted correlation data, and use the graceful worker shutdown workflow when accepted task IDs must survive a deploy.

The sixteen-test contract

The module ran with the stable Node.js v22 test runner and no external packages. Sixteen deterministic tests passed on Node.js v22.23.2 on August 22, 2026:

1. Normalize one HTTPS source and valid request limits.
2. Reject non-HTTPS and credential-bearing source URLs.
3. Reject prompts and variation counts outside the documented contract.
4. Submit once with bearer authentication and the expected multipart fields.
5. Reject a missing server key before fetch.
6. Avoid retrying an ambiguous submission failure.
7. Poll `processing` to `succeeded` and validate the brief.
8. Retry a `429` task GET after `Retry-After`.
9. Avoid retrying a `401` task read.
10. Reject overlapping timeline segments.
11. Reject invalid segment bounds.
12. Reject empty segment labels or prompts.
13. Reject a variation-count mismatch.
14. Preserve the task ID when the local deadline passes.
15. Surface a provider task failure with its task ID.
16. Parse numeric and HTTP-date `Retry-After` values.

These tests prove the local request guard, secret placement, single-submit boundary, GET retry behavior, timeline checks, variation checks, and recovery data for controlled fixtures. They do not prove production availability, billing, latency, task duration, creative quality, source safety, tenant authorization, or that a generated soundtrack will satisfy a customer.

When not to use video analysis

Do not add an analysis step when the user already supplied a final, approved music direction and the additional cost, wait, and review state have no product value.

Do not treat analysis as media generation. It returns structured direction, not playable audio or a finished video.

Do not use the returned prompts as unquestionable creative truth. They are candidate directions derived by a model and still need human or application review.

Do not retry a timed-out POST just because no task ID reached the client. Route the uncertain request through the same operations path used for other ambiguous paid actions.

Do not accept overlapping or malformed timeline data merely to keep a workflow moving. A structurally invalid brief should stop before it can produce a misleading generation request.

Do not expose the provider key or unreviewed source URLs to the browser. Build the analysis call behind a trusted server boundary.

Ship the brief as a reviewable contract

The smallest reliable workflow is: validate one source, submit the analysis once, save the returned task ID, poll the documented path, validate sections and variation prompts, and record which direction was approved. Only then open a separate music-generation request.

Review the Video Analysis endpoint for the current field limits, use the Video-to-Music API guide for the next integration stage, or talk to Sonilo about a production workflow that needs account-specific limits and review controls.