Live Transcription + Translation

This real-time workflow combines transcription, optional translation, always-on mishear correction, and optional speaker diarization for meetings and multi-speaker conversations. The endpoint path retains /notetaker for compatibility.

Connection

Endpoint: wss://api.valsea.ai/v1/realtime/notetaker

Authentication

You must authenticate the WebSocket connection by passing your API key in the HTTP headers during the handshake.

Headers:

• Authorization: Bearer YOUR_API_KEY (Recommended)
• X-API-Key: YOUR_API_KEY (Supported)

Paid Rate-Limit Bypass

If you need to exceed the realtime connection RPM limit for a session, you can bypass the rate-limit check by sending one of these opt-in flags during the WebSocket handshake:

• Header: X-Bypass-Rate-Limit: true
• Query parameter: bypass_rate_limit=true
• Query parameter: bypassRateLimit=true

Bypass applies only to the per-organization rate-limit check. Authentication, credit checks, and session billing still apply. RTT sessions using this bypass are billed at 2x the normal realtime credit cost. Since diarization is commonly used on this endpoint, note that enabling it brings the final session cost to 4x the normal realtime credit cost when combined with the bypass.

Message Flow

1. Connect: Client establishes WebSocket connection.
2. Session Created: Server sends session.created event.
3. Start Session: Client sends session.start to configure language, target language, and diarization.
4. Stream Audio: Client sends audio.append messages with base64-encoded PCM16 audio chunks.
5. Receive Transcripts: Server streams transcript.partial and transcript.final events — corrected, optionally translated, and optionally speaker-labeled.
6. Commit Audio: Client sends audio.commit when a user stops speaking (optional/VAD dependent).
7. Stop Session: Client sends session.stop to end the session.

Partial vs Final (Important)

RTT emits two transcript event types for each utterance:

• transcript.partial: low-latency, in-progress text. This can change as more audio arrives.
• transcript.final: stable, corrected text for a completed segment — translated and speaker-labeled if configured. Treat this as the committed result.

Recommended client behavior:

1. Keep a temporary currentPartial string for transcript.partial.
2. Append only transcript.final to your persisted transcript history.
3. Clear currentPartial when you receive a matching transcript.final.

Client Messages

session.start

Initialize the session with configuration.

{
  "type": "session.start",
  "model": "valsea-rtt",
  "hint_text": "Optional context or vocabulary",
  "enable_correction": true,
  "language": "auto",
  "language_hints": ["en", "vi", "ar"],
  "target_language": "english",
  "diarize": true,
  "diarization_min_speakers": 2,
  "diarization_max_speakers": 6,
  "noise_suppression": "off"
}

When diarize=true, final events include speaker-labeled words and utterances metadata. Speaker IDs are zero-based integers. Diarization is emitted only on final events.

When target_language is set and differs from the input language, transcript.final.text is translated to that target language. Supported translation targets are english, chinese, japanese, korean, vietnamese, thai, french, spanish, german, russian, indonesian, malay, filipino, tamil, khmer, and lao.

audio.append

Send audio data.

{
  "type": "audio.append",
  "audio": "BASE64_ENCODED_PCM16_DATA"
}

• Format: Raw PCM 16-bit, 16kHz (recommended), mono.
• Encoding: Base64 string.

You may also send raw binary PCM16 frames directly over the WebSocket. Binary frames are treated as audio.append messages by the server.

audio.commit

Signal the end of a speech segment (e.g., VAD triggered silence).

{
  "type": "audio.commit"
}

session.stop

End the session gracefully.

{
  "type": "session.stop"
}

Server Messages

session.created

Sent immediately upon connection.

{
  "type": "session.created",
  "sessionId": "rtt_...",
  "supportedModels": ["valsea-rtt"]
}

session.ready

Sent when the backend engine is connected and ready to receive audio.

{
  "type": "session.ready",
  "sessionId": "rtt_...",
  "engine": "valsea-7"
}

transcript.partial

Intermediate transcription results (low latency, may change; never diarized or translated).

{
  "type": "transcript.partial",
  "text": "Hello everyone, let's",
  "isFinal": false,
  "timestampMs": 1230
}

transcript.final

Finalized, corrected text for a speech segment.

{
  "type": "transcript.final",
  "text": "Hello everyone, let's get started.",
  "rawText": "hello everyone lets get started",
  "isFinal": true,
  "timestampMs": 2500,
  "corrections": []
}

With diarization enabled:

{
  "type": "transcript.final",
  "text": "Hello everyone, let's get started.",
  "rawText": "hello everyone lets get started",
  "isFinal": true,
  "timestampMs": 2500,
  "words": [{ "word": "Hello", "start": 0.4, "end": 0.8, "speaker": 0 }],
  "utterances": [
    {
      "start": 0.4,
      "end": 2.1,
      "speaker": 0,
      "transcript": "Hello everyone, let's get started.",
      "words": []
    }
  ]
}

With translation enabled:

{
  "type": "transcript.final",
  "text": "El rápido zorro marrón salta sobre el perro perezoso mientras el mercado",
  "rawText": "The quick brown fox jumps over the lazy dog while the market",
  "isFinal": true,
  "timestampMs": 3765,
  "translated": true,
  "sourceLanguage": "english",
  "targetLanguage": "spanish"
}

error

Sent when an error occurs.

{
  "type": "error",
  "code": "INVALID_MESSAGE",
  "message": "Failed to parse message"
}

Diarization misconfigured — diarization_min_speakers greater than diarization_max_speakers:

{
  "type": "error",
  "code": "INVALID_DIARIZATION_CONFIG",
  "message": "diarization_min_speakers must be less than or equal to diarization_max_speakers"
}

Error codes on this endpoint

Event Handling Pattern

Use this pattern to avoid duplicated or unstable transcript content:

let currentPartial = '';
const finalSegments = [];

ws.on('message', (raw) => {
  const msg = JSON.parse(raw);

  if (msg.type === 'transcript.partial') {
    currentPartial = msg.text || '';
  }

  if (msg.type === 'transcript.final') {
    finalSegments.push({ text: msg.text, utterances: msg.utterances });
    currentPartial = '';
  }
});

Example (Node.js)

const WebSocket = require('ws');
const fs = require('fs');

const ws = new WebSocket('wss://api.valsea.ai/v1/realtime/notetaker', {
  headers: { 'X-API-Key': 'YOUR_KEY' },
});

ws.on('open', () => {
  ws.send(
    JSON.stringify({
      type: 'session.start',
      model: 'valsea-rtt',
      language: 'auto',
      diarize: true,
      diarization_min_speakers: 2,
      diarization_max_speakers: 4,
    }),
  );
});

ws.on('message', (data) => {
  const msg = JSON.parse(data);

  if (msg.type === 'session.ready') {
    const audioStream = fs.createReadStream('meeting-audio.raw');
    audioStream.on('data', (chunk) => {
      ws.send(
        JSON.stringify({
          type: 'audio.append',
          audio: chunk.toString('base64'),
        }),
      );
    });
  } else if (msg.type === 'transcript.final') {
    for (const utterance of msg.utterances || []) {
      console.log(`Speaker ${utterance.speaker}: ${utterance.transcript}`);
    }
  }
});

Model Selection Guide

Default: Auto-Detect

Use language: "auto" to automatically detect and transcribe supported speech, with built-in support for code-switching — the language can change mid-conversation (or mid-sentence) without losing accuracy. This is the recommended setting for multi-participant meetings where speakers may not all use the same language.

{
  "language": "auto"
}

For a Fixed Language

If you already know the language being spoken, set it explicitly for the most consistent routing:

{
  "model": "valsea-rtt",
  "language": "english"
}

Language Hints

Use "language": "auto" or omit language when the source language is unknown. Use a supported code such as english, singlish, vietnamese, arabic, arabic-egypt, or arabic-uae when you want to constrain routing. Add target_language only when translated final transcripts are required.

With automatic detection, language_hints can contain up to 20 expected two- or three-letter codes such as ["en", "vi", "ar"]. Hints bias detection but do not restrict it, and they are ignored when language is fixed. See Bias Automatic Detection.

VALSEA supports 181 languages, dialects, and variants overall. Translation and diarization coverage vary by endpoint and language. See the current supported-language matrix for the authoritative list.