🎬 Chapter 32: Video in HTML

Subtitle Kinds, Chapters & Accessibility

`subtitles` vs `captions` (WCAG 2.1 AA), `descriptions`, `chapters` Navigation, and `metadata` Data Streams

LEARNING OBJECTIVES ⌵

Understand the legal, technical, and semantic differences between kind="subtitles" and kind="captions" under WCAG 2.1 AA accessibility standards.
Master all 5 distinct kind attribute values: subtitles, captions, descriptions, chapters, and metadata.
Implement interactive, clickable video chapter navigation menus driven entirely by kind="chapters" WebVTT tracks.
Use kind="metadata" tracks to transmit synchronized, machine-readable JSON payloads to JavaScript runtime consumers.

🎬 INTERACTIVE VISUAL PIPELINE Core Architecture Simulation

🌐

1. Input

Directives & Tags

⚙️

2. Parse

Tokenizer & AST

🌳

3. Layout

Box Model & Flow

🎨

4. Render

GPU Paint & Composite

PHASE 1: INPUT & DIRECTIVES

Browser receives declarative markup stream, parsing tag tokens and initializing component state.

📖 The Mental Model & Story (Intuitive Foundation)

Imagine watching a tense mystery movie:

Scenario A (Subtitles): You are a hearing person watching a French movie in Paris. You hear footsteps in the dark and ominous violin music building tension. You only need the French spoken dialogue translated into English.
Scenario B (Captions / SDH): You are a deaf or hard-of-hearing viewer. If the subtitle only displays spoken words, you will miss the critical audio clues—the eerie creaking floorboard upstairs, the distant gunshot, and the tense musical crescendo that signals impending danger.

+-----------------------------------------------------------------------------------+
|                        SUBTITLES VS CAPTIONS MENTAL MODEL                         |
+-----------------------------------------------------------------------------------+
|  kind="subtitles" (For Hearing Viewers in Another Language):                      |
|  "Did you hear that sound in the hallway?"                                        |
|                                                                                   |
|  kind="captions" (For Deaf / Hard-of-Hearing Viewers - WCAG AA Requirement):     |
|  [EERIE VIOLIN MUSIC BUILDS]                                                      |
|  [FLOORBOARD CREAKS UPSTAIRS]                                                     |
|  Detective Miller: "Did you hear that sound in the hallway?"                      |
|  [THUNDER CRACKS OUTSIDE]                                                         |
+-----------------------------------------------------------------------------------+

The kind attribute specifies the exact semantic purpose of the timed text track. Choosing the wrong kind can violate international accessibility laws (such as ADA Title III and the European Accessibility Act) or cause assistive technologies to misinterpret the track.

Technical Deep Dive & Specifications

The 5 `<track>` Kinds Specification Matrix

`kind` Value	Primary Target Audience	Payload Requirements	UI Render Behavior
`subtitles` (Default)	Users who can hear audio but do not speak the source language.	Spoken dialogue translation only.	Rendered as text overlay inside video canvas.
`captions`	Deaf or hard-of-hearing users (WCAG 2.1 AA compliance).	Dialogue + Sound Effects `[EXPLOSION]`, Music `[UPBEAT JAZZ]`, and Speaker IDs.	Rendered as high-contrast overlay inside video canvas.
`descriptions`	Blind or visually impaired users.	Text description of visual actions, intended for Text-to-Speech (TTS) synthesis.	Not visually displayed; consumed by screen readers.
`chapters`	All users seeking content navigation.	Timestamps paired with section/chapter titles.	Browser or custom UI builds an interactive jump menu.
`metadata`	JavaScript client applications.	Raw JSON, XML, or sensor coordinates synchronized with video time.	Invisible to user; accessed via `TextTrack.cues` in JS.

WCAG 2.1 AA Accessibility Requirements

To achieve WCAG 2.1 Success Criterion 1.2.2 (Captions - Prerecorded) and 1.2.4 (Captions - Live):

Captions Must Be Synchronized: Text must appear at the exact moment audio is heard (within 100ms tolerance).
Equivalent Content: All dialogue, non-speech audio cues (screams, doorbells, applause), and emotional tone in music must be transcribed.
Speaker Identification: When multiple speakers are on screen or off screen, captions must identify each speaker by name or unique color.

Interactive Chapter Navigation with `kind="chapters"`

A WebVTT chapters file provides a timeline table of contents:

WEBVTT - Course Syllabus Navigation

00:00:00.000 --> 00:01:30.000
Chapter 1: Introduction to GPU Decoding

00:01:30.000 --> 00:04:15.000
Chapter 2: The MP4 moov Atom & FFmpeg

00:04:15.000 --> 00:08:00.000
Chapter 3: Custom Media Controls Engineering

When loaded via <track kind="chapters" src="chapters.vtt">, JavaScript can access the TextTrack cues and dynamically generate a clickable sidebar:

const video = document.querySelector('video');
const chapterTrack = video.textTracks[0];

// Force track to load cues into memory
chapterTrack.mode = 'hidden'; 

chapterTrack.oncuechange = () => {
  const activeCue = chapterTrack.activeCues[0];
  if (activeCue) {
    highlightActiveChapter(activeCue.text);
  }
};

Machine-Readable Data Streams with `kind="metadata"`

You can embed structured JSON inside WebVTT cues for interactive e-commerce, interactive quizzes, or sports analytics:

WEBVTT - E-Commerce Synchronized Overlay

00:00:10.000 --> 00:00:25.000
{ "productId": "sku_982", "name": "Wireless Noise-Canceling Headphones", "price": "$199.99", "url": "/shop/sku-982" }

When oncuechange fires, JavaScript parses JSON.parse(cue.text) and dynamically pops up an interactive "Buy Now" widget over the video!

💻 Interactive Code Playground

Starter Code

Line-by-Line Code Breakdown

Line 66 (kind="captions"):
- Contains full sound effect annotations ([UPBEAT MUSIC PLAYING]), meeting WCAG AA requirements for deaf and hard-of-hearing viewers.
Line 78 (kind="chapters"):
- Provides synchronized time ranges mapping to descriptive section titles.
Lines 102–108 (item.addEventListener('click', ...):
- Enables instant jumping to specific chapter timestamps when clicked by the user.

Expected Browser Render Output

SYS: ACTIVE

HULL: 98%

CORE: STABLE

NET: ONLINE

HTML STARSHIP CODE TERMINAL playground.html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Chapter Navigation & Caption Systems</title>
  <style>
    body {
      font-family: system-ui, sans-serif;
      max-width: 840px;
      margin: 30px auto;
      padding: 0 20px;
      background: #090d16;
      color: #f1f5f9;
    }
    .player-layout {
      display: grid;
      grid-template-columns: 1fr 260px;
      gap: 20px;
      background: #131b2e;
      border: 1px solid #1e293b;
      border-radius: 12px;
      padding: 20px;
    }
    @media (max-width: 768px) {
      .player-layout { grid-template-columns: 1fr; }
    }
    video {
      width: 100%;
      height: auto;
      border-radius: 8px;
      background: #000;
    }
    .chapter-panel {
      background: #0b0f19;
      border-radius: 8px;
      padding: 16px;
      border: 1px solid #1e293b;
    }
    .chapter-panel h2 {
      font-size: 1rem;
      color: #38bdf8;
      margin-top: 0;
      margin-bottom: 12px;
      text-transform: uppercase;
      letter-spacing: 0.05em;
    }
    .chapter-list {
      list-style: none;
      padding: 0;
      margin: 0;
    }
    .chapter-item {
      padding: 10px 12px;
      border-radius: 6px;
      font-size: 0.85rem;
      cursor: pointer;
      margin-bottom: 6px;
      background: #1e293b;
      color: #cbd5e1;
      transition: all 0.2s;
    }
    .chapter-item:hover {
      background: #334155;
      color: #ffffff;
    }
    .chapter-item.active {
      background: #0284c7;
      color: #ffffff;
      font-weight: 600;
    }
  </style>
</head>
<body>

<h1 style="color: #38bdf8; font-size: 1.4rem;">Synchronized Chapter Navigation System</h1>

<track 
          kind="captions" 
          srclang="en" 
          label="English (CC)" 
          default
          src="data:text/vtt;charset=utf-8,WEBVTT%0A%0A00%3A00%3A00.500%20--%3E%2000%3A00%3A02.500%0A%5BUPBEAT%20MUSIC%20PLAYING%5D%0A%3Cv%20Instructor%3EWelcome%20to%20the%20lecture.%0A%0A00%3A00%3A03.000%20--%3E%2000%3A00%3A05.000%0A%5BGENTLE%20BREEZE%20RUSTLING%5D%0A%3Cv%20Instructor%3EWe%20examine%20chapter%20segmentation.">

<track 
          id="chaptersTrack"
          kind="chapters" 
          srclang="en" 
          label="Chapters" 
          src="data:text/vtt;charset=utf-8,WEBVTT%0A%0A00%3A00%3A00.000%20--%3E%2000%3A00%3A02.000%0A1.%20Course%20Introduction%0A%0A00%3A00%3A02.000%20--%3E%2000%3A00%3A04.000%0A2.%20Core%20Concepts%0A%0A00%3A00%3A04.000%20--%3E%2000%3A00%3A06.000%0A3.%20Summary%20%26%20Next%20Steps">
      </video>
    </div>

<aside class="chapter-panel">
      <h2>Video Chapters</h2>
      <ul class="chapter-list" id="chapterList">
        <li class="chapter-item" data-time="0">1. Course Introduction (0:00)</li>
        <li class="chapter-item" data-time="2">2. Core Concepts (0:02)</li>
        <li class="chapter-item" data-time="4">3. Summary & Next Steps (0:04)</li>
      </ul>
    </aside>
  </div>

// Click Chapter -> Seek Video
    items.forEach(item => {
      item.addEventListener('click', () => {
        const seekTime = parseFloat(item.getAttribute('data-time'));
        video.currentTime = seekTime;
        video.play();
      });
    });

// Update active highlight on timeupdate
    video.addEventListener('timeupdate', () => {
      const cur = video.currentTime;
      items.forEach((item, index) => {
        const time = parseFloat(item.getAttribute('data-time'));
        const nextTime = items[index + 1] ? parseFloat(items[index + 1].getAttribute('data-time')) : Infinity;
        if (cur >= time && cur < nextTime) {
          item.classList.add('active');
        } else {
          item.classList.remove('active');
        }
      });
    });
  </script>

</body>
</html>

LIVE RENDER & DIAGNOSTICS CORE TEMP: 45°C

INSPECTING DOM: VALID

TAGS: SCANNING...

+------------------------------------------------------------------------+
| Synchronized Chapter Navigation System                                 |
| +------------------------------------+  +----------------------------+ |
| |                                    |  | VIDEO CHAPTERS             | |
| |         [ VIDEO CANVAS ]           |  | [1. Course Intro (0:00)]*  | |
| |  [UPBEAT MUSIC PLAYING]            |  |  2. Core Concepts (0:02)   | |
| |  Instructor: Welcome to lecture.   |  |  3. Summary (0:04)         | |
| |                                    |  +----------------------------+ |
| | [ > ] [===o==================] 0:01|                                 |
| +------------------------------------+                                 |
+------------------------------------------------------------------------+

🏋️ Hands-On Exercise

🎯 The Challenge: Build a Timed E-Commerce Product Overlay using `kind="metadata"`

Instructions:

Create a <video> element with controls, crossorigin="anonymous", and the flower MP4 video.
Embed a <track kind="metadata"> with a Data URI containing JSON metadata payloads for two time ranges:
- 0.0s to 2.5s: {"product": "Botanical Fertilizer 500ml", "price": "$14.99"}
- 2.8s to 5.0s: {"product": "Ceramic Plant Pot", "price": "$24.50"}
Use JavaScript to listen for track cue changes (track.oncuechange) on the metadata track (set track.mode = 'hidden').
When a cue triggers, parse the JSON payload and render an interactive pop-up badge on top of the video showing the product name and price with a "Buy Now" button.

🏁 Starter Code Sandbox

SYS: ACTIVE

HULL: 98%

CORE: STABLE

NET: ONLINE

STARTER CODE SANDBOX exercise.html

LIVE RENDER & DIAGNOSTICS CORE TEMP: 45°C

INSPECTING DOM: VALID

TAGS: SCANNING...

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Metadata Track Challenge</title>
  <style>
    body {
      font-family: system-ui, sans-serif;
      max-width: 680px;
      margin: 2rem auto;
      padding: 0 1rem;
      background: #0b0f19;
      color: #f1f5f9;
    }
    .video-stage {
      position: relative;
      width: 100%;
      aspect-ratio: 16 / 9;
      background: #000;
      border-radius: 12px;
      overflow: hidden;
    }
    video {
      width: 100%;
      height: 100%;
      display: block;
    }
    .product-popup {
      position: absolute;
      top: 16px;
      right: 16px;
      background: rgba(15, 23, 42, 0.95);
      border: 1px solid #38bdf8;
      border-radius: 8px;
      padding: 12px 16px;
      box-shadow: 0 8px 20px rgba(0,0,0,0.6);
      display: none;
      z-index: 10;
      animation: fadeIn 0.3s ease;
    }
    @keyframes fadeIn {
      from { opacity: 0; transform: translateY(-10px); }
      to { opacity: 1; transform: translateY(0); }
    }
    .product-title {
      font-size: 0.9rem;
      font-weight: 700;
      color: #38bdf8;
      margin-bottom: 4px;
    }
    .product-price {
      font-size: 0.85rem;
      color: #4ade80;
      margin-bottom: 8px;
    }
    .buy-btn {
      background: #0284c7;
      color: white;
      border: none;
      padding: 6px 12px;
      border-radius: 4px;
      cursor: pointer;
      font-size: 0.8rem;
      font-weight: 600;
    }
    .buy-btn:hover { background: #0369a1; }
  </style>
</head>
<body>

  <h1>Metadata-Driven Interactive Video</h1>

  <div class="video-stage">
    <video 
      id="productVideo"
      controls 
      crossorigin="anonymous"
      poster="https://images.unsplash.com/photo-1518709268805-4e9042af9f23?auto=format&fit=crop&w=800&q=80">
      
      <source src="https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4" type="video/mp4">

      <!-- Timed JSON Metadata Track -->
      <track 
        id="metaTrack"
        kind="metadata" 
        default
        src="data:text/vtt;charset=utf-8,WEBVTT%0A%0A00%3A00%3A00.500%20--%3E%2000%3A00%3A02.500%0A%7B%22product%22%3A%20%22Botanical%20Fertilizer%20500ml%22%2C%20%22price%22%3A%20%22%2414.99%22%7D%0A%0A00%3A00%3A02.800%20--%3E%2000%3A00%3A05.000%0A%7B%22product%22%3A%20%22Ceramic%20Plant%20Pot%22%2C%20%22price%22%3A%20%22%2424.50%22%7D">
    </video>

    <!-- Synchronized Floating Card -->
    <div class="product-popup" id="popupCard">
      <div class="product-title" id="pName">Product Name</div>
      <div class="product-price" id="pPrice">$0.00</div>
      <button class="buy-btn" onclick="alert('Added to cart!')">🛒 Quick Buy</button>
    </div>
  </div>

  <script>
    const video = document.getElementById('productVideo');
    const popupCard = document.getElementById('popupCard');
    const pName = document.getElementById('pName');
    const pPrice = document.getElementById('pPrice');

    // Wait until tracks are ready
    video.addEventListener('loadedmetadata', () => {
      const metaTrack = video.textTracks[0];
      // Setting mode to 'hidden' loads cues into JavaScript memory without native UI rendering
      metaTrack.mode = 'hidden';

      metaTrack.addEventListener('cuechange', () => {
        const activeCue = metaTrack.activeCues[0];
        if (activeCue) {
          try {
            const data = JSON.parse(activeCue.text);
            pName.textContent = data.product;
            pPrice.textContent = data.price;
            popupCard.style.display = 'block';
          } catch (e) {
            console.error('Invalid JSON cue:', e);
          }
        } else {
          popupCard.style.display = 'none';
        }
      });
    });
  </script>

</body>
</html>

Why this works:

kind="metadata" is specifically reserved for non-visual application data.
Setting track.mode = 'hidden' forces the browser to download and parse the cues so activeCues fires cuechange events in JavaScript.
Dynamic parsing with JSON.parse() allows real-time interactive UI updates synchronized to video timestamps.

⚠️ Common Pitfalls

Using kind="subtitles" When Transcribing Sound Effects: kind="subtitles" indicates a pure spoken dialogue translation. If your track includes descriptions of sound effects (e.g., [THUNDER ROARS]), it must be declared as kind="captions" to meet WCAG AA standards.
Forgetting track.mode = 'hidden' for Metadata: If you create a kind="metadata" or kind="chapters" track without setting track.mode = 'hidden' or 'showing' in JavaScript, some browser engines will skip fetching the file to save bandwidth, leaving textTrack.cues empty (null).
Multiple default Attributes: Marking more than one <track> element with the boolean default attribute is invalid HTML and leads to undefined browser behavior.

💡 Pro Tips

Automated Live Transcripts for SEO: Search engine bots index <track kind="captions"> files directly. Serving comprehensive WebVTT closed captions provides rich textual indexing for your video content on Google Search.
Track Mode Lifecycle States: A TextTrack in JavaScript has three distinct modes:
- 'disabled': The track is inactive and cues are not fetched.
- 'hidden': The track is parsed and fires cuechange events in JavaScript, but the browser does NOT render captions on the video canvas.
- 'showing': Cues are parsed AND rendered on screen by the browser.

📌 Key Takeaways

kind="subtitles" is for dialogue translation; kind="captions" is for deaf/hard-of-hearing accessibility (including sound effects and speaker IDs).
Providing synchronized captions for pre-recorded video is mandatory for WCAG 2.1 Level AA compliance.
kind="chapters" allows building interactive, clickable table-of-contents navigation menus.
kind="metadata" allows streaming synchronized JSON data payloads directly to JavaScript applications.
Use TextTrack.mode = 'hidden' to parse cues in JavaScript without native visual rendering.
--

⭐ LEARN: HTML 🌟 ⚔️ QUIZ BATTLE ARENA // ACTIVE

? Question 1 / 3

What is the critical difference between `kind="subtitles"` and `kind="captions"`?

Topic: HTML Fundamentals

? Question 2 / 3

Which `track` kind should be used when transmitting synchronized JSON coordinates or e-commerce payloads to JavaScript?

Topic: HTML Fundamentals

? Question 3 / 3

What `TextTrack.mode` state must be assigned in JavaScript to receive `cuechange` events while suppressing native visual caption rendering?

Topic: HTML Fundamentals

👨‍🚀 CADET FLIGHT DOSSIER

CADET EXPLORER

📖 The Mental Model & Story (Intuitive Foundation)

Technical Deep Dive & Specifications

The 5 <track> Kinds Specification Matrix

WCAG 2.1 AA Accessibility Requirements

Interactive Chapter Navigation with kind="chapters"

Machine-Readable Data Streams with kind="metadata"

💻 Interactive Code Playground

Starter Code

Line-by-Line Code Breakdown

Expected Browser Render Output

🏋️ Hands-On Exercise

🎯 The Challenge: Build a Timed E-Commerce Product Overlay using kind="metadata"

🏁 Starter Code Sandbox

⚠️ Common Pitfalls

💡 Pro Tips

📌 Key Takeaways

What is the critical difference between kind="subtitles" and kind="captions"?

Which track kind should be used when transmitting synchronized JSON coordinates or e-commerce payloads to JavaScript?

What TextTrack.mode state must be assigned in JavaScript to receive cuechange events while suppressing native visual caption rendering?

The 5 `<track>` Kinds Specification Matrix

Interactive Chapter Navigation with `kind="chapters"`

Machine-Readable Data Streams with `kind="metadata"`

🎯 The Challenge: Build a Timed E-Commerce Product Overlay using `kind="metadata"`

What is the critical difference between `kind="subtitles"` and `kind="captions"`?

Which `track` kind should be used when transmitting synchronized JSON coordinates or e-commerce payloads to JavaScript?

What `TextTrack.mode` state must be assigned in JavaScript to receive `cuechange` events while suppressing native visual caption rendering?