Minimize processing time

Mux Video accepts most modern video formats and CODECs. However, certain types of inputs need to be normalized in order for Mux to do further operations on them, and this can add time before the video is ready to be streamed. If you want to normalize your content before sending it to Mux, and potentially improve performance, this guide will show what you need to do.

Standard input specs

Standard input has the following attributes.

• 1080p/2K or smaller. Video up to 2048x2048 is considered standard, including 1080p (1920x1080) video. Video larger than this is considered non-standard.
• H.264 video CODEC. H.264 is the dominant video CODEC in use today and almost every device supports H.264. While Mux accepts other CODECs as input, other CODECs must be normalized to H.264 and are considered non-standard.
• Max 10-second keyframe interval. To stream well using HTTP-based streaming methods like HLS, Mux requires all keyframes intervals to be less than 10 seconds.
• Closed GOP (group-of-pictures). (Warning: video jargon ahead. You can likely ignore this.) In closed-GOP video, all B frames reference other frames in the same GOP. Closed GOP always begins with an IDR (Instantaneous Decoder Refresh) frame. This means that every GOP can be played independently, without reference to another GOP. Standard input must be closed-GOP, which means that open-GOP video will be treated as non-standard and will be normalized to standard.
• 8Mbps or below. While Mux accepts higher bitrate inputs, bitrates higher than 8Mbps (and the bitrate should not exceed 16Mbps for any single GOP) are generally challenging for most viewer's connections and are considered non-standard.
• 8-bit 4:2:0 or below. This refers to the color depth and chroma subsampling. If you don't know what this is, you can probably ignore this, since most streaming video is 8-bit 4:2:0. This means that high dynamic range video (HDR) is currently considered non-standard, and will be normalized to SDR.
• Simple Edit Decision Lists. Edit Decision List (EDL) is typically added during post-production and defines how certain segments are used to build the track timeline for playback. A good example of a Simple Edit Decision List is to fix out of order frames in the video. Video with more complex uses of EDLs are considered non-standard.
• Frame rate between 10 and 120. Video with average frames per second (fps) less than 10 or greater than 120 is considered non-standard. Video frame rates within this range will be preserved. Video with less than 10 fps or greater than 120 fps will be normalized to 30 fps.
• Square Pixel Aspect Ratio. Pixel Aspect Ratio is a ratio of pixel's width to the height of that pixel. The value 1:1 represents Square Pixel Aspect Ratio. Video with non-square pixel aspect ratio is considered non-standard.
• AAC audio CODEC. AAC is the dominant audio CODEC in use today and almost every device supports this audio CODEC. While Mux accepts other CODECs as input, Mux only delivers AAC audio and non-AAC audio inputs are considered non-standard..

How to create standard input (ffmpeg)

As a starting point, here is a sample ffmpeg command for creating video that complies with Mux standard input. Feel free to modify this by using things like 2-pass encoding, different presets, or different bitrates (as long as the total bitrate ends up below than 8Mbps).

copy
ffmpeg -i input.mp4 -c:a copy -vf "scale=w=min(iw\,1920):h=-2" -c:v libx264 \
-profile high -b:v 7000k -pix_fmt yuv420p -maxrate 16000k out.mp4  

How to create standard input (mobile)

Most mobile devices capture H.264 8-bit 4:2:0 video by default. Here are the main things to watch out for:

• Ensure that the total file bitrate is below 8 mbps.
• Ensure the output file uses SDR (standard dynamic range) imaging. Some newer devices capture video in HDR (High Dynamic Range), which requires 10-bit 422 color, and must be re-encoded by Mux Video to support most devices.
• Ensure the output file is smaller than 1080p (1920x1080) or 2K (2048x1152). Some cameras shoot 4K video, which is currently converted down to 1080p when using Mux Video.
• If possible, choose a keyframe interval of 5s or so, but certainly between 2 and 10 seconds, and enable closed-GOP encoding. (If you don't see these options in your app or camera, it's probably the default already.)

Non-standard input

Mux Video works fine with video outside of the standard input specs. But because other videos cannot be easily streamed to many modern devices, Mux Video must perform an initial encoding operation on non-standard input to create a mezzanine file. This means that non-standard input will be slower to ingest.

Mux Video adds all the reasons why the input file is considered non-standard. The reasons are added to the video.asset.ready webhook event and the asset object when retrieved using the Asset GET API API. As an example, a UHD/4K input file encoded with HEVC/H.265 video CODEC and a frame rate of 60 fps is non-standard for two reasons:

1. UHD/4K video resolution is 3840x2160 pixels and so this resolution is greater than 2048 pixels for the input file to be standard.
2. Video CODEC is HEVC/H.265 which is considered non-standard (only AVC/H.264 video CODEC is considered standard). Note that, however, the frame rate is within the bounds of the standard input specification. So Retrieving the AssetAPI lists both video_resolution and video_codec as non-standard reasons:

copy
{
  "id": "123456789012345678",
  "created_at": "1506642742",
  "status": "ready",
  "duration": 120.000,
  "max_stored_resolution": "HD",
  "max_stored_frame_rate": 30.000,
  "aspect_ratio": "16:9",
  "per_title_encode" : true,
  "playback_ids": [
    {
      "id": "a1B2c3D4e5F6g7H8i9",
      "policy": "public"
    }
  ],
  "tracks": // tracks info
  "non_standard_input_reasons" : {
    "video_codec" : "hevc",
    "video_resolution" : "3840x2160"
  },
  "mp4_support": "none",
  "master_access": "none",
  "passthrough": "You shall pass!"
}  

General limits

The max duration for any single asset is 12 hours.