Ancillary data

Ancillary data is data that has been added to given data and uses the same form of transport. Common examples are cover art images for media files or streams, or digital data added to radio or television broadcasts.

Television

Ancillary data (commonly abbreviated as ANC data), in the context of television systems, refers to a means which by non-video information (such as audio, other forms of essence, and metadata) may be embedded within the serial digital interface. Ancillary data is standardized by SMPTE as SMPTE 291M: Ancillary Data Packet and Space Formatting.

Ancillary data can be located in non-picture portions of horizontal scan lines, known as Horizontal ANCillary data (HANC). Ancillary data can also be located in non-picture regions of the video frame, known as Vertical ANCillary data (VANC).

Technical details

Location

Ancillary data packets may be located anywhere within a serial digital data stream, with the following exceptions:

They should not be located in the lines identified as a switch point (which may be lost when switching sources).
They should not be located in the active picture area.
They may not cross the TRS (timing reference signal) packets.

Ancillary data packets are commonly divided into two types, depending on where they are located—specific packet types are often constrained to be in one location or another.

Ancillary packets located in the horizontal blanking region (after EAV but before SAV), regardless of line, are known as horizontal ancillary data, or HANC. HANC is commonly used for higher-bandwidth data, and/or for things that need to be synchronized to a particular line; the most common type of HANC is embedded audio.
Ancillary packets located in the vertical blanking region, and after SAV but before EAV, are known as vertical ancillary data, or VANC. VANC is commonly used for low-bandwidth data, or for things that only need be updated on a per-field or per-frame rate. Closed caption data and VPID are generally stored as VANC.

Note that ANC packets which lie in the dataspace which is in both the horizontal and vertical intervals, is considered to be HANC and not VANC.

VANC packets should be inserted in this manner:

(SMPTE 334M section 3): VANC data packets can appear anywhere between the SAV and EAV TRS packets in any line from the second line after the line specified for switching to the last line preceding active video, inclusive. Given the spec for switch points (set RP168 figure 2), the first allowed lines are 12 and 275 (for 525-line/59.94 Hz systems) or 8 and 321 (for 625-line/50 Hz systems). This conflicts with SMPTE 125M, and does not address requirements for carrying DVITC (Digital Vertical Interval TimeCode) and video index packets.
(SMPTE 125M section 3.6.2): VANC should appear only in lines 1-13, 15-19, 264-276, and 278-282, with lines 14 and 277 reserved for DVITC and video index data. This conflicts with SMPTE 334M, and does not address 625-line/50 Hz systems.

Packet format

All ANC packets must start with a start sequence; for component interfaces (the only kind of serial digital interface in widespread use today), the start sequence is 0x000 0x3FF 0x3FF. This sequence is otherwise illegal in the serial digital interface. (In the obsolete composite versions of SDI, the ANC start sequence is a single word, 0x3FC).

Three words immediately follow the start sequence in the header. The first word after the start sequence is the Data Identifier or DID, followed by either a 'Secondary Data Identifier (SDID) or a Data Block Number (DBN), followed by a Data Count (DC). After the Data Count word are 0 - 255 (inclusive) User Data Words (UDW), followed by a Checksum (CS) word.

DID

The Data Identifier word (along with the SDID, if used), indicates the type of ancillary data that the packet corresponds to. Data identifiers range from 1 to 255 (FF hex), with 0 being reserved. As the serial digital interface is a 10-bit format, the DID word is encoded as follows:

Bits 0-7 (bit 0 being the LSB), are the raw DID value.
Bit 8 is the even parity bit of bits 0-7.
Bit 9 is the inverse of bit 8.

Thus, a DID of 0x61 (01100001) would be encoded as 0x161 (0101100001), whereas a DID of 0x63 (01100011) would be encoded as 0x263 (1001100011). Note that this encoding scheme ensures that the reserved values in the serial digital interface (0-3 and 1020-1023) are never used.

If the DID is equal to 128 (0x80) or greater, then the packet is a Type 1 packet, and the DID is sufficient to identify the packet type, and the following word is a Data Block Number. If the DID is less than 128, it is a Type 2 packet, and the following words is the Secondary Data Identifier; the DID and SDID together identify the packet type.

SDID

The SDID is only valid if the DID is less than 0x80. The SDID is nominally an 8-bit value, ranging from 0 to 255. It is encoded in the same fashion as the DID.

DID/SDID words of 161 101 (hex) correspond to a DID of 61 hex, and a SDID of 1 (once the two high bits are removed); these values would indicate that the packet type is defined by SMPTE 334M, and contains DTV closed captions data.

DBN

The DBN is only valid if the DID is 80 hex or greater. It is (optionally) used to identify multiple packets of the same type within a field; each subsequent packet of the indicated type has a DBN which is one higher than the previous packet, wrapping around as necessary. The DBN is an 8-bit value, encoded in the same fashion as the SDID.

DC

The Data Count word is an 8-bit value, encoded in the same fashion as the DID, which indicates how many user data words are to follow. It can range from 0 to 255.

UDW

User data words are the "payload" present in the ANC packet. They are defined according to the packet type, SMPTE 291M does not define their use or impose any restrictions on the values which may be present in the UDW space. The only restriction is that the reserved values in the serial digital interface (0-3 and 1020-1023) may not be included in the UDW. Many ANC formats, though not all, are essentially 8-bit formats, and encode data in the same manner that the header words are encoded.

Example

SMPTE 352M (Video Payload ID) defines four UDW:

Bits	Byte 1	Byte 2	Byte 3	Byte 4
Bit 7	1	Interlaced (0) or Progressive (1) transport	Reserved	Reserved
Bit 6	0	Interlaced (0) or Progressive (1) picture	Horizontal Y´/Y sampling 1920 (0) or 2048 (1)	Reserved
Bit 5	0	Reserved	Reserved	Reserved
Bit 4	0	Reserved	Reserved	Dynamic range 100% (0h), 200% (1h), 400% (2h), Reserved (3h)
Bit 3	1	Picture Rate (see SMPTE 352M table 2)	Sampling structure (see SMPTE 352M table 3 and Note 1)	Dynamic range 100% (0h), 200% (1h), 400% (2h), Reserved (3h)
Bit 2	0	Picture Rate (see SMPTE 352M table 2)	Sampling structure (see SMPTE 352M table 3 and Note 1)	Reserved
Bit 1	0	Picture Rate (see SMPTE 352M table 2)	Sampling structure (see SMPTE 352M table 3 and Note 1)	Bit depth 8-bit (0h), 10-bit (1h), 12-bit (2h), Reserved (3h)
Bit 0	1	Picture Rate (see SMPTE 352M table 2)	Sampling structure (see SMPTE 352M table 3 and Note 1)	Bit depth 8-bit (0h), 10-bit (1h), 12-bit (2h), Reserved (3h)

Checksum

The last word in an ANC packet is the Checksum word. It is computed by computing the sum (modulo 512) of bits 0-8 (not bit 9), of all the other words in the ANC packet, excluding the packet start sequence. Bit 9 of the checksum word is then defined as the inverse of bit 8. Note that the checksum word does not contain a parity bit; instead, the parity bits of other words are included in the checksum calculations.

Usage

Embedded audio

Embedded audio is audio payload which is (typically) the soundtrack (music, dialogue, and sound effects) for the video program. Two standards, SMPTE 272M (for SD) and SMPTE 299M (for HD and 3G) define how audio is embedded into the ancillary space. The SD and HD standards provide for up to 16 channels of PCM audio, while 3G allows up to 32 channels, typically encoded in the AES3 format. In HD, the embedded audio data packets are carried in the HANC space of Cb/Cr (chroma) parallel data stream.

In addition, both standards define audio control packets. The audio control packets are carried in the HANC space of the Y (luminance) parallel data steam and are inserted once per field at the second video line past the switching point (see SMPTE RP168 for switching points of various video standards). The audio control packet contains audio-related metadata, such as its timing relative to video, which channels are present, etc.

Embedded audio packets are Type 1 packets.

EDH

EDH packets are used for error detection in standard definition interfaces (they are not necessary in HD interfaces, as the HD-SDI interface includes CRC checkwords built in).

References

SMPTE: SMPTE 291M-1998: Ancillary Data Packet and Space Formatting
SMPTE: ANSI/SMPTE 125M-1995: Component Video Signal 4:2:2; Bit-Parallel Digital Interface
SMPTE: ANSI/SMPTE 334M-1995: Vertical Ancillary Data Mapping for Bit-Serial Interface
SMPTE: RP168-2002: Definition of Vertical Interval Switching Point for Synchronous Video Switching
SMPTE: SMPTE 299-1:2010: 24-Bit Digital Audio Format for SMPTE 292 Bit-Serial Interface
SMPTE: SMPTE 299-2:2010: Extension of the 24-Bit Digital Audio Format to 32 Channels for 3 Gb/s Bit-Serial Interfaces
SMPTE: Data Identification Word Assignments for Registered DIDs