SMPTE ST 2117-1,[1] informally known as VC-6, is a video coding format.[2]
| Organization | SMPTE |
|---|---|
| Domain | Video compression |
Overview
editThe VC-6 codec is optimized for intermediate, mezzanine or contribution coding applications.[2] Typically, these applications involve compressing finished compositions for editing, contribution, primary distribution, archiving and other applications where it is necessary to preserve image quality as close to the original as possible, whilst reducing bitrates, and optimizing processing, power and storage requirements. VC-6, like other codecs in this category[3][4] uses only intra-frame compressions, where each frame is stored independently and can be decoded with no dependencies on any other frame.[5] The codec implements lossless and lossy compression, depending on the encoding parameters that have been selected. It was standardized in 2020. Earlier variants of the codec have been deployed by V-Nova since 2015 under the trade name Perseus. The codec is based on hierarchical data structures called s-trees, and does not involve DCT or wavelet transform compression. The compression mechanism is independent of the data being compressed, and can be applied to pixels as well as other non-image data.[6]
Unlike DCT based codecs, VC-6 is based on hierarchical, repeatable s-tree structures that are similar to modified quadtrees. These simple structures provide intrinsic capabilities, such as massive parallelism[7] and the ability to choose the type of filtering used to reconstruct higher-resolution images from lower-resolution images.[8] In the VC-6 standard[2] an up-sampler developed with an in-loop Convolutional Neural Network is provided to optimize the detail in the reconstructed image, without requiring a large computational overhead. The ability to navigate spatially within the VC-6 bitstream at multiple levels[2] also provides the ability for decoding devices to apply more resources to different regions of the image allowing for Region-of-Interest applications to operate on compressed bitstreams without requiring a decode of the full-resolution image.[9]
History
editAt the NAB Show in 2015, V-Nova claimed "2x–3x average compression gains, at all quality levels, under practical real-time operating scenarios versus H.264, HEVC and JPEG2000.".[10] Making this announcement on 1 April before a major trade show attracted the attention of many compression experts.[11] Since then, V-Nova have deployed and licensed the technology, known at the time as Perseus,[10] in both contribution and distribution applications around the world including Sky Italia,[12] Fast Filmz,[13][14] Harmonic Inc, and others. A variant of the technology optimized for enhancing distribution codec will soon be standardized as MPEG-5 Part-2 LCEVC.[15][16][17]
Core concepts
editPlanes
editThe standard[2] describes a compression algorithm that is applied to independent planes of data. These planes might be RGB or RGBA pixels originating in a camera, YCbCr pixels from a conventional TV-centric video source or some other planes of data. There may be up to 255 independent planes of data, and each plane can have a grid of data values of dimensions up to 65535 x 65535.[18] The SMPTE ST 2117-1 standard focuses on compressing planes of data values, typically pixels. To compress and decompress the data in each plane, VC-6 uses hierarchical representations of small tree-like structure that carry metadata used to predict other trees. There are 3 fundamental structures repeated in each plane.[2]
S-tree
editThe core compression structure in VC-6 is the s-tree. It is similar to the quadtree structure common in other schemes. An s-tree is comprised nodes arranged in a tree structure, where each node links to 4 nodes in the next layer. The total number of layers above the root node is known as the rise of the s-tree. Compression is achieved in an s-tree by using metadata to signal whether levels can be predicted with selective carrying of enhancement data in the bitstream. The more data that can be predicted, the less information that is sent, and the better the compression ratio.[6][2]
Tableau
editThe standard[2] defines a tableau as the root node, or the highest layer of an s-tree, that contains nodes for another s-tree. Like the generic s-trees from which they are constructed, tableaux are arranged in layers with metadata in the nodes indicating whether or not higher layers are predicted or transmitted in the bitstream.[6]
Echelon
editThe hierarchical s-tree and tableau structures in the standard[2] are used to carry enhancements (called resid-vals) and other metadata to reduce the amount of raw data that needs to be carried in the bitstream payload. The final hierarchical tool is an ability to arrange the tableaux, so that data from each plane (i.e. pixels) can be dequantized at different resolutions and used as predictors for higher resolutions. Each of these resolutions is defined by the standard[2] as an echelon. Each echelon within a plane is identified by an index, where a more negative index indicates a low resolution and a larger, more positive index indicates a higher resolution.
Bitstream overview
editVC-6 is an example of intra-frame coding, where each picture is coded without referencing other pictures. It is also intra-plane, where no information from one plane is used to predict another plane. As a result, the VC-6 bitstream contains all of the information for all of the planes of a single image.[2] An image sequence is created by concatenating the bitstreams for multiple images, or by packaging them in a container such as MXF or Quicktime or Matroska.
The VC-6 bitstream is defined in the standard.[2] by pseudo code, and a reference decoder has been demonstrated based on that definition. The primary header is the only fixed structure defined by the standard.[2] The secondary header contains marker and sizing information depending on the values in the primary header. The tertiary header is entirely calculated, and then the payload structure is derived from the parameters calculated during header decoding[2]
Decoding overview
editThe standard[2] defines a process called plane reconstruction for decoding images from a bitstream. The process starts with the echelon having the lowest index. No predictions are used for this echelon. Firstly, the bitstream rules are used to reconstruct residuals. Next, desparsification and entropy decoding processes are performed to fill the grid with data values at each coordinate. These values are then dequantised to create full-range values that can be used as predictions for the echelon with the next highest index. Each echelon uses the upsampler specified in the header to create a predicted plane from the echelon below which is added to the residual grid from the current echelon that can be upsampled as a prediction for the next echelon.[19]
The final, full-resolution, echelon, defined by the standard, is at index 0, and its results are displayed, rather than used for another echelon.[2]
Upsampler options
editBasic options
editThe standard[2] defines a number of basic upsamplers[20] to create higher-resolution predictions from lower-resolution echelons. There are two linear upsamplers, bicubic and sharp, and a nearest-neighbour upsampler.
Convolutional Neural Network Upsampler
editSix different non-linear upsamplers are defined[2] by a set of processes and coefficients that are provided in JSON format.[20] These coefficients were generated using Convolutional Neural Network[21] techniques.
References
edit- ^ "IEEE Xplore Search Results". IEEE. Retrieved 17 September 2020.
- ^ a b c d e f g h i j k l m n o p q r "ST 2117-1:2020 - SMPTE Standard - VC-6 Multiplanar Picture Format — Part 1. Elementary Bitstream". St 2117-1:2020: 1–156. July 2020. doi:10.5594/SMPTE.ST2117-1.2020. ISBN 978-1-68303-219-9. Archived from the original on 28 October 2020.
- ^ "ST 2042-1:2012 - SMPTE Standard - VC-2 Video Compression". St 2042-1:2012: 1–137. August 2012. doi:10.5594/SMPTE.ST2042-1.2012. ISBN 978-1-61482-890-7. Archived from the original on 13 June 2018.
- ^ "ST 2019-1:2016 - SMPTE Standard - VC-3 Picture Compression and Data Stream Format". St 2019-1:2016: 1–108. June 2016. doi:10.5594/SMPTE.ST2019-1.2016. ISBN 978-1-68303-020-1. Archived from the original on 6 March 2017.
- ^ "ST 2073-1:2014 - SMPTE Standard - VC-5 Video Essence - Part 1: Elementary Bitstream". St 2073-1:2014: 1–50. March 2014. doi:10.5594/SMPTE.ST2073-1.2014. ISBN 978-1-61482-797-9.[dead link]
- ^ a b c "SMPTE Ratifies V-Nova's AI-Powered VC-6 Video Codec". Digital Media World. 7 October 2020.
- ^ Hung, Yubin; Rosenfeld, Azriel (1 August 1989). "Parallel processing of linear quadtrees on a mesh-connected computer". Journal of Parallel and Distributed Computing. 7 (1): 1–27. doi:10.1016/0743-7315(89)90049-X. ISSN 0743-7315.
- ^ Samet, Hanan (1988), "An Overview of Quadtrees, Octrees, and Related Hierarchical Data Structures", Theoretical Foundations of Computer Graphics and CAD, Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 51–68, doi:10.1007/978-3-642-83539-1_2, ISBN 978-3-642-83541-4, retrieved 9 September 2020
- ^ S., V. G. (5 October 2020). "SMPTE Publishes New VC-6 Video Production Codec Standard". Sports Video Group.
- ^ a b "Review: V-Nova Perseus: Does its Compression Live Up to the Hype?". Streaming Media Magazine. 17 June 2016. Retrieved 4 September 2020.
- ^ "Historical timeline of video coding standards and formats". Vcodex. Retrieved 30 July 2021.
- ^ "Sky Italia chooses V-Nova to extend IPTV reach". Digital TV Europe.
- ^ "India's FastFilmz taps V-Nova to deliver OTT to 2G phones". Digital TV Europe. 7 April 2016. Retrieved 9 September 2020.
- ^ "SHAREit Acquires Fastfilmz To Increase Video Content, Regional Users". Inc42 Media. 8 May 2018. Retrieved 17 September 2020.
- ^ "Low Complexity Enhancement Video Codec". LCEVC - A New Approach to Video Compression.
- ^ "V-Nova announces MPEG-5 Part 2 LCEVC". TVB Europe.
- ^ "Perseus politics leak out at NAB in wake of MPEG-5 revelation". Rethnk Research. 11 April 2019.
- ^ "VC-6 Overview". mrmxf.com.
- ^ ST 2117-1:2020 - SMPTE Standard - VC-6 Multiplanar Picture Format — Part 1. Elementary Bitstream. July 2020. pp. 1–156. doi:10.5594/SMPTE.ST2117-1.2020. ISBN 978-1-68303-219-9. Archived from the original on 28 October 2020.
{{cite book}}:|journal=ignored (help) - ^ a b ST 2117-1 upsampler media element. 21 July 2020. pp. 1–156. doi:10.5594/SMPTE.ST2117-1.2020. ISBN 978-1-68303-219-9. Archived from the original on 15 June 2021.
{{cite book}}:|journal=ignored (help) - ^ Arabshahi, P. (May 1996). "Fundamentals of Artificial Neural Networks [Book Reviews]". IEEE Transactions on Neural Networks. 7 (3): 793. doi:10.1109/tnn.1996.501738. ISSN 1045-9227. S2CID 6576607.