An Introduction To Video Compression
An Introduction to VideoCompression
Typical CameraAn Introduction to rface4Image Sensors (CCD/CMOS)AGENDA Video Basics– Analog Video– Digital Video– Scanning Formats Video Compression– Intra-Frame Coding– Inter-Frame Coding Video Quality Video Coding Standards Audio Compression2 Each cell is a monochromedetector A filter array is used to makethem sensitive to different colorfrequencies More green cells are used thanred and blue cells becausehuman vision is more sensitiveto green frequencies This is called a Bayer pattern5Image ProcessingVideo Basics3 Image processing internal to the cameraconverts the Bayer pattern to the desiredcamera output format Demosaicing and Interpolating algorithmsare used to generate the output pixels Typical outputs include RGB and YCrCb6
Video InterfaceAnalog Composite Video Formats National Television Standard Committee (NTSC) Converts the camera’s pixel array data to anoutput signal Horizontal/Vertical scanning of image array Parallel to Serial conversion of pixel data Synchronization, timing and status insertion Electrical signal generation–––––Interlaced: 2 fields/frame525 lines/frame, 262.5 lines/field29.97 frames/second, 59.94 fields/second63.5 uSeconds/lineColor burst: 3.58 MHz Phase Alternate Line (PAL)–––––Interlaced: 2 fields/frame625 lines/frame, 312.5 lines/field25 frames/second, 50 fields/second64 uSeconds/lineColor burst: 4.43 MHz Séquentiel couleur à mémoire (SECAM)710Digitized Composite Video FormatsAnalog Composite Video Waveform NTSC––––720 horizontal x 480 vertical pixels2:1 Interlaced29.97 Frames per Second (59.94 Fields per Second)165.888 M Bits per Second PAL––––8720 horizontal x 576 vertical pixels2:1 Interlaced25 Frames per Second (50 Fields per Second)165.888 M Bits per Second11Other Analog FormatsAnalog Video Image S-Video––––Y – Luminance SignalC – Chrominance SignalEliminates need for color separation filteringImproved quality over Composite Video YPrPb Component––––Y – Luminance SignalPr – Red-Green Color Difference SignalPB – Blue-Green Color Difference SignalSupports High Definition Resolutions RGB Component––––912Red, Green, and Blue SignalsSeparate Sync signalSync on GreenPrimarily a computer monitor format
Digital Broadcast Interfaces Serial Digital Interface– SMPTE 259 SDI (270Mbps)New Digital Broadcast Interfaces New Serial Digital Interfaces– SMPTE 2081 6G-SDI (5.94Gbps) 480i30 (720x480) 576i25 (720x576) 4Kp30 (3840x2160)– SMPTE 2082 12G-SDI (11.88Gbps)– SMPTE 292 HD-SDI (1.485Gbps, 1.485/1.001Gbps) 720p60 (1080x720) 1080i30 (1920x1080) 1080p30 (1920x1080) 4Kp60 (3840x2160)– SMPTE 2083 24G-SDI (23.76Gbps) 8Kp30 (7680x4320)– SMPTE 424 3G-SDI (2.970Gbps, 2.970/1.001Gbps) 1080p60 (1920x1080)1316Serial Digital Interface LineBroadcast Interface Fixed resolutions andframe rates Unidirectional interface No control channel Fire and forget Format indicated withinthe signal Coax cable No trigger capability No camera power14Machine Vision Interface Random resolutions andframe rates Bidirectional interface Control channel Often requires handshake Format configured incontrol channel Most use complex cables Triggers Camera power17Serial Digital Interface FrameMachine Vision Interfaces GigE Vision– Uncompressed video over UDP on 1-Gigabit and 10-GigabitEthernet networks– Cat5 cabling, Ethernet infrastructure– 125MB/s (1250MB/s) transfer rate USB3 Vision– Same API as GigE Vision over USB 3.0 connections1518
Interlaced Scanning FormatMachine Vision Interfaces (Cont)1 CoaXPress2642– 6.25 Gbps over coax cable– Group multiple cables (4 cables 25 Gbps)– Future expansion to 10 and 12.5 GbpsRetrace2653Odd fieldscan line CameraLink––––26-pin Mini-D ribbon connectorLVDS signal pairs, 2.04 GbpsGroup multiple cables (2 cables 4.08 Gbps)Video data, discrete signals, control channel19524262525Even fieldscan line26322Progressive Scanning FormatMachine Vision Interfaces (Cont)1 IEEE-1394 FireWire23– Tree bus topology, bus master negotiationsand self identification– 400 and 800 Mbps (1600 and 3200 Mbpsdefined but not widely supported).– 4, 6 and 9 conductor connectorsScan line DVI/HDMI20––––52219 pin connector48 Gbps data rateI2C Control channelCopy Protection negotiation52352452523Interlace vs ProgressiveVideo Resolutions21Retrace4524
Interlace vs ProgressiveVideo Compression2528Color Spaces Monochrome Red, Green, Blue (RGB)– Separate sync, Sync on Green YIQ (NTSC, PAL) YUV YPbPr (Analog), YCbCr (Digital)The need for Video Compression Digital Transmission.– Need to match video data rate to digitalcommunications system bandwidth. Digital Storage.– Need to match video data rate to digital storagesystem bandwidth.– Need to reduce storage capacity or increase storagetime.– Pb/Cb Blue - Luma– Pb/Cr Red - Luma HSV, HSB, HSL CMYK26 Multiplexing29– Send more programs or other data over the samechannel.Other Characteristics Pixel Bit DepthVideo Data Rates Uncompressed SD Video– 720x480, 30 fps, 16 bpp– 8, 10, 12, 14, 16 bits per component Chroma Sampling 166 Mbps Uncompressed HD Video– 1920x1080, 60 fps, 16 bpp– 4:4:4, 4:2:2, 4:2:0, 4:0:0 1,990 Mbps Uncompressed UHD Video– 7680x4320, 120 fps, 16 bpp 63,701 Mbps2730
Types of CompressionDigital Transmission Rates Lossless– Output image is numerically identical to the original imageon a pixel-by-pixel basis.– Only statistical redundancy is reduced– Compression ratio is usually low – 2:1 to 4:1– Reversible (infinite compress/decompress cycles) Lossy– Output image is numerically degraded relative to theoriginal.– Statistical and perceptual redundancy is reduced– High compression due to reduction of perceptualredundancy– Can be visually lossless– Irreversible (compress/decompress degrades images) Transmission System Data Rates––––––––Ethernet: 10/100 Mbps, 1/10 GbpsOC12: 622 MbpsOC3: 155 MbpsDS3: 45 MbpsT1: 1.544 MbpsDSO: 64 KbpsModem: 33.6 KbpsCellular: 96003134Video Compression StandardsDigital Storage Capacity Uncompressed SD Video: 166 Mbps– 1 Min. Clip 1.24 G Bytes– 30 Min. TV Show 37.3 G Bytes– 2 Hour Movie 149.3 G Bytes Uncompressed HD Video: 1,990 MbpsPRE-PROCESS- Noise Filter- Scaling ENCODEFunctions Standardized– Bit stream syntax– How the decoder interprets the bitstream– 1 Min. Clip 14.9 G Bytes– 30 Min. TV Show 447.8 G Bytes– 2 Hour Movie 1.79 T Bytes– 1 Min. Clip 477.8 G Bytes– 30 Min. TV Show 14.3 T Bytes– 2 Hour Movie 57.3 T BytesDigital Storage DevicesFunctions Not Standardized– Pre-processing Filtering, scaling, noise reduction– Encoding strategyMode SelectionQuantizer SelectionBlock Pattern SelectionMotion Estimation Scaling, block filtering, errorconcealmentPreprocessing (1/5) Noise Reduction– Filter out high frequency information andimproves compression efficiency– Spatial Noise Reduction– 185 to 870 M Bytes (various formats) DVD– 1.46 to 17.08 G Bytes (SS, SD, SL, DL) Reduces high frequency information within apicture BluRay Disk– 25 G Bytes (50 GB for dual layer disks)– Temporal Noise Reduction Disk Drives Reduces high frequency information betweenframes– 1 T Byte33POSTPROCESS- SignalEnhancement- Filter- ErrorConcealment– Post-filter35 CD DECODE Uncompressed UHD Video: 63,701 Mbps32BITSTREAM36
Preprocessing (2/5)Preprocessing (5/5) Color representation Resolution Reduction– Reduces the amount of spatial informationthat needs to be compressed– Reduces the amount ofspatial information thatneeds to becompressed– Scaling Maintains field of s/PixelY40Preprocessing (3/5)Cr CbIntra-Frame Coding Resolution Reduction Removes the redundancies within a frame Each frame is encoded separately withoutregard to adjacent frames Similar to JPEG– Reduces the amount ofspatial information thatneeds to becompressed– Cropping Maintains pixelresolution3841Preprocessing (4/5)Encoding Stages Frame Rate Reduction– Reduces the amount of temporal informationthat needs to be compressed12345 Signal Analysis– Transform from spatial domain to anotherdomain that is easier to compress6 QuantizationX Variable Length Coding– Discard less important informationVideo Sequence – 30 Frames per Second1X3X5– Last stage of efficient codingVideo Sequence – 15 Frames per Second3942
Transform Coding by DiscreteCosine Transform (DCT)Video Compression SystemSignalAnalysisINTRAFRAME- Predictive 1D, 2D predictor Fixed, adaptive- Block transform DCT Karhunen LoeveUncompressed Hadamardinput Fouriere.g. 8 bits/pixel Haar- Sub-band filtering Unconstrainedrectangular blocks QMF, other filters Wavelets Pyramidal- Block patternmatching (VQ)- FractalsQuantization- Variable precision- Predictionerror- Transformcoefficients- Filter energylevels- Visibility matrixSegmentation of a frameinto blocks of pixelsVariable length coding(entropy coding)- Fixed Huffman B Comma 2 dimensional(run length/COEFF)COMPRESSEDOUTPUT- Fixed, adaptive- Lossy, lossless- Scalar, vector- Adaptive Conditional Arithmetic One, two passes8x8Coefficients8x8 Pixels- LosslessDCTINTERFRAME- Application ofintraframe techniques- Motion estimation- 3-D transform4346DCT Basis FunctionSignal Analysis PhaseSignalAnalysisINTRAFRAME- Predictive 1D, 2D predictor Fixed, adaptive- Block transform DCT Karhunen LoeveUncompressed Hadamardinput Fouriere.g. 8 bits/pixel Haar- Sub-band filtering Unconstrainedrectangular blocks QMF, other filters Wavelets Pyramidal- Block patternmatching (VQ)- FractalsQuantization- Variable precision- Predictionerror- Transformcoefficients- Filter energylevels- Visibility matrixVariable length coding(entropy coding)- Fixed Huffman B Comma 2 dimensional(run length/COEFF)COMPRESSEDOUTPUT- Fixed, adaptive- Lossy, lossless- Scalar, vector- Adaptive Conditional Arithmetic One, two passes- LosslessINTERFRAME- Application ofintraframe techniques- Motion estimation- 3-D transform4447Effect of DCT CoefficientsDiscrete Cosine Transform Converts spatial information to frequencyinformation Important information is concentrated inlower frequency bands Operations typically performed on 8x8blocks of pixels Lossless and reversible4548
More EffectsQuantization Converts 12 bit DCT coefficients to 8 bitvalues Try to force less important informationtoward zero values Takes less bits to code zero Most of loss occurs here4952Scalar QuantizationCoefficient LevelsOutput432-1024 -768 -512 -256 10-1256512768-453Scanning Order in a BlockQuantization Phase1341011212236SignalAnalysisINTRAFRAME- Predictive 1D, 2D predictor Fixed, adaptive- Block transform DCTUncompressed Karhunen Loeve Hadamardinput Fouriere.g. 8 bits/pixel Haar- Sub-band filtering Unconstrainedrectangular blocks QMF, other filters Wavelets Pyramidal- Block patternmatching (VQ)- FractalsQuantization- Variable precision- Predictionerror- Transformcoefficients- Filter energylevels- Visibility matrix- Fixed, adaptive- Lossy, lossless- Scalar, vectorVariable length coding(entropy coding)- Fixed Huffman B CommaCOMPRESSED 2 dimensional(run length/COEFF) OUTPUT- Adaptive Conditional Arithmetic One, two passes- LosslessINTERFRAME- Application ofintraframe techniques- Motion estimation- 3-D transform51InputQuantize 803 3Inv Quant 3 768Quantization Error803 – 768 35660632943445455616264
Entropy Coding PhaseHuffman Variable Length CodingSignalAnalysisINTRAFRAME- Predictive 1D, 2D predictor Fixed, adaptive- Block transform DCT Karhunen LoeveUncompressed Hadamardinput Fouriere.g. 8 bits/pixel Haar- Sub-band filtering Unconstrainedrectangular blocks QMF, other filters Wavelets Pyramidal- Block patternmatching (VQ)- FractalsQuantization- Variable precision- Predictionerror- Transformcoefficients- Filter energylevels- Visibility matrixVariable length coding(entropy coding)SymbolProbabilityVLC Code- Fixed Huffman B Comma 2 dimensional(run length/COEFF)A½000B¼1001COMPRESSEDOUTPUT- Fixed, adaptive- Lossy, lossless- Scalar, vectorFixed Length Code- Adaptive Conditional Arithmetic One, two passesC1/811010- LosslessD1/811111For 128 symbols: Fixed Length 256 bitsINTERFRAME- Application ofintraframe techniques- Motion estimation- 3-D transform55Variable Length 224 bits (12.5% savings)58Sample Intra Block CodingVariable Length Coding A technique for assigning shorter codes tohigh probability symbols, and longer codes tolower probability symbols Entropy Coding, Huffman Encoding,Arithmetic Coding Modest compression gain (2:1-4:1) Lossless 184868889-10000-1-10000-100-1-1-1-10-10-1-10688 -21-39 0000000000000-1-10-1000-1-200-10-100000-100-1-1b) TRANSFORM 0000000000000000000000000000000000000c) QUANTIZED COEFFICIENTS59VLC 9596f) RECONSTITUTED BLOCKa) ORIGINAL PIXELS684 -19-37 000000000000000000000000InverseDCT00000000e) INVERSE QUANTIZED COEFFICIENTSRUN LEVEL CODE8601010110-3001011-6001000011EOB 10TOTAL CODE LENGTH 25InverseQuantized) VARIABLE LENGTH CODINGVariable Length CodeInter-Frame Coding Removes the redundancies betweenframes Uses one or more previous or futureframes as a prediction of the current frame Produces a difference between the currentframe and the prediction Only encode the differences5760
Encoding StagesDifference Image Signal Analysis– Transform from spatial domain to another domain that is easierto compress Quantization– Discard less important information Variable Length Coding– Last stage of efficient coding Prediction– Create a prediction for the frame being encoded and onlycompress the difference6164Generic Inter-Frame Coding SystemPredictionInput-VideoErrorDCT andQuantizerDifference Image HistogramsOutputDataPredictedPredictorSignal Inverse DCTand Quantizer6265Pictures TypesOriginal Image Histograms Intra Frame (I)– Full picture coded independent of other frames– Key frame Predicted Frame (P)– Inter coded frame predicted from a previous frame Bidirectional Frame (B)– Inter coded frame predicted from a future frame GOP– # of P/B frames between I frames– # of B frames between I/P frames– Open or Closed GOP6366
Improved Prediction with MotionVectorsGroup of Pictures (GOP)Transmission Order: IPBBPBB6770Motion Vector ExamplePicture Type Efficiency I Frames: 1 bit per pixelP Frames: 0.25 bits per pixelB Frames: 0.03 bits per pixelAll I Frames: 10MbpsIP15: 6 MbpsIPB15: 2.5 Mbps6871Improving Prediction Objects in a scene do not usually changeinstantaneouslyBlock Sizes H.261/H.263 use 16x16 block for motionestimation H.264 uses variable block sizes– Objects move– Camera pans and zooms1 macroblock partition of16*16 luma samples andassociated chroma samples Improve prediction by accounting for themotion of blocks Extremely computation intensiveMacroblockpartitions2 macroblock partitions of16*8 luma samples andassociated chroma samples001 sub-macroblock partitionof 8*8 luma samples andassociated chroma samplesSub-macroblockpartitions2 sub-macroblock partitionsof 8*4 luma samples andassociated chroma samples722 sub-macroblock partitionsof 4*8 luma samples andassociated chroma samples0004 sub-macroblocks of8*8 luma samples andassociated chroma samples0123111692 macroblock partitions of8*16 luma samples andassociated chroma samples04 sub-macroblock partitionsof 4*4 luma samples andassociated chroma samples01231
Block Size ExampleVideo Quality7376Intra-Frame PredictionRate Control Algorithm (1/2) 4x4 intra prediction modes This is the encoder magic sauce Determines how to code each macro-block Rate Control Modes– Constant Quality (CQ) – Attempt to product a consistent videoquality regardless of the data rate required– Constant Bit Rate (CBR) – Attempt to product the best qualityvideo while holding the data rate constant. May use fill data tomaintain the constant data rate. May still be some minorvariability in data rate.– Variable Bit Rate (VBR) – Attempt to produce the best qualityvideo while not exceeding some maximum data rate, butallowing the data rate to drop when not needed Also 16x16 intra prediction modes7477PerformanceRate Control Algorithm (2/2) For VBR and CBR, encoder attempts to use thefinest quantizer (sharpest quality pictures) If the encoder is generating too much data forthe selected data rate, the encoder increasesthe coarseness of the quantizer up to themaximum user selected value (picture qualitydegrades) If the encoder is still generating too much datafor the selected data rate, the encoder beginsdropping frames (reduced frame rate)7578
Video Quality EffectsEncoder Trade-offs Video Frame Rate Multidimensional trade-off space– Data rate– Resolution– Frame Rate– Quality– Latency– Error resilience79– Reduced Frame Rate– Jerky Video (irregular frame rate) Video Artifacts––––––Multi-color blocksBlack or white stripesTearing or melting imagesGhosts and shadows behind moving objectsBlock edgesEdge noise around sharp edges Decoder Issues– Loss of sync, cannot detect start codes– Video freeze– Decoder crash82Compression RatioCompression Artifacts - Blockiness Old compression algorithms had a fixed bits per pixeland therefore had a defined Compression Ratio. New compression algorithms have a variable number ofbits per pixel depending on a number of factors, mostnotable is picture quality.– 1000:1 compression is very possible, but picture quality will beunusable– 100:1 is reasonable Compression efficiency and picture quality are alsohighly dependent on scene complexity.8083Typical Video Quality ProblemsCompression Artifacts - Blockiness Data rate too low to support desired quality– Communications link bandwidth– Codec efficiency– Resolution and frame rate Too many errors– Bit errors– Packet loss– Burst errors8184
Compression Artifacts - Blurriness85Error Artifacts - I-Frame Data88Compression Artifacts - Edge Noise86Error Artifacts – P/B-Frame Data89Compression Artifacts - Edge Noise87Error Artifacts – P/B Frame Data90
Packet Loss Artifacts1% Packet Loss Rate5% Packet Loss Rate91Error Concealment Features Decoder functions to reduce the effect of errors Intra-frame concealment – use other pixelswithin the frame to predict the value of themissing pixels Inter-frame concealment – use pixels fromprevious frames to predict the value of themissing pixels Estimation of missing motion vectors94Compressed Data Error SensitivityError Resilience Schemes Sparse Start Codes – usually at thebeginning of a picture Variable length codes – a bit error not onlychanges the value, but also the length ofthe code and the following values Inter-frame coding propagates an errorover the rest of the GOP 9295Error Resilience Features GOP Structure – distance between I-Frames Intra Refresh – Intra-coded macroblocksdistributed throughout long GOPs Slices – partitioning image into sub images withre-sync points Flexible Macroblock Ordering, Arbitrary SliceOrdering, Redundant Slices Forward Error Correction SchemesFlexible Macroblock Ordering (FBO)Arbitrary Slice Ordering (ASO)Data Partitioning (DP)Redundant Slices (RS)SP/SI Frames for bit rate switchingReference Frame SelectionIntra-block refreshingVideo Encoder Settings Video Resolution– Autodetect– Scaling/Cropping Frame Rate Decimation GOP Size/Structure– Infinite GOP 0 (only P-Frames)– GOP 1(only I-Frames)– Normal GOP 2 to N (I and P-frames) Quantizer Settings9396
Contact InformationVideo Encoder Settings747 Dresher RoadHORSHAM, PA 19044Ph: 215-657-5270FAX: 215-657-5273www.deltadigitalvideo.com Slice Modes––––Error RecoveryFull FrameMacrobl
An Introduction to Video Compression 2 AGENDA Video Basics – Analog Video – Digital Video – Scanning Formats Video Compression – Intra-Frame Coding . –Need to match video data rate to digital storage system bandwidth. –Need to reduce storage capacity or increase storage time.
Using Cross Products Video 1, Video 2 Determining Whether Two Quantities are Proportional Video 1, Video 2 Modeling Real Life Video 1, Video 2 5.4 Writing and Solving Proportions Solving Proportions Using Mental Math Video 1, Video 2 Solving Proportions Using Cross Products Video 1, Video 2 Writing and Solving a Proportion Video 1, Video 2
Introduction Lossless Compression Algorithms There will be no data loss in this type of compression as it is defined by the name. Both original data and the compressed data are the same in this compression. The algorithms for the compression and decompression are exact inverse of each other in the Lossless Compression. The main mechanism in
2) A new neural codec for video compression. We employ the above paradigm towards building an end-to-end trainable codec. To the best of our knowledge, this is the first work to utilize a deep generative video model together with discretization and entropy coding to perform video compression. 3) High compression ratios.
4 COMPRESSION THERAPY BANDAGES Comprilan – Short Stretch Compression Bandage 24 JOBST Compri2 – 2-Layer Bandage System 25 JOBST Comprifore LF – Multi-Layer Compression Bandage 25 Tensopress – Long Stretch Compression Bandage 26 Gelocast – Zinc Paste Compression Bandage (Unna Boot) 26 COMPRESSION SYSTEMS JOBST UlcerCARE – Ready
Image Compression Model Image compression reduces the amount of data from the original image representation. There are two approaches to compress an image. These are: (a) Lossless compression (b) Lossy compression Fig.2.2 shows a general image compression model. Image data representation has redundancy (also called pixel
The result is a lower compression ratio for video sequences compared to "real" video compression techniques like MPEG. The benefit is its robustness with no dependency between the frames, which means that, for example, even if one frame is dropped during transfer, the rest of the video will be un-affected. JPEG 2000
How to edit the imported video on Mac Leawo Video onverter for Mac can also serve as a video editor to help users make a unique video. This page will tell the detailed steps of how to edit a video on mac. Five main video editing function will meet all your basic needs to edit a video. 1. Edit video on Mac . Import the file
enFakultätaufAntragvon Prof. Dr. ChristophBruder Prof. Dr. DieterJaksch Basel,den16. Oktober2012, Prof. Dr .
