HEVC核心编码技术之六.环路滤波，SAO及其它

Overview of
the High Efficiency Video Coding(HEVC) Standard之四

K. 环内滤波

In-Loop Filters

​

In HEVC, two processing steps, namely a deblocking

filter (DBF) followed by an SAO filter, are applied to the

reconstructed samples before writing them into the decoded

picture buffer in the decoder loop. The DBF is intended

to reduce the blocking artifacts due to block-based coding.

在HEVC中，对要写入解码环的DPB的重建像素使用了两个处理：

先SAO滤波，再去块效应滤波；

DBF的目的是为了减少基于块编码而人工产生的块效应；

The DBF is similar to the DBF of the H.264/MPEG-4 AVC

standard, whereas SAO is newly introduced in HEVC. While

the DBF is only applied to the samples located at block

boundaries, the SAO filter is applied adaptively to all samples

satisfying certain conditions, e.g., based on gradient. During

the development of HEVC, it had also been considered to

operate a third processing step called the adaptive loop filter

(ALF) after the SAO filter; however, the ALF feature was not

included in the final design.

HEVC的DBF和H.264/MPEG-4 AVC相似，而SAO则是新引入的；

因为DBF只用于处理块边界上的像素，

而SAO滤波器可用于满足某种统计条件(如基于梯度图)的所有像素；

1) 去块效应滤波

Deblocking Filter:

The deblocking filter is applied to

all samples adjacent to a PU or TU boundary except the case

when the boundary is also a picture boundary, or when deblocking

is disabled across slice or tile boundaries (which is an

option that can be signaled by the encoder). It should be noted

that both PU and TU boundaries should be considered since

PU boundaries are not always aligned with TU boundaries in

some cases of interpicture-predicted CBs. Syntax elements in

the SPS and slice headers control whether the deblocking filter

is applied across the slice and tile boundaries.

除了当边界位于图像边界，或对跨片，或瓦片的去块效应滤波被关闭，

去块效应滤波被用于所有PU或TU边界相邻的像素；

应当注意的是，PU和TU边界都应考虑，

因为PU边界在帧间预测CB中并不是总和TU边界对齐的；

在SPS和片头中有语法元素提示去块效应滤波是否用于跨片和瓦片的边界；

Unlike H.264/MPEG-4 AVC, where the deblocking filter

is applied on a 4×4 sample grid basis, HEVC only applies

the deblocking filter to the edges that are aligned on an 8×8

sample grid, for both the luma and chroma samples. This

restriction reduces the worst-case computational complexity

without noticeable degradation of the visual quality. It also

improves parallel-processing operation by preventing cascading

interactions between nearby filtering operations.

和H.264/MPEG-4 AVC只对4x4像素块边界用去块效应滤波不同，

HEVC只对8x8像素块的亮度和色度的边界使用DBF；

这种限制减少了计算复杂度且并没有对图像质量有很大的降低；

而且能提高并处处理效率；

The strength of the deblocking filter is controlled by the

values of several syntax elements similar to the scheme in

H.264/MPEG-4 AVC, but only three strengths are used rather

than five. Given that P and Q are two adjacent blocks with a

common 8×8 grid boundary, the filter strength of 2 is assigned

when one of the blocks is intrapicture predicted. Otherwise,

the filter strength of 1 is assigned if any of the following

conditions is satisfied.

去块效率的滤波强度参数控制和H.264/MPEG-4 AVC相邻，

但参数个数减少到了3个，

计算方式如下：

1) P or Q has at least one nonzero transform coefficient.

P或Q至少有一个非零变换系数

2) The reference indices of P and Q are not equal.

P和Q的参考索引不相等

3) The motion vectors of P and Q are not equal.

P和Q的MV不相等

4) The difference between a motion vector component of

P and Q is greater than or equal to one integer sample.

P和Q的MV分量的差值大于或等于一个整像素

​

If none of the above conditions is met, the filter strength of 0

is assigned, which means that the deblocking process is not

applied.

如果上述的条件都不满足，则滤波强度为0，意味着不需要进行去块效应处理；

According to the filter strength and the average quantization

parameter of P and Q, two thresholds, tC and β, are determined

from predefined tables. For luma samples, one of three cases,

no filtering, strong filtering, and weak filtering, is chosen based

on β. Note that this decision is shared across four luma rows

or columns using the first and the last rows or columns to

reduce the computational complexity.

依据滤波强度，P和Q的平均量化参数，可以在预定义的表中确定两个阈值：tC and β；

对于亮度像素，基于β选择下列三种情况中的一种：

不滤波，

强滤波，

弱滤波。

并且这个选择可以跨过四个亮度行和列，在第一和最后一行或列共用，以减少计算复杂度；

There are only two cases, no filtering and normal filtering,

for chroma samples. Normal filtering is applied only when the

filter strength is greater than one. The filtering process is then

performed using the control variables tC and β.

对于色度像素，只有不滤波和正常滤波两种情况；

正常滤波只有在滤波强度大于1时使用，滤波处理使用控制变量tC and β来处理；



Fig. 10. Four gradient patterns used in SAO. Sample labeled “p” indicates

a center sample to be considered. Two samples labeled “n0” and “n1”

specify two neighboring samples along the

(a) horizontal (sao_eo_class = 0),

(b) vertical (sao_eo_class = 1),

(c) 135° diagonal (sao_eo_class = 2), and

(d) 45° (sao_eo_class = 3) gradient patterns.

In HEVC, the processing order of the deblocking filter is

defined as horizontal filtering for vertical edges for the entire

picture first, followed by vertical filtering for horizontal edges.

在HEVC中，DBF的处理顺序是

先对整个图像的垂直边进行水平滤波；

再对水平边界进行垂直滤波；

This specific order enables either multiple horizontal filtering

or vertical filtering processes to be applied in parallel threads,

or can still be implemented on a CTB-by-CTB basis with only

a small processing latency.

这种方式是很方便在并行处理架构同时进行多水平行或多垂直列的滤波；

也可以逐个CTB进行来减少时延；

2) SAO:

SAO is a process that modifies the decoded

samples by conditionally adding an offset value to each sample

after the application of the deblocking filter, based on values

in look-up tables transmitted by the encoder. SAO filtering is

performed on a region basis, based on a filtering type selected

per CTB by a syntax element sao_type_idx. A value of 0 for

sao_type_idx indicates that the SAO filter is not applied to

the CTB, and the values 1 and 2 signal the use of the band

offset and edge offset filtering types, respectively.

In the band offset mode specified by sao_type_idx equal

to 1, the selected offset value directly depends on the sample

amplitude. In this mode, the full sample amplitude range is

uniformly split into 32 segments called bands, and the sample

values belonging to four of these bands (which are consecutive

within the 32 bands) are modified by adding transmitted values

denoted as band offsets, which can be positive or negative. The

main reason for using four consecutive bands is that in the

smooth areas where banding artifacts can appear, the sample

amplitudes in a CTB tend to be concentrated in only few of

the bands. In addition, the design choice of using four offsets

is unified with the edge offset mode of operation which also

uses four offset values.

SAO是在去块效应之后，基于由编码器传输的查找表的值，

依据条件给每个解码后的像素加一个偏移值来改变像素值的处理；

SAO滤波基于每个CTB的语法元素sao_type_idx来选择滤波类型，并且基于区域执行；

sao_type_idx 等于 0时，表示不对CTB使用SAO滤波器；

sao_type_idx 等于 1时，表示使用频带偏移滤波类型，偏移值的选择依靠于像素幅值。

sao_type_idx 等于 2时，表示使用边界偏移滤波类型；

在频带偏移滤波类型中，事个像素幅度的范围被统一划分成32个段，称为频带，

如果像素值属于这些频带中的四个，则加上变换值，称为频带偏移，来修改，

这个偏移值可以是正或负；

使用四个连续频带的理由是在平滑的区域，频带痕迹就会出现，

在CTB中的像素幅值会趋向于集中在少数几个频带；

In the edge offset mode specified by sao_type_idx equal

to 2, a syntax element sao_eo_class with values from 0 to 3

signals whether a horizontal, vertical or one of two diagonal

gradient directions is used for the edge offset classification

in the CTB. Fig. 10 depicts the four gradient patterns used

for the respective sao_eo_class in this mode. Each sample in

the CTB is classified into one of five EdgeIdx categories by

comparing the sample value p located at some position with

the values n0 and n1 of two samples located at neighboring

positions as shown in Table IV. This classification is done for

each sample based on decoded sample values, so no additional

signaling is required for the EdgeIdx classification. Depending

on the EdgeIdx category at the sample position, for EdgeIdx

categories from 1 to 4, an offset value from a transmitted

look-up table is added to the sample value. The offset values

are always positive for categories 1 and 2 and negative for

categories 3 and 4 - thus the filter generally has a smoothing

effect in the edge offset mode.

sao_type_idx等于2时，使用边界偏移模式；

语法元素sao_eo_class等于0到3，表示对CTB使用水平，垂直，还是对角线方向的边界偏移；



TABLE IV

Sample EdgeIdx Categories in SAO Edge Classes

EdgeIdx Condition Meaning

0 Cases not listed below Monotonic area

1 p < n0 and p < n1 Local min

2 p < n0 and p = n1 or p < n1 and p = n0 Edge

3 p > n0 and p = n1 or p > n1 and p = n0 Edge

4 p > n0 and p > n1 Local max

Thus, for SAO types 1 and 2, a total of four amplitude

offset values are transmitted to the decoder for each CTB. For

type 1, the sign is also encoded. The offset values and related

syntax elements such as sao_type_idx and sao_eo_class are

determined by the encoder - typically using criteria that

optimize rate-distortion performance. The SAO parameters can

be indicated to be inherited from the left or above CTB using

a merge flag to make the signaling efficient. In summary,

SAO is a nonlinear filtering operation which allows additional

refinement of the reconstructed signal, and it can enhance the

signal representation in both smooth areas and around edges.

L. 特殊编码模式

Special Coding Modes

HEVC defines three special coding modes, which can be

invoked at the CU level or the TU level.

HEVC定义三种特殊编码模式，可以用在CU级和TU级；

1) In I_PCM mode,

the prediction, transform, quantization

and entropy coding are bypassed, and the samples are

directly represented by a pre-defined number of bits. Its

main purpose is to avoid excessive consumption of bits

when the signal characteristics are extremely unusual

and cannot be properly handled by hybrid coding (e.g.,

noise-like signals).

预测，变换，量化和熵编码都旁路，像素直接表示成预定义的比特数；

它主要的目的是避免当信号特性(如噪声信号)完全不能用混合编码时的情形；

2) 无损模式

In lossless mode,

the transform, quantization, and other

processing that affects the decoded picture (SAO and

deblocking filters) are bypassed, and the residual signal

from inter- or intrapicture prediction is directly fed into

the entropy coder (using the same neighborhood contexts

that would usually be applied to the quantized transform

coefficients). This allows mathematically lossless

reconstruction, which is achieved without defining any

additional coding tools.

变换，量化和其它影响解码图像(SAO和DBF)的处理都被旁路,

帧内或帧间预测的残差信号被直接送到熵编码器；

这种方式实现了算术上的无损重建；

3) In transform skipping mode,

only the transform is bypassed.

This primarily improves compression for certain

types of video content such as computer-generated images

or graphics mixed with camera-view content (e.g.,

scrolling text). This mode can be applied to TBs of 4×4

size only.

只有变换被旁路，它主要用来压缩一些特定类型的视频内容，

如计算生成的图像或有摄像角度内容的图形。

这种模式只能用在4x4变换尺寸；

SAO and deblocking filtering are not applied to lossless mode

regions, and a flag controls whether they are applied to I_PCM

regions.

SAO和DBF不能用于无损模式，

有标志位来控制它们是否能用于I_PCM模式；

用六，利永贞。

【白话】“用六”这一爻，利于永远保持中正。

《象》曰：用六“永贞”，以大终也。

【白话】《象辞》说：用六的爻辞说“利于永远保持中正”，即是指阴盛到了极点就会向阳转化。