Adaptive CU Mode Selection in HEVC Intra Prediction: A Deep Learning Approach



Journal Title

Journal ISSN

Volume Title


Birkhauser Boston


The computational time of HEVC encoder is increased mainly because of the hierarchical quad-tree-based structure, recursive coding units, and the exhaustive prediction search up to 35 modes. These advances improve the coding efficiency, but result in a very high computational complexity. Furthermore, selecting the optimal modes among all prediction modes is necessary for subsequent rate-distortion optimization process. Therefore, we propose a convolution neural network-based algorithm which learns the region-wise image features and performs a classification job. These classification results are later used in the encoder downstream systems for finding the optimal coding units in each of the tree blocks, and subsequently reduce the number of prediction modes. The experimental results show that our proposed learning-based algorithm reduces the encoder time saving up to 66.89% with a minimal Bjøntegaard delta bit rate (BD-BR) loss of 1.31% over the state-of-the-art machine learning approaches. Furthermore, our method also reduces the mode selection by 45.83% with respect to the HEVC baseline. ©2019, Springer Science + Business Media, LLC, part of Springer Nature.


Due to copyright restrictions full text access from Treasures at UT Dallas is restricted to current UTD affiliates (use the provided Link to Article).


Neural networks (Computer science)--Convolutional, Coding theory, Electric distortion, Image segmentation, Machine learning


©2019 Springer Science + Business Media, LLC, part of Springer Nature.