Screen Content Coding
Screen content videos have become popular in recent years with the advances in mobile technologies and cloud applications, such as shared screen collaboration, remote desktop interfacing, cloud gaming, wireless display, animation streaming, online education, etc. These emerging applications create an urgent demand for better compression technologies and low-latency delivery solutions for screen content videos.
To exploit the unique signal characteristics of screen content and develop efficient SC compression solutions, the ISO/IEC Moving Picture Expert Group and the ITU-T Video Coding Experts Group, also referred as the “Joint Collaborative Team on Video Coding” (JCTVC), has launched the standardization of SCC extension on top of the latest video standard – High Efficiency Video Coding (HEVC) since January 2014 and this extension is concluded in 2016 with significant research efforts involved from both academia and industry.
The official JCTVC Screen Content Model software (SCM) is reported to provide >50% BD-Rate saving over the HEVC Range Extension (RExt) for computer-generated contents. Novel coding tools and algorithms (e.g., palette coding mode, intra block copy, adaptive color transform, adaptive motion compensation precision, etc.) were introduced and adopted during the standardization.
In this project, we are collaborating with Huawei researchers and exploring fast screen content encoding and transcoding solutions to accelerate screen content compression, while simultaneously preserving the rate-distortion performance, including: (1) Fast Screen Content Encoding System Design using Machine Learning Techniques; (2) Fast HEVC-SCC Transcoding System Design using Machine Learning Techniques; and (3) Fast SCC-HEVC Transcoding System Design using Statistical Mode Mapping Techniques.
Related Publications: (P: Patent; J: Journal; C: Conference)
[P1] H. Yu, M. Xu, W. Wang, F. Duanmu, and S. Minaee, “Advanced Coding Techniques For High Efficiency Video Coding (HEVC) Screen Content Coding (SCC) Extensions”, 2016.
[J1] F. Duanmu, Z. Ma, M. Xu, and Y. Wang, “An HEVC-Compliant Fast Screen Content Transcoding Framework Based on Mode Mapping”, IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2018 (Under Review).
[J2] F. Duanmu, Z. Ma, and Y. Wang, “Fast Mode and Partition Decision Using Machine Learning for Intra-Frame Coding in HEVC Screen Content Coding Extension,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS), 2016 Aug; Vol: PP, Issue 99, Page:1-15. doi: 10.1109/JETCAS.2016.2597698.
[C1] F. Duanmu, M. Xu, Y. Wang, and Z. Ma, “HEVC-Compliant Screen Content Transcoding Based on Mode Mapping and Fast Termination,” in Proc. of IEEE Visual Communications and Image Processing (VCIP), Petersburg, Florida, USA, 2017.
[C2] F. Duanmu, Z. Ma, and Y. Wang “A Novel Screen Content Fast Transcoding Framework Based on Statistical Study and Machine Learning”, in Proc. International Conference of Image Processing (ICIP), pp. 4205-4209, Phoenix, Arizona, USA, 2016.
[C3] F. Duanmu, Z. Ma, and Y. Wang, “Fast CU partition decision using machine learning for screen content compression”, Proc. IEEE International Conference on Image Processing (ICIP), pp. 4972 – 4976, Quebec City, Canada, 2015.
[C4] Y. Xu, W. Huang, W. Wang, F. Duanmu, and Z. Ma “2-D Index Map Coding for HEVC Screen Content Expression”, Proc. Data Compression Conference (DCC), pp. 263-272, Snowbird, Utah, USA, 2015.
Last Update by Fanyi Duanmu, 02/14/2018