Compression Artifacts Removal for Improved Video Analytics



Journal Title

Journal ISSN

Volume Title



Video compression algorithms are pervasively applied at the camera level prior to video transmission due to bandwidth constraints, thereby reducing the quality of video available for automated video analytics. These artifacts may introduce undesired noise and complex structures, which remove textures and high-frequency details in video frames, leading to decreased performance of some core applications in video surveillance systems such as object detection. To improve video quality, either for human viewers or machine vision algorithms, it is required to recover original video frames from distorted ones. This means that one ideally should invert the compression process through a complicated non-linear image transformation. Most of the current research on compression artifact removal addresses eliminating a specific type of artifacts with prior knowledge of the noise distribution, typically by adding small-magnitude perturbations to input images. These artifacts present an interesting problem; however, in automated video surveillance systems, such carefully chosen noise is unlikely to be encountered. The goal of this dissertation is to develop and evaluate algorithms for the compression artifact removal task in automated video surveillance systems. Solving this problem requires addressing three separate subproblems: (i) creating a dataset representing common scenarios where video surveillance cameras are deployed, (ii) understanding the impact of video quality on computer vision algorithms, and (iii) developing and evaluating algorithms for removing adversarial perturbations stemming from compression artifacts during video acquisition. By learning to eliminate these distortions, such techniques can have a broader impact by improving video quality available for video analytics without making any changes to the existing compression pipelines. The methods introduced in this dissertation can accurately estimate the image manifold, thus producing higher video frames with finer, sharper, and consistent details. Moreover, results obtained show that our methods can be used as a pre-processing step for computer vision tasks in practical, non-idealized applications where quality distortions may be present.



Compression Artifacts, Deep Learning, Video Quality, Video Surveillance


Portions of this document appear in: Aqqa, Miloud, Pranav Mantini, and Shishir K. Shah. "Understanding How Video Quality Affects Object Detection Algorithms." In VISIGRAPP (5: VISAPP), pp. 96-104. 2019. And in: Aqqa, Miloud, and Shishir K. Shah. "CAR-CNN: A Deep Residual Convolutional Neural Network for Compression Artifact Removal in Video Surveillance Systems." In VISIGRAPP (4: VISAPP), pp. 569-575. 2020. And in: Aqqa, Miloud, and Shishir K. Shah. "CAR-DCGAN: A Deep Convolutional Generative Adversarial Network for Compression Artifact Removal in Video Surveillance Systems." In VISIGRAPP (4: VISAPP), pp. 455-464. 2021.