A Multi-objective Optimization Framework for Subjectively Optimized Stereoscopic Video Coding
M.A. Ismail, E.A. Edirisinghe
Department of Computer Science, Loughborough University, UK
M.A. Ismail, E.A. Edirisinghe
Department of Computer Science, Loughborough University, UK
ABSTRACT: In the recent past many successful research attempts have been made in proposing video CODECs that are capable of efficiently coding stereoscopic video sequences. Of these, CODECs that extend the use of international monoscopic video coding standards such as MPEG-4, H.264 AVC/SVC to stereoscopic video sequence coding, play a significant role due to the use of optimized coding algorithms directly derived from monoscopic video coding standards [1]. These research efforts can be classified as attempts to extend the use of efficient monoscopic video coding algorithms to stereoscopic video coding, rather than efforts that aim to tailor the CODEC’s design to best fit requirements of stereoscopic vision. In the proposed research we aim to address two fundamental questions in the design of a stereoscopic video CODEC. Firstly, given the design of a stereoscopic video CODEC, which coding parameters (both encoder and decoder specific) have the most significant effect on rate, distortion, CPU cycles and memory usage? Secondly, given the significant parameters, how can we design an optimization framework where multiple constraints/objectives, i.e. rate, distortion, CPU cycles and memory related, can be met. A specific feature of importance of the proposed approach is that the distortion is measured using a subjective metric (in a scale of 1-5) rather than using the objective metric, PSNR, which often fails to meet important subjective needs of a stereoscopic entertainment system [2].
We carry out the experiments on a state-of-the-art, H.264 AVC based stereoscopic video CODEC [1]. Comprehensive profiling tests are carried out to determine the most significant encoder/decoder parameters (QP, number of reference frames, resolution, use of fast motion estimation, search range, intra period, slice group structure, intra mode prediction etc.) that affect rate, subjective quality, memory usage, CPU cycles. The experiments are performed on three stereoscopic video sequences having different statistical properties. The profiling tests for the CPU usage were carried out using Intel’s VTune performance analyser. Subjective tests were designed using a stereoscopic video system that displays stereo video on shutter-glasses controlled by an IR-emitter. Twenty subjects were used in a controlled environment for subjective quality profiling tests. The profiling results are initially used to derive objectives functions for each objective/constraint. These objective functions are subsequently used within a multi-objective optimization strategy that adopts a Genetic Algorithm (GA) to produce optimal results. We use the well established GA software tool, Non-dominated Sorting Genetic Algorithm (NSGA-II) for this purpose. The final outcome of the optimisation is the determination of coding parameter values that will jointly optimize rate, subjective quality, CPU and memory usage of the given CODEC. We show that the proposed framework can be extended to optimize any given design of a stereoscopic video CODEC and has the feasibility to be further extended to include further constraints, such as networking and data transmission constraints such as packet loss and delay. Instead of the usual approach of jointly optimising rate-distortion only, the consideration of memory usage and CPU cycle constraints enables the framework to be used in determining the optimal coding parameters for stereoscopic videos to be displayed in multi-modal devices. The research also aims to bridge a significant research gap in stereoscopic image coding, namely the subjective quality optimisation. We present some interesting experimental results in support of our conclusions.
This paper will be submitted to the The 9th IASTED International Conference on Visualization, Imaging & Image Processing 2009 that will be held on 13 - 15 July 2009, at Cambridge, UK.
We carry out the experiments on a state-of-the-art, H.264 AVC based stereoscopic video CODEC [1]. Comprehensive profiling tests are carried out to determine the most significant encoder/decoder parameters (QP, number of reference frames, resolution, use of fast motion estimation, search range, intra period, slice group structure, intra mode prediction etc.) that affect rate, subjective quality, memory usage, CPU cycles. The experiments are performed on three stereoscopic video sequences having different statistical properties. The profiling tests for the CPU usage were carried out using Intel’s VTune performance analyser. Subjective tests were designed using a stereoscopic video system that displays stereo video on shutter-glasses controlled by an IR-emitter. Twenty subjects were used in a controlled environment for subjective quality profiling tests. The profiling results are initially used to derive objectives functions for each objective/constraint. These objective functions are subsequently used within a multi-objective optimization strategy that adopts a Genetic Algorithm (GA) to produce optimal results. We use the well established GA software tool, Non-dominated Sorting Genetic Algorithm (NSGA-II) for this purpose. The final outcome of the optimisation is the determination of coding parameter values that will jointly optimize rate, subjective quality, CPU and memory usage of the given CODEC. We show that the proposed framework can be extended to optimize any given design of a stereoscopic video CODEC and has the feasibility to be further extended to include further constraints, such as networking and data transmission constraints such as packet loss and delay. Instead of the usual approach of jointly optimising rate-distortion only, the consideration of memory usage and CPU cycle constraints enables the framework to be used in determining the optimal coding parameters for stereoscopic videos to be displayed in multi-modal devices. The research also aims to bridge a significant research gap in stereoscopic image coding, namely the subjective quality optimisation. We present some interesting experimental results in support of our conclusions.
This paper will be submitted to the The 9th IASTED International Conference on Visualization, Imaging & Image Processing 2009 that will be held on 13 - 15 July 2009, at Cambridge, UK.
