A Multi-objective Optimization Framework for Subjectively Optimized Stereoscopic Video Coding
M.A. Ismail, E.A. Edirisinghe
Department of Computer Science, Loughborough University, UK
ABSTRACT: In the recent past many successful research attempts have been made in proposing video CODECs that are capable of efficiently coding stereoscopic video sequences. Of these, CODECs that extend the use of international monoscopic video coding standards such as MPEG-4, H.264 AVC/SVC to stereoscopic video sequence coding, play a significant role due to the use of optimized coding algorithms directly derived from monoscopic video coding standards [1]. These research efforts can be classified as attempts to extend the use of efficient monoscopic video coding algorithms to stereoscopic video coding, rather than efforts that aim to tailor the CODEC’s design to best fit requirements of stereoscopic vision. In the proposed research we aim to address two fundamental questions in the design of a stereoscopic video CODEC. Firstly, given the design of a stereoscopic video CODEC, which coding parameters (both encoder and decoder specific) have the most significant effect on rate, distortion, CPU cycles and memory usage? Secondly, given the significant parameters, how can we design an optimization framework where multiple constraints/objectives, i.e. rate, distortion, CPU cycles and memory related, can be met. A specific feature of importance of the proposed approach is that the distortion is measured using a subjective metric (in a scale of 1-5) rather than using the objective metric, PSNR, which often fails to meet important subjective needs of a stereoscopic entertainment system [2].
We carry out the experiments on a state-of-the-art, H.264 AVC based stereoscopic video CODEC [1]. Comprehensive profiling tests are carried out to determine the most significant encoder/decoder parameters (QP, number of reference frames, resolution, use of fast motion estimation, search range, intra period, slice group structure, intra mode prediction etc.) that affect rate, subjective quality, memory usage, CPU cycles. The experiments are performed on three stereoscopic video sequences having different statistical properties. The profiling tests for the CPU usage were carried out using Intel’s VTune performance analyser. Subjective tests were designed using a stereoscopic video system that displays stereo video on shutter-glasses controlled by an IR-emitter. Twenty subjects were used in a controlled environment for subjective quality profiling tests. The profiling results are initially used to derive objectives functions for each objective/constraint. These objective functions are subsequently used within a multi-objective optimization strategy that adopts a Genetic Algorithm (GA) to produce optimal results. We use the well established GA software tool, Non-dominated Sorting Genetic Algorithm (NSGA-II) for this purpose. The final outcome of the optimisation is the determination of coding parameter values that will jointly optimize rate, subjective quality, CPU and memory usage of the given CODEC. We show that the proposed framework can be extended to optimize any given design of a stereoscopic video CODEC and has the feasibility to be further extended to include further constraints, such as networking and data transmission constraints such as packet loss and delay. Instead of the usual approach of jointly optimising rate-distortion only, the consideration of memory usage and CPU cycle constraints enables the framework to be used in determining the optimal coding parameters for stereoscopic videos to be displayed in multi-modal devices. The research also aims to bridge a significant research gap in stereoscopic image coding, namely the subjective quality optimisation. We present some interesting experimental results in support of our conclusions.
This paper will be submitted to the The 9th IASTED International Conference on Visualization, Imaging & Image Processing 2009 that will be held on 13 - 15 July 2009, at Cambridge, UK.
Today we bring Puteri to Measham Carboot sale since it was such a nice and sunny day. This carboot is among the biggest carboot in the Midland. They had moved to a new and bigger piece of land. There are variety of items sold there, I mean, everything including the kitchen sink. We went there just to enjoy the weather and pick up some odd items. As usual, Puteri is fast asleep in her pushchair.
Having breakfast at Pret - A - Manger, King Cross St. Pancras station
Before checking out of the hotel room
Got my passport & birth certificate
Lunch at Bonda Cafe, Edgware Road, London
Pak Su merenung
Taking a break on the park bench. Had my nappy changed as well.
King Albert Memorial Statue
Went to Imperial College
Some statue at the park
Some statue
Breakfast with very spicy nasi lemak at Hyde Park bench
Me & Pak Su again
Me & Pak Su
Map of Hyde Park
Going for a walk after handing in passport application form to Malaysian Embassy.
Sound asleep after travelling
Checking in at Travelodge Hotel, Ilford
After eating lunch at MSD
Hyde Park Tube Station
Reached King Cross St. Pancras Train Station
Asleep on the train
Lboro Train Station
Lboro Train Station
Today Puteri had her 4 month's immunization shot. She smiled at the nurse before they gave her the shot on both of her thigh. We then gave her some medication to help her ease the pain...
Today Atie brought Puteri to the Cobden Children Centre for a Stay & Play session. They had lots of fun activities there. Puteri also did a hand print. They also sang nursery songs as well as playing with toys with other children...
