- Open Access
Quality of multimedia experience
EURASIP Journal on Image and Video Processing volume 2012, Article number: 5 (2012)
Typically, multimedia-processing algorithms are evaluated using objective metrics or through subjective testing in a controlled environment. However, perceived user experience is psychological in nature and it changes in different environmental conditions and with different multimedia devices. Therefore, new techniques are needed to assess and enhance multimedia experience from the user perspective.
The goal of this special issue is to address the recent advances in theory, algorithms, challenges, and applications in the field of quality of multimedia experience. We have received numerous articles addressing a wide variety of topics to enhance multimedia quality experience including subjective assessment of images and video, perceptual image and video quality assessment, full/reduced/no-reference objective quality metrics, speech quality, and audio-visual quality models. Among the ten articles accepted for publication in this issue, two of them are extended versions of the articles that have received Best Paper and Best Student Paper Awards at the 1st International Quality of Multimedia EXperience (QoMEX) Workshop held in San Diego in 2009. The accepted articles are classified based on the type of multimedia content that they target in the following order: general, audio-visual, visual, and speech .
The first article "Gradient ascent subjective multimedia quality testing", by S. Voran and A. Catellier, is an extended version of the article that won the Best Paper Award at QoMEX'09. It proposes the Gradient Ascent Subjective Testing (GAST) method, as an alternative solution to a subjective evaluation strategy covering the full range of test cases; GAST reduces the number of subjective experimental trials, gathering more information from a limited number of them .
The second article "Impairment-factor-based audio-visual quality model for IPTV: Influence of video resolution, degradation type and content type", by M. N. Garcia, R. Schleicher, and A. Raake, proposes different audio-visual content-aware quality models for IPTV video services. Applications of the presented quality models include network planning and video quality monitoring. The authors provide useful comparison of other proposed models in the literature as well highlight further suggestions to address current limitations of the proposed quality models .
The third article "The extended-OPQ method for user-centered quality of experience evaluation: A study for mobile 3D video broadcasting over DVB-H", by D. Strohmeier, S. Jumisko-Pyykko, K. Kunze, and O. Bici, is dedicated to the evaluation of a service from the point of view of the user experience. The Open Profiling of Quality mixed method is proposed, combining a conventional quantitative psycho-perceptual evaluation and a qualitative descriptive quality evaluation based on naïve participants' individual vocabulary .
The fourth article "Quality of Experience for large ultra-high-resolution tiled displays with synchronization mismatch", by S. Deshpande and S. Daly, studies the quality of visual experience for large ultra high-resolution displays made from individual display tiles. They define experiments to measure synchronization mismatch perception between individual tiles. Besides helping construction of low cost tile systems by using a distributed synchronization mechanism for bezeled displays, this work can further benefit the standardization of subjective experiments when multiple panels are used .
The fifth article "Subjective quality assessment of H.264/AVC video streaming with packet losses", by F. De Simone, M. Naccari, M. Tagliasacchi, F. Dufaux, S. Tubaro, and T. Ebrahimi, is an extended version of the article that won the Best Student Paper Award at QoMEX'09. It describes the creation of a database of videos that are affected by transmission errors and their associated MOS values. This dataset is made publicly available to promote the development of objective video quality metrics .
The sixth article "A packet-layer video quality assessment model with spatio-temporal complexity estimation", by N. Liao and Z. Chen, presents a packet-layer video quality monitoring model that is based on the spatiotemporal complexity of the video content and that takes into account interactions between content features, error concealment, and error propagation effects. In addition, the authors present criteria for designing subjective evaluation video databases for video system planning and for video quality monitoring .
The seventh article "Comparing apples and oranges: assessment of the relative video quality in the presence of different types of distortions", by U. Reiter, J. Korhonen, and J. You, presents a subjective quality assessment method that is suitable for comparing different types of quality distortions. The proposed method is also used to evaluate the performance of existing objective quality metrics for visual content with different types of quality distortions .
The eighth article "A framework for measuring sharpness in natural images captured by digital cameras based on reference image and local areas", by M. Nuutinen, O. Orenius, T. Saamanen, and P. Oittinen, proposes a framework for objective image quality assessment, where the important areas used for assessment in the reference image are automatically determined by using structural energy attributes of images. This method will be particularly useful for digital camera phone benchmarking where a pixel-wise full reference image does not exist. The proposed framework has been proven to be promising based on the experiments done for evaluating the sharpness feature of images .
The ninth article "No-reference image blur assessment using multiscale gradient", by M.-J. Chen and A. C. Bovik, describes a no-reference image and video blur assessment algorithm that uses natural scenes statistics models combined with multi-resolution decomposition methods. The performance of the algorithm is tested on the LIVE Image Quality Database and the Real Blur Image Database providing results that are highly correlated with human judgments .
The last article in this special issue "A study of artificial speech quality assessors of VoIP calls subject to limited bursty packet losses" by S. Jelassi and G. Rubino focuses on studying the performance of artificial speech quality assessment algorithms with a focus on VoIP applications. Detailed analysis of the estimated perceived quality of voice calls under bursty packet loss conditions is presented. This study highlights the effectiveness as well the weakness of the studied algorithms under various test conditions .
Finally, we would like to thank the authors for their contributions and the administrative and publication staff and Editor-in-Chief of the EURASIP Journal on Image and Video Processing for their efforts in the preparation of this special issue. We are especially grateful for the reviewers whose comments and suggestions helped to improve the quality of the articles.
We hope that you would enjoy reading this special issue and that you would gain valuable insights into this fast growing area of quality of multimedia experience.
Gökçe Dane received the B.S. degree in Electrical and Electronics Engineering from Boğaziçi University, Türkiye in 1999, the M.S. degree in Electrical Engineering from University of California, Riverside in 2001 and the Ph.D. degree in Electrical and Computer Engineering from University of California, San Diego in 2005. She is currently working at Qualcomm as a Staff Engineer in Multimedia R&D team specializing in "Video and Display Systems and Algorithms"; specifically working on algorithms related to signal acquisition, pre-processing, compression, transmission, post-processing, and display panel characterization to improve end-to-end visual quality for mobile, tablet, and DTV platforms for a wide range of input/outputs including 2D, 3D, LDR, and HDR. She worked in Thomson Corporate Research in Princeton on high definition video coding, video quality assessment, and enhancement during 2005 and 2006. She was with HP Labs, Palo Alto in Imaging Systems Department during 2000 working on problems related to face detection and recognition. She has around 20 technical publications, and filed more than 15 US and international patents on display, video processing and compression systems and architectures. She has served in Technical Program Committees of various IEEE conferences covering multimedia processing areas. She is one of the founding members of the International Workshop on Quality of Multimedia Experience (QoMEX) and served as the Technical Program Co-Chair for QoMEX 2009.
Lina J. Karam received the B.E. degree from the American University of Beirut (AUB), in 1989, and the M.S. and Ph.D. degrees in electrical engineering from the Georgia Institute of Technology, Atlanta, in 1992 and 1995, respectively. She is currently a full Professor at Arizona State University, Tempe, where she directs the Image, Video, and Usability (IVU) and the Real-Time Embedded Signal Processing (RESP) Laboratories. She worked at Schlumberger Well Services on problems related to data modeling and visualization, and in the Signal Processing Department of AT&T Bell Labs on problems in video coding during 1992 and 1994, respectively. Prof. Karam is the recipient of an NSF CAREER Award, a NASA Technical Innovation Award, and an AUB Distinguished Alumnus Award. She served as the Chair of the IEEE Communications and Signal Processing Chapters in Phoenix in 1997 and1998. She also served as an Associate Editor of the IEEE Transactions on Image Processing from 1999 to 2003 and from 2006 to 2010 and of the IEEE Signal Processing Letters from 2004 to 2006, and as a member of the IEEE Signal Processing Society's Conference Board from 2003 to 2005. She served as the lead guest editor of the IEEE Journal on Selected Topics in Signal Processing, Special Issue on Visual Media Quality Assessment, as a Technical Program Chair of the 2009 IEEE International Conference on Image Processing, and as the General Chair of the 2011 IEEE International DSP/SPE Workshop. She was one of the founding members of the International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM) and of the International Workshop on Quality of Multimedia Experience (QoMEX). She currently serves on the editorial board of the Foundations and Trends in Signal Processing journal and as a guest editor for the IEEE Signal Processing Magazine. She is an elected member of the IEEE Circuits and Systems Society's DSP Technical Committee, the IEEE Signal Processing Society's IVMSP Technical Committee, and the Signal Processing Society's Signal Processing EducationTechnical Committee. She is also serving on the Signal Processing Society's Nominations and Awards Committee.
Khaled El-Maleh received his M. Eng. and Ph.D. from McGill University, Canada, and B. Sc. in EE and B. Sc. in Applied Math from King Fahd University of Petroleum and Minerals (KFUPM), Saudi Arabia. He has been with Qualcomm, San Diego, since 2000. Dr. El-Maleh's areas of expertise and interests include design, implementation, and quality evaluation of mobile multimedia systems with focus on speech and video compression/processing algorithms. He was a key member of the team who designed the first generation of mobile camcorder and video telephony solutions for both UMTS and CDMA networks. He was the video quality lead for Qualcomm chip unit and contributed to the design of media quality assessment tools with focus on automated on-chip quality testing. He is currently leading the multimedia engineering team in the Strategic IP Department of Qualcomm. In addition to his technical publications, he has filed more than 200 US and international patents covering video\image\speech processing and compression technologies. He is a founding member of the International Workshop on Quality of Multimedia Experience (QoMEX) and served as the General Co-Chair for QoMEX 2009. He is actively involved in organizing various IEEE conferences covering multimedia processing areas (ICIP, ICME, MMSP, and ICASSP).
Vittorio Baroncini Education bac ca Laureate in Physics, in digital electronics. Work experience beg an work in 1976 at the ITT R&D Laboratory of Pomezia (Italy), as HW designer. And in 1984, began to work at the "Instituto Superiore delle Poste e Telecomunicazioni", the technical branch of the Italian Government's Telecommunication Ministry. Worked in the area of national regulatory activity, dealing with line transmission system on digital networks (PDH). In 1986, began work at the Fondazione Ugo Bordoni (FUB), and moved to the TV group of FUB, with responsibility for the design of HW. At the same time, began participating in international standards work, by contributing to the works of the CCIR (now ITU-R) Committee. Began activity in the International Standards Organization (ISO) in 1992, participating in the works of the SC29-WG11 (MPEG) group dealing with the MPEG-2 verification tests. In 1995, he took responsibility of FUB involvement in the EC projects "QUOVADIS", and "MOSQUITO", both of them related to the automatic control of QoS in TV digital telecommunication networks. In these projects he contributed with a new algorithm for a "reduced reference" QoS metric to be applied on digital TV network capable of a return channel on the "control network".
Co-founder (1997) of the Video Quality Expert Group (VQEG), a joint group of T and R Sector experts of ITU, aiming at the promotion of new metrics for the objective video quality assessment, he lead the group to the first world wide international standard of objective quality metric for digital TV (ITU-R Recommendation BT-1683). Currently, co-chair of the HDTV and of the Multimedia II project and member of the ILG (Independent Laboratory Group). He is the designer new subjective evaluation methods. Among them the "Single Stimulus with Multiple Repetition" and the Double Stimulus Hidden Reference"; he is also the designer of the test method to evaluate the vertical resolution of 35-mm film, and has been the technical manager of the first quality evaluation test of D-Cinema in Hollywood (2001). He also contributed to the design of other new test methods developed inside MEPG; among them the SSCQE and DSCQE, subsequently standardised by ITU-R. During this period became the responsible of the video testing laboratory in FUB.
In 1999, he was appointed by the Italian "Authority for the Communications" to the Commission in charge of preparing the "White Book on the Digital Terrestrial Television". Within this framework, contribute to the specification of rules for access to resources made available by new digital technologies. From 2000 in the steering board of two prominent Working Parties in the ITU-R: chair of WP6Q (Performance assessment and quality control) and vice-chair of TG6/9 (Digital Cinema). On 2008 he became Vice-Chair of ITU-R WP6C and Chair of the MPEG Test Group. On 2009 he was nominated Coordinator of the Subjective Quality Evaluation test issued by ISO and ITU-T in the joint effort for the definition of a new standard in video coding for ultra high-resolution TV and MM services. He is the technical responsible for video quality assessment and of the HDTV, 3DTV and D-Cinema projects in FUB. Author of many articles to conferences and journals he is also co-author of a book on MPEG-4.
Touradj Ebrahimi received his M.Sc. and Ph.D., both in Electrical Engineering, from the Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland, in 1989 and 1992, respectively. In 1993, he was a research engineer at the Corporate Research Laboratories of Sony Corporation in Tokyo, where he conducted research on advanced video compression techniques for storage applications. In 1994, he served as a research consultant at AT&T Bell Laboratories working on very low bitrate video coding. He is currently Professor at EPFL heading its Multimedia Signal Processing Group. He is also adjunct Professor with the Center of Quantifiable Quality of Service at Norwegian University of Science and Technology (NTNU). Prof. Ebrahimi has been very active in the field of multimedia signal processing with a special emphasis on quality evaluations and metrics. In February 2001, he was the first to suggest the notion of Quality of Experience (QoE) be used in multimedia communication systems as a dual and complement to Quality of Service (QoS).
He has been author, or co-author of more than 40 publications in the field of quality metrics for various multimedia content, an initiator of Advanced Image Coding within JPEG standardization committee, which develops subjective and objective quality metrics for future imaging systems, one of the responsible for the subjective evaluations of MPEG's next generation video compression standard, in March 2010, was one of the founding members, and the first general co-chair of International Workshop on Quality of Multimedia Experience (QoMEX), as well as a guest co-editor of the IEEE Journal on Selected Topics in Signal Processing, Special Issue on Visual Media Quality Assessment. Prof. Ebrahimi is a member of IEEE MMSP and IVMSP Technical Committees, has been associate editor of various scientific journals, including the IEEE Trans. on Image Processing, IEEE Trans. on Multimedia, and a past member of the editorial board of the IEEE Signal Processing Magazine, in which he served as a guest editor for a Special Issue on Universal Multimedia Access.
Dane G, Karam L, El-Maleh K, Baroncini V, Ebrahimi T: Quality of multimedia experience. Eurasip Journal on Image and Video Processing 2011.
Voran S, Catellier A: Gradient ascent subjective multimedia quality testing. Eurasip Journal on Image and Video Processing 2011.
Garcia MN, Schleicher R, Raake A: Impairment-factor-based audio-visual quality model for IPTV: Influence of video resolution, degradation type and content type. Eurasip Journal on Image and Video Processing 2011.
Strohmeier D, Jumisko-Pyykko S, Kunze K, Bici O: The extended-OPQ method for user-centered quality of experience evaluation: a study for mobile 3D video broadcasting over DVB-H. Eurasip Journal on Image and Video Processing 2011.
Deshpande S, Daly S: Quality of Experience for large ultra-high-resolution tiled displays with synchronization mismatch. Eurasip Journal on Image and Video Processing 2011.
De Simone F, Naccari M, Tagliasacchi M, Dufaux F, Tubaro S, Ebrahimi T: Subjective quality assessment of H.264/AVC video streaming with packet losses. Eurasip Journal on Image and Video Processing 2011.
Liao N, Chen Z: A packet-layer video quality assessment model with spatio-temporal complexity estimation. Eurasip Journal on Image and Video Processing 2011.
Reiter U, Korhonen J, You J: Comparing apples and oranges: assessment of the relative video quality in the presence of different types of distortions. Eurasip Journal on Image and Video Processing 2011.
Nuutinen M, Orenius O, Saamanen T, Oittinen P: A framework for measuring sharpness in natural images captured by digital cameras based on reference image and local areas. Eurasip Journal on Image and Video Processing 2011.
Chen M-J, Bovik AC: No-reference image blur assessment using multiscale gradient. Eurasip Journal on Image and Video Processing 2011.
Jelassi S, Rubino G: A study of artificial speech quality assessors of VoIP calls subject to limited bursty packet losses. Eurasip Journal on Image and Video Processing 2011.
The authors declare that they have no competing interests.