- Research Article
- Open Access
Quality of Experience for Large Ultra-High-Resolution Tiled Displays with Synchronization Mismatch
© Sachin Deshpande and Scott Daly. 2011
- Received: 11 November 2010
- Accepted: 7 February 2011
- Published: 8 March 2011
This paper relates to quality of experience when viewing images, video, or other content on large ultra-high-resolution displays made from individual display tiles. We define experiments to measure vernier acuity caused by synchronization mismatch for moving images. The experiments are used to obtain synchronization mismatch acuity threshold as a function of object velocity and as a function of occlusion or gap width. Our main motivation for measuring the synchronization mismatch vernier acuity is its relevance in the application of tiled display systems, which create a single contiguous image using individual discrete panels arranged in a matrix with each panel utilizing a distributed synchronization algorithm to display parts of the overall image. We also propose a subjective assessment method for perception evaluation of synchronization mismatch for large ultra-high-resolution tiled displays. For this, we design a synchronization mismatch measurement test video set for various tile configurations for various interpanel synchronization mismatch values. The proposed method for synchronization mismatch perception can evaluate tiled displays with or without tile bezels. The results from this work can help during design of low-cost tiled display systems, which utilize distributed synchronization mechanisms for a contiguous or bezeled image display.
- Video Sequence
- Original Video
- Object Velocity
- Vernier Acuity
- Tiled Display
Displays with large screen size and high resolution are increasingly becoming affordable and ubiquitous. Large displays are often used in certain niche markets such as public displays and digital signage markets. These include displays at public places such as airports, museums, hotels, stadiums, hospitals, malls. These displays are often created using individual small display tiles. Also, in universities, research institutes, and corporations, large wall-sized displays are often built from small-sized individual display panels. Such large-sized tiled displays are used for scientific, medical visualization applications . Examples of such tiled displays include LambdaVision display at University of Illinois at Chicago's Electronic Visualization Laboratory , Stallion tiled display at Texas Advanced Computing Center , and Stanford School of Medicine tiled display . Also, prior work exists on building projection based tiled displays [5, 6].
In the typical tiled display architecture as described above, rendering nodes send parts of overall image to be displayed to individual display nodes. The display nodes then utilize a distributed synchronization algorithm and individually display parts of the image on their display tiles to provide the overall perception of a single continuous image. In some cases, the display nodes can utilize advanced graphics cards such as Nvidia Quadro family of cards and can utilize Genlock frame lock  to achieve a better frame synchronization amongst them. However, these cards are expensive. Furthermore, the Genlock frame lock techniques cannot completely solve the synchronization problem as each display node is receiving its image parts over the network. Thus, the image part ready to be displayed by each node at a given refresh instance (e.g., at 60 Hz) may not belong to the same overall image depending upon the network stream reception performance of each node.
define synchronization mismatch vernier acuity and define an experiment to obtain synchronization mismatch acuity threshold as a function of object velocity and as a function of occlusion (tile bezel width),
define a method for perception evaluation of synchronization mismatch for ultra-high-resolution large displays and to perform a subjective quality evaluation using this method to arrive at conclusions regarding synchronization mismatches among individual tiles/panels which will still result in an acceptable picture quality for the overall tiled display.
As a result of a time offset, a moving object edge will be spatially offset. Since the spatial offsets vary across tiles due to the synchronization mismatches, the edges of moving objects will suffer spatial offsets across the tiles. This is analogous to vernier acuity psychophysical experiments, where a line is presented with a break due to an orthogonal spatial offset. The distortions in our application are more general, where the offset break direction depends on the motion direction relative to the edge orientation. Further, in our application, the offset disappears with a nonmoving image, since the tiles are then essentially synchronized. The magnitude of the spatial offset increases with both edge speed and synchronization temporal offset.
In the prior art, Westheimer and McKee  defined several experiments and measured spatial vernier acuity. Mostly, static images were used for these experiments, as the goal was only to obtain spatial vernier acuity thresholds. In one study , vernier acuity under retinal motion of up to 3.5 deg/sec (= 3.2 pixels/frame at 60 fps input for HD resolution, ) was studied, and only a slight loss of acuity was found. However, motion imagery is known to have much higher velocities. In our application, such higher velocities will cause larger spatial offsets for any synchronization delay, but the acuity may be less as the velocity increases. So, it is unknown to us which effect will dominate and whether high levels of motion will make the synchronization artifacts more or less visible. Gorea and Hammett defined experiments to study spatiotemporal vernier acuity . Their experiments resulted in determination of the smallest instantaneous displacement discriminable from a continuous drift and shortest motion stop discriminable from a continuous drift. Their study assumed perfect synchronous and contiguous display of the object under consideration. In comparison with these prior works, we define experiments for measuring spatial, temporal, and synchronization mismatch vernier acuity.
With respect to our second goal, we are not aware of any existing method for interpanel synchronization mismatch perception evaluation. ITU-R Recommendation BT.500-11  describes methods for the subjective assessment of the quality of television pictures. Also, ITU-R Recommendation BT.710-4  describes methods for subjective assessment for image quality in high-definition television. The above methods [12, 13] describe subjective evaluation which is conducted on a single display screen. The main distinctions of our proposed subjective evaluation method are the following. We describe a method for creation of a synchronization mismatch measurement test video set for particular tile configuration and for various interpanel synchronization mismatch values. This synchronization mismatch test video set allows performing synchronization mismatch perception evaluation for a target tiled system using a typical single-screen display (e.g., a 46′′ LCD display). That is, the tiled display is simulated on a single display panel. The method supports perceptual evaluation to correspond to a tiled display with or without tile bezels. The method creates a higher frame rate video from an original video for playback using a typical video player. The method uses a display which can provide higher frame rate playback, such as 120 Hz display, since 60 Hz (16.66 ms) is too close to the human visual system threshold.
In a typical tiled display system, each panel may have a bezel (mullion). Typically, tile bezels may be black/dark in color, as seen in Figure 2. When displaying the overall image, the tile bezels behave in a manner similar to occluding virtual pixels underneath them. Thus, the middleware removes those pixels which fall "underneath" the tile bezels. This helps humans perceive the overall displayed image as natural (such as seen through a French window). Thus, circles stay as circles and not as ellipses. Similarly, a human face retains its proportion. In this paper, the tile bezel width is treated as a occlusion and is referred to as gap width.
The rest of this paper is organized as follows. Section 2 describes our proposed experiment for measuring synchronization mismatch vernier acuity of moving images. In Section 3, we describe our subjective method for synchronization mismatch perception evaluation. Section 4 provides the details about our subjective quality evaluation using the method described in Section 3. In Section 5, we provide the conclusions from our subjective quality evaluation.
threshold for the detection of direction the of the vertical misalignment of two vertical lines as a function of length of each line,
threshold for the detection of the direction of the vertical misalignment of two vertical lines as a function of separation of the lower end of the top line and upper end of the bottom line (for four different lengths of line),
threshold for the detection of the direction of the vertical misalignment of two short gaps each in one long horizontal line as a function of vertical separation of the lines,
threshold for the detection of the direction of the vertical misalignment of a line with the point of a chevron pattern,
threshold for difference in the spatial interval between two vertical lines as a function of their separation,
threshold of the distance discrimination for different configurations (bright lines, dark lines, bright edges, dark edges, bright edge and bright line), dots versus lines,
threshold for detection of differences in width of bars (made from individual lines).
The work of Westheimer and McKee was focused on spatial vernier acuity only and utilized static images. Other important vernier acuity work from Klein and Levi  and Morgan and Regan  was only spatial.
smallest instantaneous displacement (infinite velocity) discriminable from a continuous drift,
shortest motion stop discriminable from a continuous drift.
Two Gaussian blobs of opposite polarity drifting at equal speeds in opposite directions disappear, and only one blob reappears at a variable spatial offset relative to true position. Observers decide positive or negative offset.
Two Gaussian blobs drifting at unequal speeds in opposite direction. Observers decide higher speed blob pair.
Two Gaussian blobs disappear after being flashed simultaneously followed by asynchronous reappearance at positions corresponding to different speeds. Observers decide which blob jump (reappearance position with respect to original position) has higher velocity.
An object (e.g., a Gaussian blob) moves as a single contiguous object with a constant velocity , disappears at at time . The object reappears at as two partial objects: at position at time and at position at time , with and moving with constant velocity and where . The objects and disappear at time and reappear at time at as single contiguous object moving at a constant velocity .
Observer is shown the reference video which has no synchronization mismatch (i.e., = 0) followed by the test video with synchronization mismatch (i.e., > 0). Observer is then asked to rate the test video with respect to reference video (see Section 3.2 for double stimulus impairment scale method used).
Additionally, in a set of experiments, a gap of width is placed at a fixed position (e.g., at the center of the frame). The gap width control parameter is varied in the range corresponding to minimum and maximum gap width. This allows studying synchronization mismatch acuity threshold as a function of occlusion (gap or bezel width).
synchronization mismatch acuity threshold as a function of object velocity,
synchronization mismatch acuity threshold as a function of occlusion (gap width).
create a synchronization mismatch measurement test video set for particular tile configuration for various interpanel synchronization mismatch values,
use a subjective visual quality measurement method for evaluating the synchronization perception at different interpanel synchronization mismatch values.
3.1. Synchronization Mismatch Measurement Test Video Set Creation
Create a new synch measurement video with frames in it (e.g., ), where the additional frames are created as described below. Set the video frame rate of the synch measurement video to frames per second. Thus, the new video will be played at frame rate (e.g., at 120 frames per second assuming , ) when playing back with a typical video player.
- (iii)The new synch measurement video above is created as follows.
Then, for each of the new video frames copy the left % portion of original video (Part in Figure 2) in the left % portion of new video with each frame repeated times to create total frames.
For each of the new video frames, copy the right % portion of original video (Part in Figure 2) in the right % portion of the new video with each frame repeated times, but with a copy offset with a value between .
Value for can be set based on the measurements obtained from individual nodes (i.e., from a recorded trace) or can be set in absolute terms, for example, ms corresponding to synch discrepancies up to ms.
3.2. Subjective Visual Quality Evaluation of Interpanel Synchronization Mismatch
Subjective visual quality evaluation can be conducted by playing back and evaluating the videos from the above created synchronization mismatch video measurement set using procedure as follows. Methods similar to ITU-R BT.500-11 recommendation  can be used to playback the videos from synchronization mismatch video measurement set and obtain their subjective quality evaluation. The following methods adapted from  can be used.
3.2.1. Double-Stimulus Impairment Scale (DSIS) Method
The original video will be used as unimpaired reference. Each of the videos in the synchronization mismatch video measurement set will be used sequentially as impaired videos.
The subject will be shown the reference video followed by one of the impaired video. The subject then uses a 5 point impairment scale (shown below) for assessment of impaired video with respect to the reference video. The reference video is the original video which was used to create the synchronization mismatch video set.
The above step is repeated for each of the videos in the synchronization mismatch set as an impairment video in random order.
A 5-point impairment scale can be used
4: perceptible, but not annoying,
3: slightly annoying,
1: very annoying.
3.2.2. Double-Stimulus Continuous Quality-Scale (DSCQS) Method
In this case, the original video and each of the videos in the synchronization mismatch video measurement set are played back in pair.
Subjects can switch between the two videos in the pair and can also repeat video playbacks for those two videos any number of times. The subject does not know which is reference (original) video and which is impaired video.
After the playback, subject uses a continuous rating scale (to avoid quantization errors), but the scale is divided into five equal lengths which correspond to the normal ITU-R five-point quality scale (excellent, good, fair, poor, bad).
The above step is repeated for each of the videos in the synchronization mismatch video measurement set as an impairment video together with the reference (original) video in random order.
3.2.3. Single-Stimulus (SS) Method or Single-Stimulus with Multiple Repetition (SSMR) Method with Adjectival Categorical Judgment Method
In this case, the original video and each of the videos in the synchronization mismatch video measurement set are played back one at a time (single stimulus).
After playback of each video the subject provides rating for it (SS method).
If using SSMR method, the subject can repeat the same video multiple times.
A 5-grade scale (e.g., ITU-R quality impairment scale) can be used to provide adjectival category judgment.
In other cases, instead of adjectival categorical judgment method, numerical categorical judgment method with 11-grade numerical categorical scale (SSNCS)  or noncategorical judgment method with numerical scale (e.g., 0–100) can be used.
For all the subjective quality tests, the viewing conditions are set as described in Annex 1 of . The video can be played back on single contiguous display (preferred) or on a tiled display. The video is played back locally or using a mechanism, where no additional jitter/synchronization mismatch is introduced.
Tile with no bezels: in this case, the video from the synchronization mismatch measurement set created above is played back using a standard video player.
- (ii)Tile with bezels: one of the following two methods can be used to simulate bezels (mullions).
A black/bezel color bar pattern for the target tile configuration can be created and embedded on top of the video player to cover parts of the video being played back to simulate the effect of tile bezels.
The synch mismatch video measurement set can be preprocessed to remove pixels corresponding to bezels.
We conducted subjective tests using the synchronization mismatch perception video set generated using our proposed method. The test setup used a 120 Hz single-panel display. This is because a 60 Hz display does not reach the human visual system temporal cutoff frequency (especially in parafovea and periphery). A computer runs video player which decodes 120 frames per second video and displays it on 120 Hz display. The viewing distance for a subject is set to 3 picture heights (standard viewing distance for HD, ).
synchronization Mismatch Vernier Acuity tests: are done on a Gaussian Blob sequence as described in Section 2,
natural video content tests are done using typical natural video sequences as shown in Algorithm 1.
4.1. Natural Video Content Tests
Video sequences used.
Type of motion
Springboard diving sequence
Camera panning motion
From "digital video essentials"  test material. The sequence has a fence with regular structure
Fixed camera with object motion, followed by camera and object motion
4.2. Synchronization Mismatch Vernier Acuity Tests
The gap width control parameter (Bezel width) is tested for the values of (0 0.18, 0.36, 0.54, 0.72, and 0.90) deg. For a tiled display system, these correspond to a bezel-bezel size of (0 10 20 30 40 50) pixels.
We defined experiments to measure synchronization mismatch vernier acuity. The experimental results can used to obtain synchronization mismatch acuity threshold as a function of object velocity and as a function of occlusion (gap width).
Let us define synchronization mismatch discomfort threshold (SMDT) as the synchronization mismatch value (in ms) above which the synchronization mismatch perception between tiles has a DSIS score at or below certain value. As an example, in a stringent system, DSIS score below 4 can be chosen to define this threshold. This is the value we will use below to obtain example SMDT values.
Synchronization mismatch discomfort threshold (in ms) is larger with bezels compared to without bezels.
This can be observed by comparing the DSIS scores for Figures 10(a)–10(c) to those from Figures 10(d)–10(f), respectively. In general, people prefer a tiled display with smaller or no bezels. Our results show that a tiled display system which has bezels can allow a more lax synchronization among tiles compared to one without bezels. The results in terms of lower distortion visibility for the bezels case corresponds well with that from standard spatial vernier acuity threshold elevation as gap width increases.
Synchronization mismatch discomfort threshold (in ms) decreases as number of tiles increases. This can be observed by comparing the DSIS values in Figures 10(a)–10(f) for the case of tile to that of or tiles.
It is unclear why the increase in number of tiles increases the distortion visibility and annoyance. It could be due to the increases in opportunities of mismatches along the varying edges or to a more overall sense of independent tile images.
Synchronization mismatch discomfort threshold (in ms) as a function of object velocity (motion).
It can be observed that as the amount of motion increases (from slow motion = 6 deg/s to fast motion 24 deg/s), the DSIS score increases for the same synchronization mismatch value. Thus, SMDT increases with object velocity even though the larger motion causes larger spatial offset, which would be more visible according to standard spatial vernier acuity.
Synchronization mismatch discomfort threshold (in ms) as a function of gap (bezel) width.
It can be observed that as the gap (bezel) width increases (from 0 to 0.9 deg), the DSIS score increases for the same synchronization mismatch value as well as for the same object velocity. Thus, SMDT increases with gap width.
The results from our evaluation can help during design of a tiled display system to meet a certain acceptable synchronization mismatch tolerance at the design stage. As a future work, we are developing a quantitative model which can predict the DSIS scores as a function of intertile synchronization mismatch values for any tile configuration with or without bezels.
Scott Daly is now with Dolby Laboratories. The above work was done while he was at Sharp Laboratories of America.
- Gaba D, Stringer J, Kung SSY: display walls in healthcare education what why how. Stanford School of Medicine, http://summit.stanford.edu/pdfs/DisplayWallGIRfinal.pdf
- University of Illinois at Chicago's Electronic Visualization Laboratory's LambdaVision Tiled Display http://www.evl.uic.edu/cavern/lambdavision
- Texas Advanced Computing Center's Stallion tiled display http://services.tacc.utexas.edu/index.php/stallion-user-guide
- Stanford Display Wall http://summit.stanford.edu//research/displaywall.html
- Roman P, Lazarov M, Majumder A: A scalable distributed paradigm for multi-user interaction with tiled rear projection display walls. IEEE Transactions on Visualization and Computer Graphics 2010, 16(6):1623-1632.View ArticleGoogle Scholar
- Dominick S, Ruigang Y: Anywhere pixel router. Proceedings of the ACM/IEEE 5th International Workshop on Projector Camera Systems (PROCAMS '08), August 2008Google Scholar
- Humphreys G, Houston M, Ng R, Frank R, Ahern S, Kirchner PD, Klosowski JT: Chromium: a stream-processing framework for interactive rendering on clusters. Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '02), July 2002 693-702.View ArticleGoogle Scholar
- NVIDA Quadro : G-Sync. http://www.nvidia.com/page/quadrofx_gsync.html
- Westheimer G, McKee SP: Spatial configurations for visual hyperacuity. Vision Research 1977, 17(8):941-947. 10.1016/0042-6989(77)90069-4View ArticleGoogle Scholar
- Westheimer G, McKee SP: Visual acuity in the presence of retinal image motion. Journal of the Optical Society of America 1975, 65(7):847-850. 10.1364/JOSA.65.000847View ArticleGoogle Scholar
- Gorea A, Hammett ST: Spatio-temporal vernier acuity. Spatial Vision 1998, 11(3):295-313. 10.1163/156856898X00040View ArticleGoogle Scholar
- ITU-R Recommendation : Methodology for the subjective assessment of the quality of television pictures. ITU; 2002.Google Scholar
- ITU-R Recommendation : Subjective assessment methods for image quality in high-definition television. ITU; 1998.Google Scholar
- Klein SA, Levi DM: Hyperacuity thresholds of 1 sec: theoretical predictions and empirical validation. Journal of the Optical Society of America 1985, 2(7):1170-1190. 10.1364/JOSAA.2.001170View ArticleGoogle Scholar
- Morgan MJ, Regan D: Opponent model for line interval discrimination: interval and vernier performance compared. Vision Research 1987, 27(1):107-118. 10.1016/0042-6989(87)90147-7View ArticleGoogle Scholar
- Digital Video Essentials http://www.jkpi.net/products_main.php
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.