Multi-view display is the most widely used autostereoscopic 3D display method. It separates the view images spatially with the optical devices and provides binocular disparities to observers [1-19]. A multi-view display system is usually implemented with 2D display systems such as flat panel displays or projectors, and takes advantages of their high lateral resolution [1-4]. An optical structure such as a lenticular or a parallax barrier is used to control the light rays, so the observer watches different view images at different positions. Multi-view displays can be categorized into two groups according to the base 2D display: projection-type multi-view displays and panel-type multi-view displays. A projection-type multi-view display is based on 2D projectors and a screen. In projection-type multi-view display, multiple projectors are often spatially multiplexed for higher data capacity. A panel-type multi-view display is composed of a 2D display panel and an optical structure, so it takes a small volume, but it is hard to achieve both high resolution 3D images and a large number of views due to the limited number of pixels [1-4]. Since both types of multi-view display system are composed of several complete optical elements, the calibration between the elements is an important factor for the quality of the reconstructed 3D image.

The calibration of a projection-type multi-view display has been studied intensively, because the system is often implemented with multiple projectors [11-19]. Multiple projectors have their own mechanical calibrators inside, so one can calibrate them with the optical movement and with the image compensation method. Furthermore, the automatic calibration methods for multiple projectors have been presented [20-23]. On the other hand, the calibration method of a panel-type multi-view display has not been studied as much as that of the projection-type. Basically it is a calibration just between two devices: a flat panel display and an optical structure, and a slight misalignment causes changes of viewing characteristics rather than a significant degradation of 3D image quality [5, 6, 24-26]. Especially for the panel-type multi-view display with a large number of views, the pitch of the optical structure is much bigger than the pixel pitch, so small changes in pixel scale are not a big problem to the whole system [5-7]. There have been several previous approaches to compensate the misalignment in panel-type multi-view display systems, but they focused on the image processing method rather than the actual optical calibration [24-26].

However, the accurate alignment is becoming important not only for the projection-type but also for the panel-type multi-view display. In recent high-resolution multi-view displays, the pitch of the optical structure is similar to the pixel pitch [9, 10]. Furthermore in slanted lenticular display, the alignment of slant angle is crucial, otherwise the views are scrambled [8-10]. To build a personal multi-view display with the individual display panel, it is essential to have a standard calibration method [9]. Some calibration methods were introduced in previous studies such as using a white/black image as an alignment pattern [24-26] or using a view image of numbers as an alignment pattern [27, 28]. However, to the best of our knowledge, there have been no studies on a panel-type multi-view display which analyze the sensitivity of the various alignment patterns, verify the alignment patterns, and present the whole calibration process, until now.

In this paper, we propose a novel calibration method which can be applied to all kinds of panel-type multi-view display. Firstly, we analyze how the misalignment in panel-type multi-view display affects the 3D image quality. We categorize the alignment between the panel and the optical structure into three parts: angular, axial, and lateral alignment. We demonstrate the ray optics simulation for the quantitative calculation. Based on the analysis, we propose a new alignment pattern for all the types of the panel-type multi-view display. We theoretically show the generality of the proposed alignment pattern and verify the pattern with image simulations. Furthermore we also implement a slanted parallax barrier system to show the validity of the proposed calibration method.

A panel-type multi-view display is one of the methodologies to construct 3D display systems, and it does not indicate a specific structure. Various types of display panel and various types of optical structure can be utilized as the elements of the panel-type multi-view display system. Nevertheless, a panel-type multi-view display system can be generally interpreted with the periodic structures such as pixels and optical structures (lenses or barriers) as shown in Fig. 1, and starting from this microstructure level, a general analysis on the alignment issue can be derived.

Figure 1 shows the schematic diagram of a panel-type multi-view display system. A display panel located at z=−g plane and an optical structure at z=0 plane are in parallel as shown in Fig. 1(a). In the display panel as shown in Fig. 1(b), pixels started from the origin O are repeated periodically with the vertical pixel pitch p_px and horizontal pixel pitch p_py, and in the pixel, the vertical subpixel pitch is p_sx and horizontal subpixel pitch is p_sy. In Fig. 1(b), RGB stripe subpixel structures are assumed. However, the analysis can be easily applied to the other types of subpixel structure such as a pentile [10] or a slanted RGB stripe [7-9].

An optical structure is located at the z=0 plane with pitch l_p and slant angle θ_l. O_offset(Δ_x, Δ_y, 0) is the position of the top left vertex of the optical structure, and l_px and l_py are the vertical and the horizontal optical structure pitches, respectively, where the triangular relation exists between them as follows:

Suppose that this optical structure is ideally designed, so that the view images can be clearly separated at the viewpoint V₁, V₂, …, V_N on observing plane z=D if there is no misalignment. A slanted optical structure is assumed here, however, this analysis is still valid with a non-slanted optical structure where θ_l=0.

In the ideal case, the perfectly calibrated situation, the following equations are satisfied:

and

where θ₀ and g₀ are the designed slant angle and the designed gap between the display panel and the optical structure, respectively. However in a practical case, slight misalignments do exist. Non-zero value of Δθ (=θ_l−θ₀), Δg (=g−g₀), and Δx and Δy cause angular, axial, and lateral misalignment, respectively. The reasons for these three independent alignment values are complicated. However, the result is identical: the degradation of the view image quality. From that point of view, it can be helpful to know how the image quality is degraded by the misalignment for the analysis of the alignment problem and for the designing the alignment pattern.

For instance, suppose that a subpixel P shown in Fig. 1(a) is designed to form V₁, which means the light rays started from subpixel P converge to viewpoint V₁ at the observing distance, so P is not observed at V₂, V₃, …, V_N. Q_n is a point of contact between the line and the optical structure as shown in Fig. 1(a). If the system is perfectly aligned, Q₁ is the center of one of the optical structures, while Q₂, Q₃, …, Q_N are not, so the light ray from P is transmitted to V₁ by the ray optics. For instance, in a panel-type parallax barrier system, Q₁ is supposed to be in the opening, while Q₂, Q₃, …, Q_N are supposed to be in the barrier. In this paper, we call subpixel P an intended subpixel of V₁, and P_n the group of the intended subpixels for V_n. The total number of P_n is derived as follows:

where R_x and R_y are the horizontal and vertical resolution of the display panel respectively, N_s is the number of the supixels in a pixel, and N is the number of views. In a practical case with the misalignment, the light rays from the intended subpixels do not always reach the intended viewpoint, so the number of the observed intended subpixels is smaller than n(P_n). From that point of view, the number of the observed intended subpixels of one view can be the measure of the view image quality, and quantitative analysis on the calibration issue is possible.

The ray tracing simulation is presented to calculate the changes of the number of the intended subpixels with the angular, the axial, and the lateral misalignment. A 3-view slanted parallax barrier with a pentile display system is assumed, which was implemented in our previous work [10]. This system verifies that our analysis is valid not only for an RGB stripe structure but also for a pentile structure, and the simulation results can be verified with the implemented system easily. A parallax barrier system is adopted because of the simplicity of calculation, but the generality is still preserved because all calculations are on and off: whether the subpixel is observed or not at the certain viewpoint. If a subpixel is an intended subpixel of a viewpoint with a parallax barrier system, then it is also an intended sub-pixel of the same viewpoint with a lenticular system with the same pitch, the same focal length, and the same slant angle.

Figure 2 is the diagram for the 3-view slanted parallax barrier system with a pentile display. In Fig. 2(a), the numbers on the subpixels indicate which view they belong to. The intended subpixels of a certain view are along the optical structure borders, because the relative position between the subpixel and the optical structure concludes the direction of the refracted light rays. The detailed specifications of the system are listed in Table 1. For the system, θ₀ and g₀ are 45° and 796.15 μm, respectively [10].

The simulation process is straightforward. First, draw a line from a certain viewpoint V_n to a subpixel Pⁱ ∈P_n. Then, calculate the position of the contact between the line and the parallax barrier. After that, identify whether is in the opening or in the parallax barrier. returns 1 if is in the opening and returns 0 otherwise. Finally, repeat these processes for all Pⁱ ∈P_n and the total number of the observed intended subpixels of V_n, , is represented as follows:

The function can be calculated from the relative position of and the parallax barrier. Since the barrier structure is repeated with l_p and θ_l, the equation of the right and the left border lines of the k^th barrier can be derived respectively as follows:

where l_oy is the horizontal pitch of the opening of the barrier. Therefore, if is located between two lines, then returns 1, otherwise it returns 0. This relation can be interpreted as follows:

where mod function means the modulo operation which returns the remainder after division.

Figure 3 shows the simulation results of the total number of the observed intended subpixels of V₂, , with the change of Δθ, Δg, and Δx. The small images in the graphs are the region of the observed intended subpixels at several points. The simulation result shows that the basic tendencies of the graphs are similar. is the same with n(P₂) at the ideally calibrated situation, but it decreases as the angular, axial, and lateral misalignments increase. However, the appearances and the sensitivities of the graphs are all different.

At first, Fig. 3(a) shows that the angular alignment is the most important for the panel-type multi-view display system. The system works perfectly in a very narrow range of angular misalignment, 0.019°, and the intended view image is totally scrambled when |Δθ| becomes about 0.05°. Even though it can be said that the exact values depend on the specifications of the individual systems, the scale shows that the angular alignment is relatively sensitive. The reason why the angular alignment is the most important for the multi-view display system is that the angular misalignment influences view image creation. To form a viewpoint, the light rays from all of the intended subpixels should converge at a point. The angular misalignment does not allow to form a viewpoint, while the axial and lateral misalignment shift the viewpoint.

Secondly, Fig. 3(b) shows the influence of the axial misalignment. The system provides the intended view images completely when −43 µm < Δg < 144 μm. As the axial misalignment gets severe, decreases gradually without any points of inflection. This indicates that the observer can watch intended view images somewhat even if the system is axially misaligned. The results explain why the panel-type multi-view system provides 3D images well without delicate axial alignment in μm scale and why the 3D images are shown not only at the exact optimal distance, but also around it. The system is quite tolerant to axial misalignment.

Finally, the view image shift caused by lateral misalignment is shown in Fig. 3(c). The lateral misalignment affects the image quality very sensitively. The toggles between n(P₂) and 0 in every 57.5 μm, which is almost impossible to calibrate in a practical case without a linear stage. However, the view images are not scrambled but just shifted laterally. These results explain why the panel-type multi-view system provides 3D images well without delicate lateral alignment in μm scale. The observer moves the eyes laterally to find the position where the 3D images are shown correctly.

From the simulations, it is derived that the lateral and the axial calibrations are less important than the angular calibration. However, it is still important to calibrate the system perfectly for the image reconstruction as designed. In this section, we compare the existing calibration patterns with the simulation, and propose a new pattern for the panel-type multi-view display system.

The most widely used alignment patterns for the panel-type multi-view display are the white/black (w/b) image and the image of numbers. Figure 4(a) shows the w/b alignment pattern for a 3-view slanted multi-view system (see Table 1). One view image is the white image and the others are the black images. At a certain viewpoint the white image is shown clearly, while it is not at other viewpoints. The alignment pattern with the image of numbers is shown in Fig. 4(b), where the different number images are shown at the different viewpoints.

Figures 5-7 show the view images of alignment patterns at a viewpoint V₂ with the angular, the axial, and the lateral misalignment, respectively. These results can be obtained with the element by element multiplication between the image of the intended subpixels and the base image of the alignment pattern. The sensitivity of the alignment patterns depends on the types of misalignment. The simulation results show that the image of numbers is less sensitive to the angular and the axial misalignment than the w/b image.

The simulation results show that conventional w/b image is sensitive enough to all types of misalignment. However, there are some limitations with the conventional w/b images’ alignment pattern. At first, w/b image is not a proper option for the lateral alignment. The w/b image is as sensitive as image of numbers on the lateral misalignment of one view (V₂), but it cannot distinguish V₁ and V₃. Since only one view is white in w/b image, it is difficult to check the other views at the corresponding viewpoints. The black view images are identical and it is impossible to distinguish the differences between them. When the panel-type multi-view display has a large number of views, then this problem becomes severe.

Secondly, the w/b image is less sensitive to the angular calibration than the proposed pattern. Since we sweep the angle during the angular calibration, the sensitivity of the calibration is decided by the image intensity differences between the nearby misalignment. For example, if the misalignment Δθ = 0.025°, we would try to make Δθ = 0.000°. Then the image intensity between Δθ = 0.025° and Δθ = 0.024° and Δθ = 0.026° and Δθ = 0.025° should be different enough to be observed with human eyes, so the presenter can rotate the system to the right direction in 0.001° unit. Therefore, the sensitivity of the alignment pattern can be analyzed through relative image intensity difference dI/I, which is defined as follows:

where dθ is the unit of the angular misalignment and the I(Δθ) is the intensity of the image at the viewpoint with the angular misalignment Δθ.

Figure 8 shows the simulation results of dI/I with the w/b image and the proposed alignment pattern where dθ = 0.001°. Both images do not change around Δθ = 0° because the system is aligned well and the view image does not change. However when the misalignment gets severe, the relative image intensity difference is higher with the proposed alignment pattern. The simulation results show that the proposed calibration pattern is more sensitive to the angular misalignment during the calibration process. This higher image intensity difference helps the angular alignment, which is the most important calibration in the panel-type multi-view display.

Thirdly, it is hard to recognize the difference between the perfectly aligned case and the misaligned case with the w/b image. The ideal view image (center) and the first distorted view image as the misalignment increases (center left/right) shown in Figs. 5(a), 6(a), and 7(a) are hard to distinguish because the pattern changes gradually and the nearby white pixels disturb the observer’s recognition in the practical case. For comfortable calibration, it is better to use the on and off calibration pattern, so the observer can judge whether the system is well-aligned or misaligned.

Lastly, it is hard to figure out which type of alignment is in trouble with the w/b image or the image of numbers. As shown in Figs. 5(a), 5(b), 6(a), 6(b), 7(a), and 7(b), the resultant images all look similar, so the observer cannot judge the types of misalignment only from the distorted view image. This problem gets severe when the three types of misalignment exist together. The view image is scrambled and the observer feels difficulty to find the reason of the image degradation.

From that point of view, we propose a new calibration pattern for the panel-type multi-view display system. First, the proposed pattern covers all types of misalignment. It is sensitive to all of the axial, the angular, and the lateral misalignment, so the system is valid when the pattern is wellaligned. Secondly, the proposed pattern is filled with complicated images with the high spatial frequency, so the observer can easily recognize the difference between the aligned case and the misaligned case. For the axial alignment, we use the on and off alignment pattern to help the observer. Finally, the proposed pattern consists of the axial, the angular, and the lateral part, so the observer can find which alignment is in trouble. This enables the observer to calibrate the system in series: the angular first, then the axial, and finally the lateral alignment, and the observer feels more comfortable with the calibration procedure.

Figure 4(c) shows our proposed alignment pattern. The pattern is composed of three different parts for the three different types of alignment. The center region is the image of numbers as shown in Fig. 4(c). Since the resultant of the lateral misalignment only toggles between n(P_n) and 0, we do not have to use the whole region for the lateral alignment. Therefore the small patch of the number images is enough for the lateral calibration. For the angular alignment, we develop the w/b image principle. The repeated white lines are shown in V₁, V₂, and V₃, and they sensitively react to the angular misalignment. This high frequency line image is helpful to check whether the system is well-aligned or not. For the axial alignment, we use 4 white marks at every corner in V₂. Since the region of the intended subpixels decreases from the corners as the axial misalignment increases, marking an image in every corner is enough for the axial calibration. This axial calibration method takes advantage of on and off judgment. The arrows in Fig. 3 indicate the sensitivity of the three alignment patterns to each type of alignment. As shown in Fig. 3, the proposed pattern reacts to the angular and the axial misalignment as sensitively as the w/b image, and reacts to the lateral misalignment like the image of numbers as expected.

The proposed method can be applied to any type of panel-type multi-view display system: repeated white line images in all view images, small number patches in all view images, and 4 white marks at the corners in one view image. The line images, number images, and marks are generated with the MATLAB (MathWorks Inc., R2012a) easily [29].

We verify the proposed alignment pattern with an experiment. The 3-view slanted parallax barrier system is implemented for the experiment [10]. Detailed specifications are identical to the system in previous simulations, as listed in Table 1. Figure 9 shows the experimental results of the calibration with the proposed alignment pattern. Since the proposed pattern is composed of three regions for the three types of alignment, we verify them separately. Figure 9(a) shows the angular alignment with the proposed white line images. Since the lines are repeated with the high spatial frequency, the observer can distinguish the angularly misaligned case more easily than the w/b image. After angular alignment, the lines are shown clearly in Fig. 9(a).

For the axial alignment, we verify the proposed method by changing the viewing distance rather than the gap between the panel and the parallax barrier. Since the gap mismatch in the parallax barrier system corresponds to the change of the viewing distance, we can verify the axial alignment more gradually with the variation of the viewing distance. Figure 9(b) shows the experimental results of the axial alignment with the proposed axial alignment patterns. At the optimal viewing distance the 4 mark images are shown clearly, while only 2 or 3 mark images are observed if the viewing distance is too close or too far. The close and the far situations correspond to the larger and the smaller gaps, respectively. Finally, the lateral alignment with the image of numbers at the center works well as shown in Fig. 9(c).

After the calibration process, the resultant view image V₂ at the optimal viewing distance is shown in Fig. 9(d). The example of the calibration process with two hands is shown in Fig. 9(e). The calibration process is done in series: the angular, the axial, and the lateral alignment. At first, calibrate the system angularly with the hands. After angular calibration, find the distance that the 4 mark images are observed together clearly by viewing distance change. If the distance is different from the designed viewing distance, then change the gap between the panel and the parallax barrier and repeat the angular calibration until the designed viewing distance is obtained for the 4 mark images. After the axial calibration, calibrate the system laterally with the hands until the corresponding view image is shown at the each viewpoint. The calibration clamps are helpful as shown in Fig. 9(e).

We performed the whole calibration experiment with the proposed calibration pattern. We utilized the same 3-view slanted parallax barrier system. Figure 10(a) is the observed image before calibration. It is easy to verify that this system is not aligned angularly, axially, and laterally. The lines are not clear as well as the 4 marks at the corner. The numbers at the center are not shown clearly.

Figure 10(b) is the observed image after angular alignment. As shown in the Fig. 10(b), the 4 mark images are not shown clearly so that the system is not axially aligned. Although the system is axially misaligned, the center region shows clear line patterns as expected. This clear line image guarantees this system is angularly calibrated at least. One can proceed with axial calibration maintaining angular alignment by preserving these clear line images.

Figures 10(c), (d), and (e) are observed images at each viewpoint after angular and axial calibration. One can laterally calibrate this system by comparing these viewpoints and actual viewpoints. The experimental results correspond to the simulation results shown in Figs. 5(c), 6(c), and 7(c) as expected.

The proposed alignment pattern assumes that the optical structure is perfectly manufactured. In practice, there must be errors and the optical structure is not uniform. However the proposed alignment pattern is still valid with the local non-uniformities, because some fluctuations cannot change the number of observed intended subpixels significantly. If the error is big enough to change the viewpoints or viewing distance, other correction methods should accompany [30-32]. In that case, the proposed alignment pattern is compatible with other correction methods.

In this paper, we proposed a novel calibration method which can be applied to all types of panel-type multi-view displays. We analyzed how the misalignment in panel-type multi-view displays affects the 3D image quality, and categorized the alignment between the panel and the optical structure into the angular, the axial, and the lateral alignment. We demonstrated the ray optics simulation for the quantitative calculation. Based on the analysis, we proposed a new alignment pattern for all types of panel-type multi-view displays. The proposed pattern covers all types of the misalignment. It is sensitive to all of the axial, the angular, and the lateral misalignments. Especially for the angular calibration, the relative image intensity difference is higher with the proposed alignment pattern, which can help more accurate calibration. The proposed pattern also helps observers to calibrate the system easily with the high spatial frequency images and on and off calibration methods. We theoretically showed the generality of the proposed alignment pattern and verified the pattern with the image simulations. Furthermore we also implemented a slanted parallax barrier system to show the validity of the proposed calibration method. This proposed method can be applied to any type of the panel-type multi-view display system, and it would be applied to the 3D display industry as well as the research field.