Design of a See-Through Head-Mounted Display with a Freeform Surface

  • cc icon
  • ABSTRACT

    This paper described an off-axis head-mounted display system. It is composed of a 7-piece coaxial relay lens group and a freeform surface combiner. This configuration has a simple structure and a wide field of view (FOV). In this design, a freeform surface is chosen as the combiner, to simplify the structure and attain good image quality. We generated this freeform surface by considering both coordinates and normals of discrete data points. Moreover, we realize a coaxial structure in the relay lens group, which is compact and benefits from a loose tolerance requirement. The HMD system we finally realized has a 40° × 30° FOV and 15-mm exit pupil diameter.


  • KEYWORD

    Head-mounted display , Freeform surface , Combiner , Coaxial relay lenses

  • I. INTRODUCTION

    Developing a see-through head-mounted display (HMD) system with wide field of view (FOV) and good image quality has been a great challenge to designers for decades, since the design process of an HMD system requires an interdisciplinary approach [1]. Nowadays HMDs have undergone continuous and rapid development, due to the rapid advance of these interdisciplinary technologies, and thus have been widely used in many applications, such as virtual reality, augmented reality, training, medical visualization, and entertainment industries [2-5]. Figure 1 is the structure of a see-through HMD system. We can see that a see-through HMD system usually consists of an image source, an optical relay lens group, and a combiner. The image source is used to display information generated by the computer. The intermediate image is formed between the relay lenses and the combiner, and is positioned in the line of sight by reflection from the combiner. Through the combiner, the outside scene and the information generated by the computer can be simultaneously observed. The relay lens group is used to correct off-axis aberrations introduced by the tilted combiner.

    Since being introduced decades ago, many different structures and methods to achieve see-through HMD systems with good imaging and wide FOV have been reported. There are many designs using a freeform surface to correct the aberrations and thus attain good image quality. However, they use a non-axial relay lens group or very complicated structures to correct the off-axis aberration introduced by the tilted combiner. These introduce extra difficulties during assembly of the HMD system. In 2010 Zhenrong Zheng et al. presented a design for an off-axis see-through HMD system with 40° × 30° FOV and 70-mm eye relief by using an x-y polynomial surface [6]. In 2013 Jianming Yang et al. reported a lightweight, see-through, distortion-free HMD system with 160° × 150° binocular FOV by using two similar ellipsoids [7].

    A design for a see-through HMD system with compact structure and large FOV is proposed. We achieved a coaxial structure in the relay lens group with a freeform surface as the combiner. The coaxial structure is simple and has loose tolerance requirements. The see-through HMD system we finally realized has a 40° × 30° FOV and 15-mm exit pupil diameter.

    II. DESIGN PROCESS

       2.1. Combiner Design

    There are two design concepts for optical see-through HMD systems: One is based on a flat combiner, often providing an on-axis system, while the other has optical power, often providing an off-axis system. The off-axis design has many advantages: It minimizes the size of the system, avoids a ghost image, and provides a wider FOV. However, the off-axis system also has many disadvantages: It is difficult to correct the off-axis aberrations introduced by the tilt combiner, the structure of the relay lens group is complicated, and the tilt and decentering in the element make fabrication and assembly very tough.

    According to earlier structures of the HMD, the difficulty in correcting off-axis aberrations is due to the chief rays not converging to a single point after reflection from the combiner [8]. As shown in Fig. 2, in our design of the HMD system, the chief rays exiting from point S are meant to converge at a single point T after reflection from the freeform surface. In this paper, we generate the freeform surface by considering both coordinates and normals of the discrete data points [9].

    According to Ref. [10], a set of partial differential equations (PDEs) is introduced to describe the freeform surface as:

    image

    (ρ , φ , θ) are the spherical coordinates of any point Q on the freeform surface. I=(Ix, Iy, Iz) or O=(Ox, Oy, Oz) is the unit direction vector of the incident rays or the emergent rays respectively, which can be easily determined from the points S, T, and Q. nO and nI are refractive indices; here nO = nI = 1.

    By solving Eq. (1) numerically via the finite differential method (FDM), global coordinates of a series of discrete data points on the freeform surface are obtained. From Snell’ law:

    image

    where Ni (i=1,2…I) is the corresponding normal of these data points.

    In our design, an x-y polynomial surface is chosen to express the freeform surface. The sag for the x-y polynomial surface is shown as:

    image

    where Z(x, y) is the sagittal plane, k is the conic constant, c is the curvature of the surface, and r is the height above the optical axis; then r2 = x2 + y2. Ai is the coefficient of the x-y polynomial, which can be written as:

    image

    The number of polynomial terms is N=i(i+1)/2+j. Tilt of the freeform surface causes asymmetry about the XOZ plane of the optical system, but symmetry still exists about the YOZ plane; thus we keep even terms of x in x-y polynomials. The freeform surface is generated by fitting both the coordinates Qi and the normals Ni with Eq. (3). The ultimate freeform surface we found is shown in Fig. 3. We can see that the off-axis freeform mirror attained the required imaging quality of the chief rays.

       2.2. Relay Lens Design

    In the HMD system, the relay lens group plays an important role in correcting off-axis aberrations, such as coma, astigmatism, and asymmetrical distortion introduced by the tilted combiner [11]. To correct the off-axis aberrations, the tilt and decentering introduced in the relay lens element make fabrication and assembly very tough. To reduce assembly difficulty, the relay lenses are desired to be in a coaxial configuration. As shown in Fig. 1, the rays will form a bent image with tilt and decentering, which we call an intermediate image, between the relay lens group and the combiner. According to the Scheimpflug principle [12], we can introduce the tilt and decentering in the relay lens group about the axis AB to correct the tilted bent image.

    Figure 4 is the original structure of the relay lens group. It consists of coaxial spherical lenses in six pieces with tilt and decentering about the optical axis AB. The image plane also has a small tilt about the optical axis.

       2.3. The Original Structure of the HMD

    The original structure of the HMD system is presented in Fig. 5: The combiner is a freeform surface with a 24.6° tilt and 29.2-mm decentering, and the relay lens group consists of coaxial spherical lenses in six pieces. The structure is compact, but has poor imaging quality. As shown in Fig. 6, the Modulation Transfer Function (MTF) is unacceptable low. Thus, further optimization must be performed.

    III. OPTIMIZATION

    The optimization of the see-through HMD system can be mainly divided into two steps. First, we set the tilt and decentering of the relay lens group, the curvature of each relay lens, and the lens interval as the variables to optimize the system. By performing this step, we can correct the off-axis aberrations introduced by the combiner and meet the FOV requirement. Second, we aim to optimize the system to attain a good image quality. In a practical application, the pupil diameter of the human eye is approximate 3 mm; meanwhile, due to movement of the helmet, the 3-mm pupil is at a random position inside the 15-mm exit pupil. As shown in Fig. 7, when evaluating image quality, the reasonable MTF is not the value at the 15-mm exit pupil diameter, but the value at the random 3-mm eye pupil inside the 15-mm circle. All the parameters of the HMD system are set as the variables. Through optimization, the HMD system is meant to simultaneously satisfy the imaging requirements at the 15-mm exit pupil and the random 3-mm eye pupil, located within the exit pupil. Moreover, a 6-piece relay lens structure cannot correct off-axis aberrations well, so we add one more lens behind the second lens in the group, to correct the off-axis aberrations introduced by the combiner.

    IV. DESIGN RESULTS

       4.1. The Ultimate Structure

    Figure 8 shows the ultimate structure of the see-through HMD system, which consists of a freeform surface combiner and 7-piece coaxial relay lens group. This HMD system has a compact structure with a 40° × 30° FOV and 15-mm exit pupil diameter.

       4.2. Imaging Quality Analysis

    Figure 9 shows the best and worst MTFs among the different 3-mm-diameter evaluation points. The best MTF is about 0.7 at 30 lp/mm, corresponding to area I in Fig. 5. The worst MTF is about 0.5 at 30 lp/mm, corresponding to area II in Fig. 5. The average MTF is greater than 0.5 at 30 lp/mm. As shown in Fig. 10, the distortion is about -16.76%, while the FOV is 40° × 30°. The distortion is caused mainly by the tilt of the combiner and is difficult to correct optically. Correction of the distortion by digital-domain methods is strongly recommended [6].

       4.3. Tolerance Analysis

    Tolerance analysis is an essential process in optical design for evaluating the processing difficulty of a system [13]. Table 1 shows the tolerance value of the HMD system calculated by the data using the Zemax software. We can see that the HMD system has a loose tolerance and thus can be easily assembled. Moreover, during the tolerance analysis we ran a Monte Carlo simulation 1000 times and used the normal distribution to analyze the MTF. Figure 11 is the cumulative probability for different MTF values under the conditions in Table 1. There is an 80% probability that the MTF is higher than 0.5 at 30 lp/mm. We can see that the HMD system has a loose tolerance and can be fabricated easily.

    V. CONCLUSIONS

    A design for a see-through head-mounted display (HMD) with a freeform surface is proposed. We realize a simple HMD system with 40° × 30° FOV and 15-mm exit pupil diameter. The HMD system we have realized includes a combiner and 7-piece coaxial relay lens group. We use a freeform surface as the combiner, which benefits a compact structure and good image quality. Moreover, the coaxial structure of the relay lenses avoids decentering and tilt in the group, and benefits the assembly of the relay lenses.

  • 1. Rolland J. P, Thompson K 2011 “See-through head worn displays for mobile augmented reality,” [Proc. China National Computer Conference] google
  • 2. Hua H, Hu X. D, Gao C. Y (2013) “A high-resolution optical see-through head-mounted display with eyetracking capability,” [Opt. Express] Vol.21 P.30993-30998 google doi
  • 3. Hu X. D, Hua H (2014) “High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics,” [Opt. Express] Vol.22 P.13896-13903 google doi
  • 4. Rolland J. P, Henry F (2000) “Optical versus video see-through head-mounted displays in medical visualization,” [Presence: Teleoperators and Virtual Environments] Vol.9 P.287-309 google doi
  • 5. Piao J. A, Li G, Piao M. L, Kim N (2013) “Full color holographic optical element fabrication for waveguide-type head mounted display using photopolymer,” [J. Opt. Soc. Korea] Vol.17 P.242-248 google doi
  • 6. Zheng Z. R, Liu X, Li H. F, Xu L (2010) “Design and fabrication of an off-axis see-through head-mounted display with an x-y polynomial surface,” [Appl. Opt.] Vol.49 P.3661-3668 google doi
  • 7. Yang J. M, Liu W. Q, Lv W. Z, Zhang D. L, He F, Wei Z. L, Kang Y (2013) “Method of achieving a wide field-of-view head-mounted display with small distortion,” [Opt. Lett.] Vol.38 P.2035-2037 google doi
  • 8. Wang J. H, Liang Y. C, Xu M (2015) “Design of panoramic lens based on ogive and aspheric surface,” [Opt. Express] Vol.23 P.19489-19499 google doi
  • 9. Zhu J, Wu X. F, Yang T, Jin G. F (2014) “Generating optical freeform surfaces considering both coordinates and normals of discrete data points,” [J. Opt. Soc. Am. A] Vol.31 P.2401-2408 google doi
  • 10. Ding Y 2009 “Design of freeform optical devices and their applications in illumination systems,” Ph. D. Thesis P.25-29 google
  • 11. Rolland J. P (2000) “Wide-angle, off-axis, see-through head-mounted display,” [Opt. Eng.] Vol.39 P.1760-1767 google doi
  • 12. Gross H 2005 Handbook of Optical Systems P.475-479 google
  • 13. Li X. L, Xu M, Ren X. D, Pei Y. T (2012) “An optical design of off-axis four-mirror-anastigmatic telescope for remote sensing,” [J. Opt. Soc. Korea] Vol.16 P.243-246 google doi
  • [FIG. 1.] Structure of the see-through HMD system.
    Structure of the see-through HMD system.
  • [FIG. 2.] (a) Schematic representation of the optical imaging system with the freeform mirror. (b) Geometric relationship of the freeform mirror and the rays.
    (a) Schematic representation of the optical imaging system with the freeform mirror. (b) Geometric relationship of the freeform mirror and the rays.
  • [] 
  • [] 
  • [] 
  • [] 
  • [FIG. 3.] Ultimate freeform mirror designed by fitting both the coordinates Qi and the normals Ni.
    Ultimate freeform mirror designed by fitting both the coordinates Qi and the normals Ni.
  • [FIG. 4.] Original structure of the relay lenses.
    Original structure of the relay lenses.
  • [FIG. 5.] The original structure of the HMD system.
    The original structure of the HMD system.
  • [FIG. 6.] The MTF of the original HMD system.
    The MTF of the original HMD system.
  • [FIG. 7.] 3-mm-diameter evaluation points inside the 15-mm exit pupil.
    3-mm-diameter evaluation points inside the 15-mm exit pupil.
  • [FIG. 8.] Ultimate structure of the HMD structure.
    Ultimate structure of the HMD structure.
  • [FIG. 9.] The best and worst MTFs among the random 3mm eye pupil.
    The best and worst MTFs among the random 3mm eye pupil.
  • [FIG. 10.] The grid distortion of the HMD.
    The grid distortion of the HMD.
  • [TABLE 1.] Tolerance values of the HMD system
    Tolerance values of the HMD system
  • [FIG. 11.] Possible values of the MTF at different evaluation points.
    Possible values of the MTF at different evaluation points.