3D Reconstruction of Human Face Using the Derivative Moire Topography
 Author: Bae Yoon Jae, Ha Byeong Wan, Park Ji An, Cho Choon Sik
 Organization: Bae Yoon Jae; Ha Byeong Wan; Park Ji An; Cho Choon Sik
 Publish: Journal of the Optical Society of Korea Volume 18, Issue5, p500~506, 25 Oct 2014

ABSTRACT
A new 3D reconstruction algorithm for the human face is proposed using the derivative Moiré topography which ensures fast and robust reconstruction even for rough surfaces. The Moiré interference fringe pattern is initially obtained through the projection Moiré topography based on phase shifting, and then differentiated to provide a full unwrapped phase map for a human face. 2π ambiguity, which has been a chronically unsolved problem with Moiré topography, is successfully surmounted by differentiating the Moiré fringe patterns both in x and ydirections when the object is located in the xy plane. A real human face is used for verifying the proposed derivative Moiré topography. A human face of 4 different phaseshifted images taken in the fixed plane is almost fully reconstructed in 3D format in 0.1 mm lateral resolution.

KEYWORD
Moire topography , Derivative Moie interference fringe pattern , 3D Reconstruction algorithm , Phase unwrapping , Phase Shifting

I. INTRODUCTION
Noncontact 3D surface reconstruction techniques have been extending their applications to various industrial fields such as medical, manufacturing and biometric areas. A large number of noncontact 3D reconstruction techniques developed so far show their advantages and drawbacks as listed in [1]. Among them Moiré topography has been of prime interest because it provides fast operation for smooth surfaces since it was introduced [24]. For Moiré topography, when the projected grating of the projection system falls on the object, the deformed grating is generated on its surface, Moiré interference fringe patterns are then generated on the imaging system, yielding contours of the same heights expressed in wrapped phases. Due to diffraction, a virtual grating is often employed for replacing reference and/or projected gratings. Projection Moiré topography was thereafter developed to increase the resolution significantly using the virtual grating for the projection system. In the meantime, the phase shifting method has been devised to ensure repeatability of the projection Moiré topography where multiple phase shifts are carried out to easily calculate the phase map for objects [5, 6].
Although Moiré topography usually ensures fast and efficient 3D reconstruction in smooth surfaces [7, 8], two big obstacles of noise and imperfect phase unwrapping make one hesitate to capitalize on this. To eliminate noise occurring in a wrapped phase map, cosine transform or wavelet transform has been tried [912]. The projection Moiré topography with phase shifting method provides simple and reliable wrapped phase images for objects with –π/2 to π/2 range, called 2π ambiguity, however it is very difficult to convert this wrapped phase to a noise free unwrapped phase map. Abrupt phase change more than π cannot be detected easily, so that very few techniques have been tried for achieving more exact phase unwrapping. The weighted least squares method with discrete cosine transform (DCT) was introduced and utilized where the least squares method was used for removing discontinuity (or noise) in the phase image and DCT was employed for phase unwrapping in a strict sense, however this method shows difficulty in tracking a sudden change of phase while unwrapping the wrapped phase image, and it requires huge computing power because of a large amount of computation [13].
In this work, the derivative Moiré topography by differentiating the wrapped phase map separately in x and ydirections is proposed to resolve the 2π ambiguity problem. Compared with the least squares method with DCT, the proposed technique tracks sudden phase change very efficiently using the simple differentiation in x and ydirections. This also does not introduce a large amount of computational burden, leading to very fast attainment of phase unwrapping.
II. THE PROPOSED MOIRE TOPOGRAPHY
The classical measurement setup to obtain the Moiré fringe pattern using the phase shifting method is shown as illustrated in Figure 1 where
I_{d} (·) is the light intensity of the deformed grating at (x, y) under test,I_{r} (·) is the light intensity of the reference grating with phaseshifting, and Ik indicates the resulting Moiré intensity at (x, y) when Δ(t ) = (k – 1)×90°. To avoid diffraction virtual projected and reference gratings for the phase shifting method can be established as shown in Figure 2(a) where the object is located in the xy plane, projection and imaging systems are apart byd , and the object is away byL from the projection system. Figure 2(b) describes the real measurement setup used for this work where four different virtual gratings for the projection and the reference are devised by applying the phase difference of 90° between consecutive gratings. A light source with four virtual projected gratings is emitted in sequence from the projection system and creates the Moiré interference fringe patterns equivalently to Fig. 1. The Moiré fringe pattern is therefore created when the deformed grating in (1) is superimposed on the reference grating in (2). In principle, the intensity of the Moiré pattern,I_{Moiré} (x, y ), is extracted from the fourth term of (3) which is the product of intensity of the deformed grating (I_{1} ) and that of the transmitted imaging through the reference grating of the imaging system (I_{2} ) where Is is the intensity of the light source,A is the modulation of gratings,R is the reflectance of the object surface,M is the magnifying rate of the projection system,G is the pitch of gratings, andh (x, y ) is the height of the object as in (4) [2 – 4]. If the argument within cos (∙) is represented byϕ (x, y ) as described in (5), the height of the object can be calculated as shown in (6) since in Fig. 2(a). To obtainϕ (x, y ) to a high precision, the phase shifting method as described in Fig. 2 is capitalized using the traditional equation as expressed in (3) whereϕo (x, y ) means the principal value of the wrapped phase, 90° to 90°. For the phase shifting method, projected and reference gratings (I_{r} ) can be generated virtually with a computer, the deformed grating (I_{d} ) is obtained on the surface object as seen in (7), and the resulting interference pattern in (8) can be extracted by multiplyingI_{d} andI_{r} with the computer, yielding no more diffraction and interference wherea_{d} is the magnitude of the deformed grating,a_{r} is the magnitude of the reference grating, D(x,y) is the average intensity of the resulting interference pattern, γ(x,y) shows the visibility of the interference pattern, and ϕ(x,y) = ϕd(x,y)  ϕr(x,y) yields the phase of the interference pattern. As seen in (9),ϕ° (x, y ) can only have the principal values from 90° to 90° which is called 2π ambiguity, therefore phase unwrapping should be carried out, in other words, the absolute Moiré order is required to be found accordingly.In this work, we perform the phase unwrapping by differentiating (9) twice in x and ydirections, leading to (10) where a primed subscript means the differentiation with respect to the subscript. Since the derivative phases as expressed in (10) are continuous functions even around the big phase jump, phase can be tracked precisely without difficulty even if it is greater than 90º or less than 90º. The principal phase is at first calculated using (9) and then one differentiation is carried out in one direction (
i.e . x or ydirection) and thereafter another differentiation is taken in the other direction.The absolute Moiré order at the (
i ,j )th pixel,n_{i,j} , to obtain the exact phase can be determined using (11) at first in the xdirection. In Moiré topography, the unwrapped phase map is composed of many contour lines where the absolute order difference between a contour and its neighboring contour is always 1 for the regular case in which no discontinuous contours exist. Even in the case for which the height of the object changes abruptly, contours are distributed densely not discontinuously. The irregular case occurs when two different heights are overlapped or black spots such as the pupil of eye, nostril or eyebrow are created, providing discontinuous contours, in which no Moiré patterns are generated and then no continuous phase contours are obtained. Therefore, the absolute order can be calculated using the derivative of phase change based on current phase contours. A sudden phase change more than nπ yields addition of n from the current absolute order, where the order is updated in x and ydirections while scanning all of the pixels of the object. But since the irregular case provides incorrect phase derivative, this error is accumulated through scanning up to the final pixel point.where
fix (•) function removes the fractional part and returns the integer part, adding the corresponding integer (n) value when the phase becomes greater than nπ. Figure 3 illustrates an example showing the case for n equal to 1 at thei th pixel. With thisn_{i,j} , the phase is unwrapped in the xdirection as expressed in (12),where the calculated phase from (12) may accumulate errors due to discontinuous contours. To resolve this error accumulation, phase compensation is carried out by calculating a different derivative with respect to wider step in the xdirection. Accumulated phase error is detected and incorrect phase becomes recovered by computation of intermediate order (
n^{xc} ) as represented in (13) wherea needs to be predefined as a big number (usually ≥ 50). The left term forj ≥a in (13) indicates accumulation of errors by summing phase variations from [(k 1)a +1]^{th} pixel to [ka +1]^{th} pixel, and the right term expresses accumulation of errors by summing phase variation from [ka +1]^{th} pixel to jth pixel as shown in Fig. 4. Forj ＜a , phase variation from 1st pixel toj ^{th} pixel is calculated forn^{xc} . However, it should be noted that although phase error is not accumulated using (13) and (14), the absolute order cannot be obtained correctly in the vicinity of discontinuous contours due to overlapping heights or black spots.The absolute Moiré order is now calculated once more in the ydirection using (15),
Eventually the phase is unwrapped using (16),
III. MEASUREMENT RESULTS
To validate the proposed method, a human face as shown in Fig. 5 is taken for an object which is located 60 cm away from the projection and imaging systems. The projection system used here has 1200×800 pixels and provides 28 cm×35 cm image to the plane 60 cm (
L ) away from the projection and imaging systems which are 11.5 cm (d ) apart from each other. Projected and reference gratings are generated virtually with pitch (G ) of 1 mm. The imaging system consisting of a CCD camera and a lens has 2592×1944 pixels, therefore the minimum resolution is calculated to be 0.144 mm in the xdirection and 0.135 mm in the ydirection. Four deformed gratings obtained on the surface of the object using four different projected gratings are exhibited in Fig. 6 where the original images are converted to the corresponding gray values to provide the light intensity (I_{k} ). Using deformed and reference gratings, Moiré fringe patterns are calculated initially using (9) where the phase is not unwrapped. The initial phase for Moiré fringe patterns is calculated by subtracting the phase of the reference grating from that of the deformed grating as deduced in (8). Due to abrupt phase variation, (10) to (16) are used for producing the exact phase unwrapping specifically in the vicinity of sudden phase change.Figure 7 exhibits initial phase map obtained with (9) which shows the principal phase image from 90° to 90°. Figure 8(a) displays the unwrapped phase map applying the phase derivative in the xdirection, Fig. 8(b) shows the compensated phase map employing the additional derivative as introduced in (13), and Fig. 8(c) shows the finally unwrapped phase map which applied the phase derivatives in x and ydirections as introduced in (10) to (16). Discontinuous contours due to overlapping two different heights and black spots lead to failure of phase unwrapping as shown in Fig. 8(c) where black spots provide errors in obtaining exactly unwrapped phase, as expected in Sec. II. A detailed procedure from (11) to (16) is offered for verification of the proposed method as displayed in Table 1 where phases before unwrapping are obtained from (9), and from these data the final unwrapped phases are successfully calculated using (11)(16). The negative phases illustrate relatively lowered phases with respect to the highest altitude of cheek in Fig. 4.
The final 3D picture of a human face is reconstructed as illustrated in Fig. 9. For a clear view, the background image is removed and textured for vivid visualization. Finally a moving average with 50×50 window size is performed for removing ripples specifically around black spots such as eyebrow and nostril, providing smooth 3D surface and adding a bit more CPU time. Vague stripes still appear on the cheek as seen in Fig. 9 since the light intensity of projected gratings are not perfectly sinusoidal, leading to some discontinuity for phase unwrapping.
Compared with the least squares method with DCT, the proposed derivative Moiré technique shows fast computation and precise reconstruction for even a rough surface such as a human face. DCT has been applied to unwrap the phase image of the human face as taken in Fig. 5, however it does not produce the complete 3D reconstruction. Therefore a very simple and periodic principal phase image was used to compare DCT and the proposed method as seen in Fig. 10. CPU times have been computed for three cases as summarized in Table 2, where the derivative Moiré topography shows much faster time taken for phase unwrapping.
IV. CONCLUSION
3D reconstruction has been successfully implemented using the derivative Moiré topography based on phase unwrapping using differentiation of phase in x and ydirections. Abrupt phase change is identified successfully by this method, leading to the precise and fast height computation of objects containing rough surfaces. Differentiation is performed both in x and ydirections for obtaining noise free phase unwrapping. From this, errors occurring from inexact phase unwrapping are eliminated as the differentiation iterates. The classical phase shifting method is replaced by building four phaseshifted virtual reference gratings. A human face is exploited to validate the proposed method. This proposed method will provide a potential direction for systems necessitating robust and fast 3D reconstruction.

[FIG. 1.] The conventional Moire topography based on the phase shifting method.

[FIG. 2.] Measurement setup for the proposed Moire topography. (a) Phase shifting method based on virtually generated 4 reference gratings. (b) Real measurement setup.

[]

[]

[]

[]

[]

[]

[]

[]

[]

[]

[]

[FIG. 3.] Principle of phase unwrapping.

[]

[FIG. 4.] Detection of phase error accumulation.

[]

[]

[]

[]

[FIG. 5.] A human face photo used for this work. FIG.

[FIG. 6.] 4 different deformed grating images obtained with virtual reference gratings.

[FIG. 7.] Initial phase wrapped image.

[FIG. 8.] Phase unwrapped images based on (a) xdirected differentiation, (b) phase compensation, (c) ydirected differentiation.

[TABLE 1.] Results for phases and orders at 10 pixels between brows in Fig. 4

[FIG. 9.] The final 3D reconstructed image.

[FIG. 10.] A principal phase image used for comparison.

[TABLE 2.] Comparison of two phase unwrapping methods in terms of CPU time