For the free space optical communication (FSO) system, the performance of the coherent detection scheme is better than that of the intensity modulation direct detection, with higher spectral efficiencies and data rates, a greater ability to decrease both background and thermal noise, and more sensitive receivers [1-4]. However, the atmospheric turbulence greatly degrades the performance of the coherent FSO systems [5]. The wavefront distortion, scintillation, beam wandering and spreading caused by a turbulent atmosphere will not only degrade the entrance efficiency of the receiving antenna but also cause mismatch of the field of the signal beam and the local oscillator (LO) [6].

An adaptive optics (AO) system is identified as an effective way to compensate atmospheric turbulence in a coherent FSO system [7, 8]. And as the main component of an AO system, the wavefront sensor is attracting extensive attention [9]. The Shack-Hartmann wavefront sensor (SHWFS) is widely used for turbulence compensation for a coherent FSO system [10]. Belmonte put emphasis on elucidating how the addition of AO to the transmitter or receiver can reduce the effects of atmospheric propagation and on quantifying the improvement on the performance of FSO systems regarding coherent detection [11, 12]. Zuo investigated the bit-error-rate (BER) performance of the FSO links in weak non-Kolmogorov turbulence and showed that BER decreased sharply with AO corrections. Considering the influence of both the amplitude fluctuation and spatial phase aberrations, when the ratio of receiving aperture diameter D to the coherent length r₀ (D/r₀) was large enough, the Zernike mode was accurate [13, 14]. Li evaluated the performance of the coherent FSO system employing quadrature array phase-shift keying modulation over the maritime atmosphere with AO system based on SHWFS for atmospheric turbulence compensation [15]. C. Liu and J. Huang analyzed the improvement of BER for the coherent FSO system with AO system based on SHWFS through numerical simulation and experimental data of the 1.8 m telescope with a 127-subaperture SHWFS, under different D/r₀ [16-19]. However, the SHWFS was so sensitive to laser scintillation that the performance was limited seriously for a coherent FSO system. The focal plane wavefront sensor solved this problem. It had higher power usage ratio, but it was much more costly in time [20]. In 2000, Ghebremichael et al. applied computer-generated holographic elements to wavefront sensing and corrected the response curve [21-25]. Changhai et al. used the plane wave in combination with the Fourier lens for the replacement of spherical waves to improve the efficiency of energy utilization [26-28].

In this paper, based-on our previous work, HMWFS is applied to the coherent FSO systems to evaluate the BER performance improvement. First, the principle of HMWFS and the BER of the coherent FSO communication are given in brief in Section 2. Then, the numerical simulation is used to verify the feasibility of the HMWFS for the coherent FSO system, and the BER improvement for the coherent FSO system with HMWFS based on our previous works are presented in Section 3. Finally, we summarize and present our conclusions in Section 4.

The theoretical layout of HMWFS is shown in Fig. 1 [29].

An object beam with the minimum expected amplitude of the aberration A_minZ_i and a reference beam focused at point A on a distant detector record one hologram, as shown in Fig. 1(a). Another hologram is recorded by an object beam with the maximum amplitude of the aberration A_maxZ_i and a reference beam focused at a different point B , as shown in Fig. 1(b). Then a multiplexed hologram is generated with the pair of holograms [29].

The hologram will be reconstructed with an input beam with some amplitude, A_i , which is between the maximum and minimum values, The beams will be focused at point A and point B , as shown in Fig. 1(c). The relative intensity of at these points is related to the coefficient of Zernike item. Thus, after a calibration, the amplitude can be obtained simply by sensing the spot intensities.

In addition, the multiplexed hologram with more pairs of holograms is needed to increase the number of aberration modes detected. Considering the pairs of spots are spatially separate, the amplitudes of these spots can be read out in parallel by a sensor array. Thus, the Zernike modal amplitude can be obtained through comparing the relative intensities of pairs of spots [29].

According to Fourier optics, suppose the amplitude of the signal beam is U_S,0 , the phase function is ϕ(r_s ) , then the complex amplitude of the signal beam can be expressed as:

And the complex amplitude of the reference beam can be expressed as:

Where, U_R,0 is the amplitude of the reference beam, and ϕ(r_R ) is the phase function of the reference beam.

The two waves interfere on the recording medium, and generate a holographic pattern. When a replay beam with random aberrations illuminates the holographic pattern, the wavefront of the signal beam can be reconstructed on the focal plane. The diffraction field distribution on the focal plane shall be expressed as [29]:

Where c.c is a constant, is the conjugate of U_S( r) , U_P(r_P, r) is the replay beam, and the G(r_d, r) is the Green’s Function. In free space, G(r_d, r) can be approximately expressed as:

Where, k_d is the wave vector of laser beam (U_d)r_d and ( x_d , y_d , z_d ), ( x, y, z) are space coordinates.

Assume the Fresnel approximation conditions are satisfied, the thickness of the hologram is ignored, and λ =λ_S =λ_P =λ_d , (U_d)r_d can be expressed as [29]:

Where, Π is the pupil function, and the range of integration is limited within the pupil. a and b are respectively the length and width of the hologram. F{.} is the Fourier transform. Thus, the Fresnel integral is simplified into a Fourier transform with thin hologram conditions. Thus, the distribution of complex amplitude on the holographic plane can be solved by a numerical method.

As mentioned in our previous work, for a coherent FSO system, the received beam is mixed with an LO signal. The received beam and the LO signal are given by [16]:

where, U_S(r, φ) and U_LO(r, φ) are the amplitudes of received beam and LO signal. ψ_S(r, φ) and ψ_LO(r, φ) are the variable phases related to spatial position of the laser carrier and LO signal, respectively. ω_S and ω_LO are their angular frequencies, respectively. The current after frequency mixing is given by:

where H_T is the magnetic field distribution after frequency mixing, E_T × H_T is the Poynting vector, η is the quantum efficiency of the detector, c is constant, and σ is the area of the detector.

Since the intensity of the LO signal is much larger than the intensity of the received beam, the detector noise is mainly shot noise. Thus, the mean square value of the noise current is given by [16]:

where B is the noise bandwidth of the detector, and I_LO is the direct current coursed by the LO signal. The signal to noise ratio (SNR) is defined by the ratio of the mean square value of the intermediate frequency signal current and the mean square value of the noise current [16]:

where P_S is the laser signal power. Specially, when ω_IF = 0, which is considered as homodyne detection, we can obtain mixing efficiency as [16]:

According to the Eq. (12), the mixing efficiency is related to both the amplitude of received beam and LO signal, and spatial phase distribution difference. Assuming the received beam and the LO signal are plane waves with uniform intensity, the mixing efficiency is influenced directly by the spatial phase distribution difference. The AO technology is generally regarded as an effective way to decrease the spatial phase distribution difference between the carrier signal and the LO signal.

In the coherent detection system, the BER can be given by [16]:

where erfc is the complementary error function, . For synchronous binary phase shift keying (BPSK) modulation, the optical power of the received beam is expressed as:

where N_p is the number of photons received within a single bit. For the BPSK modulation, the BER of the homodyne detection can be given as [16]:

Based on our previous work, in terms of the computergenerated holograph, the center wavelength of the laser is λ = 632.8 nm, the number of sampling points of the holographic grating is 600×600 pixels, the aperture of holographs is 60 mm, the distance between image plane and holographic plane is 1500 mm, the off-axis distance of the spot on focal plane is larger than 1 cm. The low eight order Zernike aberrations (Z3, Z4, …, Z10) are coded in this simulation. Where the peak-to-valley (PV) value of Z3, Z4, Z5 is λ / 2 , and the PV value of Z6, Z7, Z8, Z9, Z10 is λ / 4 . The holograph we eventually obtained is shown in Fig. 2 [29].

According to the theory of the HMWFS, the relative intensity distribution on the holographic image plane is shown in Fig. 3 [29]. By comparing the relative intensity on the holographic image plane, the Zernike aberrations are obtained quantitatively.

The mixing efficiency is approximate to the Strehl ratio of the far field of the received beam for homodyne detection, and it is defined by the ratio of the far field encircled energy and the diffraction limited encircled energy [18]. The intensity distribution on the focal plane before correction (detail view) is shown in Fig. 4.

Thus, it can be seen that the holographic modal wavefront sensor has smaller residual in low eight orders Zernike aberrations detection. The intensity distribution on focal plane after aberrations correction is shown in Fig. 5.

Then, we analyze the BER before and after aberrations correction based on Eq. (14) when N_p = 10 and η = 1. The results are shown in Table 1.

We make another five groups numerical simulation by changing the introduced aberrations. In our simulation, we introduce the Zernike mode aberrations into the laser beam before correction. The root-mean-square (RMS) values of the aberrations before correction are between 0.15λ and 0.25λ . And RMS values of the residual aberrations after correction are nearly below 0.1λ [29]. Thus, we get the five images before and after correction as shown in Fig. 6.

Where, the Fig. 6(a) shows the images before correction, and the Fig. 6(b) shows the images after correction. The BER before and after correction are shown in Table 2.

As shown in Table 2, according to homodyne detection coherent FSO system, we can see that before correction, the BER is above 10⁻⁴, and after correction, the BER is below 10⁻⁵. According to the BER before correction, we can estimate the atmospheric turbulence intensity. Under some weak atmospheric turbulence conditions, the BER is about 10^-4 before correction. And after AO correction based on HMWFS, the BER can reach below 10⁻⁷. In the meanwhile, under some strong atmospheric turbulence conditions, the BER is about 10⁻² before correction. And after correction, the BER can achieve below 10⁻⁵. Above all, the AO system based on HMWFS is an effective way to improve the BER performance of a coherent FSO system with homodyne detection conditions. According to Table 2, we can also verify that the wavefront sensing we proposed is better for weak atmospheric turbulence with fast detecting rates and insensitivity to beam scintillation.

In this paper, we analyzed the principle of HMWFS in theory and gave the BER expression of the coherent FSO system, and then first applied HMWFS to a coherent FSO system, and then we analyze the BER performance improvement through numerical simulation based on HMWFS. Finally, we discussed the impact of HMWFS on BER of the coherent FSO system before and after the aberrations correction based on HMWFS. According to the results of numerical simulation, the wavefront sensor we proposed is better for weak atmospheric turbulence. The most obvious advantage of HMWFS is fast detecting rates and insensitivity to beam scintillation.

The disturbance strength is only described by the value of the introduced aberrations in our work. The relationship between the introduced aberrations and the structure constant of refractive index will be carried out in the near future.