The hybrid optical amplifier (HOA), composed of a Raman and an erbium-doped fiber amplifier (EDFA), has emerged as a promising solution for extending the span and transmission capacity of dense wavelength division multiplexed (DWDM) systems [1]. It is attractive because of its ability to tailor the gain profile, compensate for fiber dispersion loss, and suppress spontaneous noise [2]. Although Raman-only amplifiers have demonstrated the capability to improve system BER performance, Raman–EDFA HOAs are comparatively more power-efficient and cost-effective [3, 4]. However, a drawback of a Raman amplifier is that nonlinear effects such as stimulated Brillouin scattering, self-phase modulation, cross-phase modulation, and four-wave mixing degrade the signal when the amplifier has large output power [5]. On the other hand, the nonuniform gain spectrum in conjunction with the saturation effects in EDFAs cause an increase in signal power and decrease the optical signal-to-noise ratio unacceptably [6]. Therefore, both the amplifiers and their applied systems must be carefully designed to install an HOA in a DWDM system at reduced channel spacing.

Thus it is mandatory to optimize the important parameters (such as Raman length, EDFA length and pump power, etc.) before an HOA is deployed. In the literature, various optimization techniques are used to optimize the parameters of HOAs to improve system performance in the terms of gain, gain flatness, transient effect, etc.

Carena et al. [7] presented a mathematical optimization technique to yield the optimal configuration of a Raman-EDFA HOA for a desired optical signal-to-noise ratio. It was reported that the span distance is around 30 km when only EDFAs are used, while it increases to 50-60 km in the case of an HOA. In [8] Kaler investigated a simulationbased DWDM system to optimize the Raman-EDFA HOA. Unfortunately this investigation is based on single-parameter optimization, i.e. only one parameter of the HOA is varied at a time. In the previous research papers [9-11], various global optimization techniques (such as genetic algorithm, hybrid genetic algorithm, neural network learning) were presented to optimize a Raman or EDFA amplifier only. Until now no global optimization technique has been applied to optimize the Raman-EDFA HOA in an optical communication system. As far as the mathematical model of the HOA is concerned, according to [5] the total gain is the multiplication or addition of individual gains of cascaded amplifiers, but for better understanding the net-gain model of the HOA has to be calculated, which has not been done in the literature.

Recently we have proposed various combinations of optical amplifiers and investigated the impact of reduced channel spacing at high bit rates [12]. It was observed that the Raman-EDFA HOA is the best combination, but in this investigation optimization of HOA was not performed. In this paper we extend the previous results of [8-12] by designing a net-gain mathematical model and optimizing the Raman-EDFA HOA using a genetic algorithm.

This paper is organized as follows. A model for the gain of the Raman-EDFA HOA is described in Section II. In Sections III and IV the proposed genetic algorithm and its analogy for an optical system is presented to provide the optimal parameters. The system setup and results are described in Section IV. Section V summarizes the conclusions.

As mentioned, previously the total gain of cascaded amplifiers was considered to be the sum or product of their individual gains, but the gain of the second cascaded amplifier (net gain) depends on the first amplifier's gain. Thus in this section we derive a net-gain model by considering those actual conditions. The mathematical model is divided into two parts: In part A an expression for variation of pump power and signal power along the EDFA length is determined, while in part B, after considering the effect of Raman output power on EDFA power, an expression for the net signal gain is established.

In this investigation, the EDFA is assumed or modeled as two-level system, so

where N₁ is the population density in the ground state, N₂ is the population density in the meta stable state, and N_t is the total erbium ion density in the core of the EDFA. Then the rate of change of population N₁ at ground-level energy E₁ is given as [13]:

where P_PE is the pump power, PSE is the signal power, σ pa is the absorption cross section at pump frequency 𝜐_p, σ_sa is the absorption cross section at signal frequency 𝜐_s , σ_se is the emission cross section at signal frequency 𝜐_s, a_p is the cross-sectional area for the fiber modes for λ_p, a_s is the cross-sectional area for the fiber modes for λ_s, and 𝜏_sp is the spontaneous emission lifetime for transition from E₂ to E₁.

On the right-hand side of equation (2), the first term is the rate of absorption per unit volume from E₁ to E₃ due to pumping, the second term is the rate of absorption per unit volume from E₁ to E₂ due to the signal, the third term is the rate of stimulated emission per unit volume from E₂ to E₁ due to the signal, and the fourth term is the rate of spontaneous emission per unit volume from E₂ to E₁ due to the signal.

Similarly, the rate of change of population N₂ at the upper amplifier level is [13]:

Under the steady-state condition Substituting this expression into (3), after rearrangement the expression becomes:

By neglecting the contribution of spontaneous emission, the variations of the pump power P_p and the signal power P_s along the length of amplifier are calculated as:

In equations (5) and (6) the fiber losses (𝛼 and 𝛼′) are also neglected for small erbium-fiber length.

To determine the net gain, first we have to calculate the Raman output power, and then final gain is calculated by substituting this power into equations (5) and (6). The Raman-amplification process is governed by the following set of two coupled equations by considering a single continuous wave pump beam to amplify an optical signal [14].

where P_SR is the signal power for the Raman amplifier, g_R is the Raman gain coefficient, P_PR is the pump power for the Raman amplifier, a_p is the cross-sectional area of the pump beam inside the fiber, 𝛼_s and 𝛼_p are fiber losses at signal and pump frequencies 𝜔_s & 𝜔_p respectively, with ‘+’ and ‘−’ signs for forward and backward, and the ‘−’ sign for backward pumping.

For practical situations, the pump power is so large compared to signal power that pump depletion can be neglected by setting g_R=0 in equation (8) and considering forward pumping only [15],

Integrating both sides,

After solving we have:

From this equation the Raman output power can be calculated as:

Substituting the value of P_PR = P_Pout-R in equation (7),

Integrating both sides,

Now the net gain (G_net) can be calculated as:

where

Now dividing and multiplying the second term of equation (11) by L_R:

Also, P_sout-R = P_SE is the input signal to the EDFA.

Substituting the value of P_SE into (5) and rearranging:

For convenience in solving this equation, let us denote signal power by P_s only,

Integrating both sides for L_E as the length of the EDFA,

From equation (1) we see that N₁=N_t−N₂.

Using the expression for N₁ in the above equation we get:

After a long calculation and taking the exponential of both sides we have:

where pump rate 𝜔_sa is the stimulated absorption rate, and 𝜔_se is the stimulated emission rate.

This section starts with an overview of the GA, followed by its implementation for optimization of a Raman-EDFA HOA. A GA is a global search optimization algorithm based on biological evolution. A GA allows a population composed of many individuals to evolve to a state that maximizes “fitness” under specified selection rules. The details and implementation of GA can be found in [10].

Applying this simple GA in the optimization of an HOA can be broadly subdivided into the following steps, and their sequence can be represented by the flow diagram of Fig. 1.

Step 1: Initialization of GA parameters and population for the various system parameters, i.e. Raman length, EDFA length, and pump powers. During this stage the ranges for the parameters, i.e. the limits of the search space, are defined. The parameters for the GA and ranges of values for the HOA are given in Tables 1 and 2 respectively.

Step 2: The counter is started for the number of generations at the beginning. The generations proceed iteratively towards the desired results.

Step 3: For each generation there are substages to evaluate the fitness value (i.e. gain) and then modify the set of parameters to achieve maximum gain. Evaluation of amplifier gain for the various combinations of parameters is performed by calling the HOA model as a subprogram. The set of parameters obtained from a randomly generated population is passed within the function call to the subprogram one by one, for the whole population. This subprogram, upon receiving the combination of all parameters to be optimized, repeatedly evaluates the gain using equation (12) and returns the average value. The current fitness value is compared to the previous fitness, and if it is greater then its set of parameters is taken as the better solution

Step 4: During the next substage the current population of individuals is modified. The fitness value and selection probability are passed to the selection function. Here the fitness is an array of gain values respective to the set of all parameters in the current population. Tournament selection chooses a random value for chromosomes depending on a small probability, as defined in Table 1. New chromosome pairs are obtained from these selected chromosomes by a crossover method. These newly generated chromosomes form a temporary new population which replaces the original population after a mutation operation has been performed on each of the new chromosomes. Finally a new, improved population is obtained.

Steps 3 and 4 are repeated until the final generation is reached. It can be seen that the amplifier gain increases with the succession of generations. Since the proposed method for employing the GA includes tournament selection, crossover method, and mutation adopted collectively, it converges on the maximum gain in a few generations, after which further modification is undesirable.

Table 3 shows selected results obtained from the genetic algorithm as a function of the expected gain of more than 30 dB. The maximum gain (31.1 dB) is obtained for the optimal solution presented in Case 5.

In this section we describe an analogy of the GA to an optical communication system. In general the GA is a biological evolution method to search among a large number of possibilities for a solution by varying the combinations of individuals. As the proposed optimization technique is inspired by the GA, we have taken the individuals to be combinations of HOA parameters (such as Raman length, EDFA length, Raman pump power, and EDFA pump power). In the current investigation, by using GA selection rules the individuals/amplifier parameters are varied until the maximum gain is achieved. The flow chart in Fig. 2 clearly shows the analogy of the GA in which the following terms (GA terms and optical-system terms) are associated with each other: individuals ↔ HOA parameters, fitness ↔ gain calculated from equation 12, generation ↔ combinations of individuals/amplifier parameters.

To check the performance of the optimized HOA at the system level, a numerical simulation is carried out. The system setup consists of 50 DWDM channels, according to the ITU-T G.694.1 standard, with a wavelength spacing of 0.2 nm using continuous-wave lasers, as shown in Fig. 3. We have investigated this system with per-channel input laser powers of -10 dBm. The data stream from a 10 Gbps pattern generator with an NRZ binary sequence is pre-coded and drives a sine-squared amplitude modulator. The optimized parameters of different fibers and pumps are taken from Case 5, as described in Table 3.

Figure 4 shows the gain and noise spectra for the optimized Raman-EDFA HOA over the ITU-T channels. The variation of gain with wavelength is not uniform, as each amplifier induces its own nonlinearities. It can be observed that each wavelength has a gain of more than 17 dB. In the simulation various losses have been considered, such as attenuation due to fiber and other optical components, coupler losses, etc., unlike in the mathematical analysis represented in Section 2. This is the reason for the reduction of gain in the simulation. Using the optimized parameters we have obtained ̴1.5 dB of gain flatness with less than 7.7 dB of noise per channel, which shows an improvement even at this compact channel spacing [1,5]. This noise figure is observed due to the pumping scheme used (counterpropagating pump) and large Raman fiber length (140.44 km), as also reported in [16].

Due to the gain dynamics induced by the optimized HOA, distortion of pulse shapes and crosstalk between channels are present. These crosstalk effects are due to the induced nonlinearities such as stimulated Raman scattering, four-wave mixing, self- and cross-phase modulation, etc. The induced crosstalk directly affects the bit error rate of the system. The BER also varies according to the signal format, dispersion, and transmission speed, as well as noise accumulation, saturation, and reflection effects through the amplifiers.

In our simulation the BER is calculated using the following equation:

where the quality factor Q can be calculated from:

in which m1 and σ1 (m₀ and σ₀) are respectively the mean and the standard deviation of the received signal at the sampling instant when a logical “1” (“0”) is transmitted.

In Fig. 5 the variation of bit error rate among the DWDM channels is detected due to crosstalk between those channels having the different data symbols. Even so, the proposed system provides an acceptable bit error rate <1×10⁻⁹ [10] over the effective bandwidth.

In this paper we have proposed a new analytical model for the gain of a Raman-EDFA HOA and the final gain equation is used to optimize the multiple parameters using a genetic algorithm. This algorithm has proven to be robust in refining the search for the optimal Raman length, EDFA length, and pump powers required for the best configuration of the proposed HOA. In a simulation using the optimal parameters (Raman length 140.4 km, EDFA length 12.5 m, Raman pump power 232.3 mW, and EDFA pump power 390.6 mW), the proposed Raman-EDFA provides a flat gain >17 dB with a gain variation of less than 1.5 dB.