Nowadays，the infrared imaging system is based on infrared focal-plane array (IRFPA) technology. However, the system has an inherent nonuniformity problem in that that the detectors in the array have different responses to the same input signal, causing the observed images to be degraded by fixed-pattern noise (FPN). In this case, many nonuniformity correction (NUC) techniques have been proposed to compensate for FPN. The NUC techniques include two classes that are reference-based and scene-based. Evidently, the scene-based methods are more attractive because the reference-based ones need to employ blackbody sources as uniform infrared targets while scene-based methods are conveniently updating the correction coefficients and avoiding scene interruptions.

So far, numerous scene-based nonuniformity correction (SBNUC) methods have been proposed. Assuming that each detector’s irradiance in the array has the same mean and variance, Y. M. Chiang and J. G. Harris proposed constant statistics (CS) methods [1,2]. References [2,3] aimed to increase convergence speed and reduce ghosting. S. N. Torres and M. M. Hayat presented a Kalman filter based method by a discrete-time Gauss-Markov process [4]. Another adaptive SBNUC technique is based on a neural network (NN) which is described in [5]. S. N. Torres, E. M. Vera improved the speed of the method by modifying the learning rate in the gain and offset parameter estimation process [6]. However, it is proved in reference [2] that the CS algorithm with appropriate coefficient has faster convergence rate and better performance than the LMS algorithm. Registration methods are proposed in [7,8]. But they all need complex registration algorithms and cannot work well when the nonuniformity is serious.

This paper puts forward a novel SBNUC algorithm called the med-CS method. This method is based on an improvement of the CS method. Due to the small influence of the median filter by an infrequent artifact and to the efficient capability for smoothing of impulsive-type noise, the proposed method utilizes the characteristics of median filter and applies it in the CS method to estimate the nonuniformity parameters. In the proposed algorithm, a temporal median is adopted combining with the Gaussian kernel. In this way, improvement in removal of ghosting artifacts and convergence speed can be achieved and the thorough analysis of effects on these two aspects is launched.

The remainder of the paper is organized as follows. In Section II the detector-level model and the readout amplifier model are given. Then the proposed algorithm is described and the superiority of the proposed algorithm is demonstrated. In Section III results obtained from simulated and real infrared data are presented. Finally, some conclusions are given in Section IV.

The relationship between the observed infrared signal response and the real world is generally nonlinear. In literature [9]，the output pixel value of each detector is modeled nonlinearly as a high order polynomial. To simplify the problem formulation for SBNUC, the linear approximation model is adopted which contains the first-order of the polynomial. Then the output of a single pixel is given by:

where, n is the frame index and a(n) and b(n) are respectively the real gain and offset of the detector. And x(n) stands for the real incident infrared signal collected by the respective detector. Without losing generality, x(n) can be assumed to have fixed mean and unity mean deviation. The expressions are now written as:

Where, m is the temporal mean and s is the deviation of the observed signal. Then the aim of the NUC is transferred to estimate the gain s and the bias m. So the FPN can be corrected as:

To estimate the gain and bias, the CS algorithm uses the mean of the measured output y(n), based on assumption that the temporal mean and standard deviation of each pixel is constant over time and space. Mean deviation is used instead of standard deviation to simplify the implementation and provide more robust estimation. Mean and mean deviation of y are calculated by the following recursive equations:

CS is improved to better estimate changes of gain and offset with a fixed-length rectangular and exponentially shaped window.

It can be seen from (5) that coefficient α controls the size of the window and it should be less than 1 but usually very close to 1.

The main drawback of the CS NUC method originates in the basic constraint that is related to the supposition that the scene is constantly moving. However, when the scene is not moving for a short while, the ghosting effect appears and the convergence rate will be affected. In this section, the proposed method takes a temporal median value with the Gaussian kernel and the ability of the method is analyzed.

2.2.1. Parameters estimation

In the med-CS NUC method, the parameters for each pixel are calculated by computing weighted means and variances based on medians from the values with length of L frames. And the form of the weighted means expression is similar to the bilateral filter in [10] as follow

The weighted variance is expressed as:

The weights are computed from a Gaussian distribution centered at y_med

where σ is the standard deviation parameter defining the Gaussian kernel. It can be observed that the weights depend on differences in intensity value between y_n (i, j) and y_med (i, j). The y_med (i, j) is the median value of the pixel (i, j) over L frames

where the weights ω_k (i, j) for each pixel location are used to minimize the effect of outliers (values far from the median y_med (i, j)). The weight is controlled by σ which controls the decay of the weight factor as the range parameter σ varies.

2.2.2. Analysis of convergence

An assumption that the parameters of the nonuniformity are constant is employed, which means the offsets of sensors are considered to be fixed over time. According to the derivation in the literature [3] the mean-square error (MSE) of the recursive nonuniformity parameters estimation with the conventional CS method can be written as

where σ_y is the standard deviation of y(n). To explain intuitionisticly, ΔMSE(n) is defined as the difference of MSE between the adjacent frames as follow

From (11), it can be seen that if σ_y(n) is equal to σ_y (n-1), the MSE of the conventional CS-method decays as n goes to infinity and ΔMSE(n) shrinks to zero with recursive coefficient α². However, the actual scene consists of multiple parts with different deviations. So the ΔMSE will oscillate when σ_y changes and the convergence will be influenced. In this section, the superiority of the med-CS method in estimating the average of scene which contains different deviations is demonstrated.

The CS method is based on the assumption that every pixel has the same distribution and the temporal mean and standard deviation of each pixel is constant over time and space. But the distribution tendency characteristic of each pixel over time has not been considered. The mean value that the CS method adopted is an appropriate measure of the central tendency. For instance the measurements of each pixel in a period of time has a Gaussian distribution. In fact, the temporal statistics of each pixel approximate mixture distributions rather than following a single distribution strictly. The mixture distribution contains more than two distributions. Regularly, there is more than three times difference among the deviations of the different distributions. Consequently, the deviation of the measurements cannot be constant over time and it will bring the problem according to (11) which was analyzed previously. To solve this problem a median value is utilized and proved to be effective by Monte Carlo simulation.

The median method, however, should have more perfect performance and this hypothesis is verified using Monte Carlo simulation. For this study, several groups of statistics were examined. Without loss of generality, an assumption is employed that the statistics consist of two Gaussian distributions with different deviations.

The simulation consists of 10,000 samples and each sample contains N values, where N is set to 50 and 500. As in (12), each sample is made up of two kinds of different distribution values whose deviations are σ and ζσ. The coefficient ζ is set to 3, 5 and 10 and the proportion coefficient λ is set from 0% to 100% with interval of 10%. The mean and median of samples are calculated and their mean-square errors (MSE) are calculated respectively by:

The result of the simulation is showed in Table 1,Table 2. Table 1 is generated by 10,000 samples with 50 values per sample, Table 2 is generated by 10,000 samples with 500 values per sample. The curves in Fig. 2(a) are drawn with the data in Table 1 and Fig. 2(b) is drawn with the data in Table 2.

The results display that the MSE of the mean method performs significantly worse than the median method especially when distributions are various. The proportion coefficient λ expresses the diversity of the sample distribution. The sample distribution is unitary when the coefficient λ is approaching 0% or 100% and various when the coefficient λ is approaching 50%. The mean method performs well when the distribution is unitary but the median method has the obvious advantage when the distribution is various like the behavior of the blue line while λ is approaching 50%.

Simultaneously, the coefficient ζ indicates the degree of the difference between the distributions. When the coefficient ζ is from 3 to 10 the difference between the two kinds of MSE becomes larger, the greater the difference of the distributions’ deviations changes, the more outstanding the median method performs.

Finally, comparing Table 1 to Table 2 or comparing Fig. 2(a) to Fig. 2(b), it shows that the more sufficient the amount of data is, the maore superior the MSE is. However, the general statistical characteristic of the data is constant.

From the above analysis the median method has a remarkable effect on eliminating interference of deviation variation. The median value is an accepted measure of the various tendencies

of a group of values even when their distribution is non- Gaussian and the median method is valid even when applied to a small number of samples. Moreover the median method is less sensitive to the variation of sample distribution. Therefore, the med-CS using a temporal median is more robust and suitable to convergence than the conventional CS method.

In this section, the proposed med-CS method is applied to both simulated and real data to correct the nonuniformity compared with the various SBNUC algorithms and the method’s capability is verified.

Several tests have been executed over an IR sequence of frames with simulated nonuniformity in this work. The deghosting capability has been demonstrated by visually comparing the frames obtained with the original CS method [1] and the improved CS method in [2]. Here, we call the method in [2] the gated-CS method because the algorithm sets a gated threshold to deghost.

As an evaluation criterion of correction performance, the Root Mean Square Error (RMSE) [8] is used to measure the difference between the true infrared image and the corrected image. Smaller values for the RMSE indicate better performance. The RMSE is calculated by:

where X (i, j) is the (i, j) th pixel’s value of the true frame, while

is the pixel’s value of the corrected frame and M × N is the number of detectors in the FPA.

The corrupted sequence with artificial nonuniformity is generated from a clear frame infrared video sequence with Simulated Nonuniformity added. The added FPN is composed of a synthetic gain with a unit-mean Gaussian distribution with standard deviation of 0.2, and a synthetic offset with a zero-mean Gaussian distribution with standard deviation of 15.

The CS method uses an exponential window parameter of α = 0.99 (N？100). The gated-CS method uses the same α and a change threshold of T = 20 and the med-CS method uses the same frame length L = 100. The results in Fig. 3 are very typical of the numerous video sequences tested. In the first 200 frames the algorithms converge and then the RMSE curves oscillate because of ghosting artifacts. The curves in Fig. 3 show that the med-CS method significantly outperforms the original CS method and the gated-CS method and that the oscillation of the med-CS method is notably weaker than the other methods. Although the gated-CS performs well when the sequences have strong object higher than the threshold but burns in image with original CS method as in 350th frame, the med-CS gets the best result in removing ghosting artifacts when the burn-in object is lower than the change threshold of the gated-CS method as in the 630th and 780th frame.

Figure 4 shows the images for the 780th frame. Fig. 4(a) shows the true clear infrared image corrupted with simulated nonuniformity. The NUC results of original CS method, gated-CS and med-CS algorithms are shown in Fig. 4(b) - (d). It is shown that the med-CS algorithm effectively generates much fewer ghosting artifacts than the other techniques.

In this section, the original CS, gated-CS, and med-CS are applied to a noisy sequence collected by using a 320 × 256 HgCdTe IRFPA camera, operating in the 8 to 14 μm range. The video contains 2000 frames captured at a rate of 25 fps.

The coefficients used in the experiment are: the original CS method uses an exponential window parameter of α = 0.992 (N？125). The gated-CS method uses the same α and a change threshold of T = 25 and the med-CS method uses the same frame length L = 125.

To indicate good correction performance or examine whether there is the presence of artifacts, a visual evaluation is performed by watching a video sequence. Fig. 5 shows the original image and the corrected images using the original CS, gated-CS, and med-CS algorithm of the 379th frame. It is very noticeable that the med-CS compensates the FPN and performs the best over the sequence. Besides, it effectively generates much fewer ghosting artifacts than the other techniques.

In this paper, based on the analysis of property of scene statistical distribution, a new NUC method called med-CS has been proposed, which adopts a temporal median filter combining with Gaussian kernel to estimate the NUC parameters. Taking advantage of the median filter’s characteristics properly, the proposed algorithm has the ability to improve the nonuniformity correction and eliminate ghosting artifacts efficiently. Experiments carried out with simulated nonuniformity and the real infrared data have shown that the proposed method offers the best performance compared with the conventional methods tested.