Scenebased Nonuniformity Correction Algorithm Based on Temporal Median Filter
 Author: Geng Lixiang, Chen Qian, Qian Weixian, Zhang Yuzhen
 Organization: Geng Lixiang; Chen Qian; Qian Weixian; Zhang Yuzhen
 Publish: Journal of the Optical Society of Korea Volume 17, Issue3, p255~261, 25 June 2013

ABSTRACT
Scenebased nonuniformity correction techniques for infrared focalplane arrays have been widely considered as a key technology, and various algorithms have been proposed to compensate for fixedpattern noise. However, the existed algorithms’ capability is always restricted by the problems of convergence speed and ghosting artifacts. In this paper, an effective scenebased nonuniformity correction method is proposed to solve these problems. The algorithm is an improvement over the constant statistics method and a temporal median is utilized with the Gaussian kernel to estimate the nonuniformity parameters. Also theoretical analysis is conducted to demonstrate that effective ghosting artifacts elimination and superior convergence speed can be obtained with the proposed method. Finally, the performance of the proposed technique is tested with infrared image sequences with simulated nonuniformity and with infrared imagery with real nonuniformity. The results show the proposed method is able to estimate each detector’s gain and to offset reliably and that it performs better in increasing convergence speed and reducing ghosting artifacts compared with the conventional techniques.

KEYWORD
Infrared imaging , Nonuniformity correction , Temporal median filter

I. INTRODUCTION
Nowadays，the infrared imaging system is based on infrared focalplane array (IRFPA) technology. However, the system has an inherent nonuniformity problem in that that the detectors in the array have different responses to the same input signal, causing the observed images to be degraded by fixedpattern noise (FPN). In this case, many nonuniformity correction (NUC) techniques have been proposed to compensate for FPN. The NUC techniques include two classes that are referencebased and scenebased. Evidently, the scenebased methods are more attractive because the referencebased ones need to employ blackbody sources as uniform infrared targets while scenebased methods are conveniently updating the correction coefficients and avoiding scene interruptions.
So far, numerous scenebased nonuniformity correction (SBNUC) methods have been proposed. Assuming that each detector’s irradiance in the array has the same mean and variance, Y. M. Chiang and J. G. Harris proposed constant statistics (CS) methods [1,2]. References [2,3] aimed to increase convergence speed and reduce ghosting. S. N. Torres and M. M. Hayat presented a Kalman filter based method by a discretetime GaussMarkov process [4]. Another adaptive SBNUC technique is based on a neural network (NN) which is described in [5]. S. N. Torres, E. M. Vera improved the speed of the method by modifying the learning rate in the gain and offset parameter estimation process [6]. However, it is proved in reference [2] that the CS algorithm with appropriate coefficient has faster convergence rate and better performance than the LMS algorithm. Registration methods are proposed in [7,8]. But they all need complex registration algorithms and cannot work well when the nonuniformity is serious.
This paper puts forward a novel SBNUC algorithm called the medCS method. This method is based on an improvement of the CS method. Due to the small influence of the median filter by an infrequent artifact and to the efficient capability for smoothing of impulsivetype noise, the proposed method utilizes the characteristics of median filter and applies it in the CS method to estimate the nonuniformity parameters. In the proposed algorithm, a temporal median is adopted combining with the Gaussian kernel. In this way, improvement in removal of ghosting artifacts and convergence speed can be achieved and the thorough analysis of effects on these two aspects is launched.
The remainder of the paper is organized as follows. In Section II the detectorlevel model and the readout amplifier model are given. Then the proposed algorithm is described and the superiority of the proposed algorithm is demonstrated. In Section III results obtained from simulated and real infrared data are presented. Finally, some conclusions are given in Section IV.
II. NONUNIFORMITY CORRECTION METHODS
2.1. Observation Model and Conventional CSNUC Method
The relationship between the observed infrared signal response and the real world is generally nonlinear. In literature [9]，the output pixel value of each detector is modeled nonlinearly as a high order polynomial. To simplify the problem formulation for SBNUC, the linear approximation model is adopted which contains the firstorder of the polynomial. Then the output of a single pixel is given by:
where,
n is the frame index anda (n ) andb (n ) are respectively the real gain and offset of the detector. Andx (n ) stands for the real incident infrared signal collected by the respective detector. Without losing generality,x (n ) can be assumed to have fixed mean and unity mean deviation. The expressions are now written as:Where,
m is the temporal mean ands is the deviation of the observed signal. Then the aim of the NUC is transferred to estimate the gains and the biasm . So the FPN can be corrected as:To estimate the gain and bias, the CS algorithm uses the mean of the measured output
y (n ), based on assumption that the temporal mean and standard deviation of each pixel is constant over time and space. Mean deviation is used instead of standard deviation to simplify the implementation and provide more robust estimation. Mean and mean deviation ofy are calculated by the following recursive equations:CS is improved to better estimate changes of gain and offset with a fixedlength rectangular and exponentially shaped window.
It can be seen from (5) that coefficient
α controls the size of the window and it should be less than 1 but usually very close to 1.2.2. The Proposed MedCS NUC Method
The main drawback of the CS NUC method originates in the basic constraint that is related to the supposition that the scene is constantly moving. However, when the scene is not moving for a short while, the ghosting effect appears and the convergence rate will be affected. In this section, the proposed method takes a temporal median value with the Gaussian kernel and the ability of the method is analyzed.
2.2.1. Parameters estimation
In the medCS NUC method, the parameters for each pixel are calculated by computing weighted means and variances based on medians from the values with length of
L frames. And the form of the weighted means expression is similar to the bilateral filter in [10] as followThe weighted variance is expressed as:
The weights are computed from a Gaussian distribution centered at
y_{med} where
σ is the standard deviation parameter defining the Gaussian kernel. It can be observed that the weights depend on differences in intensity value betweeny_{n} (i ,j ) andy_{med} (i ,j ). They_{med} (i ,j ) is the median value of the pixel (i ,j ) overL frameswhere the weights
ω_{k} (i ,j ) for each pixel location are used to minimize the effect of outliers (values far from the mediany_{med} (i ,j )). The weight is controlled byσ which controls the decay of the weight factor as the range parameterσ varies.2.2.2. Analysis of convergence
An assumption that the parameters of the nonuniformity are constant is employed, which means the offsets of sensors are considered to be fixed over time. According to the derivation in the literature [3] the meansquare error (
MSE ) of the recursive nonuniformity parameters estimation with the conventional CS method can be written aswhere
σ_{y} is the standard deviation ofy (n ). To explain intuitionisticly,ΔMSE (n ) is defined as the difference ofMSE between the adjacent frames as followFrom (11), it can be seen that if
σ_{y} (n ) is equal toσ_{y} (n 1), theMSE of the conventional CSmethod decays asn goes to infinity andΔMSE (n ) shrinks to zero with recursive coefficientα^{2} . However, the actual scene consists of multiple parts with different deviations. So theΔMSE will oscillate whenσ_{y} changes and the convergence will be influenced. In this section, the superiority of the medCS method in estimating the average of scene which contains different deviations is demonstrated.The CS method is based on the assumption that every pixel has the same distribution and the temporal mean and standard deviation of each pixel is constant over time and space. But the distribution tendency characteristic of each pixel over time has not been considered. The mean value that the CS method adopted is an appropriate measure of the central tendency. For instance the measurements of each pixel in a period of time has a Gaussian distribution. In fact, the temporal statistics of each pixel approximate mixture distributions rather than following a single distribution strictly. The mixture distribution contains more than two distributions. Regularly, there is more than three times difference among the deviations of the different distributions. Consequently, the deviation of the measurements cannot be constant over time and it will bring the problem according to (11) which was analyzed previously. To solve this problem a median value is utilized and proved to be effective by Monte Carlo simulation.
The median method, however, should have more perfect performance and this hypothesis is verified using Monte Carlo simulation. For this study, several groups of statistics were examined. Without loss of generality, an assumption is employed that the statistics consist of two Gaussian distributions with different deviations.
The simulation consists of 10,000 samples and each sample contains
N values, whereN is set to 50 and 500. As in (12), each sample is made up of two kinds of different distribution values whose deviations areσ andζσ . The coefficientζ is set to 3, 5 and 10 and the proportion coefficientλ is set from 0% to 100% with interval of 10%. The mean and median of samples are calculated and their meansquare errors (MSE ) are calculated respectively by:The result of the simulation is showed in Table 1,Table 2. Table 1 is generated by 10,000 samples with 50 values per sample, Table 2 is generated by 10,000 samples with 500 values per sample. The curves in Fig. 2(a) are drawn with the data in Table 1 and Fig. 2(b) is drawn with the data in Table 2.
The results display that the
MSE of the mean method performs significantly worse than the median method especially when distributions are various. The proportion coefficientλ expresses the diversity of the sample distribution. The sample distribution is unitary when the coefficientλ is approaching 0% or 100% and various when the coefficientλ is approaching 50%. The mean method performs well when the distribution is unitary but the median method has the obvious advantage when the distribution is various like the behavior of the blue line whileλ is approaching 50%.Simultaneously, the coefficient
ζ indicates the degree of the difference between the distributions. When the coefficientζ is from 3 to 10 the difference between the two kinds ofMSE becomes larger, the greater the difference of the distributions’ deviations changes, the more outstanding the median method performs.Finally, comparing Table 1 to Table 2 or comparing Fig. 2(a) to Fig. 2(b), it shows that the more sufficient the amount of data is, the maore superior the
MSE is. However, the general statistical characteristic of the data is constant.From the above analysis the median method has a remarkable effect on eliminating interference of deviation variation. The median value is an accepted measure of the various tendencies
of a group of values even when their distribution is non Gaussian and the median method is valid even when applied to a small number of samples. Moreover the median method is less sensitive to the variation of sample distribution. Therefore, the medCS using a temporal median is more robust and suitable to convergence than the conventional CS method.
III. RESULTS AND DISCUSSION
In this section, the proposed medCS method is applied to both simulated and real data to correct the nonuniformity compared with the various SBNUC algorithms and the method’s capability is verified.
3.1. Applications to Simulated Nonuniformity
Several tests have been executed over an IR sequence of frames with simulated nonuniformity in this work. The deghosting capability has been demonstrated by visually comparing the frames obtained with the original CS method [1] and the improved CS method in [2]. Here, we call the method in [2] the gatedCS method because the algorithm sets a gated threshold to deghost.
As an evaluation criterion of correction performance, the Root Mean Square Error (
RMSE ) [8] is used to measure the difference between the true infrared image and the corrected image. Smaller values for theRMSE indicate better performance. TheRMSE is calculated by:where
X (i ,j ) is the (i ,j ) th pixel’s value of the true frame, whileis the pixel’s value of the corrected frame and
M ×N is the number of detectors in the FPA.The corrupted sequence with artificial nonuniformity is generated from a clear frame infrared video sequence with Simulated Nonuniformity added. The added FPN is composed of a synthetic gain with a unitmean Gaussian distribution with standard deviation of 0.2, and a synthetic offset with a zeromean Gaussian distribution with standard deviation of 15.
The CS method uses an exponential window parameter of
α = 0.99 (N ？100). The gatedCS method uses the sameα and a change threshold ofT = 20 and the medCS method uses the same frame lengthL = 100. The results in Fig. 3 are very typical of the numerous video sequences tested. In the first 200 frames the algorithms converge and then theRMSE curves oscillate because of ghosting artifacts. The curves in Fig. 3 show that the medCS method significantly outperforms the original CS method and the gatedCS method and that the oscillation of the medCS method is notably weaker than the other methods. Although the gatedCS performs well when the sequences have strong object higher than the threshold but burns in image with original CS method as in 350th frame, the medCS gets the best result in removing ghosting artifacts when the burnin object is lower than the change threshold of the gatedCS method as in the 630th and 780th frame.Figure 4 shows the images for the 780th frame. Fig. 4(a) shows the true clear infrared image corrupted with simulated nonuniformity. The NUC results of original CS method, gatedCS and medCS algorithms are shown in Fig. 4(b)  (d). It is shown that the medCS algorithm effectively generates much fewer ghosting artifacts than the other techniques.
3.2. Applications to Real Infrared Data
In this section, the original CS, gatedCS, and medCS are applied to a noisy sequence collected by using a 320 × 256 HgCdTe IRFPA camera, operating in the 8 to 14
μ m range. The video contains 2000 frames captured at a rate of 25fps .The coefficients used in the experiment are: the original CS method uses an exponential window parameter of
α = 0.992 (N ？125). The gatedCS method uses the sameα and a change threshold ofT = 25 and the medCS method uses the same frame lengthL = 125.To indicate good correction performance or examine whether there is the presence of artifacts, a visual evaluation is performed by watching a video sequence. Fig. 5 shows the original image and the corrected images using the original CS, gatedCS, and medCS algorithm of the 379th frame. It is very noticeable that the medCS compensates the FPN and performs the best over the sequence. Besides, it effectively generates much fewer ghosting artifacts than the other techniques.
IV. CONCLUSION
In this paper, based on the analysis of property of scene statistical distribution, a new NUC method called medCS has been proposed, which adopts a temporal median filter combining with Gaussian kernel to estimate the NUC parameters. Taking advantage of the median filter’s characteristics properly, the proposed algorithm has the ability to improve the nonuniformity correction and eliminate ghosting artifacts efficiently. Experiments carried out with simulated nonuniformity and the real infrared data have shown that the proposed method offers the best performance compared with the conventional methods tested.

[TABLE 1.] MSE of mean and median from Monte Carlo simulation for various samples (50 values per sample) from the different deviations observed distributions

[TABLE 2.] MSE of mean and median from Monte Carlo simulation for various samples (500 values per sample) from the different deviations observed distributions

[FIG. 1.] (a) A simulated sequence of scene signals of single pixel for 500 frames, (b) The results of estimating the simulated signals’ average and the green curve indicates the conventional CS method, the blue curve indicates the medCS method.

[FIG. 2.] MSE of mean and median from Monte Carlo simulation for various samples from the different deviations observed distributions. And (a) is corresponding to the Table 1, (b) is corresponding to the Table 2.

[FIG. 3.] The RMSE curves of the three SBNUC algorithms. The green curve indicates the conventional CS method, the blue curve indicates the gatedCS method and the red curve indicates the medCS method.

[FIG. 4.] The result of simulated nonuniformity correction (780th frame) with the three SBNUC algorithms, (a) the clear infrared image added with simulated nonuniformity, (b) corrected by the CS method, (c) corrected by the gatedCS method, (d) corrected by the medCS method.

[FIG. 5.] The result of true nonuniformity correction (379th frame) with the three SBNUC algorithms, (a) the clean infrared image with real FPN, (b) corrected by the CS method, (c) corrected by the gatedCS method, (d) corrected by the medCS method.