Currently, vision-based traffic surveillance is widely used across the world to recognize illegal parking and obtain information about illegally parked vehicles, which causes a significant drain in the manpower, materials, and financial resources related to traffic control. With the development of computer vision technology, vision-based surveillance systems based on an image processing technique are increasingly and extensively used in the field of transporttation. However, in recent years, research in this field has been concentrated only in the areas of transportation flow, speed detection, pedestrian surveillance, and moving object recognition.
Thus far, a few studies on illegal parking have been reported, such as [1-4]. However, only the related technol- ogy framework has been constructed and there is still a lack of a systemic, concrete analysis and an understanding of the image processing algorithm for illegal parking detection and recognition. Further, the validity and practicality of the corresponding algorithm is yet to be proven. Therefore, in this paper, we propose an illegal parking surveillance algorithm based on image processing technology: a digital camera was set in the illegal parking region to acquire the video frames. An adaptive Gaussian mixture model (GMM) is used for the background subtraction in the complex environment to identify the regions of moving objects in our test video. Stationary objects are detected by using the pixel-level features in time sequences. Stationary vehicles are recognized by using the local features of an object, and thus, information about illegally parked vehicles is successfully obtained. The result of this research work is that the illegal parking time of a vehicle is obtained. Further, an automatic alarm system is used according to the different regulations of different illegal parking regions.
The rest of this paper is organized as follows: the proposed system is described in Section II. The experimental results that show the efficiency of the proposed method are presented in Section III. Finally, the conclusion is summarized in Section IV.
A flowchart of the proposed algorithm to detect and recognize illegal parking is shown in Fig. 1.
Effective detection of moving objects is the basis of the detection of illegal parking. Currently, frame difference, optical flow field, and background subtraction are mainly used for such detection. The frame difference method leads to a transparent (easy to perceive or detect) phenomenon of moving objects, which results in a misclassification of moving objects. Further, it is difficult to satisfy the requirements of real-time detection by using the optical flow field method because of the complex calculations. Background subtraction is the most frequently used method at present. The principle of this method is to show the scene background with a model before detecting a moving object first; this model is called the background model. Then, the moving object is detected through a subtraction of the background and the observed image. The algorithm proposed in this paper has successfully implemented background subtraction in complex traffic scenes by using adaptive GMM, which robustly overcomes the effects of light and the shaking of branches and can satisfy the detection condition that there exist continuous moving objects in the detection areas.
The probability density function of the current pixel point
To simplify the calculation, set
In general, the background is the main part of GMM and the variance of background model is small. At the same time, the parameter of GMM changes along with the pixel of the current frame. To obtain the effective components of the background in GMM, the values of
Because of the changes in the external factors such as light and weather (rain or snow), the background changes as well. To recognize a long video surveillance, adaptive updating is needed to maintain the background. According to the method proposed in , we need to check whether the pixel point of each current image matches with
If there is no matched Gaussian component in the mixture model, the bottom distribution obtained after the ranking of the values of
In general, the traditional background updating process only considers the updating background model at the pixel level; that is, we assume that the time sequence of each pixel is independent to that of the others, thereby ignoring the special characteristics of the neighboring pixels. Therefore, this updating process is unable to overcome the problem of moving vehicles becoming a part of the background because of a long parking time; thus, this process cannot be used for the detection of illegal parking. To overcome the above problem, information such as movement tracking and object segmentation must be considered. This information can be obtained through the foreground, which can be represented as follows:
A regional template of an object can be obtained according to the structure information of a moving object segmented by an object region. This template reflects whether the current object is in the detection region or not and then, determines whether to update the background or not; the template can be represented as follows:
If the current pixel is detected as an object, the background does not update; otherwise, the background automatically updates according to the background of the pixel level, which can be represented as
The background can be obtained efficiently through the abovementioned background model, after which the detection of a foreground object can be achieved by using background subtraction.
The detection of illegal parking consists of two parts: first, the time sequences of pixel-level features are used for analyzing and detecting still objects, and then, the stopped vehicles are detected by using the local features of the object.
1) Detection of Stationary Object
Normally, the background value of each pixel is relatively stable over a long period of time and the pixel value of the point changes when the foreground object gets through, in which case the degree of change is bigger than the background value itself (affected by the environment). Therefore, if we detect that the gray value of a pixel point on a time sequence changes considerably in the illegal parking region, we can confirm that there is a foreground object. However, the change in the pixel value detected on the time sequences may be caused by a moving object that is just passing through the illegal parking region (the moving status does not change) and the object that has entered the illegal parking region and stopped. Therefore, to efficiently recognize illegal parking, the situation in which a pixel value changes considerably just because there is a moving foreground object passing through should be ruled out. This suggests that there is a moving object that passed through the detected region but did not stop when the pixel value of the background changed suddenly and returned to the initial background status value again within a short period of time. Further, the pixel point of the background is occupied by the foreground, which entered the illegal parking region and stopped; there might be illegal parking when the pixel value changes suddenly and the degree of change of the pixel values remains stable in the subsequent period of time. In addition, because of the effect caused by the surroundings and the change in the light conditions, the pixel value of the background might be turbulent on relatively long time sequences and may be unable to indicate the change of pixel value of background within a short period of time. This has already been discussed in the above background updating part, and here, we ignore this situation. Fig. 3 shows the change in the gray value of a certain pixel point under the following two situations: the moving object passes through the detected region and the moving object enters the detected region and stops.
2) Detection of Stationary Vehicles
Because of the mixed traffic environment, the still object detected in the illegal parking region might be a bicycle, a person, or some other non-motor vehicle. Therefore, it is necessary to recognize motor vehicles that are parked illegally. In this study, we perform regional segmentation of an object by using the seed filling algorithm , which is based on eight neighborhoods and extracts the still square of the illegal parking region. Then, the spatial property is characterized by using the duty ratio between the square of the object detected and the area of the illegal parking region. The duty ratio of the object in the region is defined as follows:
The steps of the proposed algorithm for detecting illegally parked vehicles are described below:
Step 1: Calculate the foreground pixel value of the current pixel point in the illegal parking detection region,
Step 2: After the abovementioned significant change, calculate the degree of change of the detected pixel value in the adjacent frame in the time sequence,
On the basis of the movement characteristics and the color characteristics of the object, it is impossible for the pixel value to change uniformly in a short period of time. Therefore, if
Step 3: When the pixel point becomes stable, calculate the difference value between the current pixel point and the initial background point,
Step 4: Justify the type of object according to the duty ratio of the foreground object
Step 5: To confirm that there is illegal parking and set off an automatic alarm, calculate the parking time of the vehicle as soon as it goes into a stable state according to the requirements of different parking prohibited regions.
The algorithm proposed in this paper is proven by using practical traffic scenes. The video sequences are recorded by using a digital camera with a resolution of 1024 × 576 pixels. The algorithm is developed on the MATLAB (R2013a) platform, and the CPU of the machine is Intel Core 7-2600 @ 3.40 GHz; the machine has an 8-GB memory. According to the algorithm proposed in this paper, the playing rate of a video sequence can be up to 29 frames per second; this satisfies the requirement of real-time surveillance.
Fig. 4 shows the processing results of background estimation and foreground extraction using the adaptive GMM for the background.
We assume that the dark green region in Fig. 5(b) is the illegal parking region, that is, the valid detection region of the algorithm, and the algorithm proposed in this paper is aimed at this region. When a still object is detected in the illegal parking region, the time sequence of the pixel level is used for analyzing and judging whether there is an illegal parking phenomenon or not.
[Fig. 5.] Results of the proposed illegally parked vehicle detection algorithm. (a) Background image. (b) Illegal parking region model. (c) Vehicle parking detection in 450th frame. (d) Vehicle parking detection in an illegal parking region in 450th frame. (e) Vehicle parking detection in an illegal parking region in 8120th frame. (f) Vehicle parking detection in an illegal parking region in 9105th frame.
By using the local features of an object, we can identify whether it is a vehicle or not and execute object segmentation to mark the illegally parked vehicle by using a boundary abuttal rectangle. Fig. 5(d)–(f) represent the detection effect of the 450th frame, 8120th frame, and 9105th frame of the video sequence separately, and the 8120th frame and 9105th frame are two frames in the parking process. The problem that objects become a part of the background because of long time parking can be overcome efficiently by using the algorithm proposed in this paper. The 450th frame in the figure is the time when the vehicle just enters the stable status, which can recognize the detection of an illegal parking incident and automatically calculate the parking time of the vehicle. Because of the different regulations for illegal parking in different regions (it is completely prohibited to park somewhere and is illegal to park for a long period of time elsewhere), specific time parameters for illegal parking can be set according to the illegal parking rules of different illegal parking regions in order to set an automatic alarm efficiently.
The proposed algorithm has been tested on our own recorded datasets, which contain different backgrounds and foregrounds in three time sequences. Table 1 shows the performance comparison of the proposed method with some conventional methods. Some test results with different backgrounds and fore-grounds in three time sequences are shown in Fig. 6.
Performance comparison of the proposed and other methods for three different time sequences
Because of the necessity of vehicle control, traffic surveillance systems are improved by the use of a computer vision system. Adaptive GMM background extraction and foreground extraction have been performed for a complex traffic scene. The pixel-level features in the time sequences and the local features of the objects are used for vehicle identification and the calculation of the illegal parking time of vehicles; they are also used for detecting and identifying an illegal parking incident and providing a technical means for the intelligent traffic management of urban traffic. The proposed algorithm is very highly efficient and can be executed in real time.