License plate recognition working principle 

by:Shenzhen TGW Technology Co.,Ltd

The license plate recognition technology is based on the image segmentation, image recognition theory .The technology is used to analyzes images containing a number plate to determine the position of the license plate and further extracts and recognizes the text characters.

A typical license plate recognition process includes image acquisition, image pre-processing, license plate positioning, character segmentation, character recognition, and result output. Per process is complementary to each other. All process must ensure its high efficiency and high anti-interference ability. Only in this way can the identification processing achieve a satisfactory result.

There are two main ways to implement the license plate recognition system, one still images recognition, and the other is dynamic video stream recognition. Image recognition is limited by factors such as image quality, license plate defacement, and license plate tilt. Dynamic video stream recognition requires faster recognition speed, which is limited by the performance indicators of the processor, especially when we want to realize real-time license plate recognition on mobile terminals,it requires more performance optimization.

Although the license plate recognition includes six major processes, the core algorithm is only located in three modules: license plate positioning, character segmentation, and character recognition.

License plate positioning

The main task of license plate positioning is to find the area of license plate from the still picture or video frame and separate the license plate from the image for subsequent processing module processing. License plate positioning is one of the important factors that affects system performance. At present, there are many ways to locate license plates, but in general, they can be divided into two methods:

First, the method based on graphic imaging.

There are mainly (1) color-based localization methods, such as color edge algorithm, color distance and similarity algorithm, etc .; (2) texture-based localization methods, such as wavelet texture, horizontal gradient differential texture, etc .; (3) edge detection-based Positioning method; (4) Positioning method based on mathematical form.

The positioning method based on graphics and image technology is susceptible to interference caused by external interference information and causes positioning failure. For example, in the color analysis-based positioning method, if the background color of the license plate is similar to the color of the license plate, it is difficult to extract the license plate from the background. In the method based on edge detection, the stain on the edge of the license plate can easily cause the positioning failure. The interference of external interference information will also deceive the positioning algorithm, causing the positioning algorithm to generate too many non-license plate candidate regions, which increases the system load.

Second, the method based on machine learning.

Machine learning-based methods include feature engineering-based positioning methods and neural network-based positioning methods. For example, we can train a license plate positioning system through a cascaded classifier based on haar features provided by OpenCV. But this method is very time-consuming to train, and the efficiency of classification and positioning is also low. Therefore, in the area of target localization, neural network-based methods are the mainstream methods. In neural network-based localization methods, convolutional neural networks are mainly used to learn target features. Because the convolutional neural network has translation invariance, it can be supplemented with candidate regions in the learning process and classify the candidate regions. The candidate region that is correctly classified is the location of the target. There are many implementation models for such methods, such as RCNN, faster RCNN, SSD, and so on.

Character segmentation

The task of character segmentation is to cut each character in a multi-column or multi-line character image from the entire image into a single character image. Traditional character segmentation algorithms can be summarized into the following two categories: direct segmentation methods, segmentation methods based on image morphology. The direct segmentation method is simple, based on some prior knowledge, such as the distribution of license plate characters, and also assists some basic projection algorithms to achieve segmentation; the morphology-based segmentation method uses edge detection, expansion and corrosion to determine the character image position. Traditional character segmentation algorithms are also sensitive to external disturbances, such as license plate inclination, character fouling, and adhesion. The correct segmentation of the license plate characters is very important for character recognition. Only when the segmentation is correct can the recognition accuracy be guaranteed. With the continuous development of neural network theory, end-to-end picture classification and recognition technology have also made great breakthroughs, so many OCR software gradually get rid of the traditional character segmentation processing, and multi-characters are directly recognized by the recognition network.

Character recognition

Character recognition is the process of extracting character encoding from a picture containing one or more characters. The typical method of character recognition is the method of picture classification based on machine learning. In the picture classification method, one picture can only output one classification, that is, one picture can only contain one character image. This requires a high accuracy of character segmentation. Another recognition method is the end-to-end character recognition method based on a recurrent neural network. This method inputs the entire license plate image into the network, and the neural network will directly output all characters. The end-to-end method directly removes the character segmentation process and avoids the loss of stability caused by the character segmentation error, but the end-to-end method is also sensitive to other disturbances such as license plate tilt.

we briefly discussed some technologies of the three core part of the license plate recognition system above. In the follow-up, we will make a detailed discussion of some mainstream technologies.