I conducted this research project for my bachelor's thesis, implementing an Automatic Number Plate Detection and Recognition system using YOLOv3 and Pytesseract.
The custom YOLOv3 model was trained specifically for car number plates and utilized as a detection model to identify the location of number plates on cars.
Once the number plate is detected, the image is cropped, and various image processing steps are performed using OpenCV.
The processed image is then passed through Pytesseract OCR to extract the text from the number plate.
#
| Development Tools | Description |
|---|---|
| Yolov3 | Deep learning algorithm for real-time object detection. |
| OpenCV | Computer vision library for image and video processing. |
| Python | Programming language used for the development of algorithms. |
| Flask | Web framework for creating web applications with Python. |
| Pytesseract OCR | Optical Character Recognition (OCR) tool for text. |
| Visual Studio Code | An IDE for coding. |
| Google Colab | Cloud-based platform for collaborative coding in Python. |
The images below showcase successful license plate detection on vehicles.
Figure 1: Single Car
Figure 2: Multiple Cars
These images demonstrate successful detection in scenarios with both single and multiple cars, showcasing the versatility of the detection system.
After license plate detection, a set of preprocessing steps is performed on the extracted license plate to enhance recognition accuracy. The key techniques include:
-
Grayscale Conversion: Simplifying subsequent processing steps and reducing computational complexity by converting the cropped RGB image to grayscale.
-
Gaussian Blur: Applying a 7x7 filter size for Gaussian blur to suppress noise and enhance license plate features.
-
Color Inversion: Enhancing the contrast between the license plate and background through color inversion using bitwise not operation.
-
Binarization: Deriving a binary image from the inverted grayscale image using thresholding (threshold value of 100), accentuating license plate features, and suppressing background noise.
-
Morphological Dilation: Utilizing a 3x3 rectangular kernel in the dilation process to enhance license plate boundaries. This improves contour identification, contributing to a more robust model in various scenarios, including varying angles, lighting conditions, and image noise.
The figure below illustrates the sequence of image processing steps.
Model Performance in different environmental conditions.
In the OCR phase, Pytesseract is used on preprocessed images to extract alphanumeric characters from the cropped License Plates as shown in Figure below. This is achieved by using the ‘image_to_string()’ function, which retrieves the text.
For Korean characters, the language parameter is set to ‘Hangul’ . Post-OCR, the text undergoes post-processing to correct any mistakes, which includes spellchecking and validation against known license plate formats, as well as handling unique characters or symbols found in Korean plates. The final step is to output the recognized text for further use or storage. The effectiveness of Pytesseract in recognizing Korean license plates can be significantly influenced by the image's quality, the preprocessing methods, and the OCR engine's training.
Figure: Text extraction using OCR
In this section, witness real-time license plate detection in action through the provided video demonstration. The YOLOv3 model dynamically identifies and highlights license plates in the video stream.
output_video_2.mp4
I've implemented a live license plate detection system using a webcam, and the real-time feed is streamed to a browser using Flask. During testing, I used car pictures on my phone, bringing the phone in front of the webcam, and the system successfully detected license plates. Below are the results of the detection from the webcam:
I have crafted the interface using Python, HTML, CSS, and Flask, with a primary focus on achieving a clean, simple, and user-friendly design. The homepage interface starts with a brief project introduction, as illustrated in Figure 15 and Figure 16.
Users can upload an image, and YOLOv3, based on the uploaded image, performs license plate detection and subsequently crops the license plate from the image. We utilize OpenCV to execute various post-processing steps aimed at enhancing the visibility and accuracy of the detected license plate.
The figure below shows the web interface:
To enhance the system, future efforts will concentrate on:
- Improving character recognition through the implementation of single-character detection using deep learning.
- Employing advanced techniques like CRNN and LSTM to bolster the system's robustness and achieve more accurate character recognition.
- Exploring the integration of the system into mobile devices such as smartphones and tablets to leverage mobile environments.
While initially designed for Korean car number plates, the project's robust architecture enables effective functionality with number plates from diverse regions and countries.
Feel free to reach out to me via email if you have any inquiries about this project or if you're interested in collaborating:
Email: abdijumale2@gmail.com
Looking forward to hearing from you!