Last September, we installed optical parking sensors to measure the occupancy of more than 500 street parking spots in the city of Braunschweigs university district. Thats the largest installation of parking sensors ever to be rolled out in public space in Germany and one thing is for sure:We will make it the last one.This is the story of the great efforts, an AI company made in search for ground truth data to fuel its algorithms.

This is also the story behind a beautiful piece of engineering, which happened to be the by-product of a greater mission and will eventually never be put to production again. All of that in favor of a more disruptive, data-driven technology.Since, at Bliq, we work on predictive algorithms to model parking availability based on traffic data, we have the natural need for ground truth data of the real-life parking occupancy in a reference area. To obtain that kind of data, we decided to build our very own, real-world experimental setup of parking sensors that measure the occupancy of more than 500 street parking spots, in real-time, 24 hours a day.With this article, we would like to share some insights about the engineering efforts that have gone into this project. More specifically, well talk about the test site we chose, system architecture and the actual sensor we designed and built to collect data. The test siteSometimes, AI companies need to become creative when it comes to collecting ground truth for their models. In our case, modeling car parking availability, this meant finding a district that meets certain criteria in terms of traffic flow, usage and demographics.We chose the university district in Braunschweig for its great variety of influences on comparably small space:In the south, we have the universitys main campus attracting thousands of students and hundreds of employees every day. Even further to the south, the inner city center with its shops and attractions is just a few-minutes walk (not shown on the map above). The northern part of the district forms a residential area with public, non-restriced street parking. Most people living in this area are either students or employed by the local automotive industry.

The vast majority of residents commutes to work using private cars. The district is separated by an arterial circular road, which embraces the citys center district.How the system worksThe basic system architecture of the sensor installation is pretty straightforward and basically what one would expect from most IoT applications: A little piece of hardware is deployed somewhere in the real world and transmits data to a cloud backend. The backend stores the data and makes it accessible for further processing, to serve as ground truth for machine learning endeavours or just for simple visualization in an app or web app.Hard requirement: Privacy by designWhats special about the sensors architecture is the fairly strong computational power we deployed on the edge: Because of regulatory requirements in German Public Space that is additionally fired by recent and ongoing GDPR discussions, we were not able to process images on a remote cloud with lots of computational resources.Thats why we needed to do all the heavy lifting for determining open spots directly on the sensor device rather than somewhere else. The upside about this is that this approach does not consume large data volumes for sending images back and forth. Instead, we can keep the running costs of the sensors for connectivity also within a relatively low range. On the downside, equipping the device with enough computational power to perform image analysis requires lots of extra efforts in hardware development. Why not use another parking sensor?Why have we decided to go through the pain of designing and building our own parking sensor instead of just buying one of the many finished sensor models that are already available out there?There are three answers to that:We didnt know how complex building a new device was gonna become ;)The existing parking sensors all had some flaws: The regulatory situation prohibited us from using any kind of available, optical solution, since these kind of systems would violate privacy. Surface-mounted sensors would not withstand snow removal during winter. And lastly, in-ground sensors were quite costly themselves and even more expensive to install. We were on a quite tight budget when we started this project. We entirely bootstrapped the company at this point: Our funds consisted of some government funding, first revenues and some prices we won here and there. The other sensor models with a price tag of something in between 75 and 250 EUR per spot were just simply to expensive for us at this time. The new optical parking sensorThe idea for the working principle of our sensor was simple: Deploy the same algorithm we had already developed in our previous research project on down-sized hardware, connect it to the internet, put everything in a waterproof box and mount it to a light pole.The algorithm itself is basically an image classifier, which requires pre-defined regions of interest to look at. The original purpose of the model was to automate the analysis of parking occupancy in some huge image series we collected in a previous project with offline cameras. The challenge now only was to design a suitable hardware rig with enough computational power, shrink the model so it executes on this setup and ensure continous power supply.This was our specs wishlist:Low cost per spot: Standard componentsMeasurement frequency up to 3 min down to 30 sec.Detection robust against weather, changes in lighting and inaccurately parking cars (e. g. one car occupying two spots)Monitoring health state of deviceOption for remote software updatesLow energy consumption25 years useful lifespanSoftwareThe sensor software consists of three layers: The operating system, which was a custom development for this project, the core routine controlling all the sensors function and the actual machine learning model for open spot detection.Operating systemFor running a full fledged vision model on the edge, we quickly figured that we would need a way bigger software setup than in other IoT hardware projects. The decision was made to build a custom distribution of Linux using Yocto. This way, we could have full control over everything the OS is doing. The core features were two separate partitions, in order to be able to do file system updates and swap partitions, a number of libraries, required by the core routine and a watchdog reset. The hardware watchdog of our SBC reboots the device in case anything does not operate as expected. Having smart bricks meters above the ground on light poles because of a bug in the software would literally be the worst case.Core routineThe core routine is responsible for running the detector in an adjustable time interval, monitoring the sensors state of health and communicating with the backend (retrieving config data and sending updates).The core routine is implemented in Python. This gave us great flexibility and simplified image processing a lot, since we could make use of the large existing Python code base we already had in the company.One great thing about the software design is its ability for independent remote updates of each individual component: From the detection model over source code for the core routine up to the kernel or even the entire file system each part can be replaced remotely. Facing rapid evolvements in the CV and machine learning field in general, we wanted to make sure that the code thats running the sensors will be state of the art over the entire lifespan. Machine LearningTo perform the detection task, we took a version of tensorflow and after some tweaking eventually got it to work on our setup. Once this was done, we could deploy pretty much any pre-trained tensorflow that would fit into GPU memory.We decided to use the MobileNet, since it showed the best ratio between accuracy and performance on our setup. We also looked at several other approaches based on traditional computer vision features such as HOG features, histograms, etc. in combination with conventional machine learning classifiers like SVMs. Although these tests resulted in quite high computational performance due to much simpler model design compared to MobileNet, model accuracy was lower, which may be explained by the usual downsides of standard CV feature descriptions (light invariance, scale invariance). HardwareWorking with hardware was a quite new experience for us, being a pure software company up to this point. Although Mathias, our CTO, had worked on designing electronics at his previous job with Volkswagen R&D, our company wasnt quite ready for a hardware development task and honestly looking back, it still isnt today.Nonetheless, we needed a functional design which was easy to manufacture and to iterate with the resources we had as a bootstrapped company at this time. So, our requirements list quickly turned out to be looking like this:Case needs to be waterproof and 3d-printableSensor should be able to run at least 12 hours on batteryThe camera should be protected from rain and spray water and work in darkness as wellThe design needs to hold the camera, a temperature / humidity sensor, the LTE module, the single board computer and some power electronics for converting voltage to the appropriate level.A battery is needed to continue operating when the light pole has its power switched off (during the day)The entire setup needs to be modular to facilitate installation and to be able to exchange single components in case of failure. It also needs to small and painted gray to look unobtrusive in its operating environmentOperating conditions from -20 C to 70 C (since the setup can become quite warm in summer when it is fully exposed to the sun)We started off with a design including infrared LEDs (like many outdoor cameras have them) to be able to operate in night conditions. However, this design choice turned out to come with some flaws: These LEDs were quite power-consuming (compared to the rest of the electronics), making a non-standard and thus power supply necessary. Despite the large power consumption, they werent really capable of illuminating the entire field of sight. We probably would have needed an external IR floodlight, which, again, wasnt a serious alternative. And lastly, the LED-spiced design wasnt very pretty either.To overcome the problem of night operations we decided to make use of the static camera setting: Since the cameras position is static and the objects we are trying to detect are normally also still, we can increase exposure and the sensors light sensitivity in order to work only with residual light. So we overwrote the cameras internal exposure and ISO control and wrote a simple feedback loop that adjusts lighting settings based on the luminance of last captured frame. This approach turned out to perform quite well, since in most streets there is enough residual light from the streetlights.After several more iterations, we finally ended up with the design as shown above: The camera sits inside a cone to be protected from spray water and sun reflexes as much as possible. The electronics are mounted onto a socket inside and a ribbon cable connects the camera to the main board. The bottom is removable and mounted to the case with four standard screws. Since the case is printed in ABS, quadratic nuts sit in cut-outs to make sure that the screws can be properly tightened. A GoPro-like joint connects the case to the mount, which gets attached to the light pole by using standard steel tape. All parts are optimized for 3d-prinability, which mean no heavy overhangs, import surfaces parallel for high surface quality.Lastly, the battery box is separated from the sensor for better serviceability. It is a standard injection-molded ABS box and contains a 4. 5 Ah 12V lead battery and a charging unit, which takes 230V input (which is the voltage of most street lights in Germany).Whats nextThe data, which we are collecting throughout this project is greatly helping us to improve our understanding (and also our algorithms) of how parking works in different traffic situations and various contexts of factors that influence parking availability.We are going to post more about the actual results from a data science perspective very shortly. We will also spend some time and go more into the details of the software part and are eventually even going to open-source the software, as well as the hardware designs.AcknowledgementsAt this point, we would like to express our gratitude to the city of Braunschweig for giving us access to traffic infrastructure to support this project. They not only provided all the necessary permissions, but also covered parts of the costs. We also would like to send a big thank you to the local traffic operator Bellis and the energy provider BS Energy for the support regarding the installation and power supply of the sensors.About the authorJulian is the CEO and Co-Founder of Bliq, a Berlin-based tech company. Bliq provides live parking maps for developers in mobility. RELATED QUESTION I didn't get Google Glass Explorer Edition. Is trying to learn Glass dev without the hardware a futile effort? No, you can still learn the fundamentals of Glass development without the hardware. There are three main approaches for accomplishing this: 1) Visit the Mirror API documentation, get into the playground, and start hashing up some code. Download the PHP, Java, and Python library, whichever you're most comfortable with. Familiarize yourself with the jargon and converntions (timeline, bundles, menus, etc). Read the support documentation (second link below) to see how the Glass hardware actually functions. Build some apps to this specification. Soon enough, you will find a friend with hardware to t

Image recognition technology license plate recognition technology is a technology based on computer vision processing, digital image processing, pattern recognition and other technologies to process and analyze the vehicle image or video image taken by the camera to obtain the license plate number of each vehicle, so as to complete the recognition process, which can complete the identification of vehicle identity. Cloud platform aims at valuable mining such as data storage, retrieval, association and derivation. These data are universal in itself. Parking lot cloud platform management breaks the current situation of information island of single group vehicle management system, and realizes centralized and unified management of multiple parking lots on the same platform. Online payment for the import and export management industry, third-party payment is still a new thing, but the two businesses have a strong positive correlation. Online payment is a large business field in the terminal consumer market, and as the final end industry of the security industry, import and export management can provide a convenient and efficient service foundation for terminal payment. The binding of the two will also give birth to a new business field of import and export management industry.

In general, smart parking lots mostly use card swiping or ticket taking to manage vehicles in and out. Although the management of traditional parking lots has been greatly improved, vehicles need to stop to swipe cards or tickets when entering and leaving the parking lot, which affects the speed of vehicles in and out, and may cause congestion during the peak period of traffic flow, It affects the satisfaction of users and the operation efficiency of the parking lot. In addition, in case of rain and snow, car owners have to stop, open the window and swipe their cards to enter and leave the parking lot, which many car owners can't stand. So, is there any good way to solve the above problems? The card free parking lot system can solve the above problems. The card free parking lot system takes the license plate as the access certificate and adopts video recognition technology to automatically detect and identify the license plate number of vehicles entering and leaving the site, and then carry out different management methods according to the vehicle type. Fixed vehicles are released directly without parking and swiping cards. Let's look at its workflow. When vehicles enter the site, the camera at the entrance will automatically identify the license plate number to judge whether the vehicle is a fixed vehicle or a temporary vehicle. If it is a fixed vehicle, the gate will automatically lift up and release. If it is a temporary user, you need to take the card and release it. When the vehicle leaves the site, the camera at the exit will also automatically identify the license plate number, and display the vehicle type, photos, entry and exit time, parking fee, etc. on the display screen. If it is a fixed vehicle, the road gate will open automatically. For temporary vehicles, the card and payment need to be returned, and the security guard will open the gate for release. The card free parking lot system takes the license plate as the certificate for entry and exit, one card and one vehicle to ensure the safety of vehicles. At the same time, it greatly improves the speed of vehicles. Generally speaking, the card swiping parking lot can get in and out of 6 vehicles per minute on average, while the card free parking lot system can get in and out of 10 vehicles per minute on average, eliminating the situation that car owners queue up at the peak. The card free parking lot system is used in some high-end communities, and the parking lot with more fixed users is the most suitable.

The license plate recognition system automatically "captures" illegal vehicles; From March 1, if you find driving in the restricted area again, you will be punished. On February 3, a policeman on duty stopped a blue business car with a yellow label at the Nanhu sentry box in the Binhu section of the city. Since the same day, the municipal public security traffic police and environmental protection departments have conducted joint law enforcement inspection on the illegal driving of vehicles with yellow environmental protection inspection qualification marks and vehicles without environmental protection inspection qualification marks. On the morning of February 3, in the 24-hour forbidden section of Binhu Road, Nanning City, the relevant departments spot checked 481 vehicles within one hour through the license plate recognition system, including 13 yellow cars and 128 non-standard cars, accounting for nearly 30% of the illegal vehicles. The reporter saw at the Nanhu sentry box on Binhu road that there were two mobile capture devices to identify passing vehicles on the road.

The staff told reporters that the instrument is connected to the Yellow standard vehicle restriction capture system through the wireless network. When a yellow standard vehicle or a non-standard vehicle breaks into the restricted area, the system will automatically identify whether the vehicle is a yellow standard vehicle and whether it has received the environmental protection inspection mark. A private car passed by and the system prompted it as a green label car, but the car was also stopped by the police because it was not labeled with a green label. The police told the owner that it is also against the regulations to not stick a car logo, and will be punished from March 1. According to the municipal regulations, since December 1, 2014, vehicles without environmental protection inspection qualification marks are prohibited from driving on the road within the jurisdiction of the city (including 6 counties and 6 urban areas); The closed area composed of Zhonghua Road, Yuanhu Road, Dongge Road, Xiangzhu Avenue, Zhuxi Avenue, Jiangbei Avenue and Beida road is closed to yellow labeled vehicles 24 hours a day; The closed area composed of Xingguang Avenue, Yunju Road, Beida bridge, Beida Road, University East Road, Mingxiu Road, Nanwu Road, Xiangzhu Avenue, Zhuxi Avenue, Huluding bridge and Baisha Avenue limits the passage of yellow standard vehicles at 7:00-22:00. In order to avoid being punished after March 1, Nanning traffic police reminded car owners to seize the time to get motor vehicle environmental protection signs. According to the municipal work plan for gradually restricting the passage of highly polluting vehicles, from March 1 this year, the city will punish yellow standard vehicles for breaking into the restricted area and non-standard vehicles for driving on the road.If a motor vehicle drives on the road in a restricted driving area or restricted driving time in violation of regulations, or fails to obtain or use expired environmental protection classification signs, it will be fined 200 yuan by the traffic management department of the public security organ; Whoever forges, alters or uses, transfers, lends, alters, forges or alters environmental protection classification marks shall be confiscated and fined not less than 500 yuan but not more than 1000 yuan. The initial inspection compliance rate of Yongcheng yellow standard vehicles is less than 30%. This year, the municipal license plate recognition system began to implement the regular environmental protection inspection and compulsory maintenance system for exhaust pollution of motor vehicles. It is understood that in January, 14 motor vehicle environmental protection testing stations in the city conducted exhaust environmental protection testing on 15057 in use vehicles. From the inspection, the overall compliance rate of the initial inspection is 74%, of which the compliance rate of the initial inspection of gasoline vehicles is 80%, and the compliance rate of the initial inspection of diesel vehicles is only 47%; If divided according to the yellow and green standard, the initial inspection compliance rate of green standard vehicles is 81%, and the initial inspection compliance rate of yellow standard vehicles is only 25%. The test results show that although the environmental protection inspection of Zhejiang motor vehicle working condition law temporarily implements the loosest national standard, more than 20% of vehicles with excessive exhaust emissions are still driving on the road. Among them, less than 50% of diesel vehicles with relatively large emissions of particulate matter and nitrogen oxides are on the road; As a yellow standard vehicle with a higher pollutant emission level, less than 30% of the vehicles that meet the exhaust emission standard are on the road.

In recent years, with the progress of science and technology, great changes have taken place in the parking lot system. The traditional card swiping / ticket picking parking lot system has gradually disappeared, while the non-contact and long-distance identification technology has developed by leaps and bounds and evolved into today's license plate recognition system, which has greatly improved its efficiency, management and safety, The important thing is that it is more and more widely used. Comparing the traditional parking lot system with the current license plate recognition system, not only the appearance of the product has changed, but also the technology has changed greatly. With the precipitation of time and the development of technology, products are updated iteratively. License plate recognition system has been sought after by people in parking lot applications with its unique advantages such as powerful function, tall appearance and humanized intelligent equipment. In order to develop intelligent transportation and create a good parking environment for people to park, the license plate recognition system also continues to innovate and integrate new technologies. Combined with more intelligent devices (such as parking space guidance system, online parking payment, reverse car search query machine, ultrasonic guidance, etc.), more innovative development is for both sides (managers and car owners) Make efforts to facilitate life and make parking closer to our lifestyle. The process of license plate recognition can be divided into four steps: license plate location, license plate preprocessing, character segmentation and character recognition. The license plate recognition system adopts advanced fuzzy image processing technology, which can well deal with the overall tilt, text tilt, stain and blur of the license plate, and recognize the license plate number that is difficult to distinguish by human eyes. License plate recognition system has gradually become a complete set of intelligent access management system in the existing parking lot. It has the characteristics of low cost, simple installation and maintenance, safe and fast parking. If you want to know more wonderful industry news and parking plans, follow tiger Wong and give you wonderful information at any time!