Reconstruction: the Mighty Camera

At the core of DroneDeploy is a product called Map Engine that is able to quickly and accurately reconstruct 3D scenes from a collection of photographs without any prior knowledge of where the photos were taken, or what they contain. This is one of the core problems of the field of photogrammetry which draws on techniques from computer vision and machine learning extensively. Being able to do this is powerful, as a digital camera is a lot cheaper and easier to move around than a 3D scanner but you do need some smart software.

Reconstruction: the Mighty Camera 1

This is the first of three posts describing how this process takes place from beginning to end. We will also be implementing a simple and educational pipeline that reconstructs 3D scenes from 2D images.First we are going to look at some of the fundamentals of cameras and code up some building blocks.

Our story starts with the mighty digital camera. Cameras use light to capture a 2D representation of the 3D world by exposing a sensor to light that is focused through a lens. This creates a digital image.

Photographers talk about cameras in terms of shutter speed, focal length and ISO. Here is a great interactive illustration of how these factors affect the final image.In computer vision we approach cameras differently and are concerned rather with the external and internal geometry of the camera.

The external geometry, called the extrinsics represents how the world is transformed relative to the camera when we look through it. Its usually more intuitive to specify where the camera is in the world than how it transforms the world though. This is called the pose and we specify it as a matrix [R | t]where R is a 3x3 rotation matrix and t is a 3x1 translation vector.

Reconstruction: the Mighty Camera 2

The extrinsics are then E = [R',-R'*t]. This inversion of the pose transforms the world so that we are effectively looking down the optical axis of the camera. Next we have the internal geometry of the camera, called the intrinsics which represent a conversion to pixels.

Two important values here are the camera center (also called the principal point) and the focal length. Together in 3x3 matrix as K this represents a scaling and translation. This is is as mapping pixels into the image based on the field of field of the camera and where the pinhole of the camera is.

We can now create a projection matrix as P = K * E. Together the camera projection matrix can be thought of as acting on a 3D point as a rotation translation, followed by a translation and scaling yielding the final image coordinates. Which might call off the edges of the image depending on the resolution in pixelsProjectionLets implement a simple pinhole camera.

We can now use the projection matrix of a camera to transform 3D world points to 2D image coordinates. We represent out world coordinates at 4D homogenous coordinates and then multiple by the projection matrix and normalize to get image coordinates. Lets test this out by creating some 3D world points.

Well create the vertices of a cube and place the camera backwards long the z-axis. Well then project each of the vertices of the cube into the camera and we should get a 2D representation. The resulting image shows our cube and correctly captures the perspective introduced by the back face being further from the camera than the the front.

Aside: DecompositionWe can also go backwards and decompose the projection into its K, R, t components remembering that the structure is P = K[R | -R'*C]we can first extract t and decompose the remainder to get K and R. Unfortunately this decomposition is not unique because if you take the resulting R and Q matrices from the RQ decomposition and negate a row and corresponding column of R and Q the resulting projection matrix is the same. Positioning the cameraUp until now weve just had our camera looking down the z-axis which isnt that exciting.

We want to fearlessly be able to place our camera anywhere and pointing in any direction. Placing the camera anywhere is easy we just specify the position t. Pointing it in any direction is a bit less intuitive because we have to specify the orientation as a rotation matrix.

One way to do this is to construct the rotation matrix of the pose as the axes of the camera in world coordinates with the first axis being the direction the camera is pointing and the other two: a vector through the top of the camera and a vector through the side of the camera. Heres a function to position the cameras somewhere looking at a certain point:Implementing this we can position the camera anywhere and look in any direction. Lets test it out and position the camera randomly around our cube and use our existing projection code to look at the cube from different anglesYoull notice an optical illusion here in that the cube will look skew in some images.

As your eyes sort out the orientation you will see them square up. The reason is because we arent sorting the edges by their depth from the camera meaning some lines in the 2D images are draw over other lines that they should be behind. If we wanted to clear this up we should sort our geometry by their depth from the camera and render in that order.

Some more visualizationNow that we can position and look through arbitrary cameras lets visualize the camera positions instead of just what they see. We can do this my placing another camera looking at the whole scene of cameras and render the position of each camera along with its axes and frustum. The result is our whole scene of cameras looking as our cube.

TriangulationNow we should be pretty comfortable with cameras and mapping the 3D world to 2D images. But can we go backwards? Lets try and reconstruct the 3D vertices of the cube from a 2D picture.

Unfortunately we cant do this from a single view because a pixel corresponds to a ray extending out into the real world and the 3D point could lie anywhere along that ray. But if we had a few different poses of the camera the rays should pass through the same corresponding pixel each image and intersect at a unique point in the 3D world. Take a look at this picture to convince yourself that you need at least two pixels to reconstruct a 3D point.

If we take two images of our cube and write out how two 2D pixels were computed from the 3D points using the two projection matrices ( P1 and P2 ) we end up with a linear system of equation. We could try and solve this but a couple of things go wrong here, the matrix is typically not square (unless you have two views and so the inverse doesnt work. A better and more numerically stable way is to instead look as it as a homogeneous systems of equations.

This has a trivial solution at 0 0 0 which we are not super excited about but if we take the singular-value decomposition (SVD) of this matrix and take the singular vector corresponding to the smallest singular value we get a solution to the system with some useful additional properties. In particular it constrains the solution vector to have a magnitude of 1 so avoids the trivial solution. Its also numerically stable and when use to solve over-determined systems naturally.

Lets code this up and create a function that takes a list of cameras and corresponding list of image coordinates and triangulates them.More triangulationThe SVD based triangulation makes two assumptions. The first is the error term we are looking to minimize and the second is how the camera operates on 3D points namely as a linear operation.

We can also triangulate points using a non-linear solver. We can frame this as a problem where we seek the X, Y, Z coordinates in the world such that they project into the corresponding image coordinates in each camera we are considering. We can use something like LevenbergMarquardt to solve this problem.

Although this takes longer than our analytical solution it allows use to be more flexible about some things. For example instead of least square we can use different norms like the Huber norm to handle outliers in our data. This is useful as we will see in the next post where we dont know exactly where our 3D and 2D points are.

Another advantage is that we can use more sophisticated camera models that cant just be represented as a matrix multiplication.DistortionWe just mentioned more sophisticated camera models up until now we have dealt just with an ideal camera called a pinhole camera. In the real world cameras are not as perfect.

Due to imperfections and tolerances in the manufacturing process each camera is slightly different and also the lens used can introduce types of distortion. For example a photograph of a scene with perfectly straight lines may appear to bend in the image. This can cause problems if not handled correctly as typically the reconstructions are used for measurement and planning.

Heres a simple implementation of a BrownConrady camera model:Now lets vary some of the radial and tangential distortion parameters and see what it does to our cube projection.Unfortunately the distortion parameters are not printed on the box of the camera and have to be estimated from the correspondence between between coordinates in different images. This is something we will be solving in the next post.

So that completes our brief walkthrough of camera fundamentals. Weve talked about how to position cameras in the world and view a scene from different locations. We looked at constructing and decomposing the projection matrix.

Weve also looked at reconstructing the position of 3D world points from 2D image coordinates. In the following posts we will tackle the problem of reconstructing 3D scenes using these fundamentals but without any ground truth data like our cube

HOT PRODUCTS
no data
GET IN TOUCH WITH US
recommended articles
Cases
What Will Affect the Accuracy of Face Recognition_ Taigewang Technology
What Will Affect the Accuracy of Face Recognition_ Taigewang Technology
Face recognition technology is advanced, but its recognition rate is also 100% accurate. There are also some difficult problems that are difficult to overcome, such as human expression and illumination, which will affect face recognition. Here is a brief introduction. First, illumination change is the most key factor affecting the accuracy of face recognition. Because the face is a 3D structure, Therefore, the shadow cast by ray will strengthen or weaken the characteristics of the original face. Especially at night, the face shadow caused by insufficient light will lead to a sharp decline in recognition rate, which makes the system difficult to meet the actual requirements. At the same time, theory and experiment also prove that the difference between the same individual under different light is greater than that between different individuals under the same light. 2、 Facial expression change facial expression change is composed of different expression expression areas, including mouth, cheek, eyes, eyebrows and forehead. Human emotional expression is realized through local deformation of the muscles that make up the expression area. These changes are not simple translational and rotational rigid changes. The local deformation of facial muscles caused by expression changes will be regarded as a part of facial feature patterns, so it is difficult to distinguish whether the differences between feature patterns are caused by different faces or expression changes, which makes classification and recognition difficult.
The Parking Lot System Is Developing Towards a More Intelligent and Convenient Direction_ Taigewang
The Parking Lot System Is Developing Towards a More Intelligent and Convenient Direction_ Taigewang
With the continuous improvement of people's living standards and the increasing number of vehicles, after nearly 20 years of development, the technology of the parking lot system industry has been greatly improved and gradually moved closer to the international level. Some technologies have been in the forefront of the same industry in the world. However, while making continuous progress, there are also some shortcomings and deficiencies. It can be seen from the current development status of China's parking lot system industry that although the development of China's parking lot system industry has tended to a mature stage, there are still shortcomings of insufficient intelligence and convenience, resulting in different degrees of parking difficulties in various cities in China. Therefore, it is particularly important to improve the intelligence and convenience of parking lot system equipment at present. In recent years, with the gradual maturity of license plate recognition technology, for the convenience of vehicle access, the parking lot has adopted license plate recognition parking lot system at the entrance and exit to control the access of vehicles. Now there are many parking lot management technologies such as parking space guidance system, urban parking guidance and reverse vehicle search, which have been widely used in major parking lots across the country, It enables people to park from entry to exit, fully realize intelligent and unmanned management, solve people's parking problem, and alleviate the pressure of urban traffic. As an enterprise focusing on the production of entrance and exit equipment in the security field, taigewang technology has continuously promoted new innovations in the parking lot system industry. For the convenience of parking lot management, taigewang cloud parking lot realized the collectivized management of parking lot in 2013, broke through the regional restrictions, and promoted the whole parking lot system industry to a new level.]
The Parking Lot System Has Developed From Simple Entrance and Exit Management to Multi-function_ Tai
The Parking Lot System Has Developed From Simple Entrance and Exit Management to Multi-function_ Tai
The parking lot system has experienced more than ten years of development. It has gone from simple entrance and exit management through our step-by-step research to the current multi-functional intelligent parking lot system. Now the parking lot systems we see in the parking places of the community and major shopping malls are configured by our parking lot system manufacturers according to the site conditions, Let's take a look at the whole process of the development of the parking lot system: the parking lot system is actually the equipment we often see at the entrances and exits of major parking lots such as residential areas and shopping malls to manage vehicle access. It is now considered to be an important equipment for managing vehicle access and charging in the parking lot. It is widely used, It can not only be used to manage the entry and exit of vehicles, but also help car owners quickly find the remaining parking spaces, self-service payment, etc. These are nothing. With the continuous development of society and the continuous upgrading of high technology, the emergence of card free license plate recognition parking lot system makes the parking lot system also appear frequently in highway toll stations, highways and other toll collection occasions. Because these occasions have high requirements for vehicle recognition distance and range, we also feel the power of high technology. With the development of high technology and continuous innovation, and with the change of people's needs, the parking lot system is also constantly improving its functions. The current parking lot system is not only used to manage the entrance and exit of the parking lot, but also provides people with functions such as rapid parking space search, reverse car search and self-service payment. In order to solve the problem of people's parking difficulty and arbitrary charging, At present, the parking lot system also supports the functions of parking space reservation and payment at the mobile end, so that more people will realize the convenience of parking.
Advantages of Video Parking Space Guidance and Automatic License Plate Recognition System in Parking
Advantages of Video Parking Space Guidance and Automatic License Plate Recognition System in Parking
In order to create a good car search environment for people, many parking lots adopt more advanced intelligent management methods to reduce unnecessary time waste caused by car owners' entry and exit or car search problems, so as to improve the circulation rate of parking spaces in the parking lot and enhance the satisfaction of car owners with the overall service level of the parking lot. In order to improve a good parking environment and improve user satisfaction, the parking lot is managed unmanned and intelligently by using video parking space guidance and automatic license plate recognition parking lot system. The use of video parking space guidance and automatic license plate recognition can effectively reduce the enterprise management cost. With the development of society, the human management cost will be higher and higher. The use of parking space guidance system will effectively reduce the management personnel in the parking lot, reduce the enterprise management cost and improve the fine management level of the parking lot, Through the comparison of parking space occupancy rate, the parking lot management personnel can be reasonably allocated during peak hours, and the remaining parking spaces in the yard can be understood in real time. The advantages of using video parking space guidance and automatic license plate recognition parking lot system are obvious to both parking lot management and car owners. The system adopts advanced parking space state pattern recognition algorithm, and the accuracy of parking space state recognition is more than 99%, which can accurately identify unlicensed vehicles; Video parking space guidance adopts the integration of parking space detector and indicator light, which is installed in the middle of the lane, which makes the construction easier, effectively reduces the construction cycle, and greatly reduces the material cost and labor cost of each parking space construction. The parking space detector guided by video parking space is directly installed on the bridge in front of each parking space to effectively avoid column shielding, The installation effect is also more neat and beautiful. The video parking space guidance and license plate recognition parking lot system can improve the information function of the parking lot, effectively count the traffic flow, parking space utilization and other information of the parking lot, so as to facilitate the parking lot managers to understand the use of the parking lot.
Intelligent Parking Lot System Sweeps the Market, and These Functions Cannot Be Less_ Taigewang Tech
Intelligent Parking Lot System Sweeps the Market, and These Functions Cannot Be Less_ Taigewang Tech
As the security industry continues to be valued in people's lives, the intelligent parking lot system is at a critical point in the introduction period and growth period. Many people do not have a deep understanding of the intelligent parking lot system. However, with the further implementation of the popularization of intelligent transportation, the development potential of the intelligent parking lot system in the future market is very huge, So let's have a comprehensive understanding of several functions of intelligent parking lot system used in the market. Based on the original functions, the intelligent parking lot system adopts the characteristics of modern technology and is equipped with a high-definition license plate recognition system to manage the entry and exit of vehicles through the license plate number recognition of vehicles entering and leaving the parking lot. For some places, fixed vehicles can enter and leave the parking lot at will, while for some temporary vehicles, the license plate recognition system is managed manually, The parking safety of fixed vehicles in the site is well guaranteed. Parking space guidance system is a necessary set of system in large parking lot. It can guide the vehicles entering the site to the place where there is parking space, avoid the owners looking for parking space aimlessly in the parking lot, save parking time and effectively improve the utilization of parking space. It plays an irreplaceable role in the parking lot. For car park car park car park car park, the parking fee is not only related to the interests of owners, but also closely related to the profits of parking lots. The parking fees paid by owners are not directly connected with the parking lot management providers without the third party turnover, so that they can achieve the face-to-face charges and pay for parking by the WeChat official account in the parking lot. It has effectively solved the problem of capital loss. Convenient parking for car owners only solves the problem of one party in the parking lot. For the parking lot manager, how to manage the parking lot more conveniently is very important. The parking lot cloud platform can solve this problem.
What Are the Hidden Dangers of Underground Parking Lot_ Taigewang Technology
What Are the Hidden Dangers of Underground Parking Lot_ Taigewang Technology
With the increasing number of motor vehicles in China, some system designs of many underground parking lots lack unified standards and specifications, which leads to the problem of difficult parking, which seriously affects the utilization of parking lots. Not long ago, the reporter visited some underground parking lots for many times and found that the corners at the entrances and exits of many underground parking lots are large, and some are even 90 degrees. For some car owners, they should mention the spirit of 12 points when entering and leaving the parking lot; Some parking lots not only have large corners and steep slopes, but also have a lot of sharp turns. Some car owners obviously have some difficulty when turning. There are not only these planning problems in the parking lot, but also the lack of unified standards for the road gates of the parking lot, which makes it difficult to park and pick up cards, and is prone to safety accidents. How to solve such problems in underground parking lot? According to the regulations on the planning and construction of parking lots and the management of motor vehicle parking, the design scheme of parking lots shall solicit the opinions of the traffic management department, and the construction procedures can be handled only after it is approved. However, there are no clear requirements for the setting of road gates and safety Islands in parking lots and the height of ticket boxes in parking lots. However, for some parking lots, the installation of intelligent parking lot systems is encouraged, Like the Bluetooth remote card reading or license plate recognition parking lot system, so that the owner can enter the parking lot without stopping at the entrance of the parking lot, while the card swiping parking lot system requires the owner to drive near the ticket box to pick up the card. Sometimes when the owner drives far away from the card taking machine, he has to loosen his seat belt and lean out most of his body to pick up the card, In this way, it is easy to have some safety accidents. License plate recognition and Bluetooth remote card reading parking lot system can enable car owners to enter the parking lot directly without stopping to pick up their cards. It can automatically release the import and export license plate numbers by capturing them with HD license plate recognition cameras, or it can receive wireless signals remotely through Bluetooth devices, and the gate can automatically lift its lever through these signals, In this way, car owners can also enter the parking lot directly without taking their cards. At present, there are many hidden dangers in the underground parking lot. In order to solve these hidden dangers, in addition to the reasonable planning during the construction of the parking lot, how to select the parking lot management system is also a very important part. Therefore, whether it is a new parking lot or a parking lot to be reconstructed, the selection of the parking lot system needs to be determined according to the site conditions of the parking lot.
Why Should the License Plate Recognition System Be Replaced and Upgraded in the Parking Lot_ Taigewa
Why Should the License Plate Recognition System Be Replaced and Upgraded in the Parking Lot_ Taigewa
In recent years, with China's economic development, people's living standards have been greatly improved, and cars are gradually popularized. Moreover, more and more families have one car, two cars and three cars in one family. However, in order to pay less parking fees, many families only apply for a card in the property and park multiple cars in the community with this card. Originally, the parking space in the community was very tight, one radish and one pit. As a result, these more cars either stood in other people's parking space or occupied the channel, resulting in the gradual intensification of the contradiction between residents and property, the refusal to pay property fees, and the vicious incidents of vehicle door blocking broke out frequently, which made the property industry miserable. In fact, in order to solve this problem, before the advent of the license plate recognition system a few years ago, the manufacturers of the parking lot system used the method of artificial image comparison to limit the situation of one card and more vehicles. However, because the door guards in many communities are older people, they are not familiar with computers, can not operate, and have a weak sense of responsibility, Therefore, it plays a very limited role, resulting in the virtual existence of artificial image contrast. With the development of artificial intelligence in the past two years, computer video recognition technology has gradually matured and entered the application stage. New technologies such as face recognition, license plate recognition, character recognition and expression recognition emerge in endlessly, especially the accuracy of license plate recognition has been greatly improved, which has reached 99%. After using license plate recognition in the parking lot system, the situation of one card and more vehicles is completely eliminated. One license plate number can only be used for this vehicle. The original statistical processing method based on the card is replaced by the statistical processing method based on the license plate, which completely eliminates the card. Due to the use of computer automatic identification, it also eliminates manual intervention and prevents cheating. Therefore, the upgrading of parking license plate recognition has become a general trend.
What Should Be Paid Attention to in the Purchase of Intelligent Gate in Parking Lot System_ Taigewan
What Should Be Paid Attention to in the Purchase of Intelligent Gate in Parking Lot System_ Taigewan
The barrier is the most common supporting product in the parking lot system. With the increase of the number of cars, the demand for the barrier becomes larger and larger, and its performance requirements are higher and higher, such as a series of functional requirements such as the anti smashing car of the barrier, the movement performance of the barrier, the requirements of the service environment, the take-off and landing speed of the barrier rod, so when purchasing the parking lot system, The gate is the primary consideration. As the most important part of the parking lot system, the gate must be considered from the above factors when purchasing. In the past two years, road gate smashing incidents are common. Therefore, taigewang has developed an anti smashing Lane gate according to this phenomenon. According to the customer's requirements and on-site use, different anti smashing Lane gates can be equipped respectively, such as different induction modes such as ground induction coil, pressure wave or infrared radiation to avoid the anti smashing performance of the lane gate. First of all, the most important part of the gate is the movement. The movement with good quality can not only prolong the service life of the gate, but also prolong the service life of the parking lot system. Secondly, the service environment of the gate is also one of the important factors to be considered in the purchase of the gate, including service temperature, waterproof performance, etc. because some gates will have a certain impact on the gate when the temperature is low or high, it is necessary to install a temperature compensation instrument on the gate in a high or low environment to reduce or increase the temperature in time, Ensure the normal use of the gate. Finally, for the selection of the lifting speed of the gate, the gate with different take-off and landing speeds can be installed according to different application occasions. In general highway toll stations or some parking lot systems that enter and leave through license plate recognition, some express gates need to be installed, and the take-off and landing time is generally 1 1.5 seconds; For some areas, schools, factories, commercial squares, parking lots and other places, 3-second or 6-second road gates can be installed according to the road width of the entrance and exit and the length of the gate. The specific can be customized according to the site conditions. As an important part of the parking lot management system, the correct selection of the appropriate parking lot gate can not only solve the anti smashing performance of the gate, but also prolong the service life of the parking lot system and increase the income of the parking lot.
How to Transform the Traditional Parking Lot Charging System in the Intelligent Era_ Taigewang Techn
How to Transform the Traditional Parking Lot Charging System in the Intelligent Era_ Taigewang Techn
With the rapid development of the Internet, the traditional parking lot charging model has been difficult to keep up with the pace of the times, more difficult to meet people's needs, quickly integrate into the needs of the times and realize transformation, which is the inevitable way for many parking lot charging system manufacturers. Since its establishment, taigewang technology has always paid attention to product innovation and contributed its own strength to the security field. Because of the defects in the function of the traditional parking lot system, it has not been well used in the parking lot, and sometimes it has become a kind of decoration. After the development of the Internet in recent years, combined with the current use status of the parking lot system and the convenience brought to our life by Internet technology, some manufacturers have sprouted a new innovative model, rapidly transforming the traditional parking lot system into an unattended user self-service payment model, and the owner can complete the whole parking payment process by himself. The development of the Internet is like an irreversible chariot. No matter what kind of traditional industry, it will be seriously impacted. Only by taking this chariot and continuous innovation can it develop continuously. Taigewang technology constantly studies the changes of the market and advances with the times to launch various parking payment methods to meet the needs of different customers. At present, the most widely used is the mobile terminal payment parking fee. People can use WeChat and Alipay to pay parking fees. According to the needs of customers, the company can customize different payment interfaces for users, and can connect with the users' system. The intelligent parking lot charging system has been upgraded on the original payment method, which can customize different differentiated products and solutions for users, improve people's parking efficiency and open up new routes for the development of smart cities.
Face Recognition Access Control on Campus to Improve Safety and Management Efficiency_ Taigewang Tec
Face Recognition Access Control on Campus to Improve Safety and Management Efficiency_ Taigewang Tec
The school is an important place for students to study and live. It is an important responsibility of the school management to ensure students' learning safety. Some off campus personnel often use management loopholes to enter and leave the school at will, which brings hidden dangers to the school safety. In order to strictly control the access of personnel in the school dormitory area and teaching area, prevent regardless of molecules and realize effective, safe and unified management, It is necessary to install intelligent face recognition access control system at all major entrances and exits of the school. Face recognition access control is a completely offline face recognition access control and attendance product. It is often used to replace card swiping and fingerprint access control and attendance machines in the market. Regional feature analysis algorithm, which is widely used in face recognition technology, integrates computer image processing technology and the principle of Biostatistics. It uses computer image processing technology to extract human image feature points from video, uses the principle of biostatistics to analyze and establish a mathematical model, namely face feature template. Using the built face feature template and the human face image of the tested person for feature analysis, a similarity value is given according to the analysis results. This value determines whether it is the same person. In schools with face recognition access control, only need to put the collected student photos into the database in advance, and the system camera will automatically capture the student's face, and then compare the collected face data with the database. The identity verification can be completed in less than 1 second, reaching the level of dozens of people per minute. Moreover, the specific time and photos identified by the system will be immediately pushed to parents.
no data
Shenzhen TGW Technology Co.,Ltd is the leading access control solution provider for vehicle intelligent parking system, license plate recognition recognition system and pedestrian access control turnstile and face recognition terminals.
no data
CONTACT US

Shenzhen TigerWong Technology Co.,Ltd

Tel: +86 0755-23773884 

E-Mail: info@sztigerwong.com

Add: Room 601-605, Building 6, 1980 Science and Technology Industrial Park,  Longhua Street, Longhua District, Shenzhen

                    

Copyright © 2021 Shenzhen TigerWong Technology Co.,Ltd  | Sitemap
chat online
Please message us and we’ll be sure to respond ASAP, what product you intrested in?