(source: Education News Network)Until recently, most AI was in the data center, and most of it was training. Things are changing rapidly. It is expected that by the mid-2020, the sales of artificial intelligence will grow rapidly to $10 billion, most of which will come from edge AI reasoning.Where is today's edge reasoning market for edge reasoning applications? Let's look at the market from the highest throughput to the lowest throughput. Edge server recently, NVIDIA announced that reasoning sales exceeded training for the first time. Most of them may have been delivered to the data center, but there are many applications outside the data center, often referred to as "edges". This means that the sales of PCIe reasoning boards for edge reasoning applications may reach US $100 million. One year and growing rapidly.

It has a wide range of applications: surveillance, face recognition, retail analysis, genome / gene sequencing, etc. Since training is done in floating-point numbers and quantization requires a lot of skills / investment, most edge servers infer that it may be done in 16 bit floating-point numbers, while only the maximum number of applications are done in int8. The PCIe inference board ranges from 75W (NVIDIA Tesla T4) to 200W (Habana Goya).

Autopilot a year ago, car manufacturers and suppliers were talking about using their custom chips to achieve full automatic driving. Today's plans are more modest, using off the shelf solutions (we often hear Xavier AgX and NX) for the 2020 model year, object detection and correction of megapixel images as a driver supplement to improve safety. At present, its number has reached tens of thousands of eye-catching test tools, such as Google waymo with large camera, eye-catching lidar and electronic luggage. Within 5 years, the sales volume of highly integrated mass market level 2 object detection and correction may reach millions.

The main players here are NVIDIA's Terson (nano, TX2, Xavier AgX and Xavier NX) in 5-30w and Intel movidius in countless single digit watts, but ~ daily throughput on January 10. There are a wide range of applications here: surveillance cameras, gene sequencing, home doorbells, medical systems (such as ultrasound), photonics, robot vision, and CNN is used in most cases, but various models different from image CNN can also be used.Fans are unacceptable in this market. The customers we talked to are eager for throughput. They are looking for solutions that can provide higher throughput and larger image size at the same power / price as the power / price used today: when they get the solution, their solution will be more accurate / reliable, adopted and expanded by the market. Therefore, although today's applications have thousands of units, as the availability of reasoning will grow rapidly, reasoning will provide more and more throughput / dollar and throughput / watt.Due to the wide range of applications, this market segment should become the largest market segment over time. Image CNN requires millions of Macs to be sent per second. With keyword recognition alone, speech processing can reach billions of MACS / s or even lower. These applications (such as Amazon echo) are already important in adoption and quantity, but the price of $/ chip is much lower. The participants in this market are completely different from the above markets.

It's a delay. The edge system is determining the speed of the image at 60 frames per second. For example, in cars, it is obviously important to detect objects such as people, bicycles and cars and play a role in the shortest possible time. In all edge applications, the latency is #1, which means that the batch size is almost always 1.