AI Inference

In practical terms, artificial intelligence means giving a machine the ability to analyze data without specifically programing every criteria used to perform that analysis.

The first phase in developing an AI device is called training. The system is normally presented with a large dataset, and a program with a basic model to allow the system to analyze the data and to determine if it has achieved the success criteria. The system analyzes the data and makes modifications to the model used to arrive at the final analysis in order to improve the success rate. The training phase is very processor intensive, and normally takes place in a lab or data center with access to a large number of processors.

Once the system has been trained the model that was developed in the training phase can then be loaded onto another device to analyze new data. Using the model to infer something about new data is known as the inference phase. AI inference generally takes far less processing power than training, so inference models can be found running on smaller, lower power devices that are operating in the field, analyzing real world data.

Devices using AI inference have been seen in self-driving vehicles, speech analysis, natural language translation, chat-bots, document analysis (e.g., legal and medical documents), robotics, and predictive video encoding, to name just a few uses.

Both the training and inference phases normally benefit from parallel processing and matrix processing. Since GPUs are already based on a parallel processing technology, using GPUs for AI was a natural extension of the existing technology. NVIDIA has taken that even further by implementing Tensor Cores in its latest generation of GPUs. Tensor cores are similar to normal graphics processing cores, but they allow mixed precision computations. Machine learning models do not always require the higher precision of dedicated graphics cores, making Tensor cores a more effective use of the available processing power.

WOLF offers a number of NVIDIA GPU-based modules which include Tensor Cores as well as specialized accelerator circuits for deep learning inference, machine vision, audio processing, and video encoding. These modules can also benefit from NVIDIA’s rich set of AI tools and workflows.

WOLF modules which include Tensor Cores: