Watch On:
Summary
Without inference, an artificial intelligence model is just math and does not actually execute or forecast anything. The need for GPU and inference engine abstraction. The idea of lock-in for AI hardware is not limited to just inference engines. Intel is among the leading backers of the open-source SYCL specification, which seeks to help create a unified programming layer for GPUs. Mathews explained that SYCL is closer to the GPU programming level, while AITemplate is focusing on high-performance TensorCore/MatrixCore AI primitives.
Show Notes
Without inference, an artificial intelligence (AI) model is just math and does not actually execute or forecast much, if anything.
To date, AI inference engines have been largely tethered to specific hardware for which they are designed.
The challenge of managing inference hardware has not been lost on social media giant Meta (formerly Facebook).
In early tests conducted by Meta, it is already seeing performance improvements over non-AIT inference-powered models on both Nvidia and AMD GPUs.
“For AIT the goal is to bring flexible, open, more energy-efficient AI inference for GPU users,” Mathews said.
Source
https://venturebeat.com/ai/meta-seeks-to-accelerate-ai-inference-with-open-source-aitemplate/
