Watch On:
Summary
A new technique enables AI models to continually learn from new data on intelligent edge devices like smartphones and sensors, reducing energy costs and privacy risks. However, the training process requires so much memory that it is typically done using powerful computers at a data center, before the model is deployed on a device. To address this problem, researchers at MIT and the MIT-IBM Watson AI Lab developed a new technique that enables on-device training using less than a quarter of a megabyte of memory. Moreover, the framework preserves or improves the accuracy of the model when compared to other training approaches. The first, known as sparse update, uses an algorithm that identifies the most important weights to update at each round of training. “Updating the whole model is very expensive because there are a lot of activations, so people tend to update only the last layer, but as you can imagine, this hurts the accuracy. Then the algorithm applies a technique called quantization-aware scaling (QAS), which acts like a multiplier to adjust the ratio between weight and gradient, to avoid any drop in accuracy that may come from quantized training. At the same time, they want to use what they’ve learned to shrink the size of larger models without sacrificing accuracy, which could help reduce the carbon footprint of training large-scale machine-learning models.
Show Notes
A new technique enables AI models to continually learn from new data on intelligent edge devices like smartphones and sensors, reducing energy costs and privacy risks.
Training a machine-learning model on an intelligent edge device allows it to adapt to new data and make better predictions.
Their technique can be used to train a machine-learning model on a microcontroller in a matter of minutes.
“Our study enables IoT devices to not only perform inference but also continuously update the AI models to newly collected data, paving the way for lifelong on-device learning.
This system changes the order of steps in the training process so more work is completed in the compilation stage, before the model is deployed on the edge device.
