Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps
Abstract
Abstract
In many pattern recognition or machine learning applications, deep learning models or deep neural
networks have provided superior performance relative to feature-based classifiers. Although there
exist a number of publicly available software tools that enable the development of deep learning
models to be achieved with ease, no guidelines are currently available in one place and in a unified
manner in the literature for using these tools towards real-time deployment of deep learning models
on smartphones, which have now become the most widely used computing device in the world. A
uniform flow of implementation is developed in this dissertation for deployment of deep learning
models on smartphones as real-time apps on both Android and iOS devices. A benchmarking
framework consisting of accuracy, CPU/GPU consumption, and real-time throughput is devised
for this deployment. These guidelines are applied to image and audio processing applications. As
an image processing application, six widely used deep learning models are implemented to run in
real-time on smartphones for recognizing objects based on video captured by their cameras. As an
audio processing application, a low audio-latency signal processing pipeline along with a multi-rate processing technique are developed in order to achieve a convolutional neural network-based
voice activity detection.