Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps

Abstract

In many pattern recognition or machine learning applications, deep learning models or deep neural networks have provided superior performance relative to feature-based classifiers. Although there exist a number of publicly available software tools that enable the development of deep learning models to be achieved with ease, no guidelines are currently available in one place and in a unified manner in the literature for using these tools towards real-time deployment of deep learning models on smartphones, which have now become the most widely used computing device in the world. A uniform flow of implementation is developed in this dissertation for deployment of deep learning models on smartphones as real-time apps on both Android and iOS devices. A benchmarking framework consisting of accuracy, CPU/GPU consumption, and real-time throughput is devised for this deployment. These guidelines are applied to image and audio processing applications. As an image processing application, six widely used deep learning models are implemented to run in real-time on smartphones for recognizing objects based on video captured by their cameras. As an audio processing application, a low audio-latency signal processing pipeline along with a multi-rate processing technique are developed in order to achieve a convolutional neural network-based voice activity detection.