Hybrid CNN–VLSI Background Subtraction Framework for Intelligent Edge Vision

Abstract

Background subtraction is a critical component of intelligent vision systems such as realtime (RT) surveillance, autonomous vehicles, and smart infrastructure at the network edge. Traditional methods based on deep learning are extremely accurate but highly computationally intensive (high energy) making them impractical for deployment on resource-constrained edge devices. Traditional hardware methods for background subtraction have very simple implementation requirements and typically low power consumption but may not be suitable in very fast-changing environments. This paper describes the development of a hybrid CNN–VLSI background subtraction framework that combines together lightweight convolutional neural network designs with high energy–efficient VLSI background subtraction hardware for accurate, RT extraction of foreground images from edge platforms. The framework consists of a CNN-based adaptive background modelling unit for scene understanding. It also has a custom VLSI accelerator that performs pixel-level background subtraction, thresholding, and morphological operations on the output from the CNN unit. Assessment of the performance of this approach using test benchmark datasets showed that it achieved comparable accuracy to traditional approaches while providing reduced latency and reduced power consumption of more than six times better than traditional software-only based CNN background subtraction approaches. This work represents a viable solution for developing energy-efficient intelligent edge vision systems that is scalable and suitable for use in RT applications.