Real-Time Processing and Inference in Distributed Edge-Cloud Networks

Edge computing has recently emerged as a promising paradigm to improve the latency of real-time applications in IoT and mobile networks. By using computing resources near IoT devices for local storage and preliminary data processing, edge computing can improve the delay incurred due to the bandwidth bottleneck between devices and the cloud. The training and inference tasks are increasingly done using Deep Neural Networks (DNNs), which are computationally intensive but can be run in cloud which has enormous available resources. Deep Learning and inference in edge-cloud architectures has been studied before. However, current methods do not adequately address the unique challenges related to real-time largescale distributed system, including training and inference division between end devices, edge servers, and cloud, across possibly wireless or bandwidth-limited networks. This project will design and implement an integrated Machine Learning, Networking, and Computing for edge-cloud architectures to enable real-time data processing and inference. The target case study in this DURIP is a smart-city intersection to automate traffic management and safety. The project will use sensors, such as wired and wireless cameras, to enable real-time recognition and tracking of objects (pedestrians and vehicles) at one of the intersections close to Columbia campus. The first target is to achieve latency around 33 ms for a loop which consists of: (i) sensor data acquisition; (ii) communications between end users, edge servers and the cloud, (iii) real-time DNN-inference, and (iv) feedback towards the end-users. The second target is to achieve an accuracy of above 90% for detection and tracking algorithms, within the prescribed latency constraint. Our preliminary and proposed research is based on videos with very small pedestrians captured from a birdÕs-eye view, where pedestrians loose prominent features such as body shape. In this setup, previously known detection and tracking algorithms have to be notably modified. Moreover, most prior works do not consider privacy violation concerns resulting from the use of street level cameras. Additionally, most proposed algorithms operate offline. The project will develop and implement a distributed architecture which maps sections of DNN to the cloud, the edge, and end devices. By jointly training the distributed DNN, the lower NN layers can perform local inference at the end devices, while higher NN layers at the edge or in the cloud can aggregate output from multiple local NN layers, and use more layers to improve the overall classification accuracy. Moreover, the characteristics of videos from the intersection calls for the development of specialized methods for video pre-processing and optimizing DNN models for object detection and tracking including training the models based on the smart-city intersection dataset. A prototype based on these objectives will be demonstrated. All the major equipments of this DURIP, including Software Defined Radio Nodes and servers with GPU accelerators, will be integrated to the existing COSMOS testbed at Columbia University.