Optimal DNN Partitioning and Deployment for Dependable Inference Workflow on End Devices

The end-side inference is an efficient method that implements deep neural network (DNN) models on end devices through model partitioning, enabling real-time AI processing without relying on cloud services. However, existing partitioning and deployment strategies primarily focus on end-edge-cloud (EEC) collaborative environments, overlooking the limitations of resource-constrained end devices and their unsuitability for heterogeneous FPGA-based computing architectures. Moreover, these strategies often hinge on data and resource coordination among EEC nodes, which imposes certain limitations in terms of data synchronization and communication overhead. This paper aims to construct an inference workflow service on the end device by leveraging the pipeline of the end device and the inter-layer relationship of the DNN. To enhance inference performance on the end device, we formulate the workflow problem of multi-kernel collaborative inference by optimizing both DNN model partitioning and workflow deployment while considering constraints related to heterogeneous computing resources and hardware wiring complexity, with the goal of maximizing workflow dependability and minimizing deployment costs. To this end, we propose a hierarchical game-theoretic framework. First, we develop a sparse graph mapping model for DNN partitioning based on coalition games to merge, split, and switch DNN layers. Subsequently, to minimize deployment costs further, we develop a workflow coalition deployment scheme based on matching games and regret learning to efficiently select workflow coalitions. Simulation results demonstrate that our proposed scheme is superior to other workflow deployment schemes; tests conducted on the FPGA platform show that our mechanism achieves 8.95% lower inference latency and 6.90% higher inference speed compared to the matching game method (MGM) approach.