A Deep Reinforcement Learning Approach to Multiple Streams' Joint Bitrate Allocation

Abstract

For widely used real-time applications, encoding and transmitting multiple videos jointly over a limited bandwidth has become a popular topic. Allocating different bitrates for different sources is a better way to meet different demands from applications. In this paper, we focus on providing equal quality to users by minimizing the variance of distortion among sequences, which is denoted as the minVAR problem. The state-of-the-art Look-ahead and Feed-back Allocation Model (LFAM) allocates bitrate by taking both look-ahead complexity measures and feed-back information into consideration. However, LFAM brings additional delay to real-time applications. By taking the bitrate allocation problem as a time-series decision making problem, we propose a Deep-Reinforcement-Learning-based approach to allocate bitrate with only feed-back information to solve the two-source minVAR problem. Afterward, we introduce a binary-tree-based hierarchical approach to apply our model to arbitrary number of sources. Tested with the widely used open-source x264 encoder, our approach decreases the variance compared with LFAM in all experiments under two-, three- and four-source scenarios. Furthermore, the proposed approach also outperforms LFAM in the mean quality. The proposed approach is insensitive to the order of sequences and encoders with different complexities, showing its robustness and generalization capability.