Online Multi Object Tracking Using Reinforcement Learning



Journal Title

Journal ISSN

Volume Title



This thesis presents an approach to online learning of Multi-Object Tracking (MOT). It is based on representations from a discriminatively trained Convolutional Neural Network (CNN) and controlled by sequentially pursuing actions learned by deep reinforcement learning. While tracking, bounding box drifts and brightness variations make it tough to separate unique identities amongst targets and result in frequent identification switches. Alternatively, detection outputs suffer from longtime occlusion, dynamic backgrounds, and low-resolution images. We propose a robust tracking method that can handle these difficulties effectively. We constructed a fully end-to-end network that can predict the bounding box location of the target object in every frame. First, we developed a deep CNN model in a supervised manner from a large set of videos with tracking ground truths to obtain a generic target representation and responsible for classification of an object as foreground or background. We embedded Reinforcement Learning (RL) algorithms in the tracker helping it to make sequential decisions i.e., actions to track the object. To handle dynamic change in target and background while tracking, we continuously fine-tune the network. When tracking a target in a new sequence, we construct a new network by combining the shared layers in the pre-trained CNN with a new binary classification layer, which is again updated online. The proposed tracker will reduce the computation used for tracking by reducing the search space with the help of trained RL policies. We evaluated our approach on three public data sets and validated that our model achieves three times faster tracking speed compared to existing state-of-the-art trackers.



Multi-Object Tracking, Deep Reinforcement Learning, Online learning