Multiple object tracking has numerous applications in video surveillance, human behavior analysis, visual navigation and sports video analysis. In contrast to applying independent individual trackers, the multi-object tracker handles all of the objects simultaneously in order to exploit the cross-object clues, e.g. occlusion constraints, layout constraints, motion constraints, etc. As a consequence multi-object trackers usually display better tracking performance than single object trackers.

However, most multi-object trackers to date do not utilize the identity information of the tracked objects. We argue that the joint tracking and recognition are beneficial due to their intrinsic temporal interdependencies. With all objects distinct, such as different human faces, the interdependencies become particularly appealing. For the tracker, the objects recognized as the same identity will have higher probability of belonging to the same track, while the probability for the objects with different identities will be lower. Thus the identity information could guide the adjustment of cross-object similarities, which will be used in tracking. For the recognition side, objects of one track are more likely to have the same identity. Therefore, the joint tracking and recognition alleviates the difficulty of recognition due to the insufficient visual information from one single frame. Besides the interdependencies between tracking and recognition, one could also utilize pairing constraint, i.e. one identity cannot be assigned to multiple objects within one video frame. The above mentioned relationships are illustrated in the following figure.

In this project, we are building a joint tracking and recognition system, in which the above mentioned interdependencies between the two tasks and pairing constraint could be naturally integrated. Preliminary experiments on human faces show its superior accuracy compared to independent tracking and recognition.

References

• [1] A. Cohen and V. Pavlovic. “An Efficient IP Approach to Constrained Multiple Face Tracking and Recognition”. IEEE International Workshop on Socially Intelligent Surveillance and Monitoring. 2011.