Abstract
Human eye movements indicate important spatial information in static images as well as videos. Yet videos contain additional temporal information and convey a storyline. Video summarization is a technique that reduces video size, but maintains the essence of the storyline. Here, the authors explore whether eye movement patterns reflect frame importance during video viewing and facilitate video summarization. Eye movements were recorded while subjects watched videos from the SumMe video summarization dataset. The authors find more gaze consistency for selected than unselected frames. They further introduce a novel multi-stream deep learning model for video summarization that incorporates subjects' eye movement information. Gaze data improved the model's performance over that observed when only the frames' physical attributes were used. The results suggest that eye movement patterns reflect cognitive processing of sequential information that helps select important video frames, and provide an innovative algorithm that uses gaze information in video summarization.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
