This paper presents a system for detecting road departures by comparing linguistic representations for the trajectory of the vehicle with that for the lane marks of the road. All this information is obtained from a single camera processing exclusively the H264 motion vectors extracted from the recorded video. The process of comparison between the linguistic elements allows detecting the subset of continuous frames where there is no logical correspondence between the displacement of the vehicle and the road shape. Since the videos are captured from a moving vehicle, we propose a statistically based process to use domain changing fuzzy sets adapted to traffic scenarios that continuously change. This improves the reliability of the linguistic descriptions that, once compared, are used to detect departures. Lastly, a set of experiments using traffic videos with different characteristics are presented to validate this approach.