Vehicle and human trajectory predictions are nowadays fundamental pillars of self-driving cars. Both the industry and research communities have acknowledged the need for such pillars by running public benchmarks. While state-of-the-art methods are impressive, i.e., they have no off-road prediction and without any collision, their performance in some scenarios are unknown. In this talk, Saeed will elaborate his two recent papers (accepted in CVPR and TR_C). First, he will evaluate the social-awareness of human trajectory prediction models using the socially-attended attacks. An attack is a small yet carefully-crafted perturbations guided by soft-attention. He will also describe how the proposed method can be employed to increase the social understanding of the state-of-the-art models. Second, he will talk about the generalizability to new scenes and presents a novel method that automatically generates realistic scenes causing the vehicle trajectory prediction models go off-road. The model is a simple yet effective generative model based on atomic scene generation functions along with physical constraints. He will show that more than 60% of the existing scenes from the current benchmarks can be modified in order to make prediction methods fail. Moreover, they do exist in the real world, and can be used to make existing models robust.