3 papers indexed
Yuwen Tan, Joey Huang, Jin Huang, Haoxiang Li, Boqing Gong
Understanding camera movement in natural language is critical for training and evaluating video generation models, among other applications. However, we demonstrate that existing vision-language models (VLMs) fail this task in surprising ways, frequently confusing translation wit…
Yuan Qing, Chengzhi Mao, Boqing Gong
Large Vision-Language Models (LVLMs) rely extensively on Visual Instruction Tuning (VIT) to elicit their multimodal reasoning capabilities. However, we find a discrepancy: VIT often packs multiple language tasks about the same image for conversational, multi-turn training, wherea…
Max Whitton, Zecheng Wang, Puchen Liu, Quang Tuan Truong, Shengao Wang, Manaswi Yadamreddy, et al.
Is the sense of touch a mechanism for human babies' learning of visual concepts? If so, can we quantify its importance, and to what extent do babies rely on their sense of touch for visual learning? To approach these questions in a principled way, we propose a structured coding s…