Shoubin Yu

Shoubin Yu

UNC, Chapel Hill

Hi, thanks for stopping by! I am now a second-year Ph.D. Student at The University of North Carolina at Chapel Hill, advised by Prof. Mohit Bansal. Previously, I did my undergraduate study at Shanghai Jiao Tong University.

While at UNC, I spent my summer time at Amazon Alexa (2023). Prior to UNC, I did research at SenseTime (2021), MIT-IBM Watson AI Lab (2021).

I am interested in wide topics in computer vision, especially in video, including video+X (language, audio, robotics), video understanding, generation, reasoning, representation learning.

ðŸ”Ĩ News

  • 2024.01: 🎎 I will intern at Adobe as Research Intern for Summer 2024.
  • 2023.09: ⛓ïļ We have one paper accepted to NeurIPS 2023. Check SeViLA for Video Loc+QA.
  • 2023.07: ðŸĶī We have one paper accepted to IEEE TCSVT. Check MoPRL for skeletal anomaly detection.
  • 2023.05: 🌞 I will intern at Amazon as Research Scientist Intern for Summer 2023.
  • 2022.06: 🎓 Graduate from Shanghai Jiao Tong University! (excellent graduates).
  • 2022.04: ⛩ïļ I will join UNC-CH MURGe-Lab in Fall 2022.
  • 2021.10: 🌟 We have one paper accepted to NeurIPS 2021. Check STAR for real-world situated reasoning.

📝 Pre-print

Preprint
sym

CREMA: Multimodal Compositional Video Reasoning via Efficient Modular Adaptation and Fusion

Shoubin Yu*, Jaehong Yoon*, Mohit Bansal

Code | Project Page

  • We present CREMA, an efficient & modular modality-fusion framework for injecting any new modality into video reasoning.
Preprint
sym

A Simple LLM Framework for Long-Range Video Question-Answering

Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, Gedas Bertasius

Code

  • We present LLoVi, a simple yet effective framework with LLM for long-range video question-answering.

📝 Publications

NeurIPS 2023
sym

Self-Chained Image-Language Model for Video Localization and Question Answering

Shoubin Yu, Jaemin Cho, Prateek Yadav, Mohit Bansal

Code | Demo | Talk

  • We propose SeViLA, which self-chained BLIP-2 for 2-stage video question-answering (localization + QA) & refine localization with QA feedback.
TCSVT 2023
sym

Regularity Learning via Explicit Distribution Modeling for Skeletal Video Anomaly Detection

Shoubin Yu, Zhongyin Zhao, Haoshu Fang, Andong Deng,Haisheng Su, Dongliang Wang, Weihao Gan, Cewu Lu, Wei Wu

Code

  • We propose MoPRL, a transformer-based model incorporated with skeletal motion prior for efficient video anomaly detection.
NeurIPS 2021
sym

STAR: A Benchmark for Situated Reasoning in Real-World Videos

Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B. Tenenbaum, Chuang Gan

Code | Project Page

  • We propose STAR, a benchmark for neural-symbolic video reasoning in real-world scenes.

🎖 Honors and Awards

  • CN Patent CN114724062A, 2022
  • The Hui-Chun Chin and Tsung Dao Lee Scholar, 2020
  • CN Patent CN110969107A, 2019
  • Meritorious Award in Mathematical Contest in Modeling, 2019
  • Second Prize in Shanghai, China Undergraduate Mathematical Contest in Modeling, 2019

🧐 Service

  • Conference reviewer: CVPR 2024, ACL 2023, EACL 2023, CoNLL 2023, CVPR 2023 Workshop, AAAI 2023 Workshop
  • Journal reviewer: IEEE Transactions on Circuits and Systems for Video Technology

📖 Educations

sym
  • 2022.09 - Present
  • The University of North Carolina at Chapel Hill
  • Computer Science, Ph.D.
sym
  • 2017.09 - 2022.06
  • Shanghai Jiao Tong University
  • Information Security, B.Eng.

ðŸ’ŧ Internships

sym
sym