Wenhao Chai

Wenhao Chai is currently a graduate student at University of Washington, with Information Processing Lab advised by Prof. Jenq-Neng Hwang. Previously, he was an undergradate student at Zhejiang University, with CVNext Lab advised by Prof. Gaoang Wang. He is fortunate to work with Prof. Christopher D Manning at Stanford University, and have worked with Prof. Saining Xie and Prof. Yilun Du. He has internship at Pika Labs and Microsoft Research Asia. His research primarily in large multimodal models (LMMs) for video understanding, embodied agent, and generative models. He has published related papers in top-tier conferences and journals such as CVPR, ICCV, ECCV, and AAAI. He has also organized workshops and tutorials at CVPR and AAAI, and served as a reviewer for NeurIPS, ICLR, ICML, CVPR, ECCV, and AISTATS. My current research focus on developing embodied AI agents inspired by cognitive science principles to interact with the physical world, building upon video understanding as a core perceptual mechanism. I propose a long-short term memory framework modeled after the human memory system, enabling pre-trained video LMMs to comprehend multi-hour video content without additional fine-tuning. To enhance efficiency, I introduce token merging, significantly reducing visual tokens with minimal performance degradation. I also demonstrate step-by-step agent system development in Minecraft, showcasing cognitive-inspired agent capabilities in virtual environments.

Publications

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory

Cheng-Yen Yang, Hsiang-Wei Huang, Wenhao Chai, Zhongyu Jiang, Jenq-Neng Hwang

Wenhao Chai

Publications

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory

LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound

Ego3DT: Tracking Every 3D Object in Ego-centric Videos

PAD: Personalized Alignment of LLMs at Decoding-Time

AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancement

RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark

NTIRE 2024 Image Shadow Removal Challenge Report

STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft

Learning Diffusion Texture Priors for Image Restoration

CityCraft: A Real Crafter for 3D City Generation

Boosting Online 3D Multi-Object Tracking through Camera-Radar Cross Check

MovieChat+: Question-aware Sparse Memory for Long Video Question Answering

MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection

Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model

Random bridge generator as a platform for developing computer vision-based structural inspection algorithms

VersaT2I: Improving Text-to-Image Models with Versatile Reward

Exploring Learning-based Motion Models in Multi-Object Tracking

Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation

Unsupervised Domain Adaptation Approach for Vision-Based Semantic Understanding of Bridge Inspection Scenes without Manual Annotations

User-Aware Prefix-Tuning is a Good Learner for Personalized Image Captioning

CityGen: Infinite and Controllable 3D City Layout Generation

See and Think: Embodied Agent in Virtual Environment

UniHPE: Towards Unified Human Pose Estimation via Contrastive Learning

Efficient Domain Adaptation via Generative Prior for 3D Infant Pose Estimation

Sequential Affinity Learning for Video Restoration

Devil in the Number: Towards Robust Multi-modality Data Filter

Chasing Consistency in Text-to-3D Generation from a Single Image

UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning

StableVideo: Text-driven Consistency-aware Diffusion Video Editing

PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation

MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision

Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation

MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling

Image Reference-guided Fashion Design with Structure-aware Transfer by Diffusion Models

Five A+ Network: You Only Need 9K Parameters for Underwater Image Enhancement

Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation

Blind Inpainting with Object-Aware Discrimination for Artificial Marker Removal

Deep Learning Methods for Small Molecule Drug Discovery: A Survey

DiffFashion: Reference-Based Fashion Design With Structure-Aware Transfer by Diffusion Models

Automatic Spinal Ultrasound Image Segmentation and Deployment for Real-time Spine Volumetric Reconstruction

Weakly Supervised Two-Stage Training Scheme for Deep Video Fight Detection Model

Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend

Diffusion-based Zero-Shot 3D Human Pose Estimation

Supplement Material: Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation

PAD: Personalized Alignment at Decoding-Time