I am a Research Scientist at ByteDance in San Jose, USA. Prior to this, I worked as a Research Scientist at Tencent with Dr. Gang Yu for about three years. I received my doctoral degree from University of Chinese Academy Sciences (ShanghaiTech) Visual Intelligent Center (VIC) under the supervision of Prof. Jingyi Yu.

My research interests focus on Computer Vision for Graphics. I have a great passion on new things and new ideas, my goal is to create Generative AI, which is about humans, used for humans, and benefits humans.

Interests
  • Generative AI
  • Computer Vision for Graphics
  • Human Motion Generation
  • 3D Foundation Models
  • Multimodal Models
Services
  • [Conference Reviewer] 
    SIGGRAPH, CVPR, ICCV, ECCV, AAAI
  • [Journal Reviewer]
    TPAMI, TIP, IJCV, TMM
Collaborations
  • Welcome discussions from academia and industry, especially regarding technology implementation and real-world impact. Feel free to reach out via email.
  • I currently have internship positions available with the goal of conducting cutting-edge research in artificial intelligence. If you are interested, please send me an email.
News
  • [2024/09] MeshXL and 3DET-Mamba accepted to NeurIPS 2024, congrats to Sijin Chen and Mingshen Li
  • [2024/07] Joint ByteDance as a Research Scientist in San Jose, USA
  • [2024/06] We introduce MeshXL, a 3D fundamental model for mesh generation
  • [2024/04] Vote2Cap-DETR++ is accepted by T-PAMI 2024, congrats to Sijin Chen
  • [2024/02] 3 papers accepted at CVPR 2024, congrats to Paint3D, LL3DA, OMG
  • [2024/01] TapMo is accepted by ICLR 2024, congrats to Jiaxu Zhang
  • [2023/12] We introduce Paint3D, a lighting-less texture diffusion model.
  • [2023/12] We introduce AppAgent, a multimodal agent for smartphone apps.
  • [2023/11] We introduce ShapeGPT, a multimodal LLM for 3D shape generation.
  • [2023/11] We introduce LL3DA and M3DBench, a multimodal-language 3D assistant and benchmark.
  • [2023/09] 3 papers accepted at NeurIPS 2023
  • [2023/07] 1 paper accepted at ICCV 2023
  • [2023/02] 2 papers accepted at CVPR 2023
  • [2022/02] Join Tencent as a Research Scientist
  • [2022/01] Receive my Ph.D. degree
  • [2021/10] TightCap will be presented at ACM SIGGRAPH 2022

Publications

.js-id-selected
Selected All LMM 3D Human Motion 2024 2023 2022 ...
MeshXL: Neural Coordinate Field for Generative 3D Foundation Models
Sijin Chen, Xin Chen, Anqi Pang, Xianfang Zeng, Wei Cheng, Yijun Fu, Fukun Yin, Yanru Wang, Zhibin Wang, Chi Zhang, Jingyi Yu, Gang Yu, Bin Fu, Tao Chen
NeurIPS 2024 Jun 1, 2024
Paper GitHub Arxiv Project Cite GitHub stars
Paint3D: Paint Anything 3D with Lighting-less Texture Diffusion Models
Xianfang Zeng, Xin Chen, Zhongqi Qi, Wen Liu, Zibo Zhao, Zhibin Wang, Bin Fu, Yong Liu, Gang Yu
CVPR 2024 Apr 6, 2024
Paper GitHub Arxiv Project Cite GitHub stars
MotionChain: Conversational Motion Controllers via Multimodal Prompts
Arxiv 2024 Mar 28, 2024
Paper GitHub Arxiv Project Cite
MotionChain: Conversational Motion Controllers via Multimodal Prompts
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, Tao Chen
CVPR 2024 Jan 3, 2024
Paper GitHub Arxiv Project Cite GitHub stars
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
AppAgent: Multimodal Agents as Smartphone Users
Arxiv 2024 Jan 1, 2024
Paper GitHub Arxiv Project Cite GitHub stars
Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation
Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, Shenghua Gao
NeurIPS 2023 May 1, 2023
Paper GitHub Arxiv Project Cite GitHub stars
MotionGPT: Human Motion as a Foreign Language
NeurIPS 2023 May 1, 2023
Paper GitHub Arxiv Project Cite GitHub stars
Executing your Commands via Motion Diffusion in Latent Space
CVPR 2023 Jan 3, 2023
Paper GitHub Arxiv Project Cite GitHub stars
Executing your Commands via Motion Diffusion in Latent Space
A Large-Scale Outdoor Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene Reconstruction
Chongshan Lu, Fukun Yin, Xin Chen, Tao Chen, Gang Yu, Jiayuan Fan
ICCV 2023 Jan 1, 2023
Paper GitHub Arxiv Project Cite
End-to-End 3D Dense Captioning with Vote2Cap-DETR
Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Tao Chen, Gang Yu
CVPR 2023 Jan 1, 2023
Paper GitHub Paper Code Cite
End-to-End 3D Dense Captioning with Vote2Cap-DETR
TightCap: 3D Human Shape Capture with Clothing Tightness Field
Xin Chen, Anqi Pang, Wei Yang, Peihao Wang, Lan Xu, Jingyi Yu
SIGGRAPH 2022, TOG Journal Track Apr 1, 2022
Paper GitHub Arxiv Project Cite GitHub stars
Anisotropic Fourier Features for Neural Image-Based Rendering and Relighting
Huangjie Yu, Anpei Chen, Xin Chen, Lan Xu, Ziyu Shao, Jingyi Yu
AAAI 2022 Oral Feb 1, 2022
Paper Project Cite
Anisotropic Fourier Features for Neural Image-Based Rendering and Relighting
Neural Free-Viewpoint Performance Rendering under Complex Human-object Interactions
Guoxing Sun, Xin Chen, Yizhang Chen, Anqi Pang, Pei Lin, Yuheng Jiang, Lan Xu, Jingya Wang, Jingyi Yu
ACMMM 2021 Aug 1, 2021
Paper Arxiv Cite
Few-shot Neural Human Performance Rendering from Sparse RGBD Videos
Anqi Pang, Xin Chen, Haimin Luo, Mingye Wu, Jingyi Yu, Lan Xu
IJCAI 2021 Jul 1, 2021
Paper Arxiv Cite
ChallenCap: Monocular 3D Capture of Challenging Human Performances using Multi-Modal References
Yannan He, Anqi Pang, Xin Chen, Han Liang, Yuexin Ma, Lan Xu
CVPR 2021 Oral Jun 1, 2021
Paper Video Arxiv Cite
Error: URL does not contain a valid GitHub repository.
SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos
IJCV 2021 Jan 1, 2021
Paper GitHub Arxiv Project Cite
Sparse Photometric 3D Face Reconstruction Guided by Morphable Models
Xuan Cao, Zhang Chen, Anpei Chen, Xin Chen, Shiying Li, Jingyi Yu
CVPR 2018 Jun 1, 2018
Paper Arxiv Cite
AutoSweep: Recovering 3D Editable Objects from a Single Photograph
TVCG 2018 Jan 1, 2018
Paper GitHub Arxiv Project Cite GitHub stars

Posts

Waiting for my update
Welcome 👋 …
Apr 15, 2024 3 min read Demo, 教程
Waiting for my update

Invited Talks

Human Foundation Model and Agent
Welcome. A human-like agent can not only provide human motion from motion descriptions, but also should have the ability to think out human behavior based on inputs of vision and goals…
Jun 4, 2024 miHoYo, Shanghai, China 米哈游技术分享讲座 2024
Human Foundation Model and Agent
The Next Motion Generation | 下一代人体动作生成
Welcome to The Next Motion Generation. By discretizing all text, images, and motion into tokens, could we develop the human foundation agent leveraging the fundamental strengths of VLMs…
May 26, 2024 Fudan University, Shanghai, China 复旦人工智能前沿讲座 2024
The Next Motion Generation | 下一代人体动作生成
Human Motion Generation | AIGC 虚拟人动作生成技术的发展与应用
Welcome to the talk of Human Motion Generation, presented by Xin Chen to unfold the advancement of Motion Generation, including MotionGPT and Motion-Latent-Diffusion.
Jun 30, 2023 Bytedance, Beijing, China 稀土开发者大会 2023
Follow
Human Motion Generation | AIGC 虚拟人动作生成技术的发展与应用

Experiences

 
 
 
 
 
ByteDance | 字节跳动
Research Scientist
ByteDance | 字节跳动
July 2024 – Present San Jose, Bay Area
Research Scientist at Intelligent Creation Lab, focusing on Multi-modal Human Foundation Models and Agents.
 
 
 
 
 
Tencent | 腾讯
Research Scientist
Tencent | 腾讯
February 2022 – June 2024 Shanghai
Research Scientist at QQ Image Lab, received two Outstanding performances and Tencent STAR Award 2023.
 
 
 
 
 
Tencent | 腾讯
Research Scientist Intern
Tencent | 腾讯
November 2020 – May 2021 Shanghai
Research Scientist Intern at Tencent YouTuLab in 2020, focusing on the 3D human body reconstruction.
 
 
 
 
 
DGene Digital Technology Inc. | 叠境数字
Research Engineer Intern
DGene Digital Technology Inc. | 叠境数字
July 2018 – December 2019 Shanghai
Research Engineer Intern at DGene Digital Technology Inc., my supervisor’s start-up, for world digitalization. I won the Best Outstanding Intern Award in 2018 for leading the mobile virtual fitting project.
 
 
 
 
 
Deerev | 飞路
Software Engineer Summer Intern
Deerev | 飞路
June 2014 – September 2014 Shanghai
Software Engineer Intern in the Mobile App design Group, where I worked on system design and software testing.

Contact