Zetian Wu CV

PhD Candidate at Oregon State · Open to AI research roles

Building AI systems that connect language, sign, and 3D human motion.

I am an AI research engineer working at the intersection of large language models, multilingual NLP, and multimodal generation. My current research develops direct and simultaneous translation between sign languages and geometry-aware text-to-sign generation.

PhD candidate in Computer Science & Artificial Intelligence at Oregon State University, advised by Prof. Liang Huang and collaborating with Prof. Stefan Lee.

Explore selected research Download CV

Selected research

From spoken language to expressive motion

Three complementary systems spanning direct, simultaneous, and geometry-aware sign-language generation.

01 Submitted to NeurIPS 2026 · Preprint

Direct Translation between Sign Languages

Direct ASL/CSL/DGS translation without a spoken-text pivot at inference, replacing a three-stage cascade.

mBARTVQ-VAETranslateGemma-4BBack-translationQwen3-8B

Paper / arXiv

Problem
Remove text-pivot information loss, error propagation, and three-pass latency from sign-to-sign translation.
Techniques
Created 112,324 cross-lingual pairs from How2Sign, CSL-Daily, and PHOENIX-2014T; jointly trained text-to-sign and sign-to-sign tasks with a frozen motion tokenizer.
Results
On the held-out BT-input test set, reduced mean DTW-PA-MPJPE by 19%, improved back-translation BLEU-4 by 50%, and achieved 2.3× faster inference than the cascade.
02 Submitted to EMNLP 2026

Simultaneous Translation between Sign Languages

Problem. Generate target-sign motion before the complete source utterance is available for streaming ASL/CSL/DGS translation.

Techniques. Combined mBART-large-cc25 with a frozen VQ-VAE over SMPL-X poses; implemented prefix-to-prefix training, test-time wait-k decoding, stochastic multi-path supervision, and CA-Stream-AL.

Results. Reduced computation-aware latency by 38% versus full-sentence decoding, with a measured trade-off of 9% higher DTW-PA-MPJPE and 2.1-point lower BLEU-4.

Streaming decodingSMPL-XWait-kMulti-path training
03 arXiv preprint

Geometry-Aware Text-to-Sign Generation

Problem. Generate variable-length 3D sign-pose sequences from text while preserving hand articulation and skeletal geometry.

Techniques. Used XLM-R with OpenPose, 2D-to-3D lifting, inverse kinematics, skeleton normalization, parent-relative weighting, bone-aware losses, and explicit EOS prediction.

Results. Reached 14.22 back-translation BLEU-4 on PHOENIX14T dev (+3.08 points over XLM-R) and reduced bone-length and movement-variance discrepancies by 18.76% and 5.48%.

XLM-ROpenPoseInverse kinematics3D pose

Paper / arXiv

Industry experience

Research that moves between ideas and systems

Experience in LLM reasoning, long-context multimodal generation, multilingual representation learning, and generative sequence modeling.

2024–2025

Genies

Research Scientist Intern

Built an LLM-powered user-insight application over profiles, app logs, and chat histories, with privacy-aware data normalization for behavioral analytics.

Explored GRPO post-training and MCTS-guided reasoning for efficient text-to-SQL generation.

2023

Amazon

Applied Scientist Intern

Researched Transformer-based generative models for long multimodal sequences, exploring architecture and training designs for cross-modal dependencies and long-context coherence.

2022

Amazon

Applied Scientist Intern

Developed FMDA on frozen XLM-R-base using language-identification, reconstruction, and contrastive objectives to disentangle language-specific, template-semantic, and residual-semantic representations.

Outperformed XLM-R and a strong language-disentanglement baseline on MASSIVE retrieval across EN–DE, EN–ES, and EN–JA.

2019–2020

EnjoyMusic

Machine Learning Engineer

Built Transformer-XL/discriminator models for music style transfer and a VAE for drum-conditioned MIDI generation; ported TypeScript MIDI tooling to Python for product integration.

Publications

Selected papers

Work across sign-language generation, multilingual representation learning, multimodal benchmarks, and interpretable NLP.

Technical toolkit

Models, methods, and systems

A research stack spanning modern language models, multimodal generation, rigorous evaluation, and production-minded implementation.

01

Foundation & language models

LLMs, Transformers, mBART, XLM-R, Hugging Face, machine translation, multilingual NLP, text-to-SQL

02

Multimodal & generative

VQ-VAE, VAE, multimodal fusion, sign-language translation, 3D human pose and motion, OpenPose, SMPL-X

03

Training & evaluation

Back-translation, synthetic data, contrastive learning, GRPO, MCTS, streaming decoding, model evaluation, SHAP

04

Engineering

Python, C/C++, SQL, PyTorch, scikit-learn, CUDA, multi-GPU training, Linux, Git, Bash, TypeScript

Education

2022–Present

Oregon State University

PhD, Computer Science & Artificial Intelligence · Outstanding Scholar Program Fellow

2020–2021

Johns Hopkins University

MSE, Data Science

2015–2019

Zhejiang University

BS, Physics · Minor in Finance