Antonio Guillen-Perez, Ph.D.

Recent News & Highlights

• [Oct 2026] Received the 2026 IEEE Best Land Transportation Paper Award from the IEEE Vehicular Technology Society for "Multi-Agent Deep Reinforcement Learning to Manage Connected Autonomous Vehicles at Tomorrow's Intersections", published in IEEE Transactions on Vehicular Technology (Vol. 71, No. 7, pp. 7033–7043, July 2022).
[Paper]
• [Apr 2026] Joined Uber AV Labs as a Senior Machine Learning Engineer to work on Foundation Models and long-tail data curation.
• [Feb 2026] Released "Deep-Flow", a novel unsupervised framework for AV safety validation. It leverages Optimal Transport Conditional Flow Matching on a Spectral Manifold to detect "long-tail" semantic anomalies with an AUC-ROC of 0.77. [Paper] [GitHub]
• [Apr 2026] Accepted by TMLR: "Semantic-Drive" — Trustworthy and Efficient Long-Tail Data Curation via Open-Vocabulary Grounding and Neuro-Symbolic VLM Consensus. [OpenReview] [Paper] [GitHub] [Live Demo]
• [Sept 2025] Our two new benchmarks for validating complex AI systems, DCcluster-Opt (SustainCluster) and LC-Opt (SustainLC), accepted to the NeurIPS 2025 Datasets and Benchmarks Track.
• [Aug 2025] New Research Stream: Advancing Autonomous Driving AI.
  • Developed "Efficient Virtuoso," a state-of-the-art latent diffusion Transformer for goal-conditioned trajectory planning, achieving minADE of 0.25 on Waymo Open Motion Dataset. [Paper] [GitHub]
  • Submitted new research on "Mining the Long Tail" for robust Offline RL in AVs to arXiv, demonstrating significant safety improvements via data curation. [Paper] [GitHub]
  • Submitted new research on "From Imitation to Optimization" for Offline RL in AVs to arXiv, demonstrating 3.2x higher success rates than BC baselines. [Paper] [GitHub]
• [Early 2025] Initiated a new research stream at HPE exploring LLM-based agents for dynamic system control.
• [Nov 2024] Our work on Hierarchical RL, GreenDCC, accepted to the AAAI 2025 Demonstration Track.
• [Oct 2024] Our MARL benchmark, SustainDC, accepted to the NeurIPS 2024 Datasets and Benchmarks Track.
• [May 2024] Filed a new U.S. patent on "Real-Time Carbon Footprint Reduction Controller".
• [Dec 2023] Received the Best ML Innovation Award at the NeurIPS 2023 Climate Change AI Workshop.
• [Sep 2022] Joined HPE AI Labs as an AI Research/Applied Scientist.
• [Jun 2022] Awarded Ph.D. in Computer Science with Cum Laude distinction.

About Me

I am a Senior Machine Learning Engineer at Uber AV Labs. I focus on building the Foundation Models and intelligent systems needed for safe, large-scale autonomy. My work combines deep research with production-grade engineering:

• Generative AI & Safety: My research centers on Generative World Models and Flow Matching to solve the "long-tail" data problem. I use my Ph.D. background in Reinforcement Learning and Imitation Learning to create robust decision-making systems for complex driving environments.
• Large-Scale Engineering: I design and implement the infrastructure required to process petabyte-scale vehicle logs. I am an expert in building distributed training pipelines using PyTorch and Ray to turn raw sensor data into high-quality training sets.
• Neuro-Symbolic VLMs: I specialize in using Vision-Language Models (VLMs) to automate data curation and safety validation. By combining symbolic grounding with cognitive reasoning, I build tools that help autonomous systems understand and navigate rare real-world scenarios.

Featured Projects

Manifold-Aware Flow Matching: Detecting anomalies via exact likelihood estimation.

Latent Flow Dynamics (Click to expand)

Deep-Flow: Unsupervised Safety Validation via Conditional Flow Matching

Built a physics-informed generative framework to detect "unknown unknown" safety anomalies in L4 autonomous driving. Replaces brittle heuristics with Exact Log-Likelihood estimation using Optimal Transport Conditional Flow Matching (OT-CFM) on a spectral manifold.

My Key Contributions:

• Engineered a Spectral Manifold Bottleneck (PCA-12) to enforce kinematic smoothness and enable deterministic, exact Jacobian trace computation (avoiding stochastic estimators).
• Designed a Goal-Conditioned Early Fusion Transformer with direct intent skip-connections to resolve multi-modal ambiguity at complex intersections.
• Implemented a Parallel Eager Loader to process 300k Waymo scenarios in-memory, maximizing A100/3090 GPU throughput.
• Identified the "Predictability Gap": Proved that semantic violations (e.g., illegal turns) can be detected as statistical outliers even without high kinematic energy.

[Paper] [GitHub]

Neuro-Symbolic "Judge-Scout" Pipeline running locally on RTX 3090.

System Architecture (Click to expand)

Semantic-Drive: Neuro-Symbolic VLM Mining for "Dark Data"

TMLR Accepted

Accepted by TMLR, this privacy-preserving, local-first DataOps engine mines long-tail edge cases (e.g., erratic VRUs, sensor failures) from raw video logs. It implements a "System 2" inference strategy using YOLOE-11 for grounding and Qwen/Mistral for reasoning.

My Key Contributions:

• Designed a Neuro-Symbolic architecture that reduces VLM hallucinations by 51% via "Skepticism Policy" prompting.
• Implemented an Explicit Outcome Reward Model for inference-time consensus (Best-of-N search).
• Operationalized the WOD-E2E taxonomy into a structured JSON schema for automated causal analysis.
• Released a full open-source stack (Streamlit UI) that runs on consumer hardware.

[Paper] [Code] [Live Demo] [Dataset]

Our Sparse Route model (right) generates highly unbiased precise trajectories, outperforming other goal representations.

Qualitative comparison of goal representations in a challenging turning scenario. (Click to expand)

Efficient Virtuoso: Latent Diffusion Transformer for Trajectory Planning

Developed a state-of-the-art conditional latent diffusion model for goal-conditioned trajectory planning, achieving a \textbf{minADE of 0.25} on Waymo Open Motion Dataset. Introduced novel normalization and provided key insights into optimal goal representation for AVs.

My Key Contributions:

• Pioneered a two-stage normalization pipeline for stable latent diffusion training.
• Designed a Transformer-based StateEncoder for rich scene context fusion.
• Conducted rigorous ablation on goal representation, proving multi-step routes are critical for tactical precision.
• Achieved state-of-the-art minADE of 0.25 on WOMD.

[Paper] [GitHub]

Baseline CQL: Collision in merge scenario.

Heuristic-Weighted: Suboptimal, reactive merge.

Uncertainty-Weighted: Proactive, successful merge.

Qualitative comparison of data curation strategies in a challenging highway merge.

Mining the Long Tail: Data Curation for Robust Offline RL in AVs

Systematically investigated six data curation strategies (heuristic, uncertainty, behavior-based) to tackle the long-tail problem in autonomous driving, achieving nearly a three-fold reduction in collision rate with uncertainty-based methods.

My Key Contributions:

• Developed novel, data-driven criticality metrics (e.g., model disagreement via ensemble scouts) for non-uniform data sampling.
• Designed specialized PyTorch `Dataset` implementations for timestep and scenario-level weighting.
• Conducted large-scale comparative study demonstrating all curation methods significantly outperform uniform sampling.

[Paper] [GitHub]

BC-S (MLP): Fails due to complex interactions.

BC-T (Transformer): Brittle, leads to "circling" failure.

CQL (Offline RL): Robust recovery, successfully navigates.

Qualitative comparison of Behavioral Cloning baselines vs. robust Offline RL (CQL) agent. (Click to expand)

From Imitation to Optimization: Offline Learning for Autonomous Driving

Pioneered an end-to-end pipeline applying state-of-the-art Offline Reinforcement Learning (CQL) to the Waymo Open Motion Dataset, demonstrating significantly superior robustness over Behavioral Cloning baselines for long-horizon AV control.

My Key Contributions:

• Engineered a robust, parallelized data processing pipeline for the Waymo Open Motion Dataset.
• Conducted rigorous comparative study, demonstrating CQL's 3.2x higher success rate and 7.4x lower collision rate over Transformer-based BC.
• Designed an effective multi-objective reward function for Offline RL training in autonomous driving.

[Paper] [GitHub]

MARL agents coordinating to cross an intersection safely.

Autonomous Intersection Management

Designed and implemented a MARL system where autonomous vehicles learn to coordinate and safely cross intersections without traffic lights, significantly improving traffic flow.

My Key Contributions:

• Designed the end-to-end MARL system architecture, including a novel LSTM-based state encoder.
• Engineered the multi-objective reward function to balance efficiency and safety.
• The final system reduced vehicle travel time by up to 59% in simulation.

[Paper] [GitHub]

System architecture for the SustainDC benchmark.

SustainDC: A NeurIPS Benchmark

Co-led the creation of an open-source, Gym-compatible benchmark for developing MARL controllers to optimize the energy and carbon footprint of data centers.

My Key Contributions:

• Co-led the architectural design and open-source implementation.
• Engineered the Python-based physics models for cooling and power.

[NeurIPS Paper] [GitHub]

Conceptual framework for Learning from Oracle Demonstrations.

Learning from Oracle Demonstrations (LfOD)

Developed a novel Imitation Learning paradigm to accelerate DRL training by using a learned "Oracle" agent to provide corrective demonstrations to the primary agent.

My Key Contributions:

• Engineered the core LfOD methodology from first principles.
• Implemented the TD3fOD algorithm to integrate oracle advice.
• Demonstrated a 5x speedup in training convergence on complex tasks.

[Paper]

Animation showing the CNN prediction.

3D CNN Surrogate for Accelerating Physics Simulations

Developed a 3D CNN (U-Net) to act as a fast proxy for computationally expensive CFD simulations to predict 3D heat distribution in data centers, achieving a >2800x inference speedup over the original simulator.

My Key Contributions:

• Evaluated 3D U-Net architectures for spatial heat prediction in data centers.
• Engineered the data pipeline to process and voxelize raw CFD simulation data.
• Used the surrogate model to optimize the workload placement using a genetic algorithm, reducing the maximum temperature by 7.7% and the energy consumption by 2.5%.

[NeurIPS Workshop Paper]

Core Technical Skills

Selected Publications

For a full list of publications, please visit my Google Scholar profile.