Projects

Problem

Multi-objective reinforcement learning tasks need several reward components weighted against each other. Fixed weights, set once before training, tend to produce unstable or suboptimal policies as training progresses.

What I built

• A framework that dynamically re-weights reward components during training instead of fixing them upfront
• PyTorch implementation, tracked and compared via Weights & Biases
• Evaluated across Gymnasium/MuJoCo, MO-Gymnasium, and the team's own rSoccer-gym robot-soccer simulator

Result

Peer-reviewed and published at AAMAS 2025, a top-tier venue for multi-agent systems.

Stack

Problem

RoboCup Small Size League robots need path/trajectory planning that is fast when the field is clear but still safely avoids static and moving obstacles when it isn't.

What I built

• An RL-based path planner, trained in the team's rSoccer-gym simulator and tracked with Weights & Biases
• Benchmarked against baseline planning algorithms across obstacle-free and single/moving-obstacle scenarios
• Carried from simulation into on-robot tests (the repo ships a dedicated real_tests/ suite)

Result

60% time gain over baseline algorithms in obstacle-free environments, with dynamic-obstacle avoidance demonstrated against moving obstacles.

Stack

Problem

Training and validating RL policies on a real 6-DoF industrial arm needs an accurate, fast simulator before anything touches the physical hardware.

What I built

• A Gymnasium-compatible MuJoCo simulation of the DENSO VS050 arm fitted with a Robotiq 2F-85 gripper
• Reach-pose and pick-and-place task environments, with modeled kinematics, joint limits, and reward functions
• A companion project (vs050-mjlab) for large-scale, batched-simulation RL training

Stack

Problem

Started as a way to pull a gas plume out of thermal video, frame by frame, without rewriting code every time the detector or preprocessing chain needed to change.

What I built

• A source → stages → sinks pipeline architecture, fully described in YAML — swapping a detector or background model is a config edit, not a code edit
• Structure, texture, and motion detection stage families, built on OpenCV/NumPy
• An optional PyQt6 GUI editor with a live preview of every stage and sink, plus packaged AppImage/dmg installers

Stack