Close

Sahand
Rezaei-Shoshtari

PhD Student

Download Resume

About Me

I am a PhD student in Computer Science at McGill University and Mila co-supervised by Prof. David Meger and Prof. Doina Precup. I am also working with Prof. Prakash Panangaden.


My research focus is on temporal and state abstraction in reinforcement learning, particularly for continuous control problems. During my MSc, I worked on data-efficient methods for learning robot dynamics for control and interaction inference.

Experience

Microsoft Research, Amsterdam, Netherlands

Research Intern

AI4Science

Samsung AI Center, Montreal, Canada

Research Intern

Meta imitation learning and meta reinforcement learning for continuous control and robotics

Samsung AI Center, Montreal, Canada

Research Intern

Multimodal generative modeling for learning intuitive physics using the senses of touch and vision

Ubisoft La Forge, Montreal, Canada

AI Programmer

Deep reinforcement learning for automated video game testing

Samsung AI Center, Montreal, Canada

Research Intern

Object detection neural networks for human hand-wave motion detection

Publications

Conferences and Journals

Policy Gradient Methods in the Presence of Symmetries and State Abstractions

Prakash Panangaden*, Sahand Rezaei-Shoshtari*, Rosie Zhao*, David Meger, and Doina Precup. Pre-print. 2023.

Cite Paper Code Env Code

Hypernetworks for Zero-shot Transfer in Reinforcement Learning

Sahand Rezaei-Shoshtari, Charlotte Morissette, Francois R. Hogan, Gregory Dudek, and David Meger. In Thirty-Seventh AAAI Conference on Artificial Intelligence. 2023.

Cite Paper Code Env Code Webpage

Continuous MDP Homomorphisms and Homomorphic Policy Gradient

Sahand Rezaei-Shoshtari, Rosie Zhao, Prakash Panangaden, David Meger, and Doina Precup. In Advances in Neural Information Processing Systems (NeurIPS). 2022.

Cite Paper Code Webpage

Learning Intuitive Physics with Multimodal Generative Models

Sahand Rezaei-Shoshtari, Francois R. Hogan, Michael Jenkin, David Meger, and Gregory Dudek. In Thirty-Fifth AAAI Conference on Artificial Intelligence. 2021.

Cite Paper Code Webpage

Seeing Through your Skin: Recognizing Objects with a Novel Visuotactile Sensor

Francois R. Hogan, Michael Jenkin, Sahand Rezaei-Shoshtari, Yogesh Girdhar, David Meger, and Gregory Dudek. In The IEEE Winter Conference on Applications of Computer Vision (WACV). 2021.

Cite Paper

Learning the Latent Space of Robot Dynamics for Cutting Interaction Inference

Sahand Rezaei-Shoshtari, David Meger, and Inna Sharf. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2020.

Cite Paper Dataset Video

Cascaded Gaussian Processes for Data-efficient Robot Dynamics Learning

Sahand Rezaei-Shoshtari, David Meger, and Inna Sharf. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2019.

Cite Paper Video

Workshops

Jacobian of Conditional Generative Models for Sensitivity Analysis of Photovoltaic Device Processes

Maryam Molamohammadi, Sahand Rezaei-Shoshtari, Nathaniel Quitoriano. In Machine Learning for Engineering Workshop @ NeurIPS 2020.

Cite Paper

Education

McGill University, Montreal, Canada

Sep. 2020 - Present

PhD in Computer Science

McGill University, Montreal, Canada

Sep. 2017 - Dec. 2019

Master of Engineering (Thesis) in Mechanical Engineering

Thesis: Learning Manipulator Dynamics for Control and Interaction Inference

University of Tehran, Tehran, Iran

Sep. 2012 - Sep. 2016

Bachelor of Engineering in Mechanical Engineering

Selected Awards

NSERC Canada Graduate Scholarship-Doctoral (CGS-D) Award.
Total amount of $105,000 over 3 years.

Fonds de Recherche du Quebec - Nature et Technologies (FRQ-NT) Award.
Total amount of $70,000 over 3.5 years.

NeurIPS 2022 Outstanding Reviewer.
Top 8% of all reviewers.

ICML 2022 Outstanding Reviewer.
Top 10% of all reviewers.

Selected Projects

Contextual Control Suite

Contextual MDPs for Continuous Control Problems:

  • Based on DeepMind Control Suite.
  • The MDP context characterizes the dynamics and reward settings.

    • Code

    Gym Forest Fire

    Simulation of a Forest Fire Environment for Reinforcement Learning:

  • Fully-vectorized OpenAI Gym forest fire simulation based on cellular automaton for tackling wildfires with RL.

    • Code

    Motion Planning and Control Utilities for Kinova Jaco 2 Arm

    ROS package for Kinova Jaco 2 implementing:

  • Impedance control, feedforward torque control, and velocity control utilities.
  • Motion planning utilities for joint space and Cartesian space planning.

    • Code

    Certifications

    ANITI Reinforcement Learning Virtual School (RLVS) 2021
    Hosted by ANITI. Virtual. March 2021.

    Simons Institute Mathematics of Online Decision Making Workshop
    Hosted by Simons Institute. Virtual. October 2020.

    CIFAR Deep Learning and Reinforcement Learning Summer School
    Hosted by CIFAR and Amii in Edmonton, Canada. July 2019.

    Skills

    Get in Touch