I'm a fifth year grad student at UMass Amherst working in the Resource-Bounded Reasoning Lab under Dr. Shlomo Zilberstein. My research focuses on problems in automated decision making. I work on metareasoning techniques that enable autonomous systems to optimize their own process of deliberation. I also work on computational methods that allow autonomous systems to resolve exceptions that can occur during normal operation. Before grad school, I worked as a software developer on Wall Street. Outside of all that, I'm interested in many problems in philosophy, especially free will, mind, metaphysics, and epistemology.

Education

University of Massachusetts Amherst
MS in Computer Science May 2019
PhD in Computer Science Expected September 2021
Dr. Shlomo Zilberstein Resource-Bounded Reasoning Lab Automated Decision Making Research
Marist College
BS in Computer Science and Philosophy with a Mathematics Minor May 2014
4.0 GPA Valedictorian Summa Cumme Laude

News

Dec 2020 AAAI accepted our paper that proposes ethically compliant sequential decision making! July 2020 The AREA workshop at ECAI accepted our paper that improves competence in autonomous systems. July 2020 ECAI accepted my doctoral consortium paper that outlines my dissertation on metareasoning. June 2020 The AISafety workshop at IJCAI accepted our paper that proposes ethically compliant autonomous systems. June 2020 The DynamicSlam workshop at ICRA accepted our paper that introduces agent-aware state estimation. Jan 2020 ICRA accepted our paper that uses model-free metareasoning to determine when to stop anytime algorithms! Jan 2020 AAMAS accepted our paper that introduces competence-aware systems! Jan 2020 ECAI accepted our paper that integrates decision making and ethics in autonomous systems! June 2019 IROS accepted our paper that uses belief space metareasoning for exception recovery in autonomous systems! Jan 2019 I was a finalist for the Distinguished Teaching Award for teaching assistants. Dec 2018 I passed my qualifying exam with distinction! I'm a PhD candidate now. Nov 2018 I'm the primary inventor on a patent for self-driving cars! June 2018 The AEGAP workshop at IJCAI accepted our paper that introduces adaptive metareasoning. May 2018 The AI4IoT workshop at IJCAI accepted our paper that applies belief-space planning to information security. April 2018 IJCAI accepted our paper that uses model-based metareasoning to determine when to stop anytime algorithms! April 2018 My advisor and I received funding for our NSF grant on adaptive metareasoning! April 2018 I was awarded an NSF Graduate Research Fellowship! Check out the article. Feb 2018 I got an internship with Nissan this summer! I'll be working on their self-driving cars.

Timeline

2016 – Now UMass Amherst MS/PhD Student in Computer Science Summer 2018 Nissan Research Intern in Autonomous Driving 2015 – 2016 Goldman Sachs Software Developer in Big Data 2015 – 2016 Marist College Research Assistant in Distributed Graph Databases 2014 – 2015 Goldman Sachs Software Developer in Network Engineering 2013 – 2015 Marist College Software Developer in Stimulus-Response Testing 2013 – 2014 OmniTech Software Developer in Machine Learning 2012 – 2014 Goldman Sachs Software Developer Intern in Network Engineering 2010 – 2012 Marist College Web Developer at www.marist.edu Summer 2011 Marist College Teaching Assistant at the Summer Game Design Institute 2010 – 2014 Marist College Double Major in Computer Science and Philosophy with a minor in Mathematics

Research

Ethically Compliant Sequential Decision Making

Autonomous systems rarely operate with moral sensibility in the real world. In fact, the standard stategy just involves making many small tweaks to the autonomous system until ethical behavior is produced. For example, a self-driving car with an objective that encourages completing a route can be adjusted to discourage dangerous driving. However, these tweaks often lead to unpredictable behavior that doesn't reflect the values of its stakeholders. To address this problem, we offer an approach to autonomous systems that optimally completes a task while following an ethical framework.


Metareasoning for Exception Recovery

Autonomous systems use decision-making models that rely on simplifying assumptions to reduce the complexity of the real world. However, as a result of their assumptions, these systems can encounter many different exceptions. Our paper offers a new type of autonomous system, called an introspective autonomous system, that resolves exceptions by interleaving regular decision making with exception handling based on a belief over whether any exceptions have been encountered during normal operation.


Metareasoning for Planning

We develop meta-level control techniques to determine the point at which a robot should stop thinking and start doing. In dynamic environments, a robot rarely has enough time to determine the optimal solution to a decision-making problem. To handle this limitation, a robot can use an anytime algorithm, which is a type of algorithm that slowly improves a solution over time. So, as the robot thinks more and more, the solution slowly gets better and better. Now, here's the question: how long should the robot think for?


Competence Awareness

Due to the limitations of their decision-making models, autonomous systems cannot always complete a task successfully. Under certain conditions, they may even behave dangerously in their environments. Ideally, to overcome situations like this, autonomous systems should rely on human assistance if needed. Our recent work proposes a framework that enables autonomous systems to operate at varying levels of autonomy, each of which requires a different level of human involvement, by learning through experience.


Agent-Aware State Estimation

Autonomous systems often operate in environments where their behavior and the behavior of other agents is governed by a signal. However, if autonomous systems can't observe the signal directly, the consequences can be disastrous. For example, if a self-driving car can't observe a traffic light directly, it could drive into an intersection even when the traffic light is red. In response to this problem, we develop an approach to agent-aware state estimation where autonomous systems can exploit the behavior of other agents in the enviornment to observe the signal indirectly.


Distributed Graph Databases

We explore how to best place vertices of graph snapshots across workers in a distributed dynamic graph database to speed up queries like PageRank and average degree. By extending G*, our own distributed dynamic graph database, we examine how each query performs when the vertices of graph snapshots are placed on a single worker or spread across all of the workers. It turns out that vertex placement has a big impact on query performance.


Stimulus-Response Testing

We present OpenSR, an open source stimulus-response testing framework. In our framework, you can create, manage, and track an implicit association test (IAT). Basically, an IAT can help researchers determine whether or not someone has an implicit bias for or against something. Since most researchers currently have to buy complicated, expensive software to do this, we built a free, easy-to-use, customizable alternative that's better than what's already on the market.

Projects

Deep Jammer

We built Deep Jammer, a music generator that learns how to compose classical music by listening to 320 classical piano pieces. By using deep learning, specifically two LSTMs, it can learn the spatial and temporal patterns of classical music. In a survey of over 50 participants, the classical piano pieces composed by Deep Jammer scored a 7.5 rating compared to a piece by Bach that scored an 8.1 rating. Finally, to experiment with transfer learning, we trained Deep Jammer on just twenty jazz piano pieces. While it wasn't perfect, it quicky picked up on the rhythm and sound of jazz.


Logos

To learn more about automated decision-making, I built Logos, a library that can solve MDPs. Logos offers dynamic programming methods like value iteration and policy iteration and reinforcement learning algorithms like Monte Carlo learning and TD learning. I currently use Logos in semi-autonomous system simulations to control a self-driving car and a reinforcement learning agent that learns how to optimally play tic-tac-toe by playing games against itself.


Deep Rainbow

We built an optimal Rubik’s Cube solver, Deep Rainbow, that uses IDA*. We used an admissible heuristic based on three disjoint pattern databases where each database is associated with the set of edge cubees and two different sets of corner cubees. We based our solver on Richard Korf’s approach. In just a few minutes, Deep Rainbow can find optimal solutions to problems that are twelve steps or less away from the goal.


Shallow Blue

We built a chess agent named Shallow Blue that uses minimax with alpha-beta pruning. The evaluation function incorporates many important dimensions of chess, such as piece development, material balance, mobility, and attack range. Luckily, Shallow Blue won a chess tournament that had over fifteen teams that built their own chess agents. To add to the fun, I also built a Connect Four agent and a Tic-Tac-Toe agent using a similar method.


SvegOS

In an awesome class (CMPT 424) taught by Dr. Alan Labouseur, I built SvegOS, a browser-based operating system from scratch. Just like any Unix operating system, you can manage files (create, read, update, delete, and ls), load a program (load), manage processes (run, ps, and kill), and a lot more. To top it all off, the interface shows the CPU registers, the memory, the hard drive, and the state of every process, so you can follow my operating system as it runs.


Svegliator

In another great class (CMPT 432) taught by Dr. Alan Labouseur, I built Svegliator, a compiler that compiles a C-like language into 6502alan assembly language. The compiler shows the resulting machine code, the concrete syntax tree, the abstract syntax tree, and the symbol table. Just so you can trace exactly what the compiler is doing, it records each step of the scanner, the parser, the semantic analyzer, and the code generator during compilation.

Honors

Valedictorian Awarded to the top graduating student
National Science Foundation Graduate Research Fellowship Awarded to high-potential grad students early in their career
National Science Foundation Scholarship Included a full scholarship for every year of college
Distinguished Teaching Award Finalist Awarded to top teaching assistants
Excellence Award in Computer Science Awarded to the top graduating student in computer science
Excellence Award in Philosophy Awarded to the top graduating student in philosophy
Intern of the Year Awarded to a graduating student for excellence in industry
Summa Cum Laude Awarded to students with at least a 3.85 GPA
Victor Lessor Graduate Scholarship in Artificial Intelligence Awarded to a top incoming grad student in artificial intelligence
SIGAI Student Scholar Included a grant for the 50 Years of the ACM Turing Award Celebration
IJCAI Student Scholar Included a grant for the International Joint Conference on Artificial Intelligence
Deans’ Circle Admitted to an honors organization of only the top 3% of students
Deans’ List Awarded to students with at least a 3.6 GPA
Fiovranti Memorial Scholarship for Athletics Awarded for excellence in cross country and track and field
Marist Presidential Scholar Student Speaker Selected to give a talk to the top accepted prospective students