NetHack Challenge

Logo

NeurIPS 2021 NetHack Challenge website

Welcome to the NetHack Challenge

The NetHack Challenge is a competition at NeurIPS 2021 in which teams will compete to build the best agents to play the game of NetHack, using the NetHack Learning Environment (NLE).

 

Join Competition Join Discord

 

What is NetHack?

NetHack is a ASCII-rendered single-player dungeon crawl game that is one of the oldest and most difficult computer games in history. NetHack is procedurally-generated, with hundreds of different entities and complex environment dynamics, presenting an extremely challenging environment for both current state-of-the-art RL agents and humans, while crucially being lightning-fast to simulate.

Check out the NetHack Wiki and nethack.org to learn more about NetHack, or head over to the Challenge Page for an intro.

What is the NetHack Learning Environment?

The NetHack Learning Environment (NLE) is a Reinforcement Learning environment presented at NeurIPS 2020. NLE is based on NetHack 3.6.6 and designed to provide a standard RL interface to the game, and comes with tasks that function as a first step to evaluate agents on this new environment. You can read more about NLE in the NeurIPS 2020 paper.

Example of an agent running on NLE

About the NetHack Challenge At NeurIPS 20201

The NetHack Challenge invites entrants, using any method they choose, to develop agents that can reliably either beat the full game of NetHack or achieve as high a score as possible. No restrictions are placed on how the agent is trained and contestants will use their own hardware. Evaluation will be performed in a controlled setting, thanks to our partner and co-organizer, AIcrowd who will run the competition through their platform.

We are excited that this competition offers machine learning students, researchers and NetHack-bot builders the opportunity to participate in a grand challenge in AI - without prohibitive computational costs — and we are eagerly looking forward to the wide variety of submissions.

Participation Information:

There will be three competition tracks:

  1. Best overall agent - awarded to the best-performing agent in the competition. All submitted agents qualify for this track.
  2. Best agent not using a neural network - awarded to the best-performing agent not using a neural network or significantly similar modeling technique.
  3. Best agent from an academic/independent team - awarded to the best-performing agent produced by a team predominantly led by non-industry-affiliated researchers.

The competition runs from early June through October 15, and the winners will be announced at NeurIPS in December. For more information, checkout out the Challenge Page, read our blog here. You can also follow us on Twitter, join our Discord server, or get acquainted with the NetHack Learning Environment.

Key Dates

June 9th Competition starts. The participants can submit their agents.
October 15th Submission closes. Testing phase starts.
December 13th/14th NeurIPS 2021 Competition Track. The winners are announced.

 

Partners

AIcrowd Logo