Curiosity Cabinet Lab

Verified project card

# Curiosity Cabinet

Curiosity Cabinet places an intrinsically motivated agent in three rooms:

- an already simple, predictable process;
- a nonlinear but learnable process;
- an irreducibly random "noisy TV."

Every room has its own tiny neural world model. One agent chooses the room with the
largest recent prediction error; another chooses the room producing the greatest
recent reduction in error. Repeated seeded trials measure where each agent spends
its experience and how accurately it ultimately models the learnable process.

The historical anchor is Schmidhuber's 1991
[A Possibility for Implementing Curiosity and Boredom in Model-Building Neural
Controllers](https://mediatum.ub.tum.de/doc/814958/document.pdf), which proposed
reinforcement for actions that increase a model network's knowledge. This project
is a modern noisy-TV stress test of related intrinsic rewards, not an exact
reproduction of the original controller or environment.

## Verified results

Each reward ran for 1,200 steps under 60 independent seeds:

| Intrinsic reward | Simple room | Learnable room | Noisy TV | Learnable-model MSE |
| --- | ---: | ---: | ---: | ---: |
| Prediction error | 5.01% | 5.16% | 89.83% | 0.1752 |
| Learning progress | 35.03% | 33.03% | 31.94% | 0.1605 |

Fractions cover the middle half of training, after initialization and before the
final window. Prediction error reliably preferred the process that remained
unpredictable. Learning progress reduced noisy-TV attention by 57.89 points and
kept exploration roughly balanced. It did not learn a perfect room-selection
policy, and its downstream world-model improvement was modest.

## Reproduce

```powershell
uv run python projects/curiosity-cabinet/train.py
```

Evaluation snapshot

{
  "benchmark": "Curiosity Cabinet noisy-TV stress test",
  "runs_per_reward": 60,
  "steps_per_run": 1200,
  "summary": {
    "prediction_error": {
      "runs": 60,
      "mean_middle_action_fraction": {
        "simple": 0.05011111111111111,
        "learnable": 0.051611111111111115,
        "noisy_tv": 0.8982777777777778
      },
      "mean_final_action_fraction": {
        "simple": 0.049777777777777775,
        "learnable": 0.04994444444444445,
        "noisy_tv": 0.9002777777777776
      },
      "learnable_world_model_mse": {
        "mean": 0.1751612327992916,
        "median": 0.166976660490036
      }
    },
    "learning_progress": {
      "runs": 60,
      "mean_middle_action_fraction": {
        "simple": 0.3503333333333334,
        "learnable": 0.33025,
        "noisy_tv": 0.3194166666666666
      },
      "mean_final_action_fraction": {
        "simple": 0.3388333333333334,
        "learnable": 0.34405555555555556,
        "noisy_tv": 0.3171111111111111
      },
      "learnable_world_model_mse": {
        "mean": 0.1604974603901307,
        "median": 0.15731102228164673
      }
    }
  }
}

Backed-up artifact tree

• README.md
• __pycache__/app.cpython-311.pyc
• __pycache__/experiment.cpython-311.pyc
• app.py
• artifacts/curiosity-cabinet/evaluation.json
• artifacts/curiosity-cabinet/learning_progress_trajectory.npz
• artifacts/curiosity-cabinet/learning_progress_world_model.safetensors
• artifacts/curiosity-cabinet/prediction_error_trajectory.npz
• artifacts/curiosity-cabinet/prediction_error_world_model.safetensors
• data/seeded_results.parquet
• experiment.py
• requirements.txt
• train.py