Abstract:   Aristotle suggested that “all men by nature desire to know”, but it seems men are not alone in this. Curiosity, still characterized by at least some contemporary researchers as an intrinsic desire for knowledge, is evident in humans of all sorts from early infancy; it has also been said to appear in a wide range of other animals, including monkeys, birds, rats, and octopuses. One might wonder why or indeed whether such a broad range of animals have an intrinsic desire for knowledge. Even if there is much that octopuses must know in order to survive, one might wonder why they couldn’t learn enough through processes of trial and error driven by simple incentives such as hunger, as opposed to being equipped with some kind of drive to gain knowledge for its own sake. Recent advances in reinforcement learning cast new light on this issue. Drawing on this research, I argue that curiosity is vital, not for intelligent agents as such, but for agents with the restrictions of biological creatures in environments with the complexity of our natural world. I also aim to explain why the target of natural curiosity is knowledge, rather than information or uncertainty reduction.