Even young children seem to have an early understanding of the world around them, and the people in it. Before children can reliably say "ball", "wall", or "Saul", they expect balls to not go through walls, and for Saul to go right for a ball (if there's no wall). What is the formal conceptual structure underlying this commonsense reasoning about objects and agents? I will raise several possibilities for models underlying core intuitive physics as a way of talking about models of core knowledge and intuitive theories more generally. In particular, I will present some recent ML work trying to capture early expectations about object solidly, cohesion, and permanence, that relies on a rough-derendering approach.