Previous studies have used the reinforcement learning theory to explain how animals explore a task space to maximize reward. While recent works argued that uncertainty in valuation is the key variable to guide exploration, little is known about the role of another variable - the uncertainty in state-space representation. One reason is that a simple task design consisting of only a few states and actions cannot accommodate the uncertainty of the environmental structure. Here, we hypothesize that metacognition, the human's unique ability to introspect and estimate one's level of uncertainty, guides the efficient exploration of a large state-space. For this, we designed a novel two-stage decision-making task with infinitely-many choices and sparse rewards and collected a total of 101 subjects' data (88 behavioural and 13 fMRI). We examined two key variables guiding exploration: uncertainty about the environmental structure (state-space uncertainty; SU) and the reward structure (value uncertainty; VU). We found that both variables are significantly correlated with the individual metacognitive ability measured using an independent perception task. Interestingly, we also found that high metacognitive subjects outperformed the low metacognitive subject group (test phase performance; p<1e-10). In doing so, the former group relies on SU throughout learning, while the latter uses both SU and VU, suggesting that SU might be sufficient for metacognitive exploration. This finding is confirmed by the model comparison analysis with metacognitive exploration models that combine SU and VU in various ways. The preliminary fMRI analysis suggests that IPL, one of the regions previously known for metacognition, might be engaged in resolving SU. These results elucidate the role of metacognition in fostering a sample-efficient exploration strategy. Ultimately, our work may offer a new perspective on Tolman's latent learning: metacognition might be one of the key components to facilitate efficient learning of the latent environmental structure without value information.