We study the statistics of the 3D color space of natural images and develop a model for efficiently representing this structure. Analysis of a dataset of 503 calibrated natural images reveals striking non-Gaussian structure with heavy tails in distinct directions that are asymmetrically distributed in 3D color space. An overcomplete, non-negative sparse coding model adapted to this structure recovers a basis aligned with the directions of black, white, red, green, blue, and yellow - i.e., the 'unique hues' that are proposed to constitute a psychological basis for color appearance. Moreover, the nonlinear nature of inference in the sparse coding model yields both excitatory and inhibitory interactions among latent variables; the former facilitates combining adjacent pairs of unique hues to encode intermediate hues situated between them, while the latter enforces mutual exclusivity between opposite pairs of unique hues. Together, these findings mathematically formalize and support previous intuitive reasoning that the unique hues are well-suited as a basis for describing color appearance -- that is, they efficiently code unique directions in color space which arise from the statistics of the natural visual environment.