The human brain processes speech and melody differently, with the left hemisphere predominantly processing temporal features and the right hemisphere processing spectral features. This asymmetric processing raises questions about the underlying neural dynamics of acoustic feature encoding for speech and music. To further investigate this, we recorded intracranial EEG data from seventeen epileptic patients with implants in primary and associative auditory cortical regions. We utilized a stimulus set comprising one hundred a cappella songs, each presented in three versions: temporally degraded, spectrally degraded, and original. The study involved two phases: a binary choice task where participants identified whether pairs of song excerpts were identical or different, followed by a passive listening phase while watching a silent documentary. We applied multivariate pattern analysis and time-frequency analysis to train a classifier to distinguish between sentences and melodies while examining the encoding of temporal and spectral modulations in all patients. Behavioral results indicated a decrease in sentence recognition under temporally degraded conditions and a decrease in melody recognition under spectrally degraded conditions. Decoding accuracies corroborated these findings, showing that speech processing depends predominantly on temporal modulations, whereas melody processing relies on spectral modulations. These patterns persisted consistently across different times and channels, suggesting a spatiotemporal code within the auditory system. Additionally, frequency analysis across all patients highlighted the distinct roles of various frequency bands in encoding these temporal and spectral cues. In future research, we aim to explore the specific oscillatory networks in processing speech and melody.