White matter is pivotal for efficient brain communication. Fractional Anisotropy (FA) from diffusion tensor imaging has been widely used to estimate the microstructural integrity of white matter. However, the extensive fibrous structure of the white matter, varying widely across individuals, necessitates an analytic approach capable of capturing both its macroscopic and microscopic structures effectively. We hypothesized that a deep neural network approach could learn multi-scale features of white matter in an end-to-end manner. We analyzed data from the Adolescent Brain Cognitive Development (ABCD) study (N=7842, 52% boys, aged 9-10). We used diffusion-weighted imaging for mean FA values of structural connectivity to assess microstructural features, and Track-Weighted FA images (TW-FA), integrating FA values with tractograms, to encompass both microscopic and macroscopic information. We found that the 3D Convolutional Neural Network (3DCNN) outperformed the XGBOOST baseline, known for its superior performance for feature-based analysis, in predicting biological sex, BMI, and executive function in repeated settings. Further, we separately assessed macrostructural and microstructural contributions by analyzing tractogram-masked images and spatially shuffled TW-FA images, respectively. Utilizing information in multi-scales through TW-FA with 3DCNN enhanced performance across multiple tasks, outperforming models focused solely on either aspect. These results underscore the importance of considering both macroscopic and microscopic information in white matter analysis and show the effectiveness of deep learning models in capturing individual variations in white matter with compatible interpretability through explainable AI techniques. These demonstrate their potential to enhance our understanding of the neural basis of cognitive and biological traits based on white matter.ModelinputSexBMI(mean:18.78/std: 4.14)Executive Function(mean:100.4/std: 9.79)ACCAUROCr2MAEr2MAEXGBOOSTStructural connectivity FA0.7490.8230.1312.8430.0477.5393DCNN (densenet3D121)TW-FA (spatial shuffled)0.7270.8020.0043.0180.0257.591TCK_masked0.8570.9310.2042.6550.0767.367TW-FA0.8230.9090.2192.6370.0787.368