Topic: multimodal models

Distinguishing diatom and dinoflagellate blooms in Chesapeake Bay is difficult because its waters are optically complex and field observations rarely coincide with satellite measurements. This project combines historical phytoplankton observations, NOAA satellite bloom products, geospatial processing and ecological information in a reproducible workflow. Quality control addresses mixed pixels, land contamination, cloud, missing observations, saturation and inconsistent coordinates. Of 31,531 field-to-satellite records, 860 initially pass validation; neighbouring pixels offer further recoverabl

To better understand human scene understanding, we extracted features from images using CLIP, a neural network model of visual concept trained with supervision from natural language. We then constructed voxelwise encoding models to explain whole brain responses arising from viewing natural images from the Natural Scenes Dataset (NSD) - a large-scale fMRI dataset collected at 7T. Our results reveal that CLIP, as compared to convolution based image classification models such as ResNet or AlexNet, as well as language models such as BERT, gives rise to representations that enable better prediction