CLIP, LLaVA, and the Brain | Towards Data Science

What neuroscience can teach us about the limitations of modern multimodal transformers