Reverse Engineering

This week we

In the talk james DiCarlo explained the basics of image recognition of the brain. An image is not seen at once but rather consists of many quick saccades across the image.
They did research on monkey that see an image for a fraction of a second and then select what object they have seen.

For short period of time humans are as good as primates.
primates were better than 2010 computer vision systems.

retina trancduces light into chemical signals and there is already a bottleneck at the retinal ganglion cells one connecting to 1000 photoreceptors.

100-150ms you can follow images.

take the data from IT and average the spikes = spike rate
rate codes (how the information is coded)
out of these you can render a population response vector
if you measure thousands of them, you can put linear classifiers on top of them and reproduce the behavioural accuracy of the primate.
and also the performance level.
this is better than the computer vision systems in 2010.

If you plot a specific IT site, you can see that it has a complicated response for images (some more for faces than others)

gabot patches in lgn but it repeats over different areas in later layers

Not sure what V2 does differently to V1.

lets try to do core recognition: position, scale and pose
then lets optimize the parameters of the network to solve this problem

evolving networks inside a computer
does it evolve to be a brain?

then compare it to the brain data.

do you have an IT neuron that looks like this?
Yes, the ANN predicts the neuronal data very well
Models were in the right function space.

in 2012 HMO is much better at explaining brain data than any other model

these models were matched because they run well

deep learning leads to high performance. That doesn’t necessarily mean the brain does deep learning.

Vision, audition, somatosensation, decision-making, motor planning and control, navigation

what if i want to turn on a neuron in a particular state?
called target brain state

In V4 (input to IT)

(hier sollte das bild rein, welches IT/V4 am besten reizt, diese simulierten bilder)

a stimulus that only stimulates one single neuron and no other
difficult because of overlapping receptive fields

synthetic imgaes are good at that (not perfect though)

Text for ML4CCN

In this week’s workgroup, we discussed a talk by James DiCarlo, who explained the basics of image recognition in the brain and how we can compute it with a neural network.

By measuring the activity of neurons in the inferotemporal (IT) cortex of monkeys, researchers can determine the spike rates and rate codes in response to various stimuli. If you collect data for thousands of images, you can map the specific activation patterns for each euron. By designing an Artificial Neural Network (ANN) that is optimised to mimic brain behaviour, it is possible to predict neurons’ activation in IT very well.

In a 2019 paper by Hu, researchers introduced a novel layer to enhance modelling capacity in image recognition tasks. Within a Local Relation Network (LR-Net), a layer identifies compositional structures among visual elements in a localised area. The layers are more effective than those in traditional convolutional neural networks (CNNs) with a higher kernel size 7x7.
LR-Net layers possess a flexible, bottom-up method, representing more complexity and showing more robustness to adversarial attacks.
They show strong results on ImageNet classification and might be more biologically plausible for further research in human image recognition.

Additional Sources:
194 words (source: wordcounter.io)

Hu, H., Zhang, Z., Xie, Z., & Lin, S. (2019). Local Relation Networks for Image Recognition. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 3463–3472. https://doi.org/10.1109/ICCV.2019.00356

Article

What is James DiCarlos’s lab working on?

Mind & Brain ←> Intelligence Algorithms ←> Software, Hardware, Robotics

⇒ Mein Blog

Brain Online

Explorer

Reverse Engineering