Questions ML4CCN VL1

What is the Chinese Room argument?

The Chinese room arguments says that when a person is sitting in a room and arranging chinese symbols according to a rule and passes those symbols out without knowing Chinese, for outside people it seems this person does understand Chinese.
Chinese Room thought experiment

That is kind of how AI works

What are Marr’s 3 levels?

Computational theory:
What is the goal?
What is the input and output?
What information is extracted and why?

Representation and Algorithm:
How can this be implemented?
What is the algorithm?

Hardware Implementation:
How is can be algorithm be realized physically?

What challenges does Cognitive Computational Neuroscience try to solve?

All models are wrong, some are useful - George Box

it tries to find models that explain the brain
finding the sweet spot between AI, computational neuroscience and cognitive neuroscience

AI: complex tasks
cognitive neuroscience: what behaviour can be explained
computational neuroscience: how the brain is best modeled (biological plausibility and explain neural activity)
ML4CCN: computational models

When are models useful?

make researchers formalize their thoughts in concrete systems and models.
comprehensive easy abstract concepts may be distilled from complex data and lead to a deeper understanding of mechanisms
give an idea of how the brain might work
show when/how/why something works in the brain (not only observing)
making predictions about future experiments

What are other reasons for computational modeling?

How does a drift diffusion model work?

Explains how a decision process happens.
It collects data over time and then makes a decision.
This can explain consumer behaviour for example.

What is deep learning?

Deep learning is learning from experience
Deep learning is mapping an input space to an output space of some other form through non-linear transformations

How can classic pattern recognition be compared to deep learning?

both have a trainable classifier at the end
classic pattern recognition:

feature extraction
trainable classifier

deep learning

trainable low-level features
trainable mid-level features
trainable high-level features
trainable classifier

What happens to receptive fields across DNN layers?

Receptive field sizes increase in later DNN layers.
Receptive field feature complexity increases in later DNN layers.

What aspects do DNNs take from visual neuroscience?

neurons as computing entities
retinotopy (layers like receptive fields)
hierarchy in the visual pathway
sparsity (biological sparse coding, sparseness as regularizer)
neural stochasticity (dropout)
attention
simple and complex cells (max pooling)

Text for Medium

At the beginning of my studies I was totally into neuroscience.
The study of the brain is so fascinating and there are many open questions in this field.
We do not know yet how the brain works. There is no solution to the question of how cognition works. No one knows.

So neuroscience is the way to go, isn’t it?

Then I realized, we can continue locating more and more areas for what they do but this does not seem feasible in the long long.
Yes, at some point, we will be able to map the entire brain and know every circuit.
And then? We still do not know how they interact.

What do we need Cognitive Computational Neuroscience (CCN) for?

According to the Marr’s three levels of analysis, we ask three basic questions in CCN:

what is the goal (what is the input and the output?)
How can this be implemented into an algorithm?
Where can this be found inside the brain?

So CCN combines the fields of Artificial Intelligence, Cognitive Neuroscience and Computational Neuroscience and tries to get the best out of the three worlds.

The subfields basically do these things:
AI: solving complex tasks
cognitive neuroscience: what behaviour can be explained
computational neuroscience: how the brain is best modeled (biological plausibility and explain neural activity)
CCN: What behaviour can be best explained by modeling the brain?

All models are wrong, some are useful - George Box

But when are models useful?

models make predictions about future experiments. You want to find something out that you want to test on humans or rats? Run a model first to test whether you could be correct.
models distill complex data into easy, abstract concepts which provide a deeper understanding of mechanisms in the brain
also with models researchers are forced into formalizing their ideas into specific systems.
models give a pretty good idea of how the brain might work. This gets us a step closer to understand cognition.

What are good models for understanding the brain?

Drift Diffusion models which can explain how a decision process emerges. It captures the process of someone collecting evidence until a threshold of a decision is reached which is then carried out.

Deep Learning:
Deep learning is getting more and more popular and seems to be very good at modelling the brain.

Considering the rise of AI at the moment, there is no better time to be a neuroscientist. With increasingly better Deep Neural Networks (DNNs) we get so good models to understand how the brain works.
This accelerates research by years into the future.

What is Deep Learning?

Deep Learning refers to the process of learning from experience by mapping various types of input to specific outputs using non-linear transformations. This approach enables us to analyze complex relationships and effectively handle large amounts of data.

In the brain we visual pathway we find receptive fields which capture what the eyes see.
The higher we go in the ventral stream
Ventral Visual Pathway

We can also observe these receptive fields in DNNs in which the also increase in size and complexity in higher layers.

In general DNNs take many aspects from neuroscience:

neurons as computing entities
- these are the units in a DNN
retinotopy
- the arrangement of receptive fields in both the brain and DNNs where neighbouring areas in the retina (eye) correspond to neighouring areas in a receptive field {drawing by me}
visual hierarchy in the visual pathway
- First the input from the eye reaches V1 and travels to higher layers as V4 and IT.
- This is also the case in DNNs.
sparsity (biological sparse coding, sparseness as regularizer)
- the brain works very energy efficient where only a fraction of neurons fire at the same time. This is shown in DNNs where many units have a weight of 0, so they do not contribute to the output value.
neural stochasticity (dropout)
- neurotransmitter release is probabilistic and neural firing also shows probabilistic patterns.
- In DNNs the initialization of weights is random and also with dropout is a randomness introduced by setting weights of neurons to zero.
attention
- There are several attention mechanisms in the brain, enhancing some stimuli while suppressing others
- in DNNs attention scores tell the model what part to focus on
simple and complex cells (max pooling)
- in the brain simple cells compute single low-level features like edges or orientation.
- complex cells kind of summarize what the simple cells have computed and respond to a large part of the receptive field
- in DNNs units also detect low-level features like simple cells
- max pooling layers in DNNs take the maximum response over a group of units and capture the most important features.

For now DNNs are the best we have to model the brain, but we are far from explaining cognition itself.
This field is going to be accelerated so quickly by new models, ideas, and theories that are popping up faster than mushrooms.
I can’t wait to see how these developments will increase our understanding of both artificial and natural intelligence.
It’s like being part of a giant, collaborative quest to decode the mystery of cognition, and honestly, what could be cooler than that?

Brain Online

Explorer

Questions ML4CCN VL1