Building Machines that learn and think like people

Building Machines that learn and think like people

measuring dopamine neurons in midbrain - fire when animal is rewarded
but when anticipated neurons dont fire anymore

• so it’s about the reward prediction error
• temporal difference learning algorithm (cornerstone of RL)
• object recognition by DiCarlo
• TDRL is insprired by biology and behaviour
• ewhat gain do be expect from looking into biological systems and brains?

In flight acquisition, the Wright brothers stopped imitating birds and worked on understanding of aerodynamics in windtunnels

• conclusion: we shouldnt imitate the brain, if they look like the brain its great but we shouldnt be too concerned about it
• What are the lessons from the brain?

ANNs look like neurons

we have human-level performance on important tasks and brain-inspired computational architectures. maybe this is the key to human-like AI?

1. What does it mean for a machine to learn and think like a person?

   • Five key ingredients with strong empirical support from cognitive science.
     1. Intuitive Physics
        • Humans already have an intuitive understanding of physics in the first months of life like: as objects don’t interact at a distance, continuity, cohesion
     2. Intuitivs Psychology
        • Children of the age of 1 develop an intuition for psychology. Who is a helper and who is a hinderer
     3. Compositionality
        • New representations can be constructed through the combination of primitive elements
        • Dividing goals into subgoals
     4. Learning-to-learn
        • Learning new tasks is accelerated due to previous learning
        • compositionality helps in learning
     5. Causality
        • human concepts often go beyond features to resemble causal explanations or beyond pattern recognition to resemble model building.
        • ANNs don’t get the deeper meaning in images

2. How close are contemporary Al systems to reaching this standard?

   • Deep learning systems have not yet incorporated many of the key ingredients, and thus may be solving problems in different ways than people.

3. Should Al researchers care about building more human-like machines?

   • Yes - in most cognitively natural domains, people are still much better learners and thinkers than the best Al systems, and we still have a lot to learn by reverse engineering the human mind.

Current trend in deep learning:

• selective attention mechanisms
• experience replay
• external working memories

why not higher cognitive ingredients?

• for autonomous driving, scene understanding, creative design

we need cognitive plausibility to understand human

what kind of neural network is the brain?

• a neural network
• cognitive plausible neural networks might look very different

Second talk:
how to increate IT predictivity

• because the DNNs are biological implausible and still explain IT
• self-supervising tasks
  • single image statistic tasks
    • predict relative position
    • predict colorization
    • predict depth

Text for Workgroup

In this weeks workgroup, we discussed two presentations about brain-inspired machine learning and self-supervised models.
In the first talk Gershman showed how brain-inspired machine learning frameworks, such as Temporal Difference Reinforcement Learning (TDRL), are inspired by the dopamine circuit in the midbrain.
Gershman highlighted five core ingredients needed for brain-inspired human-like AI, which include intuitive physics and psychology, as well as understanding deeper meanings of scenes.
In the second talk, Zuhang pointed out that through semi-supervised tasks researchers can improve the similarities of categorization behaviour in humans.

In model-free TDRL, ‘value’ is defined as the expected sum of future rewards. This is inspired by the prediction errors (PEs) in the brain, triggered by dopamine release in the ventral striatum.
In Langdon’s 2018 paper, the authors argue that dopamine neurons have access to model-based information about expected rewards, going beyond a simple scalar representation of value as modeled in TDRL. This might be due to the PE enhancing learning in target areas indiscriminately without signaling a distinct direction (Langdon, 2018).
These findings show that dopamine signaling is more complex than previously thought, leaving opportunities for a deeper understanding of the human brain and potentially enhancing TDRL frameworks.

Additional Source:
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018). Model-based predictions for dopamine. Current opinion in neurobiology, 49, 1-7.

see also

Type:
Tags:
Status:
Location:
Created: 18-12-24 12:23
Machine Learning for Cognitive Computational Neuroscience

Source