Neuroconnectionism and TNNs
Text for ML4CNN
The neuroconnectionist research programme is a framework with many theories and approaches to understanding cognitive functions. One of them concerns Topographical Neural Networks (TNNs).
The neurons in the V1 area of the Visual Cortex are selective to one orientation and are represented by kernels in TNNs.
Unlike kernels in a CNN, in TNNs, kernels are location-fixed to mimic the topographic orientation selectivity of neurons in V1 and aim to categorise visual input. Hypercolumns get the inputs from the same location of all layers and mimic a cortical column.
However, without a spatial smoothness constraint, TNNs do not produce orientation maps like V1, which is biologically not plausible.
There is strong support for the idea that neurons in the cortical columns of V1 show a pattern of lateral inhibition that suppresses the firing of neighbouring cortical columns in a short distance (Muir, 2014).
In TNNs, hypercolumns represent one region of the sensory input. Introducing lateral inhibition could let the strongest responding hypercolumn to an input inhibit neighbouring hypercolumns. This leads to higher contrast due to the suppressed firing of hypercolumns with similar orientation preferences.
Within a column, there could also be local competition and inhibition between neurons(Muir, 2014).
Lateral and local inhibition leads to sparseness of the network and, therefore, to more representation possibilities within a network(Hawkins, 2017).
Source
Dylan R. Muir, Matthew Cook; Anatomical Constraints on Lateral Competition in Columnar Cortical Architectures. Neural Comput 2014; 26 (8): 1624–1666. doi: https://doi.org/10.1162/NECO_a_00613
Hawkins, J., Ahmad, S., & Cui, Y. (2017). A theory of how columns in the neocortex enable learning the structure of the world. Frontiers in neural circuits, 11, 295079.
Notizen:
- categorize visual input
- spatial similarity loss (regularizer)
- the TNNs creates orientation maps as well
- without spatial smoothness constraint
- due to a lack of correlations between neurons with similar orientation preferences (Muir, 2014).
Text for Medium
The neuroconnectionist research programme is a framework with many theories and approaches to understanding cognitive functions. One of them concerns Topographical Neural Networks (TNNs).
The neurons in V1 are selective to one orientation and are represented by kernels in TNNs.
Unlike kernels in a CNN, in TNNs, kernels are location-fixed to mimic the topographic orientation selectivity of neurons in V1 and aim to categorise visual input. Hypercolumns get the inputs from the same location of all layers and mimic a cortical column.
However, without a spatial smoothness constraint, TNNs do not produce orientation maps like V1, which is biologically not plausible.
There is strong support for the idea that neurons in the cortical columns of V1 show a pattern of lateral inhibition that suppresses the firing of neighbouring cortical columns in a short distance (Muir, 2014).
In TNNs, hypercolumns represent one region of the sensory input. Introducing lateral inhibition would let the strongest responding hypercolumn to an input inhibit neighbouring hypercolumns. This leads to higher contrast due to the suppressed firing of hypercolumns with similar orientation preferences.
Within a column, there could also be local competition and inhibition between neurons(Muir, 2014).
Lateral and local inhibition leads to sparseness of the network and, therefore, to more representation possibilities within a network(Hawkins, 2017).
see also
Type:
Tags:
Status:
Location:
Created: 07-11-24 19:36
Lateral inhibition
Source
Dylan R. Muir, Matthew Cook; Anatomical Constraints on Lateral Competition in Columnar Cortical Architectures. Neural Comput 2014; 26 (8): 1624–1666. doi: https://doi.org/10.1162/NECO_a_00613
Hawkins, J., Ahmad, S., & Cui, Y. (2017). A theory of how columns in the neocortex enable learning the structure of the world. Frontiers in neural circuits, 11, 295079.