Deep Neural Networks

Deep Neural Networks

• before using the input data check:
  • are the images squared?
  • are they not distorted?
  • zoom in and crop them to a square

Why depth needs new activation functions — the vanishing gradient

A network is deep once it stacks more than two layers. Depth buys expressivity, but it also makes signal propagation fragile: during backprop the gradient is the product of one activation derivative φ’(x) per layer, so if those derivatives are consistently small the gradient shrinks exponentially toward the input layer — the vanishing-gradient problem. The figure below makes the cause visible.

🐍 Figure — Activation functions and their derivatives (why ReLU beats sigmoid in deep nets)

import micropip
await micropip.install("matplotlib")
await micropip.install("numpy")
import matplotlib.pyplot as plt
import numpy as np
 
x = np.linspace(-6, 6, 500)
 
def sigmoid(x): return 1.0 / (1.0 + np.exp(-x))
def dsigmoid(x):
    s = sigmoid(x); return s * (1 - s)
def tanh(x): return np.tanh(x)
def dtanh(x): return 1 - np.tanh(x) ** 2
def relu(x): return np.maximum(0, x)
def drelu(x): return (x > 0).astype(float)
def lrelu(x, a=0.1): return np.where(x > 0, x, a * x)
def dlrelu(x, a=0.1): return np.where(x > 0, 1.0, a)
 
funcs = [
    ("sigmoid",    sigmoid,  dsigmoid,  "#3498db"),
    ("tanh",       tanh,     dtanh,     "#27ae60"),
    ("ReLU",       relu,     drelu,     "#e74c3c"),
    ("Leaky ReLU", lrelu,    dlrelu,    "#9b59b6"),
]
 
fig, (ax_top, ax_bot) = plt.subplots(1, 2, figsize=(11, 4.5))
 
for name, f, df, col in funcs:
    ax_top.plot(x, f(x), label=name, color=col, lw=2)
    ax_bot.plot(x, df(x), label=f"{name}'", color=col, lw=2)
 
for ax in (ax_top, ax_bot):
    ax.axhline(0, color="#888", lw=0.6); ax.axvline(0, color="#888", lw=0.6)
    ax.grid(alpha=0.3); ax.legend(loc="best", fontsize=9); ax.set_xlabel("preactivation  x")
 
ax_top.set_title("Activation functions  φ(x)")
ax_top.set_ylabel("φ(x)")
ax_bot.set_title("Derivatives  φ'(x)  — what backprop multiplies")
ax_bot.set_ylabel("φ'(x)")
 
ax_bot.annotate("sigmoid & tanh → 0\nfor large |x|\n(vanishing gradient)",
                xy=(4.5, dsigmoid(np.array([4.5]))[0]), xytext=(0.6, 0.62),
                fontsize=9, color="#3498db",
                arrowprops=dict(arrowstyle="->", color="#3498db"))
ax_bot.annotate("ReLU stays φ'=1\nfor x>0",
                xy=(3.5, 1.0), xytext=(0.4, 0.30),
                fontsize=9, color="#e74c3c",
                arrowprops=dict(arrowstyle="->", color="#e74c3c"))
 
plt.tight_layout(); plt.show()

What this shows. Left: the four activation curves over x ∈ [−6, 6]. Right: their derivatives — the factor that Gradient Backpropagation multiplies once per layer. The sigmoid and tanh derivatives collapse to ≈ 0 whenever the preactivation is even moderately large (the functions saturate), so in a deep stack their product vanishes and the early layers barely learn. The ReLU derivative stays exactly 1 for every x > 0, which is why it became the default activation for deep networks — it keeps the gradient alive through many layers (Leaky ReLU additionally keeps a small slope for x < 0 to avoid dead units). This single property, together with residual connections and careful Glorot/Kaiming initialisation, is what made training very deep architectures practical. See the hub note Neural Networks & Deep Learning for the broader picture.

Memory consolidation in DNNs, LSTMs and SOAR

chatbot
Memory consolidation in deep neural networks (DNNs) refers to the process of integrating new information into existing knowledge structures without disrupting previously learned information. This is a critical aspect of learning and memory in both biological systems and artificial intelligence. When comparing DNNs, Long Short-Term Memory Networks (LSTMs), and the Symbolic and Subsymbolic Optimization Algorithm Repository (SOAR) architecture, several issues and differences in memory consolidation emerge.

Deep Neural Networks (DNNs)

Issues:

• Catastrophic Forgetting: DNNs are particularly prone to catastrophic forgetting, where the learning of new information can lead to the loss of previously learned information. This is a significant issue in scenarios where continuous learning is required.
• Limited Memory Consolidation Mechanisms: DNNs lack explicit mechanisms for memory consolidation that are comparable to those in biological systems. While techniques such as replay buffers or regularization methods (e.g., Elastic Weight Consolidation) have been proposed to mitigate forgetting, they often require additional resources or modifications to the learning process.
• Generalization vs. Memorization: DNNs sometimes struggle to generalize from past experiences without overfitting to specific instances. This balance is crucial for effective memory consolidation but can be challenging to achieve.

Long Short-Term Memory Networks (LSTMs)

Advantages over DNNs:

• Better at Handling Sequential Data: LSTMs are specifically designed to address the issue of learning long-term dependencies in sequential data, making them more adept at tasks where understanding the sequence is crucial for memory consolidation.
• Reduced Forgetting through Gates: LSTMs incorporate gating mechanisms (input, output, and forget gates) that help in selectively remembering and forgetting information. This can mitigate some aspects of catastrophic forgetting by allowing the network to maintain important information over longer periods.

Issues:

• Still Prone to Forgetting: Despite their advantages, LSTMs are not immune to forgetting, especially when dealing with very long sequences or when the amount of new information to be learned is substantial.
• Complexity and Resource Requirements: LSTMs are more complex and computationally intensive than simpler DNNs, which can limit their scalability and efficiency in some applications.

SOAR Architecture

Differences:

• Symbolic Representation: SOAR is based on symbolic AI, contrasting with the sub-symbolic nature of DNNs and LSTMs. This allows SOAR to explicitly represent knowledge and rules, potentially facilitating better memory consolidation through structured knowledge bases.
• Explicit Memory Systems: SOAR incorporates different types of memory (e.g., working memory, long-term memory, procedural memory) in a manner somewhat analogous to human cognitive architectures. This can help in organizing and consolidating knowledge more effectively than the more homogeneous memory representations in DNNs and LSTMs.
• Less Prone to Forgetting: Due to its explicit and structured representation of knowledge, SOAR can be less prone to catastrophic forgetting compared to DNNs and LSTMs. However, updating and integrating new knowledge in SOAR can require explicit reprogramming or rule adjustment.

Issues:

• Scalability and Flexibility: While SOAR can be effective for tasks with well-defined rules and structures, it may struggle with the scalability and flexibility required for processing unstructured data or learning from raw sensory inputs as DNNs and LSTMs can.
• Generalization: SOAR’s approach to memory consolidation relies heavily on the predefined structures and rules, which might limit its ability to generalize from new or unseen data compared to the more adaptable, albeit sometimes less stable, learning mechanisms of DNNs and LSTMs.

In summary, while DNNs and LSTMs offer powerful mechanisms for learning from and adapting to complex data patterns, they face significant challenges in memory consolidation, particularly regarding catastrophic forgetting and the balance between generalization and memorization. The SOAR architecture, with its symbolic approach and structured memory systems, presents a different set of trade-offs, potentially offering better memory consolidation at the expense of flexibility and scalability.

Replay Buffers

chatbot
Replay buffers are a crucial component in the training of deep neural networks (DNNs), especially in the context of reinforcement learning (RL). They help to stabilize and improve the learning process by addressing issues related to data correlation and non-stationary distributions, which are common in online learning scenarios. Here’s how replay buffers function:

Basic Functionality

1. Storing Experience Tuples: A replay buffer stores experience tuples that the agent encounters over time. An experience tuple typically consists of the current state (s), the action taken (a), the reward received (r), the next state (s'), and sometimes a done flag (d) indicating whether the episode has ended. This structure captures the essential information needed for the agent to learn from its interactions with the environment.

2. Sampling Mini-Batches: Instead of learning from consecutive experiences as they occur (which can introduce strong correlations and lead to instability in learning), the network samples mini-batches of experiences at random from the buffer. This random sampling helps to break the correlation between consecutive learning updates, making the learning process more stable and efficient.

3. Improving Sample Efficiency: By storing and reusing past experiences, replay buffers allow for each experience to be used multiple times in the learning process. This improves the sample efficiency of the learning algorithm, as valuable experiences contribute to learning more than once.

4. Mitigating Non-Stationarity: In RL, the distribution of experiences can change as the policy being learned changes. This non-stationarity can make learning difficult. Replay buffers mitigate this by providing a more stable, averaged-out distribution of experiences from which to learn.

Advanced Features and Considerations

• Prioritized Experience Replay: An extension of the basic replay buffer concept is prioritized experience replay, where experiences are sampled according to their importance, as measured by the temporal difference (TD) error. This ensures that experiences from which the agent can learn the most are sampled more frequently.

• Buffer Size and Management: The size of the replay buffer is a critical parameter. A larger buffer can store more experiences but may also contain outdated information that is less relevant to the current policy. Conversely, a smaller buffer may not provide a diverse enough set of experiences. Managing the buffer’s contents (e.g., by discarding old experiences or by selectively keeping informative ones) is an ongoing area of research.

• Balancing Exploration and Exploitation: While replay buffers help in stabilizing and improving the learning process, they must be used in conjunction with strategies that ensure sufficient exploration of the environment. Without exploration, the replay buffer may become dominated by experiences from a suboptimal region of the state-action space.

In summary, replay buffers enhance the learning process in DNNs, particularly in reinforcement learning, by providing a mechanism to store, reuse, and learn from past experiences in a way that reduces correlation, improves sample efficiency, and mitigates the effects of non-stationarity.

see also

Type:
Tags:
Status:
Location:
Created: 21-11-24 22:06

Source