BrainEncoding26 Challenge – Course Overview
This note summarises the structure, data, and goals of the challenge at the core of the course Neural Encoding with Deep Neural Networks (SoSe 2026, Prof. König / Kietzmann lab, University of Osnabrück).
Neural Encoding with Deep Neural Networks
Core Idea
When a person freely views a natural scene, their eyes make rapid fixations — each one triggering a cascade of neural responses. Using MEG, we can measure these responses with millisecond precision.
The challenge: build a computational model that predicts the MEG brain response to each fixation, based only on what was seen and where the eyes landed.
The model must then generalise to a completely held-out participant (subject 60) — predicting their neural responses from metadata alone (no MEG data provided for this subject).
Dataset: Active Vision on Scenes (AVS)
Reference: Sulewski, Amme, Hebart, König, & Kietzmann (2025). Why we linger: Memory encoding, rather than visual processing demand, drives fixation timing on natural scenes — evidence from a large-scale MEG dataset.
| Property | Value |
|---|---|
| Subjects (training) | 5 |
| Scenes | 4,080 natural images (NSD stimulus set) |
| Sessions per subject | 10 |
| Fixation epochs (total) | ~200,000 (~40,000 per subject) |
| MEG system | Elekta Neuromag TRIUX, 306 channels |
| Relevant channels | 204 planar gradiometers |
| Sampling rate | 500 Hz (resampled from 1000 Hz) |
| Epoch alignment | Fixation onset |
| Peak encoding accuracy | ~110 ms post fixation onset, posterior sensors |
Preprocessing: tSSS movement compensation, bandpass 0.2–200 Hz, ICA ocular artifact removal.
Challenge Structure
Challenge 1 — Single-Timepoint Topography Prediction
- Goal: Predict MEG gradiometer topography at 110 ms post fixation for subject 60
- Output shape:
(n_fixations, 204)— fixations × channels - Two phases: development (25% of subject 60’s scenes) → final evaluation (different 25%)
Challenge 2 — Multi-Timepoint Prediction
- Goal: Predict MEG topographies at timepoints: −50, 50, 75, 100, 125, 150 ms
- Output shape:
(n_fixations, 204, 6)— fixations × channels × timepoints - Two phases (disjoint from Challenge 1 scenes)
ℹ️ Note: The four 25% scene splits across both challenges are mutually exclusive — together they cover all of subject 60’s scenes.
Pipeline: How to Build an Encoding Model
Natural scene image
↓
Fixation crop (112×112 px, centred on gaze, DVA-scaled)
↓
DNN feature extraction (e.g. ResNet-18 intermediate layers)
↓
Ridge regression (features → MEG channels)
↓
Prediction: (n_fixations, 204)
Key steps in code (meg-encoding-course-tbd):
- Load metadata + MEG targets via
DataProcessor - Crop stimuli around fixation with
Cropper(degrees of visual angle) - Extract CNN features with
extract_features(torchvision models) - Fit
FeatureEncoder(ridge + optional PCA + cross-validation) - Export with
export_challenge_metrics→exports/predictions.zip
What Makes This Hard
- Subject generalisation: Model trained on subjects 1–5, tested on subject 60 — individual differences in MEG topography are substantial
- Low SNR: Single-trial MEG epochs are very noisy; encoding models must generalise from averaged patterns
- High dimensionality: 204 channels to predict simultaneously; features from DNN layers can be very high-dimensional
- Timing: The model must capture the right temporal window (peak at ~110 ms reflects early visual processing)
Notebooks Covered in the Course
| Notebook | Topic |
|---|---|
| 1-hello-world | Setup & environment check |
| 2-explore-challenge-data | MEG data structure, metadata, fixation epochs |
| 3-encoding-models-from-handcrafted-features | Gabor filters, pixel features as baseline |
| 4-encoding-models-from-dnns | ResNet-18 feature extraction, layer selection |
| 5-model-contrasts-and-interpretation | Comparing models, layer analysis, RSA |
| 6-luminance-encoding | Low-level visual feature baselines |
Working Repository
/Users/maxmacbookpro/Developer/GitHub/meg-encoding-course-tbd/
Package: tbdencoder — data loading, cropping, encoding, eval, Gabor models.
Neural Encoding with Deep Neural Networks
Tags: neuroscience neural-encoding meg