Mind Video

Uses spatiotemporal attention

Uses Augmented Stable Diffusion model

Trains encoder modules independently

Co-trains model and encoder

Employs a two-module structure

Adaptable and flexible structure

Employs progressive learning

Reconstructs scene dynamics accurately

Implements multi-stage brain feature learning

Achieves strong semantic accuracy

Achieves metric accuracy of 85%

Enhances understanding of the cognitive process

Shows the visual cortex is dominant

Features hierarchical encoder layer operation

Maintains volume and time-frame

Employs masked brain modelling

Employs large-scale unsupervised learning

Employs multi-modal contrastive learning

Employs progressive semantic learning

Features analytical attention analysis

Performs 45% better than prior methods

Shows higher cognitive networks are important

Encoder layers derive abstract features

Uses semantic metrics and SSIM for evaluation

Progression shown in training stages

Compresses fMRI time frames

Increases generation consistency

Aids video generation

Provides fMRI encoder attention detail

Offers a biologically reasonable interpretation

Accounts for hemodynamic response time lag

Incorporates network temporal inflation

Usable with sliding windows

Integrates CLIP space training

Extracts features related to semantics

Produces visually meaningful generated samples

Enhances understanding of semantic space

The pipeline is divided into two modules

Utilizes data from the Human Connectome Project

Analyzes encoding's layer-dependent hierarchy

Maintains scene dynamics within the frame

Improves through several training stages

Constructs a flexible and adaptable pipeline

Enables the coding of multiple learned features

Encoder emphasis changes over time

Requires long training times

Relies on an annotated dataset

Needs fine-tuning

Transformer hierarchy may complicate processes

Semantic learning is gradual

Dependent on a specific diffusion model

Focus on the visual cortex has limited use cases