Back to Module 1.10: TransformersIn Progress
AI Lesson & Submodule

Feed Forward Network

Explore multi-layer perceptrons and non-linear activations inside blocks.

Why This Matters

While attention maps connections, the FFN applies non-linear mappings to refine individual token details.

What You Will Learn

  • •Write FFN projections
  • •Understand GeLU role
  • •Map hidden dimension sizes

Concepts Covered

Two-layer MLP projectsGeLU non-linearitiesHidden dimension multipliers

Mapped Foundation Project: Mini Transformer Block Explainer

Visual deconstruction of a standard decoder block, outlining normalizations, skip links, and output projections.

Architecture Preview

Block-by-block diagram tracking vector changes as inputs pass through decoder normalizations and linear mappings.

Input TokensLayer Norm Layer 1Multi-Head Attention Block
Tech Stack Planned
ReactTypeScriptFramer Motion
GitHub: Coming SoonLive Demo: Coming Soon
Coming Soon

Technical Interview Value

  • ?Explain why a non-linear Feed Forward Network (FFN) sublayer is needed in addition to self-attention
Return to Module Lessons