Decoder Layer PyTorch

Purpose and use: This tool analyzes a PyTorch decoder layer to expose exact dimensionality, parameter counts, and attention characteristics. It is aimed at ML researchers, model engineers, and tooling teams who design sequence-to-sequence decoders and transformer-based architectures. By validating shapes and budgets early, it reduces risk when scaling models or comparing variant configurations.
How it works conceptually: users supply standard decoder hyperparameters (d_model, n_heads, d_ff, num_layers) and optional cross-attention. The system applies canonical Transformer formulas to derive Q/K/V projections, output projections, FFN parameters, and LayerNorm terms. It then aggregates per-layer results to provide a total view and a detailed layer-by-layer map. It also reports inconsistencies and potential bottlenecks for quick remediation.
Value, use cases, and differentiators: the tool supports rapid comparison of architectural choices, aiding teams in selecting dimensions that balance accuracy, latency, and memory. It yields a precise parameter budget and memory footprint, enabling cost-aware planning. Key differentiators include explicit dimensional checks (d_model divisible by n_heads), transparent assumptions, and a straightforward export-ready report for documentation. It also supports integration into model planning workflows to track changes across experiments.
Core features and outputs: per-layer and total parameter tallies, shapes for Q/K/V and FFN projections, attention head counts, and estimated RAM for forward passes. It adheres to PyTorch conventions and can be integrated into model planning workflows or reproducibility logs.