core/nn/encoder_decoder library

Full encoder-decoder Transformer for seq2seq tasks (translation, summarization, etc.).

Composes two token pipelines:

  • Encoder (bidirectional, no causal mask): source token embedding + sinusoidal PE + TransformerEncoder stack. Yields a "memory" tensor of shape [Se, embedDim] (or [B, Se, D]) that summarises the source sequence.
  • Decoder (causal self-attention + cross-attention over memory): target token embedding + sinusoidal PE + TransformerDecoder stack + Linear LM head. Yields logits [St, targetVocabSize] (or [B, St, V]).

Both sides use their own embeddings and vocab; this is the classic "Attention Is All You Need" architecture.

Same 1D/2D-token convention as TransformerLM / GPT: srcTokens and tgtTokens are [N] for a single sequence or [B, N] batched — mixed rank between src and tgt is rejected.