A non-Transformer hierarchical recurrent network with differentiable Gumbel-Softmax routing and bounded memory slots. Runs 7B+ parameter models layer-by-layer on low-budget GPUs.
deep-learning pytorch recurrence attention-mechanism hardware-acceleration memory-augmented-neural-networks llm efficient-ai memory-augmented infinite-context
-
Updated
Jan 15, 2026 - Python