Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
The usability of Auto-Regressive and Diffusion models together, and the effect of ReasonAug structures on Fusion layers.
ar5iv.labs.arxiv.org03 / RESEARCH FOUNDATIONS
Ideas we build on.
Questions we explore.
The papers, frameworks and theoretical foundations informing our work on hybrid language models, emotional conditioning and adaptive intelligence.
↗
Self-Adapting Language Models
This article evaluates TanAI's self-training structure, SAT (Self Adaptive Tuning), in terms of its similarities and processes.
arxiv.orgMemory-Prediction Framework
Jeff Hawkins' Memory Prediction Framework has given us insights into dual-core model structures and emotional memory.
en.wikipedia.orgPlutchik's Emotion Model
When creating the ECV (Emotional Conditional Vector) dimensions, we considered Plutchik's 8 main emotion constructs, created their mathematical vectors, and used them for emotion consistency.
en.wikipedia.orgHierarchical Reasoning Model (HRM) Briefly Explained
We are more interested in smaller models because they are informative and enjoyable to read, but the TanAI model will also develop; it's still a child :)
medium.comRecursive Language Models
We have always been interested in Recursive Language Models. We examined this ambitious paper and wanted to share it with you as well.
arxiv.orgFROM THEORY TO IMPLEMENTATION
Explore our own research notes.
Technical writing on the choices behind TanAI, from tokenization to conditioning.
Read the publications ↗