TanAI 3B GAT Training Begins on an 8×H100 Google Cloud Cluster

TanAI has started the next major stage of its model-development programme.

Following the completion of the 2B Base-LM and Core-SFT stages, we transitioned to a redesigned 3B Generative-Adaptive Transformer architecture and began training on a Google Cloud cluster equipped with 8×NVIDIA H100 GPUs.

The new model contains approximately:

• 2.4B parameters in the language-model backbone
• 0.6B parameters in adaptive and conditioning components
• 3B parameters in total

The current training corpus contains more than 193 billion tokens and combines Turkish-native language data with selected English, code, mathematical, reasoning and structured-text sources.

This revision introduces significant architectural changes, including an updated RoPE implementation, Grouped Query Attention, a 16K native context target, block-level attention residuals and expanded FUSED, ECV and CHRONOS condition projections.

The objective is not simply to increase parameter count. The 3B revision is designed to create a more stable separation between the language backbone and TanAI’s adaptive conditioning architecture, while improving training efficiency, context behaviour and product readiness.

Training is currently in progress. In line with our Build in Public approach, selected loss curves, training milestones, model outputs and technical findings will be published as the run advances.