02 / THE FOUNDATION MODEL
TanAI 3B Ku2.
Turkish, at the core.
A language model built and trained from scratch. Follow the model’s evolution from the first low-parameter research phase to the current 3B training run.
TRAINING IN PROGRESSCURRENT REVISIONR4TanAI 3B Ku2
TRAINING CORPUS193B+tokens · training in progress
COMPUTE8×H100Google Cloud
MODEL SPECIFICATIONS
Inside the current model.
Specifications reported in the project documentation. Effective context and time to first token depend on the system configuration; a public benchmark protocol is not provided here.
- CONTEXT WINDOW
- 16384
- EFFECTIVE CONTEXT
- ~1+M
- PARAMETERS
- 3B
- LAYERS
- 36
- RESPONSE TTFT
- <22ms
- FUSED Proj
- 256dim
- ECV Proj
- 64dim
- CHRONOS Proj
- 32dim
The training architecture of the TanAI model was designed in Turkish. The development process of the TanAI model began with the selection of a low-parameter model architecture.
R0: Designed as an initial R&D phase with 153.2 million parameters.
R1: The parameter count was increased to 182.7 million through the addition of AR-Core and Conditional Layers, and the R&D process continued.
R2: The model was expanded to the 813.16 million parameter level. Thanks to significantly improved Turkish language proficiency, the ability to maintain context in long texts, and a highly effective attention mechanism, it established itself among the best Turkish-language models in its category.
R3: We scaled the model to the 2.058 billion parameter level and trained it on 57 billion tokens. This step enabled the model to perform better reasoning, possess broader world and linguistic knowledge, and generate more robust chains of thought.
R4: After successfully completing the R&D processes, we developed our base model, TanAI 3B Ku2. We are training it on over 193 billion tokens. It is currently undergoing training on an 8xH100 cluster on Google Cloud. Subsequently, training will continue on EuroHPC Mn5 ACC supercomputers.
The model is still very new and in the development phase. At this stage, the model should not be compared to large-scale models such as ChatGPT, Gemini, or Claude. Inference and reasoning outputs can be improved by adjusting hyperparameters in the architectural infrastructure and training the model on much larger datasets.
As a result of all this, TanAI is the First Turkish LLM model and architecture developed and trained from scratch based on the Core structure.
R0: Designed as an initial R&D phase with 153.2 million parameters.
R1: The parameter count was increased to 182.7 million through the addition of AR-Core and Conditional Layers, and the R&D process continued.
R2: The model was expanded to the 813.16 million parameter level. Thanks to significantly improved Turkish language proficiency, the ability to maintain context in long texts, and a highly effective attention mechanism, it established itself among the best Turkish-language models in its category.
R3: We scaled the model to the 2.058 billion parameter level and trained it on 57 billion tokens. This step enabled the model to perform better reasoning, possess broader world and linguistic knowledge, and generate more robust chains of thought.
R4: After successfully completing the R&D processes, we developed our base model, TanAI 3B Ku2. We are training it on over 193 billion tokens. It is currently undergoing training on an 8xH100 cluster on Google Cloud. Subsequently, training will continue on EuroHPC Mn5 ACC supercomputers.
The model is still very new and in the development phase. At this stage, the model should not be compared to large-scale models such as ChatGPT, Gemini, or Claude. Inference and reasoning outputs can be improved by adjusting hyperparameters in the architectural infrastructure and training the model on much larger datasets.
As a result of all this, TanAI is the First Turkish LLM model and architecture developed and trained from scratch based on the Core structure.
RESEARCH STATUS
An ongoing development process.
This is an early-stage model. Architecture goals, completed work and future capabilities are distinguished in the roadmap. Model statements above are retained from the project’s own documentation.
View the roadmap ↗