# LLMs Are Powerful Because Language Carries More Than Words > Language carries knowledge, culture, and ways of working through problems. Aleph Alpha's German reasoning experiment reveals why fluent translation alone cannot deliver the full promise of multilingual AI. - Author: Kostas Karolemeas - Published: 2026-09-25 - Topics: large language models, language and culture, multilingual AI, AI reasoning, enterprise AI - Canonical URL: [https://www.voxelperfect.com/writing/llms-are-powerful-because-language-carries-more-than-words](https://www.voxelperfect.com/writing/llms-are-powerful-because-language-carries-more-than-words) An [Aleph Alpha post about German reasoning](https://www.linkedin.com/posts/most-ai-models-can-answer-in-german-very-share-7508901394041454592-kePv/) prompted me to ask whether a model caught in a loop was being too thorough or simply indecisive. Then I wrote the part that interested me most: > Language conveys culture and it is our most precious invention so far. LLMs are powerful because of that. “Invention” was my shorthand for something humanity developed collectively, rather than something someone designed at a particular moment. And “most precious” is a judgment I am happy to defend, not a scientific ranking. Language lets one person's experience become available to someone who was never there. We can preserve an explanation, dispute it, improve it, and pass it on. Writing extends that possibility across distance and generations. I think this helps explain why learning from language has produced such broadly useful machines. The material contains much more than names for things. It contains descriptions of how things work, arguments about why they work, instructions for changing them, and disagreements about whether they should be changed at all. The German experiment makes one part of that argument unusually concrete. ## Fluent answers and reliable reasoning are different achievements In their [September 24 research report](https://aleph-alpha.com/en/blog/through-the-valley-of-tears-cold-starting-german-reasoning-in-llms/), Niklas Finken and Simon Thel describe 13 fine-tuning runs on a mixture-of-experts model with 3 billion active parameters. Their generated German dataset contains roughly 796,000 samples; 83% include reasoning traces. On German-translated AIME 2026 problems, the comparison is striking: | Training run | German math accuracy | | ------------------------------------------------------- | -------------------: | | No German fine-tuning data (`de-0`) | 70.2% | | German data in other domains, no German math (`math-0`) | 48.3% | | German math strongly upsampled (`math-16`) | 67.3% | These are average accuracies over 16 attempts per problem on a 30-problem benchmark. The baseline mostly reasoned in English. German training switched the math traces to German, but about a quarter looped in `math-0`; roughly 15% still looped in `math-16`. Domain-matched data helped recovery without fully resolving the behavior. These are results from one controlled training setup, not a ranking of languages. [Experiment and results.](https://aleph-alpha.com/en/blog/through-the-valley-of-tears-cold-starting-german-reasoning-in-llms/) My question about thoroughness and indecision was an analogy. This experiment does not identify either human trait, or establish that German culture or syntax caused the loops. Repetition without progress is a failure to complete the task, however reflective it sounds. [Barua and colleagues](https://arxiv.org/abs/2508.14828) examine the broader transfer problem across nine non-English languages. They find that generating long reasoning traces in the target language lags behind reasoning in English, particularly on multistep mathematics. Their revised study also finds that translated reference traces can outperform target-language traces distilled from larger models. There is no simple rule that native generation always beats translation. Data quality, task, and training method matter. ## Why learning language can teach more than language Consider what someone must follow to complete these different passages: a proof with a missing step, a conversation containing an implied refusal, a bug report followed by a diagnosis, or a negotiation in which one party changes the terms. Each contains words, but the useful relationships extend beyond spelling and grammar. There are dependencies, intentions, constraints, and consequences. My interpretation is that language gives a learning system access to an unusually rich record of those relationships. The [GPT-3 paper](https://arxiv.org/abs/2005.14165) provided an early demonstration of the breadth this approach could produce. A single pretrained model performed many tasks from text instructions and examples, without updating its weights for each task. The results included translation, question answering, and some tasks requiring reasoning, alongside substantial weaknesses. That does not mean a next-token objective guarantees understanding. It means the objective can reward learning structure that is useful across tasks. Architecture, scale, data selection, and subsequent training remain essential parts of the explanation. Language also supplies a working medium during generation. In [chain-of-thought prompting experiments](https://arxiv.org/abs/2201.11903), examples containing intermediate steps improved arithmetic, commonsense, and symbolic reasoning in sufficiently large models. A written intermediate result can become context for the next step. This is why I find “just predicting words” an incomplete description of what matters. It names the prediction task without telling us which capabilities a system has learned to perform it. We still have to examine those capabilities, their limits, and their reliability. ## Culture travels with the examples An explanation carries assumptions about its reader. A request carries expectations about authority and politeness. A procedure assumes particular institutions, responsibilities, and conventions. Imagine an assistant answering a question about starting a small business. It could produce impeccable Greek while organizing the answer around institutions and procedures from another country. This is an illustrative failure case: grammatical quality and contextual relevance can come apart. [Chen and colleagues' EMNLP 2024 study](https://arxiv.org/abs/2406.12822) shows why evaluation matters here. Native and generative benchmarks revealed differences between native and translated instruction data that other test sets missed. A translated benchmark can leave important language-specific knowledge untested. There is evidence of behavioral variation too. In its [July 2026 analysis](https://www.anthropic.com/research/claude-values-models-languages), Anthropic found differences in the values Claude expressed across languages, including the balance between warmth and rigor. The analysis controlled for task, topic, and user-expressed values, but it remains an observational study of model responses, not a causal explanation of national cultures. I would therefore describe a model as inheriting cultural patterns selectively. It does not inherit one coherent culture, and speaking a language does not assign it a national personality. Its behavior reflects the examples available to it and the choices made during training. For builders, that makes multilingual quality a deeper obligation. Users need help with the world they inhabit, including knowledge and distinctions that may be poorly represented in English material. ## Language matters without being the whole of thought There is a tempting leap from this argument to the claim that all thought is language. We do not need that leap. In a [2024 Nature perspective](https://www.nature.com/articles/s41586-024-07522-w), Evelina Fedorenko, Steven Piantadosi, and Edward Gibson review evidence separating human linguistic ability from other cognitive capacities. They argue that language primarily serves communication and the transmission of cultural knowledge, rather than being a prerequisite for complex thought. That strengthens the part of my argument I care about. Language can be extraordinarily valuable because it makes knowledge shareable, even when the knowledge began in perception, action, mathematics, or experience that was not verbal. We also need to distinguish the language of a model's generated reasoning from its underlying computation. In [Anthropic's interpretability work on Claude 3.5 Haiku](https://www.anthropic.com/research/tracing-thoughts-language-model), simple multilingual tasks activated some shared conceptual features across languages. The same research program demonstrated cases where a plausible written explanation did not faithfully describe the computation behind an answer. Its methods capture only part of the computation and have important limitations. So “thinking in German” is useful shorthand for generating a German reasoning trace. It does not establish that every internal representation is German, or that the trace gives us complete access to the mechanism. Readable explanations still matter. They let people examine claims and challenge assumptions. Their value increases when we can also inspect sources, verify calculations, and test the resulting action. ## What I would ask of a multilingual AI product I would begin with the work a person needs to complete in their language. For a hypothetical support assistant serving Greek and German customers, I would evaluate three things separately: - **Task success:** Does it solve the customer's actual problem correctly, within an acceptable time and cost, and reach a usable conclusion? - **Local relevance:** Does it use the appropriate terminology, source material, institutions, and conventions? Include cases written by people who work in that context. - **Reviewability:** Can the user understand the explanation and check the evidence supporting the recommendation? I would compare direct target-language operation with translation-assisted alternatives using the same realistic tasks. I would count unfinished answers and repetition as failures, and ask local practitioners to assess what automated checks miss. That is an engineering proposal, not a result established by the German experiment. It follows from taking the distinction between fluency and usefulness seriously. Language's value is also why I remain interested in [clear expression as an engineering skill](/writing/coding-agents-reward-engineers-who-can-explain-what-they-want). When we explain the situation, name the constraint, and describe what would count as success, we make more of our own understanding available to the system. The prospect I find most compelling is that LLMs can make more of humanity's accumulated knowledge usable by more people. Realizing it means caring about whose knowledge is represented, which languages it works in, and whether the answer helps someone act wisely. We have built machines that can learn from an extraordinary human inheritance. The richness of that inheritance is a large part of their power. ## Sources - **Finken and Thel, Aleph Alpha (September 24, 2026):** [Through the Valley of Tears](https://aleph-alpha.com/en/blog/through-the-valley-of-tears-cold-starting-german-reasoning-in-llms/) — the German training experiment. - **Barua et al. (2025; revised March 2026, ICLR 2026):** [Long Chain-of-Thought Reasoning Across Languages](https://arxiv.org/abs/2508.14828) — multilingual reasoning transfer and training comparisons. - **Brown et al. (2020):** [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165) — task breadth from text-based interaction. - **Wei et al. (2022):** [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://arxiv.org/abs/2201.11903) — intermediate-step prompting experiments. - **Chen et al. (EMNLP 2024):** [Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation?](https://arxiv.org/abs/2406.12822) — native versus translated data and evaluation. - **Anthropic (July 13, 2026):** [Claude's values across models and languages](https://www.anthropic.com/research/claude-values-models-languages) — observed variation in expressed values. - **Fedorenko, Piantadosi, and Gibson (Nature, June 19, 2024):** [Language is primarily a tool for communication rather than thought](https://www.nature.com/articles/s41586-024-07522-w) — evidence and argument about human language and cognition. - **Anthropic (March 27, 2025):** [Tracing the thoughts of a large language model](https://www.anthropic.com/research/tracing-thoughts-language-model) — shared multilingual features and explanation faithfulness.