A language model was learning to think in two languages simultaneously somewhere in a research lab attached to the Oak Ridge Summit supercomputer. failing to translate between them. pondering. Hi-NOLIN, a 9-billion-parameter English-Hindi bilingual model developed by the small Indian startup Nolano, relies heavily on this distinction.AI—one of the more covertly ambitious open-source AI initiatives in recent years.
Over 600 million people speak Hindi. It is associated with a vast living culture, a sophisticated grammar, and centuries of literary tradition. However, no open-source large language model had made a serious effort to assist Hindi speakers at a level that English users already take for granted until the advent of Hi-NOLIN. Anyone who pays attention knows the reasons: infrastructure costs, data scarcity, and the unsettling fact that the majority of frontier AI research still revolves around English like a planet around its sun.
Nolano’s strategy was cunning, if not particularly simple. Instead of starting from scratch and training a model on a mixed Hindi-English dataset, which would have been very costly, researchers increased the 7B Pythia architecture’s number of parameters to 9 billion before continuing to pretrain it on a dataset that combined Hindi and English text. The decision to expand to 9B wasn’t made at random; rather, it was optimized to function well across Summit’s unique configuration of six GPUs per node, a minor hardware detail that influenced the design of the entire project. Something about that is illuminating. Even state-of-the-art AI research ignores the physical limitations of the machines it operates on.
However, what happened to Hi-NOLIN’s English performance during Hindi training was truly unexpected. A well-known issue known as catastrophic forgetting had confronted earlier attempts to expand models into new languages or domains (Code LLaMa for coding, LeoLM for German). When a model learns something new, it often forgets what it already knew. Hi-NOLIN avoided that trap by employing strategies like data replay during continuous training and learning rate re-warming. The 9B model learned more than just Hindi. On English benchmarks like TruthfulQA and ARC, it actually began to outperform Pythia’s larger 12B model, simultaneously closing the gap with LLaMa-2. Additionally, the coding benchmarks got better.
However, not everything is as it seems. The model was far from reaching convergence, and the results Nolano released were based on a checkpoint at just 600 billion tokens. Training loss was gradually decreasing, but actual arrival and steady decline are two different things. The model’s potential is still unknown, particularly when it comes to complex Hindi tasks with limited training data. Additionally, there is the more general question of whether a 9B model, no matter how well-trained, can support the entire spectrum of what Hindi speakers genuinely require from AI, including regional dialects, legal documents, and the informal code-mixed Hinglish spoken by more than 350 million people every day.
By the way, one of the more intriguing wrinkles is the Hinglish angle. Without being specifically trained on code-mixed language, Hi-NOLIN demonstrated some capacity to generalize, indicating that bilingual pretraining produces representational overlaps the model can take advantage of. It remains to be seen whether that generalizes robustly, but even a hint of it is important.
Looking more broadly, Hi-NOLIN is part of a much bigger narrative about who is allowed to use AI in their native tongue. Through institutional collaborations and crowdsourced contributions, Bhashini, India’s government-backed multilingual AI project, has been compiling datasets in more than 22 Indian languages. Full-stack Hindi AI systems are being developed by private firms such as Sarvam and Krutrim. Organizations like Masakhane and Lelapa AI are working in parallel for languages that are even more distant from the mainstream of data throughout Africa. Communities are fed up with having to wait for Silicon Valley to take notice of them.
The Hindi AI issue was not resolved by Hi-NOLIN. It most likely never made that claim. However, it proved something that needed to be proven: that you could use open-source tools on hardware that is accessible to the general public to extend a strong English model into a new language without destroying what made it strong in the first place. There are actual restrictions. The model is still in the middle of training, the data is still sparse, and there is still a significant discrepancy between benchmark performance and practical utility. However, the direction seems correct. Sometimes it’s just as important to demonstrate that a path exists as it is to get there.
London Bilingualism's content on health, medicine, and weight loss is solely meant for general educational and informational purposes. This website does not offer any diagnosis, treatment recommendations, or medical advice.
We consistently compile and disseminate the most recent information, findings, and advancements from the medical, health, and weight loss sectors. When content contains opinions, commentary, or viewpoints from professionals, industry leaders, or other people, it is published exactly as it is and reflects those people's opinions rather than London Bilingualism's editorial stance.
We strongly advise all readers to consult a qualified medical professional before acting on any medical, health, dietary, or pharmaceutical information found on this website. Since every person's health situation is different, only a qualified healthcare provider who is familiar with your medical history can offer you advice that is suitable for you.
In a similar vein, any legal, regulatory, or compliance-related information found on this platform is provided solely for informational purposes and should not be used without first obtaining independent legal counsel from a licensed attorney.
You understand and agree that London Bilingualism, its editors, contributors, and affiliated parties are not responsible for any decisions made using the information on this website.
