Most LLMs fail at Twi, Yoruba, and Hausa. Here is a practical guide for developers on fine-tuning Mistral 7B on African-language data using LoRA adapters and low-resource NLP evaluation techniques.

Fine-Tuning a Small LLM for African Language Support

Ask GPT-4 to translate a proverb from Twi, and you will likely get a confident response that is grammatically plausible and culturally wrong. Ask it to complete a sentence in Hausa or generate customer-support responses in Yoruba, and results vary from mediocre to unusable. This is not a minor gap — it is a structural problem rooted in training data imbalance. English, Spanish, and Mandarin dominate the corpora that large language models are trained on. African languages, spoken by over a billion people, are barely a footnote.

The good news: you do not need GPT-4-scale resources to fix this for your specific use case. A compact open-source model like Mistral 7B, fine-tuned on a well-prepared African-language dataset, can outperform much larger general-purpose models on targeted tasks. This article walks through how to do that — from data collection to deployment-ready adapters.

Why General-Purpose LLMs Struggle With Low-Resource Languages

The term "low-resource language" does not mean the language is simple or less expressive. It means there is limited digitised text available for training. Twi, for instance, is spoken by roughly 10 million people in Ghana, but indexed Twi text online is a fraction of what exists for Dutch, a language spoken by a comparable number of people.

When an LLM encounters a low-resource language during training, it learns surface patterns — borrowed words, code-switched phrases, partial grammar — without building the deeper semantic representations it develops for high-resource languages. The result is a model that hallucinates fluently.

Fine-tuning on a focused, high-quality dataset corrects this at a fraction of the compute cost of pretraining.

Step 1: Data Preparation — The Most Important Step Nobody Talks About Enough

Your fine-tuning dataset is the ceiling of your model's performance. For African languages, sourcing quality data requires deliberate effort.

Where to source data:

JW300 and Opus corpora — contain parallel text in many African languages, though domain coverage is narrow (religious texts)
Masakhane — a community-driven NLP project with datasets covering 50+ African languages, including Yoruba, Hausa, Igbo, Amharic, and Swahili
AfriSenti — sentiment-annotated Twitter data for 14 African languages, useful for classification tasks
Custom collection — scraping community forums, digitising printed materials, or partnering with local NGOs and media organisations

Structuring your dataset for instruction fine-tuning:

Format your data as instruction-response pairs. For a customer-support bot in Twi, each record might look like this:

{
  "instruction": "Fa message yi na tua so wɔ Twi kasa mu.",
  "input": "Your order has been shipped and will arrive in 3 days.",
  "output": "Wʼadefudeɛ no ayi afiri hɔ na ɛbɛduru wo nkyɛn nnansa mu."
}

Clean aggressively. Remove duplicates, machine-translated noise, and code-switched sentences that would confuse the model about language boundaries. Aim for at least 5,000 high-quality pairs for a narrow task, and 50,000+ for general conversational fluency.

Step 2: Choosing Your Base Model

Mistral 7B is a strong default for this task. It is compact enough to fine-tune on a single A100 GPU, has strong multilingual pretraining priors, and the open weights give you full control. Alternatives worth considering:

Aya-23 (Cohere for AI) — pretrained with multilingual intent and strong African language coverage
BLOOM-7B — designed explicitly for multilingual use, though performance on downstream tasks can lag Mistral
Llama 3.1 8B — competitive with Mistral and better documented for LoRA fine-tuning pipelines

For most teams building products in West Africa, Mistral 7B is the practical starting point.

Step 3: Fine-Tuning With LoRA Adapters

Full fine-tuning of a 7B parameter model requires significant compute and risks catastrophic forgetting — where the model loses general reasoning ability while learning the new language patterns. Parameter-efficient fine-tuning with LoRA (Low-Rank Adaptation) sidesteps both problems.

LoRA injects trainable rank-decomposition matrices into the attention layers of the transformer. You update roughly 1-3% of the total parameters, preserving the base model's knowledge while teaching it new language behaviour.

A standard LoRA configuration for this task:

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,                        # rank of the update matrices
    lora_alpha=32,               # scaling factor
    target_modules=["q_proj", "v_proj"],  # which attention layers to adapt
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(base_model, lora_config)
model.print_trainable_parameters()
# trainable params: ~20M || all params: ~7B || trainable%: ~0.28%

Use QLoRA (quantised LoRA) if you are working on a single consumer GPU. Loading the base model in 4-bit precision with bitsandbytes reduces VRAM requirements to under 12GB without meaningful quality loss for most African-language tasks.

Train with a cosine learning rate schedule, a peak LR of 2e-4, and a batch size of 4-8 depending on sequence length. Monitor validation loss closely — African-language datasets are often small enough that overfitting occurs within 2-3 epochs.

Step 4: Evaluation Metrics That Actually Matter

Standard benchmarks like MMLU or HellaSwag are useless for evaluating African-language fine-tunes. You need metrics grounded in what the model will actually do.

For translation and generation tasks:

ChrF++ — character n-gram F-score that handles morphologically rich languages better than BLEU. Twi and Yoruba have complex verb morphology that BLEU systematically under-rewards.
COMET — a neural metric trained on human judgements, more robust on low-resource pairs than n-gram methods.

For classification tasks (sentiment, intent detection):

Standard F1, but disaggregated by dialect and domain. A model that scores 85% average can still fail on specific regional variants.

Human evaluation is non-negotiable. Build a small evaluation panel of native speakers — even 5-10 reviewers providing fluency and adequacy ratings will surface errors that automated metrics miss entirely. This is especially important for tonal languages like Twi and Yoruba, where small errors can shift meaning significantly.

Common Failure Modes to Watch For

Code-switching collapse: the model starts defaulting to English mid-generation, especially for technical terms. Mitigate by including code-switched training examples and adding a language-consistency reward if using RLHF.
Transliteration errors: character-level noise from inconsistent orthographic conventions in source data. Normalise your data with a language-specific pre-processing script before training.
Cultural hallucination: the model generates grammatically correct but contextually inappropriate responses. Only human reviewers catch this reliably.

Deployment Considerations

Once your adapter is trained, merge it back into the base model weights for clean deployment, or serve it dynamically using the PEFT library for multi-language adapter switching. For SaaS products serving multiple languages, a single Mistral base with swappable LoRA adapters per language is architecturally cleaner than maintaining separate models.

Quantise to GGUF format with llama.cpp for CPU inference if you are targeting edge or mobile deployments in low-connectivity markets — a realistic constraint for many African product teams.

Why This Matters for Your Project

If you are building a product for African users — a fintech chatbot, a health information service, an agricultural advisory tool — language is not a feature you can defer. Defaulting to English excludes the majority of your potential users and signals that the product was not designed for them. Fine-tuning a compact LLM on your target language, even with a few thousand quality examples, is achievable in days, not months. The tooling has matured, the open-source models are capable, and the competitive advantage of a product that actually speaks your users' language is substantial. The gap in the market is real — closing it starts with the data.