LLMs have moved faster in the last six months than most engineering teams can track. Here is a clear-eyed breakdown of what changed, what it means, and how to act on it.

Six Months of LLM Progress: What Software Teams Need to Know

If you blinked, you missed three major model releases, two new reasoning paradigms, and at least one capability that was considered impossible eighteen months ago. The large language model space has stopped moving in quarters — it moves in weeks. For software teams and SaaS founders, that pace is both an opportunity and a liability.

Here is what has actually shifted, and what to do about it.

Reasoning Models Became the New Baseline

Until recently, "reasoning" was a marketing word plastered on standard autoregressive models. That changed. A new class of models now dedicates explicit compute to multi-step internal deliberation before producing an answer — sometimes called "thinking tokens" or chain-of-thought scratchpads that run before the visible response.

The practical effect is significant: tasks that previously required careful prompt engineering — complex debugging, multi-constraint planning, structured data extraction — now work more reliably out of the box. For teams building AI-assisted features, this means:

Fewer brittle prompt templates to maintain
Higher accuracy on tasks with more than two logical dependencies
More predictable outputs in agentic workflows

The tradeoff is latency and cost. Reasoning models are slower and more expensive per call. Choosing between a fast standard model and a slower reasoning model is now a real architectural decision, not a default.

Context Windows Crossed a Practical Threshold

Context lengths have technically been "long" for a while. What changed recently is that models have become meaningfully better at using that context rather than just accepting it. Retrieval-augmented generation (RAG) is still useful, but the calculus has shifted. Stuffing an entire codebase, a full PDF contract, or a week of chat history into a single prompt is now a legitimate strategy for many use cases — not just an edge case.

For SaaS products, this opens a design space that did not exist before:

Persistent session context without external vector stores
Document-level reasoning without chunking pipelines
Fewer round trips, simpler architectures

This does not make RAG obsolete. It does mean that teams should re-evaluate when they reach for it reflexively.

Multimodality Is Now Table Stakes

Text-in, text-out is no longer the default assumption. Leading models now handle image input, audio, and in some cases video frames as first-class inputs. This matters for product builders in ways that go beyond novelty demos.

Consider what this enables:

Mobile apps that let users photograph a form, receipt, or product label and get structured data back
Customer support flows that process screenshots directly
Accessibility tooling that describes UI states for screen-reader users

Teams in Ghana and across West Africa building for markets where documents are often photographed rather than typed — think mobile money screenshots, handwritten forms, physical receipts — have a particularly concrete reason to pay attention here.

The Open-Source Gap Narrowed Considerably

Twelve months ago, the gap between frontier proprietary models and the best open-weight alternatives was wide enough to make the choice obvious for most production workloads. That gap has narrowed. Several open-weight models now match or exceed older frontier checkpoints on standard benchmarks, and more importantly, on practical coding and instruction-following tasks.

What this means in practice:

Decision framework (simplified):
- Latency-sensitive, cost-sensitive, data-private → open-weight, self-hosted
- Complex reasoning, max accuracy required → frontier API
- Hybrid: route by task complexity at runtime

For teams with compliance requirements or customers in regulated industries, viable self-hosted options are no longer a compromise. They are a credible first choice.

Agents Moved From Demo to Production (Carefully)

Agentic systems — LLMs that call tools, browse the web, write and execute code, and loop over results — went from impressive conference demos to cautious production deployments. The word "cautious" is load-bearing. Teams that shipped agents quickly discovered that non-determinism compounds across steps. A 95% reliable single-step model becomes a 77% reliable three-step agent and a 59% reliable five-step agent, assuming independent errors.

The teams seeing real production value from agents share a few patterns:

Narrow scope: agents that do one class of task, not general assistants
Human checkpoints: approvals or reviews inserted at high-stakes steps
Structured outputs: JSON-constrained responses at every tool-call boundary to prevent format drift
Observability first: logging every prompt, tool call, and response before optimising anything else

Agents are not magic pipelines. They are distributed systems with an LLM in the loop, and they require the same discipline as any other distributed system.

What Has Not Changed

Amid the acceleration, a few things remain stubbornly constant. Models still hallucinate. They still fail on precise arithmetic without tool use. They still reflect the biases and gaps of their training data. Any product architecture that assumes otherwise will eventually surface that assumption in a production incident.

Evaluation — building rigorous test sets for your specific use case — remains the most underinvested practice in AI product development. The models are improving faster than most teams' ability to measure that improvement in ways that are meaningful for their product.

Why This Matters for Your Project

If you are building or scaling a software product today, the LLM landscape of six months ago is already a poor guide to what is possible and what is cost-effective. Decisions about model selection, architecture, and data privacy that seemed settled may be worth revisiting. The teams that stay current — not by chasing every release, but by re-evaluating their assumptions on a quarterly cadence — will compound that advantage into real product differentiation. At Code!nk Technologies, this is the lens we apply when scoping AI features for clients: not what models can do in a demo, but what they can do reliably, affordably, and safely in a production environment that real users depend on.

Source: Simon Willison — https://simonwillison.net/2026/May/19/5-minute-llms/ (via Hacker News)