By Luis Lopez, AI transportation consultant, CEO of Go Hub.io Holdings Corp and subsidiaries, and host of the Freight Guru Podcast
When people talk about AI in freight, they usually mean the biggest, most capable language models. Those models are impressive, but they are not always the right tool. A lot of logistics work is narrow and repetitive: sorting emails, pulling fields from documents, tagging exceptions. For that kind of work, small language models can be faster, cheaper and easier to control.
This article explains what small language models are in plain terms, which logistics tasks suit them, the trade-offs involved and how to decide when a large model is worth it.
Small Versus Large Language Models, in Plain Terms
A language model is software trained on large amounts of text to predict and generate language. Its size is usually described by the number of parameters, which are the internal values the model learned during training.
- Large models have a very large number of parameters. They are good at open-ended reasoning, writing, handling unfamiliar situations and following complex instructions. They typically run on powerful cloud hardware.
- Small models have far fewer parameters. They are less capable at broad reasoning, but they can be very good at a specific, well-defined task, especially after being trained further on examples of that task.
There is no sharp line between small and large. What matters is matching the model to the job.
Tasks That Suit Small Models
Small models do well when the task has a clear input, a clear output and plenty of examples. In logistics, that covers a lot of ground.
Classification
Deciding which bucket something belongs in: is this email a quote request, a status check, an invoice question or a claim? Is this exception a delay, a damage report or a documentation problem? Classification is one of the most common jobs in freight operations and one of the best fits for small models.
Extraction
Pulling specific fields out of text or documents, such as reference numbers, dates, weights, origins and destinations. When the set of fields is known and the documents are reasonably consistent, a small model can extract them reliably.
Routing Emails
Combining classification and extraction, a small model can read an incoming email, decide what it is about, pull the key references and send it to the right queue or person. Our article on AI email triage in freight operations covers this workflow in more depth.
Other Good Fits
- Normalizing location names and addresses into a standard format.
- Tagging shipment notes by topic.
- Detecting the language of a message before routing it.
- Flagging messages that need urgent attention.
The Trade-Offs: Cost, Latency and Privacy
Cost
Running a smaller model generally takes less computing power per request. For high-volume tasks, like classifying every inbound email or document, that difference adds up. It can be the difference between a workflow that makes financial sense and one that does not.
Latency
Smaller models usually respond faster. When a dispatcher is waiting on a screen or a document is being processed at the dock, speed matters.
Privacy and Control
Because small models need less hardware, they can often run on your own servers or on devices rather than through an outside service. That can keep sensitive shipment, customer and rate data inside your environment. It does not remove the need for security. It changes where the security responsibility sits.
Running On-Premises or On Devices
In general terms, a small model can run on a company server, a workstation or, in some cases, a device in the field such as a tablet or an in-cab unit. Our piece on on-device and edge AI in trucking discusses that side of the picture.
Running models yourself brings responsibilities: maintaining the hardware, updating the model, monitoring performance and handling failures. For a smaller operation, a hosted small model may be the practical middle ground.
Fine-Tuning on Your Own Data
Fine-tuning means taking an existing model and training it further on examples from your business. For a small model, this is often what turns a general tool into a dependable specialist.
For example, a model fine-tuned on a few thousand of your own emails, each labeled with the correct category, will usually learn your terminology, your customers’ habits and your internal categories far better than a general model given only instructions.
What makes fine-tuning work:
- Clear, written definitions for each label or field.
- Examples labeled consistently by people who know the work.
- Enough examples of rare but important cases, not just the common ones.
- A separate set of examples held back for testing.
Fine-tuning on poor or inconsistent labels teaches the model your mistakes.
When Large Models Are Worth It
Small models are not always the answer. Large models are often worth the extra cost and latency when:
- The task is open-ended, like drafting a detailed customer response or summarizing a complicated claim file.
- Inputs vary widely and you cannot collect enough examples of each type.
- The task needs reasoning across several documents or steps.
- Volume is low enough that per-request cost does not matter much.
Many practical systems use both. A small model handles the high-volume sorting and extraction, and passes the harder cases to a large model or a person. Some decisions are better handled by plain rules than by any model; our article on LLMs versus rules engines for freight decisions covers that choice.
Evaluation: How to Know It Works
As an AI transportation consultant, the first thing I look at before choosing between a small and large model is how the team plans to measure results. Without a test, the choice is a guess.
A simple evaluation process:
- Build a test set of real examples with correct answers, labeled by experienced staff.
- Run each candidate model on the same test set.
- Measure accuracy overall and on the cases that matter most, such as urgent emails or claims.
- Look at the errors, not just the score. Some mistakes are harmless; others are costly.
- Re-test periodically, because your business and your data change.
If a small model scores close to a large one on your test set, the small model is often the better operational choice.
The Bottom Line
Small language models are not a lesser version of AI. They are a different tool, well suited to the narrow, high-volume tasks that fill a freight operation’s day. Start by listing your repetitive text tasks, collect good examples and test before you commit.
For more conversations on freight and AI, subscribe to the Freight Guru Podcast.
About the author: Luis Lopez is a Miami-based AI transportation consultant and logistics entrepreneur, the CEO of Go Hub.io Holdings Corp and subsidiaries, and host of the Freight Guru Podcast.
