Managing ONU Firmware Across a PON Fleet: Baseline, Waves and a Version Register
Key takeaways: firmware drift is one of the most under-diagnosed fault sources on a PON. When three or four builds are running on the same OLT, symptoms appear at random, engineers cannot reproduce them on site, and nobody can roll back with confidence because nobody knows what was there before. What follows is a working method: define a version baseline first, roll out in three waves at 20% / 50% / 100%, and keep version, location, timestamp and owner in one register. It is not about how to click "upgrade" — it is about keeping three thousand ONUs under control.
1. Version drift hides better than a dead port
A typical ticket looks like this. Three buildings in one complex, the same purchase batch of ONUs. A resident in building two says channel changes on IPTV take five seconds. Someone in building five says the phone can hear but cannot be heard. The third building reports nothing at all. An engineer reboots the affected units, everything works for two days, then the same complaint comes back. A new ONU fixes it, so the case is closed as "a few bad units" — except what actually changed was the model and the firmware. The problem was covered up, not solved.
PON makes this hard because there are so many layers. Optical power is fine. Registration is fine. The subscriber dials up fine. Everything looks healthy, while the actual fault sits in OMCI modelling, multicast join behaviour, or codec negotiation inside the voice stack — layers nobody looks at during a first visit. And those layers are exactly what the firmware defines.
The real blocker is visibility. Most management views show online state, optical power and last up/down time. They do not show the firmware build. Without a version view, troubleshooting turns into guesswork, and guesswork does not belong in an acceptance report or in a conversation with a customer.
Drift is rarely caused by one decision. It accumulates from ordinary habits: replacement stock from a different batch, a failed unit swapped out with whatever build the supplier shipped that month, an installer flashing anything that registers so the service can be turned up before the end of the day. Three months later one PON carries devices that behave in three different ways, and no one remembers which action caused it.
2. The first step is a baseline, not an upgrade
The instinct on most projects is to push everything to the newest build. That is also the fastest way to create an outage. An upgrade only makes sense once you know what the target build is and why it is the target. Split builds into three tiers and write the tiers down:
- Stable — the fleet-wide build. Nothing changes it except security fixes. It should cover 90% or more of the estate.
- Pilot — a build under evaluation, allowed only in named buildings or a named batch.
- Hotfix — a build issued to close one known defect. It retires as soon as the defect is closed.
Then build the matrix: ONU model and chipset as rows, builds as columns, each cell carrying a state (stable / pilot / blocked) and the services it is approved for. It does not need to be elegant, but it does need to exist and it needs to answer the question a field engineer actually asks — "which build goes on this unit?"
Two rules hold the whole thing together. Never run more than two builds under the same splitter, because fault isolation loses its reference point. And never let a new build enter the network without a pilot, regardless of what the supplier promises.
Keep the matrix where engineers can actually reach it — a shared sheet linked from the OLT landing page beats a PDF in somebody's mailbox. And make the rule enforceable: if a build is not in the matrix, it is not approved, and that answer should not require phoning anyone at nine in the evening.
3. Five checks before you touch anything
- Model and chipset. The same enclosure does not mean the same silicon. Confirm with the equipment ID and vendor ID reported over OMCI, and exclude every unit the package does not match. "Let's just try it" is how bricked units happen.
- Optical margin. Weak links drop out mid-upgrade and stall in a half-written state. Any unit below −27 dBm gets its fibre fixed first; upgrading it is a later conversation.
- Power. Losing power during a flash is the single most common cause of a bricked ONU. Confirm UPS runtime at the headend, and confirm nobody has a power cut scheduled in the riser.
- Service window. Avoid the 20:00–23:00 peak and avoid hotel check-in and check-out. 02:00–05:00 is the only window that is broadly safe.
- Rollback image. Confirm dual-bank (A/B) support on the ONU, and confirm the rollback package and the rollback steps are written down. No rollback package, no batch.
4. Three delivery paths, three different boundaries
Choosing the wrong path is worse than not upgrading at all. All three work, but they must not be mixed inside one batch, or the failure cause becomes impossible to attribute.
| Path | Reach | Precondition | Best fit |
|---|---|---|---|
| TR-069 / ACS | Across OLTs and sites | ONU supports TR-069 and is already under management | Fleet-wide alignment, policy-driven provisioning |
| OLT-side OMCI push | Every ONU behind one OLT | Same vendor, consistent OMCI modelling | Large batches at a single site, shortest wall-clock time |
| Local web / USB | One to a few dozen units | Someone is physically there | Recovery, late installs, stubborn single units |
The combination that works in practice: OMCI carries the bulk, ACS fills gaps across sites and handles later audits, local access closes the tail. Keep the execution logs for each path separate — merged logs destroy the audit trail you built them for.
Record the path per unit, not just per batch. When one wave produces three failures and twenty successes on the same OLT, the delivery path is often the only variable that differs, and without that field you will spend a day trying to prove it.
5. Wave it: 20% → 50% → 100%
A single nationwide push turns your entire subscriber base into a test sample. Fix the three-wave pattern, and give every wave an explicit release condition and rollback trigger.
| Wave | Share | Observation | Release condition |
|---|---|---|---|
| Wave 1 | 20% | 24 hours | At least 5% of ports on every OLT covered, four metrics unchanged |
| Wave 2 | 50% | 24–48 hours | Same, plus no clustered complaints |
| Wave 3 | 100% | 72 hours | Version distribution report issued after completion |
Watch four numbers: registration success rate, optical power distribution (did the whole population shift down?), disconnects per unit time, and complaint volume. If any of them degrades, stop the wave and check whether the package matches the model before blaming hardware.
A failure rate above 2% means stop immediately. That threshold is not negotiable. Stopping is not a failed rollout; forcing it through is.
One more discipline: do not change rate-limit templates, VLAN plans or authentication method in the same window. Move one variable at a time, so that when something degrades you know whether it was the build or the config. Almost every team learns this the expensive way once.
6. What to do when an upgrade fails
The most common mistake on site is retrying immediately. Three retries later the logs are overwritten and every clue is gone. Follow a fixed order instead:
- Single unit fails. Retry once. If it fails again, roll back to the previous build, record the serial number and the symptom, and stop digging on site.
- Unit is bricked (no registration, no OMCI response). Schedule a physical replacement. Do not burn remote windows retrying it.
- Whole batch fails. Stop the wave and check three things in order: does the package actually match the model, is optical power across this batch low, and has the OLT hit its OMCI session limit?
- Capture a failure snapshot. Serial number, RX/TX power at the time, the step it failed on, and the error code. Those four fields are worth more than any verbal description.
7. Put the version in a register, not in someone's head
Version management always ends up as a table. It does not need many columns, but it must answer three questions: where is this unit, what build is on it, and who touched it when.
- ONU serial number (GPON SN)
- Location: building – riser – unit number, bound to the circuit-id already used for per-subscriber authentication, so location is never entered twice
- Model, chipset, current build, previous build
- Last upgrade time, owner, result (success / failure / rolled back)
The important design decision: an upgrade may only change three columns — build, time, result. The more fields a process requires, the less often the register is actually maintained, until it becomes an outdated document. Run a distribution report monthly and produce three lists: share by build, units on unexpected builds, units due for upgrade.
Decide who maintains it before deciding how it looks. The reliable pattern is to attach the register to commissioning: no register entry, no sign-off. Registers back-filled by operations a month later are already wrong.
8. Eight acceptance checks
- The version matrix is published, and no more than two builds are live across the network.
- Every OLT has a rollback package and written rollback steps for its ONU models.
- Wave records are complete: share, timing, release decision, and stop reason if any.
- Weak-link units below −27 dBm were cleared before the upgrade started.
- The service window was confirmed in writing with the business owner; hotel projects also notify front desk and housekeeping.
- Every failed unit has a serial number, symptom and disposition on record.
- The register exports by building and can be handed straight to a field engineer.
- A monthly version distribution report exists and feeds next quarter's purchasing and spare-parts decisions.
Final thoughts
Flashing firmware is not difficult. What is difficult is that it crosses four teams — procurement, installation, operations and customer service — and a missing record in any one of them turns the whole exercise back into "reboot it when it breaks". Treat the build as an asset rather than an action: baseline first, waves second, register last. Get those three right and three thousand ONUs are just a spreadsheet.
Contact us
If you are selecting ONUs, rolling out at scale, or trying to make sense of the builds already running in a brownfield network, send us what you have. We are happy to work through the version matrix with you.
- Email: zhenghui.xu@sczhgx.com
- WhatsApp: +86 17612849559
- Online store: shop.sczhgx.com