Most businesses I talk to are running four or five systems that don't speak to each other. An accounting package, something operational, a rostering or booking system, and a set of spreadsheets holding it all together.

Every month somebody exports from each one, copies it into a single sheet, fixes the formatting, chases whichever site sent theirs late, and checks it twice because last time there was an error. That takes weeks, and the finished report describes a month that already closed.

The fix is a data pipeline. But "we'll build you a pipeline" is one of those phrases that means everything and nothing, so here is what it actually involves — and, more usefully, what has to happen after the connecting is done.

Part one: connecting the systems

This is the part most people picture. Your systems are connected once, and from then on the data comes across on its own — usually overnight, so it's ready before anyone's at their desk.

YOUR SYSTEMS Accounting Operational system Rostering / payroll Spreadsheets Site exports The pipeline Collects it every night Cleans and standardises Makes every source mean the same thing YOUR REPORTING Board pack Site comparison Finance & P&L
Connected once, then it runs on its own. No exporting, no copying between files.

The important word in that diagram is standardises. Your accounting system might call something a cost centre, your operational system calls it a department, and two sites spell the same location differently. Until those are reconciled, joining the data together produces confident-looking nonsense.

Connecting systems is a technical job. Agreeing what the numbers mean is a business decision, and it has to be made by your people, not by whoever builds the pipeline.

That's why every build I do starts with sitting down and agreeing what each measure means — one definition, written down and signed off, applied identically everywhere. It's unglamorous and it's the difference between reporting that gets used and reporting that gets argued about.

Part two: the half that usually gets skipped

Here's the problem with automation. Once the numbers appear on their own, people stop checking them — which is the point, but it means when something goes wrong, nobody finds out.

A site changes its export format. A field starts arriving blank. A system stops sending altogether. The report still opens, still looks right, and quietly reports the wrong thing until someone notices months later.

So alongside the pipeline I build a second layer that watches it.

1 Checks every load Did it arrive? Is anything missing or out of range? 2 Traces every number Board pack back to the file it came from 3 Holds back bad figures A known problem stops the number being published All of it in one report your own team opens Updated every time your data comes in — not a document written once and filed
The monitoring layer. Built into the pipeline, so it can't fall out of date.

What that looks like in practice

Say a chart in your board pack is showing a figure that's slightly off. Someone at one site has been leaving a field blank — about one record in twenty — and it's been quietly pulling the number down for three months.

You could have found that in a spreadsheet. But you'd have had to go looking, across a dozen files, knowing in advance what you were looking for. Nobody does that every month.

With the monitoring layer, the chart carries a score. It's flagged, it names the site, and it tells you when it started. And if the problem is serious enough, the figure doesn't reach the board pack at all until it's resolved.

See it working

I've put together an interactive demonstration using synthetic data. Pick a measure, trace it back to the file it came from, and see what happens when a quality rule fails.

Open the demo →

Why this matters more than it sounds

You stop defending numbers. When someone questions a figure in a meeting, you can show where it came from instead of promising to check. Meetings move from arguing about the number to deciding what to do about it.

Problems surface in days, not quarters. A site whose data quality is slipping shows up the week it starts — and that's usually an operational finding about that site, not a technical fault.

An audit becomes a lookup. If a regulator, lender or auditor asks how a reported figure was produced, the answer is a report you open rather than a fortnight of reconstruction.

Your team isn't dependent on me. The monitoring report is theirs. They open it, see what's healthy, see what changed this week, and know whether to trust what they're looking at. That's the point — you shouldn't have to call your consultant to find out whether your own numbers are all right.

The part I'd want you to take away

Governance sounds like a separate project with its own budget line. It isn't, or it shouldn't be. Done properly it's a by-product of building the pipeline carefully in the first place — the checks happen because the pipeline runs, not because someone remembers to do them.

Which also means it doesn't decay. A document describing your data is accurate for about a quarter. A check that runs every night is accurate for as long as the platform is running.

If your monthly reporting still depends on one spreadsheet and one person, that's the conversation I'd want to have.

Zohal Zahir, Founder and Principal Consultant, DATASANJ

Zohal Zahir

Founder & Principal Consultant

I build Microsoft Fabric and Power BI reporting platforms for midsize companies across Australia — connecting your systems, governing the data, and building the reporting on top. Microsoft Certified (DP-600, PL-300), based in Canberra.

Connect on LinkedIn →