Your service portfolio has dark matter

The service catalogue was built as well as it could be. It was never going to be complete, argues Steve Fleming.

Think of a tool nobody owns.

It isn’t in the catalogue. No service owner has their name against it. No lifecycle, no review date, no box on the diagram. But if it stopped working on Monday morning, you’d know by lunchtime.

You didn’t have to think hard.

That tool isn’t an exception. It’s a sample. Every IT organisation runs two portfolios — the one it designed and the one the business actually consumes — and only one of them is written down.

Shadow IT is the wrong name

The word “shadow” does a lot of quiet work. A shadow is cast away from a light, and in that metaphor the light is us. Call something a shadow and you have already passed sentence on it: unsanctioned, uncontrolled, waiting to be brought in.

Ask the people using it. For them there’s nothing shadowy about it. It’s just the thing that works.

Physics has the better term: dark matter. It isn’t hiding — it just emits no light, so the instrument can’t see it. And yet it’s most of the mass, and the system doesn’t hold together without it. Invisible doesn’t mean absent.

Once you’re looking for dark matter rather than shadows, the scope changes. It isn’t only the uncatalogued tools. It’s the service that outgrew its box — the collaboration platform filed under Communications that is also the phone system, the file store and half the development workflow. It’s the workaround that quietly became infrastructure. All of it consumed. None of it visible in the portfolio that gets decided on.

A perfect catalogue would still be incomplete

It’s tempting to call this a maintenance problem. The catalogue drifted, someone stopped updating it, tighten the governance and catch up.

It isn’t.

A catalogue is a hierarchy, and a hierarchy holds one dimension at a time. Every service has at least two legitimate truths: what it does for the people using it, and who owns and runs it. A tree holds one or the other. Never both.

That isn’t a failure of discipline. It’s the shape of the container. Which means the effort — decades of it, in some organisations — was never going to close the gap.

Every decision inherits the blind spot

Retirement is the easiest place to see it. A tool is slated for decommission because its category is covered, and it turns out to be carrying a workflow in a department the programme never had in scope. The portfolio on record is blind to it. The people using it never stopped seeing it.

The pattern is familiar enough to be boring. A shared mailbox a finance team set up years ago — never catalogued, never owned — which is now where every non-standard approval actually gets decided. Retire it and approvals stop.

The safe move is to retire nothing, so the estate only grows. That caution is rational: with only the designed portfolio visible, a dead service and a load-bearing one look identical.

But retirement is only one consequence. The same partial picture underwrites the migration plan, the continuity plan, the critical service list and the investment case. Every one of them inherits it. A dependency that was never a service was never in scope, and the first anyone hears of it is when something stops working in a department nobody thought to consult.

And the stakes are changing. Organisations are now pointing automation and AI agents at the service estate, and agents act on the catalogue. A partial picture used to produce slow, expensive decisions. It is about to produce fast ones.

The evidence was always there. Nobody could read it.

Curation is giving way to discovery

We have watched this happen elsewhere.

Music was once organised by genre and artist — a curated hierarchy, argued over by people who knew it was imperfect, because it was the only structure the shelves could hold. Then streaming could read how people actually listened, and the structure was discovered from consumption rather than assigned in advance. The web was a hand-curated directory until search could infer its shape from how sites linked to one another.

In both cases the curated structure gave way. Nobody went back.

Enterprise IT is still Tower Records. We hand-curate the shelves.

The mess is the signal

The prevailing advice on enterprise AI is to clean up your data first. Fix the taxonomy, then bring in the machines.

For this problem it runs the other way.

Inference doesn’t need reality pre-sorted into a tree. It works on relationships, in many dimensions at once, and finds the structure actually present in evidence that is messy, cross-cutting and contradictory — rather than requiring the evidence to conform before it will look at it.

The messiness that defeated twenty years of curation is the exact condition inference was built for. A service can be five things in five contexts, because nothing forces it into a box at birth.

Portfolio discovery

Dark matter leaves traces everywhere, and the richest ones aren’t operational.

Tickets and telemetry tell you what happened. They don’t tell you what it meant — which tool carried which job, where the workarounds live, what people reached for when the sanctioned route didn’t fit. That sits in what people say about the services they use, in their own words, at scale. Read together, with the human account as the anchor, the consumed portfolio emerges.

This practice needs a name, and it has one.

Portfolio discovery is not portfolio management. Management is the discipline of the designed portfolio — pipeline, catalogue, review, retirement — and it isn’t going anywhere. Discovery is the discipline of the consumed one. It asks a different question of every service: not what did we build, but what is this carrying?

It doesn’t combat shadow IT. It maps dark matter. Nothing consumed is invisible to it.

The questions worth asking

A discovered portfolio is an inference from evidence, so it inherits every question evidence raises. Three are worth putting to anything that claims to produce one.

• Where did the evidence come from?
• Could anyone with an interest in the outcome have shaped it?
• Does the discovery tool self-audit — transparent coverage, confidence, gaps — or does it ask you to believe?

A portfolio discovered from owned operational data alone is the owners’ picture, rearranged. What anchors it is the account of the people the services exist for, gathered independently, at scale, from people with nothing to hide and everything to gain from an honest picture.

That account has a name: the ground truth of the consumed portfolio.

What has changed

The designed portfolio was never a mistake. It was what the tools could hold, built well by people who understood its limits better than anyone.

What has changed is the ability to read the evidence for the other one.

A second discipline is arriving alongside portfolio management — with its own question, its own evidence standard, and one promise the designed portfolio could never make: a picture that includes the work people actually do.

The practitioners now asking what would make a discovered portfolio worth trusting aren’t early to a feature.

They’re founding a practice.