A live bug in dead code

There is a failure mode worse than a tool that cannot see a bug: a tool that sees the bug, flags it correctly, and then quietly deletes its own finding. That is what happened here. MUGEN read VLC's network parser, spotted the dangerous read exactly where the real vulnerability lives, flagged it with high confidence, and then discarded it, because a reachability check had decided that nothing in the program could ever call it. It was wrong. The bug wasn't invisible because the tool couldn't see it. It was invisible because the tool had convinced itself it was dead code.

What the bug was

CVE-2025-51602 is an out-of-bounds read in mmstu.c, the part of VLC that speaks MMS, an old Microsoft streaming protocol, inside libaccess_mms_plugin.dll. VLC parses an MMS stream straight off the network, so the attacker is not a local user but a server: connect a victim's VLC to a malicious MMS source, send a crafted 0x01 response, and the parser reads past the end of its buffer. The consequences are a crash and a possible leak of adjacent memory, not code execution. It is a medium-severity bug, CVSS 4.8, fixed in VLC 3.0.22. None of that is the interesting part. The interesting part is what MUGEN did with it.

MUGEN found the sink, and I mean found it

This is not an "almost" or a "near the right neighbourhood." On the vulnerable 3.0.7 build, MUGEN's ordinary pipeline landed on the exact recv that pulls the attacker's bytes off the socket, tagged it a HIGH-confidence sink, and got every attribute right.

// libaccess_mms_plugin.dll 3.0.7: the sink MUGEN flagged, correctly
FUN_140004960   recv(sock, buf, len, flags)
    size_is_constant   = false      // the length is not a fixed number...
    content_control    = DIRECT_INPUT  // ...it is controlled by the attacker
    z3_verdict         = SAT          // solver calls the sink shape feasible, not proof of an over-read
    tainted_params     = [0, 1]

An attacker-controlled, unbounded read off the network, in the function the CVE is about, marked SAT by the solver. The tool had the sink, exactly right. So why did it never reach a reviewer?

Thrown away as dead code

The finding never reached the Golden queue, the short list of candidates that actually gets reviewed. Not because its confidence was low. Because it had been marked unreachable. Of the 620 dangerous calls MUGEN extracted from the plugin, 551 of them, 89%, were "dead-linked": flagged as code nothing could ever call, and demoted out of sight. The single most dangerous sink in the binary was sitting in the pile the tool had written off as unreachable.

A door with no visible hinge

Why did nine calls in ten look unreachable? Because of how a VLC plugin is wired. A plugin exposes exactly one function to the outside, an export named vlc_entry__*, and it registers everything else, every handler that does the actual work, by handing VLC's core a function pointer through an indirect call:

// how a VLC plugin registers a handler, no direct call, no pointer table
(*core)(module, cookie, 0x104, ptr, FUN_handler);
//                                        ^^^^^^^^^^^ passed as data, dispatched later

There is no direct call from the entry point to any handler, and no function-pointer table sitting in the data section to walk. So to Ghidra, and therefore to MUGEN, every handler is an island: NO_DIRECT_CALLERS, data_ref_count = 0, apparently referenced by nothing. The entire parser is reachable at runtime, through a seam a disassembler simply cannot draw. The reachability heuristic saw islands and did the "sensible" thing: marked them dead.

Teaching it to see the hinge

The fix in v0.24.17 is a small detector, about 240 lines. It recognises the vlc_entry__* export by name, reads the export's own decompiled body to pull out the handler function-pointer constants it hands to the core, and then walks the call graph forward from those handlers to rebuild the reachability MUGEN had been missing, un-demoting everything in the closure. (One careful detail: it blocklists the compiler's canary helpers, __stack_chk_fail and friends, so the closure doesn't balloon to swallow every guarded function in the DLL.) On the plugin, that closure is 152 functions.

The effect is exactly what you'd hope. Dead-linked calls drop from 551 of 620 to 14 of 620, 89% down to 2.3%. And the vulnerable recv, which had no place in the queue at all, surfaces at Golden slot 3. From invisible to the third thing a reviewer would look at.

Honest about what it can't do, measured about what it can

Three things need saying before this reads as a clean victory, because two of them are limits.

It is precise. "Trust an export whose name matches a pattern" sounds like the kind of heuristic that fires everywhere and floods the queue with junk. It doesn't. Across the 288 binaries in the corpus, the vlc_entry__ marker appears in exactly 3 functions, all of them real VLC plugins. Zero false positives on anything that isn't VLC. It is also deterministic: two runs produce the byte-identical 152-function closure.

It cannot tell the bug from its fix. This is the honest limit, and it is the same one that shows up every time a static tool matches a shape instead of proving a path. Run the identical detector on the patched 3.0.22 build and the recv still lands at Golden slot 3. At the solver level the verdict is shape-identical across 3.0.7, 3.0.22, and 3.0.23: the same four SAT parameters, the same threshold, the same tainted indices, zero parameter verdicts got any safer after the fix. The patch is a boundary check MUGEN's extractor never models, so it is invisible to the analysis. What this detector recovers is reachability, not proof. It points a reviewer at exactly the right dangerous code; it cannot tell them whether this particular build still has the bug. That is the reviewer's next move, not the tool's. It is the same discipline as afd.sys: surfaced is not the same as found.

But the ranking is real, and we measured it. If the tool cannot confirm a single CVE, a fair question is whether its ordering means anything at all, or whether "slot 3" is just noise wearing a number. So we checked. Take the patched and unpatched builds, diff them function by function with BinDiff, and ask: of the functions MUGEN ranked in its top 30, how many did VLC's developers actually rewrite? Across the whole binary, 36 of 205 matched functions, 17.6%, were substantially changed; a random handful of 30 would be expected to catch about 5 of them. MUGEN's top 30 held 13, about two and a half times the chance rate, at p ≈ 0.008. That is not "MUGEN found 13 CVEs," and I will not dress it up as one. It is something quieter and more trustworthy: proof that the tool's ranking is meaningfully aligned with the code VLC actually changed, not sprayed at random.

Not just this bug

The blind spot was never VLC-specific, and neither is the approach that fixes it. The same detector, pointed at other VLC plugins, drags a whole class of previously-invisible code back into view: on libmp4_plugin.dll it reshuffles 12 of 16 Golden slots toward plugin code, evicting a cluster of C-runtime noise that had been squatting at the top; on the much larger libmkv_plugin.dll, 15,403 dangerous calls, dead-linked falls from 69% to 53%, though on a plugin that large the reshuffle evicts as many old queue slots as it adds. Everywhere a plugin registers its handlers through that indirect hinge, MUGEN had been throwing the real work away.

The bug you can't see because you decided it's unreachable

There are two ways a static tool goes blind. One is simple and, in its way, honest: it cannot read the code. The bytes are encrypted, or obfuscated, or in a function too large to decompile, and the tool fails where you can see it fail. The other is worse. The tool reads the code perfectly, flags the bug correctly, and then throws its own finding away because a reachability heuristic decided nothing calls it. That failure is confident and it is silent. It had the finding and buried it, and nothing in its output tells you it did.

Modern software is full of the seams that cause this: plugin systems, callback registration, message pumps, virtual dispatch, anywhere a function pointer travels as data and gets called later. In all of them, the call graph a disassembler can draw is a fraction of the call graph that actually runs, and a finder that trusts only the edges it can see will keep burying real bugs in its own dead code. The work here wasn't finding the dangerous code. MUGEN already had the sink, correctly, the whole time. The work was teaching it to stop hiding its own finding. That is the flip side of the month it kept inventing bugs: there the tool believed its own noise; here it disbelieved its own signal. Both are the same discipline, learned from opposite ends, of getting a tool to trust exactly what it should and nothing more.