Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion electron/native/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ Encoder selection: by default the helper keeps the existing sink-writer path fir

Frame input path: the helper feeds the encoder from the GPU when it can. On that path it copies the WGC frame across a keyed-mutex bridge to a second D3D11 device, converts BGRA to NV12 with the D3D11 video processor, and submits an allocator-owned DXGI sample to the hardware H.264 encoder, so no frame ever passes through system memory. The alternative is the original path: a staging texture, `Map(D3D11_MAP_READ)`, and a row-by-row copy into an `IMFMediaBuffer` — which is where a driver stall costs a recording (issue #252). The GPU path is a preference, never a requirement: it is skipped outright for `preferSoftwareEncoder` and for inline webcam PiP (both need the frame in system memory), and it degrades to the CPU path on its own if the encoding device, the NV12 video processor, the shared bridge texture, the DXGI sample allocator, or the hardware sink writer is unavailable. The GPU path is OFF by default: it fixed #252 on the machine that reproduces it and broke recording outright in #336, and its fallbacks only cover failures during `initialize()`, not one that appears once frames are flowing. Set `OPENSCREEN_WGC_ENABLE_DXGI_INPUT=1` to turn it on. Because the two paths land on different encoders and hardware MFTs default to constant bitrate, the GPU path asks for VBR through `ICodecAPI`; without it a static screen spends the full configured budget (measured 16.9 Mbps against 1.95 for the same desktop).

The helper reports the outcome through the `encoder-selection` stdout event (`video` is `default`, `software-preferred`, or `software-fallback`; `videoInput` is `dxgi-nv12` or `cpu-rgb32`; `container` is `fragmented-mp4` or `mp4`; all three report what the encoder settled on rather than what was asked for). On the GPU path the helper also prints one `[frame-drops] gpu_bridge_contended=<n>` line to stderr at stop: a frame the bridge was too busy to take is skipped rather than failing the recording, and a large count there is the first thing to look at in a report about missing frames. When the app sees `software-fallback` — the default encoder failed and the helper switched on its own — it shows a small dismissible notice in the recording HUD with a "Don't show again" option, because software encoding can raise CPU usage. An explicit `software-preferred` selection shows no notice, and the event stays available for diagnostics either way.
The helper reports the outcome through the `encoder-selection` stdout event (`video` is `default`, `software-preferred`, or `software-fallback`; `videoInput` is `dxgi-nv12` or `cpu-rgb32`; `container` is `fragmented-mp4` or `mp4`; all three report what the encoder settled on rather than what was asked for). On the GPU path the helper also prints one `[frame-drops] gpu_bridge_contended=<n>` line to stderr at stop: a frame the bridge was too busy to take is skipped rather than failing the recording, and a large count there is the first thing to look at in a report about missing frames. When the app sees `software-fallback` — the default encoder failed and the helper switched on its own — it shows a small dismissible notice in the recording HUD with a "Don't show again" option, because software encoding can raise CPU usage. An explicit `software-preferred` selection shows no notice, and the event stays available for diagnostics either way.

At startup the helper also emits `capture-adapter`, naming the GPU its D3D device landed on and the one actually driving the captured display, each with its LUID, plus one `[adapters]` line per enumerated adapter on stderr. `createD3DDevice` asks for the *default* adapter and nothing checks that it is the one driving the display; when they differ every frame crosses an adapter boundary before the caller touches it. The LUIDs are there because the descriptions are not enough to tell: an IddCx virtual display driver renders through the physical GPU and inherits its description string while being a separate DXGI adapter, so the configuration this diagnostic exists to catch is precisely the one where both names are identical and only the LUIDs differ (measured: `NVIDIA Quadro RTX 4000` at LUID `0:24084` driving the display, the same string at `0:12889146` for the virtual adapter). `monitorLookup` says which of three things happened: `ok`, `no-output-claims-it` (the enumeration finished and nothing owns the captured monitor, which is what an active virtual display looks like), or `unavailable` (`EnumOutputs` refused, as it does in session 0 — the outputs were never inspected, so the absence means nothing about the hardware).

Encoder diagnostic on final sink-writer failure: when the final sink-writer attempt fails (`MFCreateSinkWriterFromMediaSink` on the fragmented container, `MFCreateSinkWriterFromURL` on the plain one; the message names which), the helper logs the registered H.264 video encoder MFT count (via `MFTEnumEx`), the registered AAC encoder count when audio was requested, and the hex HRESULT. If no H.264 encoder is registered, it additionally emits the four-bullet actionable error (missing Media Feature Pack / GPU driver registration / empty `HKLM:\SOFTWARE\Microsoft\Windows Media Foundation\Transforms` / reboot). If an H.264 encoder IS registered but the sink writer still failed, it logs a hint pointing at invalid output path, missing MP4 mux, or GPU driver incompatibility. There is still no fail-fast pre-flight gate because `MFTEnumEx` and the sink writer can disagree about which H.264 encoders are available in non-interactive / Session 0 contexts.

Expand Down
114 changes: 101 additions & 13 deletions electron/native/wgc-capture/src/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,12 @@ std::string jsonEscape(const std::string& value) {
return result;
}

// HighPart:LowPart, matching how Windows tooling and QueryDisplayConfig traces
// spell a LUID, so a value from a bug report can be grepped against them.
std::string formatLuid(const LUID& luid) {
return std::to_string(luid.HighPart) + ":" + std::to_string(luid.LowPart);
}

// Reports which GPU the capture device landed on, and which one actually drives
// the monitor being captured.
//
Expand Down Expand Up @@ -237,24 +243,41 @@ void reportCaptureAdapters(ID3D11Device* device, HMONITOR targetMonitor) {
}

std::wstring monitorAdapterName;
std::string monitorAdapterLuid;
bool monitorAdapterFound = false;
bool sameAdapter = false;
// Both loops end on FAILED(), not on DXGI_ERROR_NOT_FOUND specifically.
// Set when EnumOutputs says the outputs could not be looked at, rather than
// that there are none. The two are different answers and the event reports
// them differently -- see the comment on the inner loop.
bool enumerationUnavailable = false;
// The loops end on FAILED(), not on DXGI_ERROR_NOT_FOUND specifically.
// NOT_FOUND is itself a failure code, so one test covers the normal end of
// the enumeration and every other way it can stop -- and the other ways are
// what matter here. EnumOutputs returns DXGI_ERROR_NOT_CURRENTLY_AVAILABLE
// to a process in session 0, and neither call fills its out-pointer when it
// fails. Testing only for NOT_FOUND left a null ComPtr to be dereferenced on
// the next line, which would take down a recording from inside the one
// function in this file that promises never to.
// what matter here: neither call fills its out-pointer when it fails, so
// testing only for NOT_FOUND left a null ComPtr to be dereferenced on the
// next line, taking down a recording from inside the one function in this
// file that promises never to.
for (UINT adapterIndex = 0;; ++adapterIndex) {
Microsoft::WRL::ComPtr<IDXGIAdapter1> adapter;
if (FAILED(factory->EnumAdapters1(adapterIndex, &adapter)) || !adapter) {
break;
}
for (UINT outputIndex = 0;; ++outputIndex) {
Microsoft::WRL::ComPtr<IDXGIOutput> output;
if (FAILED(adapter->EnumOutputs(outputIndex, &output)) || !output) {
// NOT_CURRENTLY_AVAILABLE is the exception to the rule above, and it
// has to be told apart: it is what EnumOutputs answers a process in
// session 0, and it means the outputs could not be inspected rather
// than that the adapter has none. Collapsing the two would report
// "no adapter claims this monitor" for a machine we never got to
// look at -- and that is a value this diagnostic tells its readers
// to interpret as an active virtual display. Same class of lie as
// the identical descriptions this event was just fixed for.
const HRESULT outputHr = adapter->EnumOutputs(outputIndex, &output);
if (outputHr == DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) {
enumerationUnavailable = true;
break;
}
if (FAILED(outputHr) || !output) {
break;
}
DXGI_OUTPUT_DESC outputDesc{};
Expand All @@ -266,6 +289,7 @@ void reportCaptureAdapters(ID3D11Device* device, HMONITOR targetMonitor) {
continue;
}
monitorAdapterName = adapterDesc.Description;
monitorAdapterLuid = formatLuid(adapterDesc.AdapterLuid);
monitorAdapterFound = true;
// Compared by LUID rather than by description, because two adapters
// of the same model report the same string.
Expand All @@ -277,18 +301,82 @@ void reportCaptureAdapters(ID3D11Device* device, HMONITOR targetMonitor) {
}
}

// The LUIDs are reported, not just the descriptions, because on the exact
// configuration this diagnostic exists to catch the two descriptions are
// IDENTICAL. An IddCx virtual display driver renders through the physical
// GPU and inherits its description string while being a separate DXGI
// adapter with its own LUID -- measured on a rented multi-adapter box:
//
// adapter[0] NVIDIA Quadro RTX 4000 LUID 0:24084 -> \\.\DISPLAY1
// adapter[1] NVIDIA Quadro RTX 4000 LUID 0:12889146 -> the IDD
//
// So a machine with the divergence would have printed two identical names
// next to sameAdapter:false, which reads as a bug in this reporting rather
// than as the finding it is. The comparison was always by LUID; only the
// output was ambiguous.
std::cout << "{\"event\":\"capture-adapter\",\"schemaVersion\":2,\"deviceAdapter\":\""
<< jsonEscape(wideToUtf8(deviceDesc.Description)) << "\",\"monitorAdapter\":";
<< jsonEscape(wideToUtf8(deviceDesc.Description)) << "\",\"deviceLuid\":\""
<< formatLuid(deviceDesc.AdapterLuid) << "\",\"monitorAdapter\":";
if (monitorAdapterFound) {
std::cout << "\"" << jsonEscape(wideToUtf8(monitorAdapterName)) << "\",\"sameAdapter\":"
std::cout << "\"" << jsonEscape(wideToUtf8(monitorAdapterName)) << "\",\"monitorLuid\":\""
<< monitorAdapterLuid << "\",\"monitorLookup\":\"ok\",\"sameAdapter\":"
<< (sameAdapter ? "true" : "false");
} else if (enumerationUnavailable) {
// Session 0: the outputs were never inspected. Reported as its own
// state so nobody reads it as a finding about the hardware.
std::cout << "null,\"monitorLuid\":null,\"monitorLookup\":\"unavailable\",\"sameAdapter\":null";
} else {
// No output claims this monitor: it is driven by something DXGI does not
// enumerate, which on the machines in #252 means a virtual display
// adapter. Worth seeing in a report in its own right.
std::cout << "null,\"sameAdapter\":null";
// The enumeration completed and no output claims this monitor: it is
// driven by something DXGI does not enumerate, which on the machines in
// #252 would mean a virtual display adapter. Worth seeing in its own
// right -- but only distinguishable from the case above because that
// one is now labelled.
std::cout << "null,\"monitorLuid\":null,\"monitorLookup\":\"no-output-claims-it\",\"sameAdapter\":null";
}
std::cout << "}" << std::endl;

// The full enumeration, to stderr, once at startup. Two adapters sharing a
// description is the thing a reader needs to see with their own eyes before
// they will believe sameAdapter over the names, and an adapter with no
// output at all is how an inactive virtual display presents.
for (UINT adapterIndex = 0;; ++adapterIndex) {
Microsoft::WRL::ComPtr<IDXGIAdapter1> adapter;
if (FAILED(factory->EnumAdapters1(adapterIndex, &adapter)) || !adapter) {
break;
}
DXGI_ADAPTER_DESC1 desc{};
if (FAILED(adapter->GetDesc1(&desc))) {
continue;
}
std::cerr << "[adapters] " << adapterIndex << " luid=" << formatLuid(desc.AdapterLuid)
<< " \"" << wideToUtf8(desc.Description) << "\"";
UINT outputCount = 0;
bool outputsUnavailable = false;
for (UINT outputIndex = 0;; ++outputIndex) {
Microsoft::WRL::ComPtr<IDXGIOutput> output;
const HRESULT outputHr = adapter->EnumOutputs(outputIndex, &output);
if (outputHr == DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) {
outputsUnavailable = true;
break;
}
if (FAILED(outputHr) || !output) {
break;
}
DXGI_OUTPUT_DESC outputDesc{};
if (SUCCEEDED(output->GetDesc(&outputDesc))) {
std::cerr << (outputCount == 0 ? " outputs=" : ",") << wideToUtf8(outputDesc.DeviceName)
<< (outputDesc.Monitor == targetMonitor ? "(captured)" : "");
}
++outputCount;
}
if (outputsUnavailable) {
// Not the same as none: session 0 refuses the question entirely.
std::cerr << " outputs=unavailable";
} else if (outputCount == 0) {
std::cerr << " outputs=none";
}
std::cerr << std::endl;
}
}

bool hasVisibleBgraContent(const std::vector<BYTE>& frame) {
Expand Down
Loading