fix(wgc): GPU DXGI encode path for Windows capture, with a fallback at every step

electron/native/README.md

@@ -85,7 +85,9 @@ The current helper implementation supports display/window video capture, system
 
 Encoder selection: by default the helper keeps the existing sink-writer path first. If that path fails while setting up H.264, it retries with the Microsoft software H.264 encoder (`mfh264enc.dll`). The key of this retry is registering that encoder locally in the helper process via `MFTRegisterLocalByCLSID`, which makes a software H.264 encoder available even when the machine's hardware encoders are missing or broken; hardware transforms are disabled for the retry only as a secondary guard so the sink writer prefers the locally registered software encoder, not as the fallback mechanism itself. Set `preferSoftwareEncoder: true` in the helper JSON, or set `OPENSCREEN_WGC_PREFER_SOFTWARE_ENCODER=true` before launching Electron, to force the software path from the first attempt.
 
-The helper reports the outcome through the `encoder-selection` stdout event (`video` is `default`, `software-preferred`, or `software-fallback`). When the app sees `software-fallback` — the default encoder failed and the helper switched on its own — it shows a small dismissible notice in the recording HUD with a "Don't show again" option, because software encoding can raise CPU usage. An explicit `software-preferred` selection shows no notice, and the event stays available for diagnostics either way.
+Frame input path: the helper feeds the encoder from the GPU when it can. On that path it copies the WGC frame across a keyed-mutex bridge to a second D3D11 device, converts BGRA to NV12 with the D3D11 video processor, and submits an allocator-owned DXGI sample to the hardware H.264 encoder, so no frame ever passes through system memory. The alternative is the original path: a staging texture, `Map(D3D11_MAP_READ)`, and a row-by-row copy into an `IMFMediaBuffer` — which is where a driver stall costs a recording (issue #252). The GPU path is a preference, never a requirement: it is skipped outright for `preferSoftwareEncoder` and for inline webcam PiP (both need the frame in system memory), and it degrades to the CPU path on its own if the encoding device, the NV12 video processor, the shared bridge texture, the DXGI sample allocator, or the hardware sink writer is unavailable. Set `OPENSCREEN_WGC_DISABLE_DXGI_INPUT=1` to force the CPU path. Because the two paths land on different encoders and hardware MFTs default to constant bitrate, the GPU path asks for VBR through `ICodecAPI`; without it a static screen spends the full configured budget (measured 16.9 Mbps against 1.95 for the same desktop).
+
+The helper reports the outcome through the `encoder-selection` stdout event (`video` is `default`, `software-preferred`, or `software-fallback`; `videoInput` is `dxgi-nv12` or `cpu-rgb32`, and reports what the encoder settled on rather than what was asked for). On the GPU path the helper also prints one `[frame-drops] gpu_bridge_contended=<n>` line to stderr at stop: a frame the bridge was too busy to take is skipped rather than failing the recording, and a large count there is the first thing to look at in a report about missing frames. When the app sees `software-fallback` — the default encoder failed and the helper switched on its own — it shows a small dismissible notice in the recording HUD with a "Don't show again" option, because software encoding can raise CPU usage. An explicit `software-preferred` selection shows no notice, and the event stays available for diagnostics either way.
 
 Encoder diagnostic on final sink-writer failure: when the final `MFCreateSinkWriterFromURL` attempt fails, the helper logs the registered H.264 video encoder MFT count (via `MFTEnumEx`), the registered AAC encoder count when audio was requested, and the hex HRESULT. If no H.264 encoder is registered, it additionally emits the four-bullet actionable error (missing Media Feature Pack / GPU driver registration / empty `HKLM:\SOFTWARE\Microsoft\Windows Media Foundation\Transforms` / reboot). If an H.264 encoder IS registered but the sink writer still failed, it logs a hint pointing at invalid output path, missing MP4 mux, or GPU driver incompatibility. There is still no fail-fast pre-flight gate because `MFTEnumEx` and `MFCreateSinkWriterFromURL` can disagree about which H.264 encoders are available in non-interactive / Session 0 contexts.
 

electron/native/wgc-capture/src/main.cpp

@@ -607,6 +607,23 @@ int main(int argc, char* argv[]) {
     MFEncoderOptions encoderOptions{};
     encoderOptions.preferSoftwareEncoder = config.preferSoftwareEncoder;
     encoderOptions.injectDefaultSinkWriterFailureOnce = injectDefaultSinkWriterFailureOnce;
+    // Keep the CPU path for software encoding and inline webcam PiP: both need
+    // the frame in system memory, which is the one thing the DXGI path does not
+    // produce. The env var is the escape hatch for a machine where the GPU path
+    // misbehaves in a way the encoder's own probes do not catch -- a support
+    // answer instead of a hotfix.
+    //
+    // config.webcamEnabled, not webcamActive: the latter is only set once the
+    // webcam capture has started, which happens well after this. Reading it
+    // here made the PiP condition dead code -- always false, so always
+    // permitting the GPU path -- and an inline-PiP recording would have run on
+    // DXGI and silently dropped the overlay, reporting success either way.
+    // config.webcamEnabled is already cleared above when webcam init fails,
+    // and writeSeparateWebcam is assigned there too, so both are final here.
+    encoderOptions.useDxgiInput =
+        !config.preferSoftwareEncoder &&
+        (!config.webcamEnabled || writeSeparateWebcam) &&
+        readEnvInt("OPENSCREEN_WGC_DISABLE_DXGI_INPUT", 0) == 0;
 
     MFEncoder encoder;
     if (!encoder.initialize(
@@ -622,15 +639,22 @@ int main(int argc, char* argv[]) {
         std::cerr << "ERROR: Failed to initialize Media Foundation encoder" << std::endl;
         return 1;
     }
+    // `videoInput` reports what the encoder settled on, not what was asked for:
+    // it silently degrades to the CPU readback on any machine the GPU path does
+    // not fit, and a bug report that cannot tell the two apart is a bug report
+    // about the wrong path.
+    const bool usesDxgiInput = encoder.usesDxgiInput();
     std::cout << "{\"event\":\"encoder-selection\",\"schemaVersion\":2,\"video\":\""
               << encoder.videoEncoderSelection()
+              << "\",\"videoInput\":\"" << (usesDxgiInput ? "dxgi-nv12" : "cpu-rgb32")
               << "\",\"preferSoftwareEncoder\":"
               << (config.preferSoftwareEncoder ? "true" : "false")
               << "}" << std::endl;
     MFEncoder webcamEncoder;
     if (writeSeparateWebcam) {
         MFEncoderOptions webcamEncoderOptions = encoderOptions;
         webcamEncoderOptions.injectDefaultSinkWriterFailureOnce = false;
+        webcamEncoderOptions.useDxgiInput = false;
         const int webcamPixels = std::max(1, webcamCapture.width()) * std::max(1, webcamCapture.height());
         const int webcamBitrate = webcamPixels >= 1280 * 720 ? 8'000'000 : 4'000'000;
         if (!webcamEncoder.initialize(
@@ -652,6 +676,10 @@ int main(int argc, char* argv[]) {
     CaptureControl control;
     std::atomic<bool> firstFrameWritten = false;
     std::atomic<bool> encodeFailed = false;
+    // Frames the GPU bridge was too busy to take. Reported at stop rather than
+    // per frame: a handful over a recording is normal contention, a stream of
+    // them is the next bug report, and neither is worth a log line each.
+    std::atomic<uint64_t> contendedFrames = 0;
     Microsoft::WRL::ComPtr<ID3D11Texture2D> latestFrameTexture;
     int64_t latestFrameTimestampHns = 0;
     int64_t firstFrameTimestampHns = -1;
@@ -802,22 +830,40 @@ int main(int argc, char* argv[]) {
                     std::this_thread::sleep_for(std::chrono::milliseconds(testStallReadbackMs));
                 }
                 if (latestFrameTexture) {
-                    // captureVideoSample performs the GPU readback
-                    // (CopyResource/Map) from latestFrameTexture, which must
-                    // stay serialized (via `mutex`) against the WGC
-                    // frame-arrival callback above, which writes new data
-                    // into the same texture on another thread.
-                    hasVideoSample = encoder.captureVideoSample(
-                        latestFrameTexture.Get(),
-                        frameTimestampHns,
-                        !writeSeparateWebcam && webcamFrame.data ? &webcamFrame : nullptr,
-                        videoSample);
-                    if (!hasVideoSample) {
+                    // Both entry points do their GPU work on latestFrameTexture,
+                    // which must stay serialized (via `mutex`) against the WGC
+                    // frame-arrival callback above, which writes new data into
+                    // the same texture on another thread. Which one is live is
+                    // the encoder's answer, not this struct's request: it falls
+                    // back to the CPU path on its own when the GPU path does
+                    // not fit the machine.
+                    bool captured = false;
+                    if (usesDxgiInput) {
+                        captured = encoder.captureDxgiSample(
+                            latestFrameTexture.Get(),
+                            frameTimestampHns,
+                            videoSample);
+                    } else {
+                        captured = encoder.captureVideoSample(
+                            latestFrameTexture.Get(),
+                            frameTimestampHns,
+                            !writeSeparateWebcam && webcamFrame.data ? &webcamFrame : nullptr,
+                            videoSample);
+                    }
+                    if (!captured) {
                         encodeFailed = true;
                         control.requestStop();
                         break;
                     }
-                    lastEncodedVideoTimestampHns = frameTimestampHns;
+                    // The DXGI path returns success with no sample when the
+                    // GPU bridge was momentarily busy. That costs one frame,
+                    // which beats ending a recording that is otherwise fine.
+                    hasVideoSample = videoSample != nullptr;
+                    if (hasVideoSample) {
+                        lastEncodedVideoTimestampHns = frameTimestampHns;
+                    } else {
+                        contendedFrames += 1;
+                    }
                 }
             }
 
@@ -1080,8 +1126,19 @@ int main(int argc, char* argv[]) {
             // sleep still be killed.
             if (stopElapsedMs() >= currentStepDeadlineMs.load() && !shutdownComplete.load()) {
                 const char* step = currentStopStep.load();
+                // The encoder stage is what turns "video-writer-join was
+                // abandoned" into something actionable: it names the call the
+                // writer thread is sitting in, instead of leaving the next
+                // report to guess the way issue #252 had to.
+                // Both threads are named, because either can be the one that is
+                // stuck and each has its own slot: encode_stage is the video
+                // writer, audio_stage the mixer. A report showing audio_stage
+                // parked on write-audio while encode_stage sits at a bridge
+                // call says the two are fighting over writerMutex_, which no
+                // single-slot breadcrumb could ever have shown.
                 std::cerr << "[stop-timing] step=" << step << " elapsed_ms=" << stopElapsedMs()
-                          << " phase=abandoned" << std::endl;
+                          << " phase=abandoned encode_stage=" << encoder.encodeStage()
+                          << " audio_stage=" << encoder.audioStage() << std::endl;
                 std::cout << "{\"event\":\"stop-timeout\",\"schemaVersion\":2,\"step\":\"" << step
                           << "\"}" << std::endl;
                 std::cout.flush();
@@ -1124,6 +1181,9 @@ int main(int argc, char* argv[]) {
     beginStopStep("video-writer-join", stepBudgetMs);
     stopVideoWriter();
     logStopStep("video-writer-join");
+    if (usesDxgiInput) {
+        std::cerr << "[frame-drops] gpu_bridge_contended=" << contendedFrames.load() << std::endl;
+    }
     // No frame lock here, and the ordering above is what makes that safe rather
     // than incidental: stopVideoWriter() joined the only thread that calls into
     // the encoder's GPU readback, and audioMixer->stop() joined the only other