Fix Swish layer

[andrewdalpino]

Swish layer: missing β in input derivative, wrong dBeta, 0/0 NaN — src/NeuralNet/Layers/Swish.php:176, 260-271. dBeta = Σ dOut·x is not Σ dOut·x²·σ(βx)(1−σ(βx)).

[Copilot]

Pull request overview

This PR fixes the Swish activation layer’s backprop math by correcting the β gradient and the input derivative, eliminating a potential 0/0 path that could produce NaNs. It also updates tests and the changelog to reflect/cover the corrected behavior.

Changes:

• Correct Swish backprop: compute dβ as Σ(dOut · x² · σ(βx)(1−σ(βx))) and fix dInput to include β while avoiding 0/0.
• Extend/adjust Swish unit tests (including coverage for β > 1) and update expected numeric results.
• Update MLP regressor test to include a Swish layer, and document the fix in the changelog.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/NeuralNet/Layers/Swish.php	Fixes Swish derivatives/gradients and removes the 0/0 derivative formulation by using σ(βx) directly.
tests/NeuralNet/Layers/SwishTest.php	Updates expected values and adds a β>1 regression test for forward/back/infer correctness.
tests/Regressors/MLPRegressorTest.php	Swaps one SiLU activation for a Swish layer to exercise the layer in an end-to-end model.
CHANGELOG.md	Notes the Swish beta-parameter gradient fix.

Suppressed comments (2)

tests/NeuralNet/Layers/SwishTest.php:159

• This floating-point gradient assertion is using strict equality. Consider switching to assertEqualsWithDelta (as used in many other neural-net tests) to avoid flaky failures from small rounding differences.

        $this->assertInstanceOf(Matrix::class, $gradient);
        $this->assertEquals($expected, $gradient->asArray());

tests/NeuralNet/Layers/SwishTest.php:170

• The infer output is asserted with strict float equality; using assertEqualsWithDelta here would make the test less sensitive to minute numeric differences.

        $this->assertInstanceOf(Matrix::class, $infer);
        $this->assertEquals($expected, $infer->asArray());

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.