Skip to content

Use Windows auth for local perf runs #4608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mdaigle wants to merge 1 commit into
base: main
Choose a base branch
from
+80 −6
Open

Use Windows auth for local perf runs #4608

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions eng/pipelines/perf/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,23 @@ supplies the isolated dedicated host, the tuned SQL instance, and the disjoint c
| Interleaving | In `interleaved` mode the harness runs **one benchmark unit at a time, baseline then candidate back-to-back**, so both sides see the same host state (see below). |
| Best-of-N confirmation | A unit flagged in the first interleaved pass is re-run `confirmationRuns` times; a regression is **confirmed** only on a strict majority. Unconfirmed flags are reported but never fail the gate. |

### Windows physical opens

The Windows connection probe measured raw TCP at about 0.06 ms but a physical
`SqlConnection.Open()` at about 158 ms when the runner used the `sa` SQL login, with the same result
from native and managed SNI. Changing TLS and TCP acknowledgment settings did not change that
latency. The Windows image runs SQL Server 2025, which verifies SQL-login passwords with 100,000
PBKDF2 iterations; Microsoft documents the resulting login-performance impact.

The benchmark process runs through a public-key SSH token, which cannot delegate Windows credentials
to the VM's private network address. When the injected SQL endpoint resolves to a local interface,
the Windows harness connects to the same SQL Server 2025 instance over local TCP loopback with
integrated authentication. It grants the ephemeral VM identity `db_owner` in the perf database; `sa`
remains unchanged and is used only for one-time setup. External SQL endpoints continue to use the
injected address and SQL authentication. This stays on SQL Server 2025's supported authentication
path while preventing password hashing from dominating local tests that intentionally create
physical connections.

### Interleaving + best-of-N (run model)

`benchmarkRunMode` selects how the two variants are measured:
Expand Down
69 changes: 63 additions & 6 deletions eng/pipelines/perf/scripts/run-perf-tests.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,67 @@ if ($sqlcmd) {
throw "sqlcmd was not found on the VM; cannot create the perf database [$DbName]."
}

# SQL Server 2025 verifies SQL-authentication passwords with 100,000 PBKDF2 iterations. That work
# costs about 150 ms per physical login and overwhelms connection benchmarks that intentionally
# disable or clear pooling. Use the supported Windows-authentication path over local TCP loopback.
function Test-IsLocalSqlServer {
param([string] $Server)

try {
$serverAddresses = [System.Net.Dns]::GetHostAddresses($Server)
} catch [System.Net.Sockets.SocketException] {
Write-Warning "Could not resolve SQL Server [$Server] while checking whether it is local."
return $false
}

$localAddresses = [System.Net.NetworkInformation.NetworkInterface]::GetAllNetworkInterfaces() |
ForEach-Object { $_.GetIPProperties().UnicastAddresses } |
ForEach-Object { $_.Address }
return $null -ne ($serverAddresses | Where-Object { $localAddresses -contains $_ } |
Select-Object -First 1)
}

if (Test-IsLocalSqlServer $SqlServer) {
$BenchmarkSqlServer = "localhost"
$benchmarkIdentity = [System.Security.Principal.WindowsIdentity]::GetCurrent().Name
$identityLiteral = $benchmarkIdentity.Replace("'", "''")
$configureBenchmarkIdentity = @"
DECLARE @ddl nvarchar(max);
IF SUSER_ID(N'$identityLiteral') IS NULL
BEGIN
SET @ddl = N'CREATE LOGIN ' + QUOTENAME(N'$identityLiteral') + N' FROM WINDOWS';
EXEC sys.sp_executesql @ddl;
END
USE [$DbName];
IF USER_ID(N'$identityLiteral') IS NULL
BEGIN
SET @ddl = N'CREATE USER ' + QUOTENAME(N'$identityLiteral') +
N' FOR LOGIN ' + QUOTENAME(N'$identityLiteral');
EXEC sys.sp_executesql @ddl;
END;
IF IS_ROLEMEMBER(N'db_owner', N'$identityLiteral') <> 1
BEGIN
SET @ddl = N'ALTER ROLE [db_owner] ADD MEMBER ' + QUOTENAME(N'$identityLiteral');
EXEC sys.sp_executesql @ddl;
END;
"@
Invoke-Native {
& $sqlcmd.Source -S $SqlServer -U sa -P $SqlPassword -C -b -l 30 -Q $configureBenchmarkIdentity
} "sqlcmd failed to configure Windows benchmark identity [$benchmarkIdentity]"
Invoke-Native {
& $sqlcmd.Source -S "tcp:$BenchmarkSqlServer,1433" -E -d $DbName -C -b -l 15 `
-Q "SET NOCOUNT ON; SELECT SUSER_SNAME(), USER_NAME();"
} "Loopback integrated-authentication preflight failed for benchmark identity [$benchmarkIdentity]"
$BenchmarkConnectionString = "Server=tcp:$BenchmarkSqlServer,1433;Integrated Security=True;Initial Catalog=$DbName;TrustServerCertificate=True;Encrypt=True;"
Write-Host "Verified Windows integrated authentication over local TCP loopback."
} else {
# Preserve the injected endpoint and SQL authentication for external SQL Server deployments.
$BenchmarkSqlServer = $SqlServer
$escapedPassword = '"' + ($SqlPassword -replace '"', '""') + '"'
$BenchmarkConnectionString = "Server=tcp:$BenchmarkSqlServer,1433;User ID=sa;Password=$escapedPassword;Initial Catalog=$DbName;TrustServerCertificate=True;Encrypt=True;"
Write-Host "Using SQL authentication for external SQL Server [$BenchmarkSqlServer]."
}

####################################################################################################
# Noise-reduction controls (InternalDriverTools wiki 339, "Reducing Noise in Performance Tests").
#
Expand Down Expand Up @@ -308,11 +369,7 @@ $rawConfig = Get-Content $srcConfig -Raw
$rawConfig = ($rawConfig -split "`n" | ForEach-Object { $_ -replace '(?m)^\s*//.*$', '' }) -join "`n"
$cfg = ConvertFrom-Json $rawConfig

# SqlClient connection-string values may be wrapped in double quotes; doubling any embedded double
# quote lets a password containing ';', '=', spaces or single quotes be parsed as a single literal
# value instead of corrupting the connection string.
$escapedPassword = '"' + ($SqlPassword -replace '"', '""') + '"'
$cfg.ConnectionString = "Server=tcp:$SqlServer,1433;User ID=sa;Password=$escapedPassword;Initial Catalog=$DbName;TrustServerCertificate=True;Encrypt=False;"
$cfg.ConnectionString = $BenchmarkConnectionString
# Apply the optional SqlClient behaviour overrides supplied by the pipeline. An empty value leaves
# the checked-in default untouched; otherwise the flag is forced to the requested boolean so the
# benchmarks run with (and PerfRun.Config records) exactly the requested behaviour.
Expand All @@ -328,7 +385,7 @@ Set-CfgBool $cfg "UseManagedSniOnWindows" $UseManagedSniOnWindows
Set-CfgBool $cfg "UseOptimizedAsyncBehaviour" $UseOptimizedAsyncBehaviour
Set-CfgBool $cfg "UseConnectionPoolV2" $UseConnectionPoolV2
$cfg | ConvertTo-Json -Depth 10 | Set-Content -Path $RunnerConfig -Encoding UTF8
Write-Host "Wrote runner config to $RunnerConfig (Server=tcp:$SqlServer,1433; Initial Catalog=$DbName)"
Write-Host "Wrote runner config to $RunnerConfig (Server=tcp:$BenchmarkSqlServer,1433; Initial Catalog=$DbName)"

####################################################################################################
# 4 & 5. Run the benchmarks, pinned to the reserved client CPU set.
Expand Down
Loading