Skip to content

Perf Tests | Default Config File #4606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
benrr101 wants to merge 1 commit into
base: main
Choose a base branch
from
+18 −10
Open

Perf Tests | Default Config File #4606

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ node_modules/
**/config.json
**/config.jsonc

# Performance test runner config
/src/Microsoft.Data.SqlClient/tests/PerformanceTests/runnerconfig.jsonc

# Generated Milestone PR metadata files
.milestone-prs/

Expand Down
20 changes: 11 additions & 9 deletions BUILDGUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -448,16 +448,19 @@ $ sqlcmd -S localhost -U sa -P password
1> quit
```

The default `runnerconfig.jsonc` expects a database named `sqlclient-perf-db`,
but you may change the config to use any existing database. The benchmarks
The default `runnerconfig.default.jsonc` expects a database named
`sqlclient-perf-db`, but you may change the generated `runnerconfig.jsonc`
to use any existing database. The benchmarks
create and drop their own tables (typically prefixed with `perf_`) in this
database; other existing tables are left untouched.

### Configure Runner

Configure the benchmarks by editing the `runnerconfig.jsonc` file directly in the
Configure the benchmarks by editing the `runnerconfig.jsonc` file in the
`PerformanceTests` directory with an appropriate connection string and benchmark
settings:
settings. If the file does not exist, the build creates it from
`runnerconfig.default.jsonc`; `runnerconfig.jsonc` is ignored by git so local
connection strings and benchmark settings can stay local.

```json
{
Expand Down Expand Up @@ -504,11 +507,10 @@ copy the file into the `artifacts` directory alongside the benchmark DLL. By
default, the benchmarks look for `runnerconfig.jsonc` in the same directory as
the DLL.

Optionally, to avoid polluting your git workspace and requiring a build after
each config change, copy `runnerconfig.jsonc` to a new file, make your edits
there, and then specify the new file with the RUNNER_CONFIG environment
variable. The same approach works for `datatypes.json` via the
`DATATYPES_CONFIG` environment variable.
Optionally, to avoid requiring a build after each config change, copy
`runnerconfig.jsonc` to a new file, make your edits there, and then specify the
new file with the RUNNER_CONFIG environment variable. The same approach works
for `datatypes.json` via the `DATATYPES_CONFIG` environment variable.

PowerShell:

Expand Down
3 changes: 3 additions & 0 deletions ...ft.Data.SqlClient/tests/PerformanceTests/Microsoft.Data.SqlClient.PerformanceTests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@
</ItemGroup>

<!-- Content files to copy to output directory -->
<Target Name="CopyRunnerConfig" BeforeTargets="Compile">
<Copy SourceFiles="runnerconfig.default.jsonc" DestinationFiles="runnerconfig.jsonc" Condition="!Exists('runnerconfig.jsonc')" />
Comment on lines +28 to +29

@paulmedynski paulmedynski Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest updating the run-perf-tests scripts to build, then look at the config, then dotnet run --no-build the benchmarks.

For the Kusto script, should the benchmark results contain the config file used? Then this script would be guaranteed to see the same config that produced the results.

</Target>
<ItemGroup>
<None Include="datatypes.json" CopyToOutputDirectory="PreserveNewest" />
<None Include="runnerconfig.jsonc" CopyToOutputDirectory="PreserveNewest" />
Expand Down
2 changes: 1 addition & 1 deletion ...tests/PerformanceTests/runnerconfig.jsonc → ...rformanceTests/runnerconfig.default.jsonc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
{
"ConnectionString": "Server=tcp:localhost; Integrated Security=true; Initial Catalog=sqlclient-perf-db;",
// Enable this flag to enable managed SNI on Windows.
"UseManagedSniOnWindows": false,
Expand Down
Loading