Skip to content

Require Go 1.19 and use its encoding/binary append functions - #715

Merged
peterstace merged 2 commits into
masterfrom
go_1_19
Nov 6, 2026
Merged

peterstace merged 2 commits into
masterfrom
go_1_19

Conversation

@peterstace

Copy link
Copy Markdown
Owner

Description

This raises the minimum Go version from 1.18 to 1.19 and changes the WKB and TWKB encoders to use the encoding/binary append functions that 1.19 added.

go.mod now declares go 1.19, and the unit, pgscan, and cmppg CI jobs run on golang:1.19. The geos and proj jobs already build with golang:1.21-alpine, and golangci-lint v1.61.0 in the lint job supports 1.19, so neither changes.

Go 1.19 added AppendUint32 and AppendUint64 to binary.LittleEndian and binary.BigEndian through the new binary.AppendByteOrder interface, along with binary.AppendVarint and binary.AppendUvarint. The encoders used to encode each value into a temporary array and then append the array to the output. They now append to the output directly. In the WKB encoder the temporary array was passed to a method on the binary.ByteOrder interface, which made it escape to the heap, so each header field and each Point or MultiPoint coordinate cost an allocation. nativeOrder changes type from binary.ByteOrder to binary.AppendByteOrder to expose the append methods. The comparisons against binary.BigEndian and binary.LittleEndian in wkb_parser.go behave the same.

Nothing else in the Go 1.19 release notes fits the codebase:

  • Doc links ([Name]) are already used throughout the doc comments.

  • sort.Find would suit the binary search in internal/jtsport/jts/noding_segment_node_list.go, but internal/jtsport is out of scope here.

  • The typed atomics, fmt.Append*, the other new APIs, and the unix build constraint don't have anything to replace.

  • The new sort algorithm and the switch jump tables don't need any code changes. The unit tests pass on 1.19, so no test depends on how an unstable sort orders equal elements.

Check List

Have you:

  • Added unit tests? No. The existing WKB and TWKB tests cover the changed encoders, and the encoded bytes are the same as before. The PR does add benchmarks for the changed code, described under Benchmark Results.

  • Add cmprefimpl tests? (if appropriate?) Not appropriate, since there is no new behaviour to compare. make cmppg, which compares AsBinary output byte for byte against PostGIS, passes on Go 1.19.

  • Updated release notes? (if appropriate?) Yes. The Unreleased section of CHANGELOG.md has an entry for the new minimum Go version, marked as a breaking change, and one for the encoder performance.

  • Updated the README.md (if new functionality is added?) Not appropriate. There's no new functionality, and the README doesn't state a Go version.

Related Issue

  • None.

Benchmark Results

These come from go test -bench on linux/arm64, with 10 runs each of the base commit and this branch, interleaved. The base checkout had this PR's perf_test.go copied in so that both sides run the same benchmarks. run_benchmarks.sh wasn't used, since it runs every benchmark in the module. The existing MarshalWKB/polygon cases write coordinates with one bulk copy, which this PR doesn't change. The new MarshalWKB/multipoint and MarshalTWKB/polygon cases cover the per-value code that did change.

Click to expand
goos: linux
goarch: arm64
pkg: github.com/peterstace/simplefeatures/geom
                                │     old      │                 new                  │
                                │    sec/op    │    sec/op     vs base                │
MarshalWKB/polygon/n=10-8         150.4n ± 10%   135.8n ±  7%   -9.71% (p=0.023 n=10)
MarshalWKB/polygon/n=100-8        643.9n ± 15%   700.4n ±  4%        ~ (p=0.075 n=10)
MarshalWKB/polygon/n=1000-8       5.186µ ±  1%   5.291µ ±  2%   +2.02% (p=0.043 n=10)
MarshalWKB/polygon/n=10000-8      65.86µ ± 18%   64.86µ ±  5%        ~ (p=0.436 n=10)
MarshalWKB/multipoint/n=10-8      670.9n ±  8%   363.2n ± 26%  -45.87% (p=0.000 n=10)
MarshalWKB/multipoint/n=100-8     6.764µ ± 13%   4.665µ ±  9%  -31.04% (p=0.000 n=10)
MarshalWKB/multipoint/n=1000-8    69.97µ ± 11%   50.11µ ± 24%  -28.39% (p=0.000 n=10)
MarshalWKB/multipoint/n=10000-8   810.2µ ± 16%   610.5µ ±  5%  -24.64% (p=0.000 n=10)
MarshalTWKB/polygon/n=10-8        473.2n ±  3%   445.6n ±  6%   -5.82% (p=0.001 n=10)
MarshalTWKB/polygon/n=100-8       2.493µ ±  4%   2.066µ ± 13%  -17.13% (p=0.000 n=10)
MarshalTWKB/polygon/n=1000-8      17.06µ ±  3%   12.28µ ± 34%  -28.01% (p=0.001 n=10)
MarshalTWKB/polygon/n=10000-8     179.4µ ±  4%   142.0µ ± 16%  -20.84% (p=0.001 n=10)
geomean                           7.481µ         6.112µ        -18.30%

                                │      old      │                 new                  │
                                │     B/op      │     B/op      vs base                │
MarshalWKB/polygon/n=10-8            232.0 ± 0%     216.0 ± 0%   -6.90% (p=0.000 n=10)
MarshalWKB/polygon/n=100-8         1.789Ki ± 0%   1.773Ki ± 0%   -0.87% (p=0.000 n=10)
MarshalWKB/polygon/n=1000-8        16.04Ki ± 0%   16.02Ki ± 0%   -0.10% (p=0.000 n=10)
MarshalWKB/polygon/n=10000-8       160.0Ki ± 0%   160.0Ki ± 0%   -0.01% (p=0.000 n=10)
MarshalWKB/multipoint/n=10-8         752.0 ± 0%     504.0 ± 0%  -32.98% (p=0.000 n=10)
MarshalWKB/multipoint/n=100-8     10.594Ki ± 0%   8.242Ki ± 0%  -22.20% (p=0.000 n=10)
MarshalWKB/multipoint/n=1000-8    106.19Ki ± 0%   82.74Ki ± 0%  -22.08% (p=0.000 n=10)
MarshalWKB/multipoint/n=10000-8   1121.1Ki ± 0%   886.8Ki ± 0%  -20.91% (p=0.000 n=10)
MarshalTWKB/polygon/n=10-8           592.0 ± 0%     624.0 ± 0%   +5.41% (p=0.000 n=10)
MarshalTWKB/polygon/n=100-8        2.875Ki ± 0%   2.906Ki ± 0%   +1.09% (p=0.000 n=10)
MarshalTWKB/polygon/n=1000-8       16.62Ki ± 0%   16.66Ki ± 0%   +0.19% (p=0.000 n=10)
MarshalTWKB/polygon/n=10000-8      191.1Ki ± 0%   191.2Ki ± 0%   +0.02% (p=0.000 n=10)
geomean                            11.45Ki        10.40Ki        -9.15%

                                │      old      │                 new                  │
                                │   allocs/op   │ allocs/op   vs base                  │
MarshalWKB/polygon/n=10-8            6.000 ± 0%   3.000 ± 0%  -50.00% (p=0.000 n=10)
MarshalWKB/polygon/n=100-8           6.000 ± 0%   3.000 ± 0%  -50.00% (p=0.000 n=10)
MarshalWKB/polygon/n=1000-8          6.000 ± 0%   3.000 ± 0%  -50.00% (p=0.000 n=10)
MarshalWKB/polygon/n=10000-8         6.000 ± 0%   3.000 ± 0%  -50.00% (p=0.000 n=10)
MarshalWKB/multipoint/n=10-8        38.000 ± 0%   6.000 ± 0%  -84.21% (p=0.000 n=10)
MarshalWKB/multipoint/n=100-8       313.00 ± 0%   11.00 ± 0%  -96.49% (p=0.000 n=10)
MarshalWKB/multipoint/n=1000-8     3020.00 ± 0%   18.00 ± 0%  -99.40% (p=0.000 n=10)
MarshalWKB/multipoint/n=10000-8   30028.00 ± 0%   26.00 ± 0%  -99.91% (p=0.000 n=10)
MarshalTWKB/polygon/n=10-8           9.000 ± 0%   9.000 ± 0%        ~ (p=1.000 n=10) ¹
MarshalTWKB/polygon/n=100-8          13.00 ± 0%   13.00 ± 0%        ~ (p=1.000 n=10) ¹
MarshalTWKB/polygon/n=1000-8         17.00 ± 0%   17.00 ± 0%        ~ (p=1.000 n=10) ¹
MarshalTWKB/polygon/n=10000-8        25.00 ± 0%   25.00 ± 0%        ~ (p=1.000 n=10) ¹
geomean                              45.03        8.406       -81.33%
¹ all samples are equal

Raise the go directive in go.mod and the golang image for the unit,
pgscan, and cmppg CI jobs from 1.18 to 1.19. The geos and proj jobs
already build with Go 1.21, and golangci-lint v1.61.0 in the lint job
supports 1.19.
Go 1.19 added AppendUint32 and AppendUint64 to binary.LittleEndian and
binary.BigEndian through the binary.AppendByteOrder interface, along
with binary.AppendVarint and binary.AppendUvarint. The WKB and TWKB
encoders now append to the output buffer directly instead of encoding
each value into a temporary array first.

The WKB encoder passed its temporary array to a method on the
binary.ByteOrder interface, which made the array escape to the heap.
Each header field and each Point or MultiPoint coordinate cost an
allocation.

The TWKB bbox buffer is created with room for the largest possible
header, so appending its varints one at a time never grows it.

Add a multipoint case to BenchmarkMarshalWKB and a new
BenchmarkMarshalTWKB. The existing polygon case writes coordinates with
one bulk copy, so it doesn't exercise the per-value code.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are blocking approval.

No open issues 🎉

What changed in this PR

Raises the module’s minimum Go version to 1.19 and uses its encoding/binary append APIs to improve WKB and TWKB encoding performance without changing encoded output.

Changes:

  • Updates the module requirement and three CI jobs to Go 1.19.
  • Replaces temporary-buffer encoding with direct append calls.
  • Adds targeted benchmarks and documents compatibility and performance changes.
File Description
go.mod Raises the minimum Go version to 1.19.
geom/​wkb_marshal.go Uses append APIs for WKB header fields and coordinates.
geom/​twkb_write.go Uses append APIs for TWKB varints and headers.
geom/​perf_test.go Adds WKB multipoint and TWKB polygon benchmarks.
CHANGELOG.md Records the Go requirement and encoder performance improvements.
.ci/​compose-unit.yaml Runs unit tests on Go 1.19.
.ci/​compose-pgscan.yaml Runs PostgreSQL scanning tests on Go 1.19.
.ci/​compose-cmppg.yaml Runs PostGIS comparison tests on Go 1.19.

Review effort: Balanced

Comment thread geom/twkb_write.go
// Store bbox min and delta for each dimension.
var buf [twkbMaxDimensions * 2 * binary.MaxVarintLen64]byte
n := 0
w.twkbBBox = make([]byte, 0, 2*w.dimensions*binary.MaxVarintLen64)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assigns rather than appends, so it relies on twkbBBox being empty at this point. It always is: writeBBoxHeader only runs from writeGeometry, which runs once per writer, and the sub-writers from copytwkbWriter never write a bbox. The capacity fits the largest possible header, so appending the varints never grows the buffer. Without it, appending to a nil slice cost small geometries an extra allocation.

@peterstace
peterstace marked this pull request as ready for review November 6, 2026 03:24
@peterstace
peterstace merged commit 210317c into master Nov 6, 2026
2 checks passed
@peterstace
peterstace deleted the go_1_19 branch November 6, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants