Add -static flag

[mattnibs]

This commit adds the -static flag which when enabled, forces static type checking on all input files. By default csup and parquet are statically checked- all other file types are dynamic. If the -static flag is enabled the entire file is read and the full type information is gathered. Non-seekable inputs (e.g., /dev/stdin) cannot be statically checked an will return an error -static is enabled.

This commit removes the -samplesize flag.

[philrz]

One small finding: Right now it's possible to invoke -static and -dynamic simultaneously. It looks like -dynamic is "winning" at the moment, but some kind of guard to prevent this might be helpful.

$ echo '{"a": "b"}' > data.json &&
  super -version &&
  super -static -dynamic -c "values foo" data.json

Version: v0.3.0-379-gd55bada8f

error({message:"no such field foo",on:{}})

[mattnibs]

Thanks Phil there should be an error if both are activated

[philrz]

@mattnibs: Here's a more significant finding. Since you'd mentioned on Slack that this affected some SQL tests, I happened to do a proactive run of the sqllogic-ztests on this branch and it did hit some crashes, such as on select1/q0.yaml. I had Claude simplify the repro, and it came up with this one which also reproduces on this branch but not current tip of main.

$ printf '{"a":1,"b":2,"c":3}\n{"a":4,"b":5,"c":6}\n' > t1 &&
  super -version &&
  super -c "SELECT a FROM t1 WHERE c > (SELECT avg(c) FROM t1) ORDER BY 1"

Version: v0.3.0-379-gd55bada8f

panic: internal error: JSON object is not a DAG operator: JSON field "expr" in Go struct type "dag.FilterOp": JSON field "rhs" in Go struct type "dag.BinaryExpr": JSON field "body" in Go struct type "dag.SubqueryExpr": JSON field "type" in Go struct type "dag.FileScan": value of type "map[string]interface {}" not assignable to type "super.Type"
goroutine 1 [running]:
github.com/brimdata/super/compiler/dag.CopyOp({0x55d2640?, 0x3130f816be20?})
	/Users/phil/work/super/compiler/dag/op.go:391 +0x77
github.com/brimdata/super/compiler/dag.CopySeq(...)
	/Users/phil/work/super/compiler/dag/op.go:376
github.com/brimdata/super/compiler/optimizer.parallelizeHead({0x3130f807c070, 0x7, 0x7}, 0x4, {0x3130f8197c80, 0x1, 0x1}, 0x6)
	/Users/phil/work/super/compiler/optimizer/parallelize.go:122 +0x217
github.com/brimdata/super/compiler/optimizer.(*Optimizer).parallelizeFileScan(0x3130f8197d40, {0x3130f7fe0d90?, 0x6?, 0x7?}, 0x6)
	/Users/phil/work/super/compiler/optimizer/parallelize.go:83 +0x1b9
github.com/brimdata/super/compiler/optimizer.(*Optimizer).Parallelize.func1({0x3130f7fe0d80, 0x7, 0x8})
	/Users/phil/work/super/compiler/optimizer/parallelize.go:37 +0x20d
github.com/brimdata/super/compiler/optimizer.walkEntries({0x3130f7fe0d80, 0x7, 0x8}, 0x3130f8455700)
	/Users/phil/work/super/compiler/optimizer/optimizer.go:114 +0xea
github.com/brimdata/super/compiler/optimizer.(*Optimizer).Parallelize(0x3130f8197d40, 0x3130f8176a50, 0x3130f84a5440?)
	/Users/phil/work/super/compiler/optimizer/parallelize.go:20 +0x85
github.com/brimdata/super/compiler.Optimize({0x55e6198, 0x3130f8173a70}, 0x3130f8176a50, 0x3130f82fb560, 0xc)
	/Users/phil/work/super/compiler/package.go:38 +0xce
github.com/brimdata/super/compiler.CompileWithAST(0x3130f8173a70, 0x32?, 0x10?, 0x1, 0xc, {0x0?, 0x3130f8173a70?, 0x3130f8500008?})
	/Users/phil/work/super/compiler/package.go:73 +0x29e
github.com/brimdata/super/compiler.(*compiler).NewQuery(0x30?, 0x5568fd8?, 0x3130f8355880?, {0x0?, 0x3130f8500008?, 0x3130f85005a8?}, 0x1f8?)
	/Users/phil/work/super/compiler/compiler.go:42 +0x45
github.com/brimdata/super/runtime.CompileQuery({0x55e6240?, 0x3130f82fc910?}, 0x3130f8176190, {0x55da5c0, 0x3130f7de6358}, 0x3130f816b2a0, {0x0, 0x0, 0x0})
	/Users/phil/work/super/runtime/compiler.go:31 +0x119
github.com/brimdata/super/cmd/super/root.(*Command).Run(0x3130f80fe200, {0x3130f7f987a0, 0x0, 0x0})
	/Users/phil/work/super/cmd/super/root/command.go:95 +0x648
github.com/brimdata/super/pkg/charm.path.run({0x3130f7de62e0, 0x1, 0x1}, {0x3130f7f987a0, 0x0, 0x0?})
	/Users/phil/work/super/pkg/charm/path.go:11 +0x7b
github.com/brimdata/super/pkg/charm.(*Spec).Exec(0x57f57d0, {0x3130f7f98790, 0x2, 0x2})
	/Users/phil/work/super/pkg/charm/charm.go:74 +0x1fa
main.main()
	/Users/phil/work/super/cmd/super/main.go:40 +0x5b

$ super -version &&
  super -c "SELECT a FROM t1 WHERE c > (SELECT avg(c) FROM t1) ORDER BY 1"

Version: v0.3.0-378-g2338849bb

{a:4}

Claude also gave its stab at root cause and possible direction for a fix, so its notes are up in a Gist if that helps.

[philrz]

Looks like that's fixed. Thanks @mattnibs!

$ super -version &&
  super -c "SELECT a FROM t1 WHERE c > (SELECT avg(c) FROM t1) ORDER BY 1"

Version: v0.3.0-379-ge8589c124

{a:4}