Skip to content

add insecure to search reindex - #3505

Merged
v-scharf merged 1 commit into
mainfrom
add-insecure-to-search-index
Sep 10, 2026
Merged

add insecure to search reindex#3505
v-scharf merged 1 commit into
mainfrom
add-insecure-to-search-index

Conversation

@v-scharf

@v-scharf v-scharf commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

during the testing #3197 I noticed that

$ opencloud search index --all-spaces --force-rescan
rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"

after added --insecure works fine:

~ $ opencloud search index --all-spaces --force-rescan --insecure
[1/7] indexed space 7230b71a-5362-486c-8f35-414d443b638b$dc67b41d-fbb3-4c52-b4d7-59a0774d0ce5!dc67b41d-fbb3-4c52-b4d7-59a0774d0ce5 in 10.579733921s
[2/7] indexed space 7230b71a-5362-486c-8f35-414d443b638b$c3919acb-aa79-4d1e-aa56-2d48cb36c891!c3919acb-aa79-4d1e-aa56-2d48cb36c891 in 10.668783838s
[3/7] indexed space 7230b71a-5362-486c-8f35-414d443b638b$b5a2d63c-5223-4263-bb2f-b1efbe500f13!b5a2d63c-5223-4263-bb2f-b1efbe500f13 in 9.702030129s
[4/7] indexed space 7230b71a-5362-486c-8f35-414d443b638b$e8ca4dda-cf6c-437f-b6a8-1e31033b0a0a!e8ca4dda-cf6c-437f-b6a8-1e31033b0a0a in 9.751792088s
[5/7] indexed space 7230b71a-5362-486c-8f35-414d443b638b$9ef27369-04f4-4b34-97fc-ccff31a9985d!9ef27369-04f4-4b34-97fc-ccff31a9985d in 8.726422712s
[6/7] indexed space 7230b71a-5362-486c-8f35-414d443b638b$4702248d-7570-4b77-bedc-0364af8a45b4!4702248d-7570-4b77-bedc-0364af8a45b4 in 9.808745254s
[7/7] indexed space 7230b71a-5362-486c-8f35-414d443b638b$cae518d9-869b-4cd1-8847-2591ab7a4a70!cae518d9-869b-4cd1-8847-2591ab7a4a70 in 1m41.059535047s

also added to log:
{"level":"info","service":"search","index":"/var/lib/opencloud/search/bleve-v4","time":"2026-09-10T09:08:19Z","line":"github.com/opencloud-eu/opencloud/services/search/pkg/mapping/reconcile.go:43","message":"created a new empty search index; if this OpenCloud instance already held files, they are not in it yet, index them by running: opencloud search index --all-spaces --force-rescan --insecure"}

@v-scharf
v-scharf requested review from butonic, dschmidt and fschade and removed request for dschmidt September 10, 2026 09:42
@codacy-production

codacy-production Bot commented Sep 10, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Coverage 50.00% diff coverage · +0.00% coverage variation

Metric Results
Coverage variation +0.00% coverage variation (-1.00%)
Diff coverage 50.00% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (994086c) 88539 20995 23.71%
Head commit (f511b1a) 88539 (+0) 20997 (+2) 23.71% (+0.00%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#3505) 2 1 50.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@dschmidt

Copy link
Copy Markdown
Contributor

I'm not sure about this.

Actually I wouldn't want to encourage usage of the --insecure flag, especially not as a default.

Could we instead improve the error message (or even deduct the --insecure flag automatically from the config file?)?

@butonic

butonic commented Sep 10, 2026

Copy link
Copy Markdown
Member

I have a history with this. The search index command uses the configured search endpoint. which is an internal grps endpoint. which is not tls encrypted (unless you enable tls for grpc ... try finding the docs). we default to grpc over h2 ... not http2 ... which is why we need to use --insecure. Sucks ... I know ... and annoys me ... but also ... is not really a security issue because we just want to trigger a rescan. which happens in the opencloud process, not the opencloud search index command ... 🤪

IMO the error message should just use the --insecure flag. or we hardcode insecure ... that actually seems less more wrong to me.

@butonic

butonic commented Sep 10, 2026

Copy link
Copy Markdown
Member

documenting it like in opencloud-eu/docs#1153 (review) is the least hassle.

@dschmidt

Copy link
Copy Markdown
Contributor

That's hella confusing 😂

I think requiring --insecure always and having it all over the docs is the worst of all options. So maybe just enable it always.. or.. why not the following?

Here's what my claude suggests:

The CLI already has the server's gRPC TLS config, it just evaluates it the wrong way round. OC_GRPC_CLIENT_TLS_MODE is never set automatically (opencloud init doesn't write it, nothing derives it), so the default is "". The go-micro client treats ""/off as "no TLS" (client.go#L75-L100), and the search server is plaintext unless OC_GRPC_TLS_ENABLED=true (server.go#L16-L20). The CLI however only dials plaintext for insecure and uses TLS for everything else, including the default (index.go#L49-L56). Plaintext server + TLS client is exactly the "first record does not look like a TLS handshake" error.

Mode go-micro client (all services) index.go on main
"" / off (default) no TLS TLS
insecure TLS, no verification no TLS
on TLS, verified TLS, verified

The config is available in the CLI via PersistentPreRunE (services.go#L264-L273) and EnsureDefaults (defaultconfig.go#L89-L94). Note OC_INSECURE is unrelated here, it only maps to events TLS and the CS3 extractor in the search config.

Proposed fix, mirroring the go-micro client (the endpoint hunk is optional, today it is a hardcoded flag default):

--- a/services/search/pkg/command/index.go
+++ b/services/search/pkg/command/index.go
@@ -46,10 +46,20 @@ func Index(cfg *config.Config) *cobra.Command {
 				return fmt.Errorf("concurrency %d exceeds max allowed %d", concurrencyFlag, cfg.ReindexMaxConcurrency)
 			}
 
+			if !cmd.Flags().Changed("endpoint") {
+				endpointFlag = cfg.GRPC.Addr
+			}
+
 			var dialOpts []grpc.DialOption
-			if cfg.GRPCClientTLS.Mode == "insecure" || insecureFlag {
+			// mirror pkg/service/grpc.NewClient: ""/"off" -> no TLS, "insecure" -> TLS without verification, "on" -> TLS
+			switch {
+			case insecureFlag, cfg.GRPCClientTLS.Mode == "", cfg.GRPCClientTLS.Mode == "off":
 				dialOpts = append(dialOpts, grpc.WithTransportCredentials(insecure.NewCredentials()))
-			} else {
+			case cfg.GRPCClientTLS.Mode == "insecure":
+				dialOpts = append(dialOpts, grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{
+					InsecureSkipVerify: true, //nolint:gosec
+				})))
+			default:
 				dialOpts = append(dialOpts, grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{
 					MinVersion: tls.VersionTLS12,
 				})))

Verified against a running server built from main with an opencloud init default config: main fails without --insecure, the patched CLI connects without it. With OC_GRPC_TLS_ENABLED=true + OC_GRPC_CLIENT_TLS_MODE=insecure, main fails (error reading server preface: EOF) while the patched CLI accepts the self-signed cert, and MODE=on correctly rejects it. With this, the README/MIGRATION/log changes in this PR are not needed. on still ignores OC_GRPC_CLIENT_TLS_CACERT, that could be a follow-up.

@v-scharf
v-scharf merged commit e503c2c into main Sep 10, 2026
67 of 68 checks passed
@v-scharf
v-scharf deleted the add-insecure-to-search-index branch September 10, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants