Skip to content

Fix#8500 Align CSV Input Verify with filename-from-field and static filename modes - #8502

Open
leehaut wants to merge 2 commits into
apache:mainfrom
leehaut:hotfix/lance-common-30
Open

Fix#8500 Align CSV Input Verify with filename-from-field and static filename modes#8502
leehaut wants to merge 2 commits into
apache:mainfrom
leehaut:hotfix/lance-common-30

Conversation

@leehaut

@leehaut leehaut commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Fix #8500

Summary

  • CSV Input Verify (F7) no longer treats an incoming hop as an error. That matches the dialog and runtime: a previous transform means the filename comes from a field.
  • In filename-from-field mode, check that the filename field is set and exists in the previous row meta.
  • In static filename mode, report an error when no filename is specified (same condition as CsvInput.init()).
  • Split CsvInputMeta.check() into helpers so cognitive complexity stays under the Sonar limit.

Test plan

  • Filename from field: Generate rows → CSV Input, filename field set to a real upstream field. F7 should be OK (no “receiving input” ERROR).
  • Filename from field: hop present but filename field empty, or field name not in previous fields. F7 should ERROR.
  • Static filename: CSV Input with no incoming hop and a file path set. F7 should be OK.
  • Static filename: no incoming hop and filename empty. F7 should ERROR with “No filename is specified to read from.”
  • Downstream hop only (CSV Input → Write to log) still uses static filename mode.
  • CsvInputMetaTest check cases pass.

@leehaut

leehaut commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

@hansva CI failure

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Hop Orchestration Platform 2.20.0-SNAPSHOT:
[INFO] 
[INFO] Hop Orchestration Platform ......................... SUCCESS [  3.253 s]
[INFO] Hop Core ........................................... SUCCESS [ 30.340 s]
[INFO] Hop Engine ......................................... SUCCESS [ 25.018 s]
[INFO] Hop p2 Libraries ................................... SUCCESS [  0.831 s]
[INFO] Eclipse SWTBot Wrapper ............................. SUCCESS [ 13.401 s]
[INFO] Eclipse TM4E Core Wrapper .......................... SUCCESS [ 13.400 s]
[INFO] Hop GUI ............................................ SUCCESS [ 24.772 s]
[INFO] Hop GUI (RCP fragment) ............................. SUCCESS [  3.571 s]
[INFO] Hop Plugins ........................................ SUCCESS [  0.474 s]
[INFO] Hop Plugins Transforms ............................. SUCCESS [  0.222 s]
[INFO] Hop Plugins Miscellaneous .......................... SUCCESS [  0.209 s]
[INFO] Hop Static Schema .................................. SUCCESS [  2.019 s]
[INFO] Hop Plugins Transforms Text File ................... SUCCESS [ 13.418 s]
[INFO] Hop Plugins Engines Spark .......................... FAILURE [  0.324 s]
[INFO] Hop Plugins Miscellaneous Git ...................... FAILURE [  0.077 s]
[INFO] Hop Plugins Transforms Excel ....................... SUCCESS [ 15.507 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:02 min (Wall Clock)
[INFO] Finished at: 2026-09-21T15:56:19Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hop-misc-git: Could not resolve dependencies for project org.apache.hop:hop-misc-git:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-action-pipeline:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-action-pipeline:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-action-workflow:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-action-workflow:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-calculator:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-calculator:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-checksum:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-checksum:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-selectvalues:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-selectvalues:jar:2.20.0-SNAPSHOT
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project hop-misc-git: Could not resolve dependencies for project org.apache.hop:hop-misc-git:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-action-pipeline:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-action-pipeline:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-action-workflow:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-action-workflow:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-calculator:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-calculator:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-checksum:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-checksum:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-selectvalues:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-selectvalues:jar:2.20.0-SNAPSHOT

    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:246)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:136)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:355)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:314)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project org.apache.hop:hop-misc-git:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-action-pipeline:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-action-pipeline:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-action-workflow:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-action-workflow:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-calculator:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-calculator:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-checksum:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-checksum:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-selectvalues:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-selectvalues:jar:2.20.0-SNAPSHOT

    at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:191)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:227)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:136)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:355)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:314)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
[ERROR] Failed to execute goal on project hop-engines-spark: Could not resolve dependencies for project org.apache.hop:hop-engines-spark:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-memgroupby:jar:2.20.0-SNAPSHOT (provided)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-memgroupby:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-mergejoin:jar:2.20.0-SNAPSHOT (provided)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-mergejoin:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-sort:jar:2.20.0-SNAPSHOT (provided)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-sort:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-uniquerows:jar:2.20.0-SNAPSHOT (provided)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-uniquerows:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-filterrows:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-filterrows:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-groupby:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-groupby:jar:2.20.0-SNAPSHOT
[ERROR] dependency: org.apache.hop:hop-transform-streamlookup:jar:2.20.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact org.apache.hop:hop-transform-streamlookup:jar:2.20.0-SNAPSHOT
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project hop-engines-spark: Could not resolve dependencies for project org.apache.hop:hop-engines-spark:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-memgroupby:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-memgroupby:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-mergejoin:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-mergejoin:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-sort:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-sort:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-uniquerows:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-uniquerows:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-filterrows:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-filterrows:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-groupby:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-groupby:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-streamlookup:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-streamlookup:jar:2.20.0-SNAPSHOT

    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:246)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:136)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:355)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:314)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project org.apache.hop:hop-engines-spark:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-memgroupby:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-memgroupby:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-mergejoin:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-mergejoin:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-sort:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-sort:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-uniquerows:jar:2.20.0-SNAPSHOT (provided)
	Could not find artifact org.apache.hop:hop-transform-uniquerows:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-filterrows:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-filterrows:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-groupby:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-groupby:jar:2.20.0-SNAPSHOT
dependency: org.apache.hop:hop-transform-streamlookup:jar:2.20.0-SNAPSHOT (test)
	Could not find artifact org.apache.hop:hop-transform-streamlookup:jar:2.20.0-SNAPSHOT

    at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:191)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:227)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:136)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:355)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:314)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hop-misc-git
Error: Process completed with exit code 1.

The Hop PR Build (Code) failure does not come from the CSV Input check() changes in this PR.

What this PR changes

  • CsvInputMeta.java — align check() with dialog / runtime (filename-from-field vs static filename)
  • i18n keys for the new check messages
  • CsvInputMetaTest

Local verification

mvn -pl plugins/transforms/textfile test

CSV Input / textfile tests pass locally.

What fails on CI

Dependency resolution for downstream modules pulled in by the incremental reactor (GIB), for example:

Downstream module Missing SNAPSHOT artifacts (examples)
hop-misc-git hop-action-pipeline, hop-transform-calculator, …
hop-engines-spark hop-transform-sort, hop-transform-memgroupby, …

Those 2.20.0-SNAPSHOT artifacts are not on Maven Central. On a PR, GIB_DISABLE=false builds changed modules plus dependents, but does not always build the other SNAPSHOT dependencies those dependents need, so Maven reports:

Could not find artifact org.apache.hop:…:jar:2.20.0-SNAPSHOT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: CSV Input Verify reports errors when the filename comes from a previous transform

1 participant

@leehaut