Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-16387

Running a Job for Every Input Row, sub job does not reset variables or parameters with every execution

    XMLWordPrintable

    Details

    • Story Points:
      0
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Tatooine (Maint)
    • Steps to Reproduce:
      Hide

      1. Create a job with a parameter properties file.
      2. In the job use a Set variables step to read from the properties file with a scope of valid in the current job.
      3. Add a write to log step to write all of the properties you created to the log.
      4. Create 2 properties files. The files should set different properties.
      5. Create a transformation that gets each of the property file names and add them to the result.
      6. Create a parent job that takes the list of property files and calls the job created in step 1 for each result row.
      7. Note that properties that were set in one properties file but not the other are still set during the execution of the sub-job for the second properties file.

      Show
      1. Create a job with a parameter properties file. 2. In the job use a Set variables step to read from the properties file with a scope of valid in the current job. 3. Add a write to log step to write all of the properties you created to the log. 4. Create 2 properties files. The files should set different properties. 5. Create a transformation that gets each of the property file names and add them to the result. 6. Create a parent job that takes the list of property files and calls the job created in step 1 for each result row. 7. Note that properties that were set in one properties file but not the other are still set during the execution of the sub-job for the second properties file.

      Description

      When running a job for every input row, or using the job executor step variables that are set in previous executions of the job are still available in the next execution of the job even with the scope set to valid in the current job. It appears that when looping a job for every input row, the variable space of the sub-job does not get re-initialized from the parent job before each execution.

      This applies to both variables and parameters. If in the parent job you pass a parameter to the sub-job and the sub-job sets that parameter to a different value in a set variables step the parameter value in the second execution will be the one that was set in the first execution of the job.

      I would expect the variable space to get reset from the parent job before each execution and act like you put multiple "job" job entries in the parent job, one for each result row rather than using result rows.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              mleite Margarida Leite
              Reporter:
              cdeptula Chris Deptula
              Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: