Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-1853

As an ETL developer, I need variables in Table Output for "Commit size" and Add sequence "Maximum value"

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Severity: Medium
    • Resolution: Fixed
    • Affects Version/s: 3.0.4, 3.1.0 GA
    • Fix Version/s: 3.1.1, 3.2.0 GA
    • Component/s: Step
    • Labels:
      None
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.

      Description

      Use case:
      We built a transformation that pulls data from a web service in small sections (e.g. 10 rows at a time) where the limit is specified by the client.

      We then insert those rows into a database using the table output.

      We want to commit only when each section of data is finished.

      Additionlly, we need to call another webservice when the section is complete to let the server know that the insert was successful. I used an Add Sequence step with the same max value as the commit size to do this (see attached transformation). So this means, we'd also like the Add Sequence to allow Max Value to be a variable too unless there is a better way to do this.

        Attachments

          Activity

            People

            • Assignee:
              gdavid Golda Thomas
              Reporter:
              jbleuel Jens Bleuel
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: