Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-7907

As an ETL-Developer, I want my jobs support restartability

    XMLWordPrintable

    Details

    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint:
      ElUltimo

      Description

      Functions:

      • Support Checkpoints

      Use Case:
      I need to ensure that my job can be restarted after an interruption (e.g. stopped or by error) at a specified checkpoint with the same variables and arguments. Checkpoints ensure that the status is serialized (variables, arguments, files and rows from the result set) and the job can be continued after a safe state that is defined at design time. This avoids unwanted runs of previous tasks that succeeded successful.

      Success Criteria:

      • I can define checkpoints within my job
      • After a job failed, I can resume a job after the following job entry of the last successful checkpoint

      Notes:

      • Add a clear warning box to the user (similar to the warning box for running job entries in parallel).
      • checkpoints need an expiration date/time or combine with dependencies (PDI-7910)
      • Have an attempt_id or similar
      • Provide a Attempts-Log table
      • Save the state in the file system (e.g. attempt_state folder)

        Attachments

        1. checkpoints_test1.zip
          18 kB
        2. checkpoints_test2.zip
          23 kB
        3. checkpoints_tests_final.zip
          30 kB
        4. CP_UI_issue.pdf
          97 kB
        5. CP_UI-final-issue.png
          CP_UI-final-issue.png
          14 kB

          Issue Links

            Activity

              People

              • Assignee:
                jbleuel Jens Bleuel
                Reporter:
                jbleuel Jens Bleuel
              • Votes:
                2 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: