Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-15452

Kettle Crashes With OoM When Running Jobs with Loops

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Severity: Unknown
    • Resolution: Won't Fix
    • Affects Version/s: 6.0.0 GA, 6.1.0 GA
    • Fix Version/s: None
    • Component/s: Core (Engine)
    • Labels:
      None
    • Environment:
      Windows Here, but Will Crash with Any
    • Story Points:
      0
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Steps to Reproduce:
      Hide

      Run the attached job in spoon/kitchen/carte. Eventually the JVM will crash with OoM

      Show
      Run the attached job in spoon/kitchen/carte. Eventually the JVM will crash with OoM

      Description

      After running a job with a loop for some time, Kettle stores "Result" objects for each iteration with a growing "logText" property. The logText contains the log's text from previous iterations of the loop with the additional text from the current iteration of the loop.

      Example:

      Iteration 1: Result.logText = "Hello World\n"
      Iteration 2: Result.logText = "Hello World\nHello World\n"
      Iteration 3: Result.logText = "Hello World\nHello World\nHello World\n"

      Also, something in Kettle is hanging onto these org.pentaho.di.core.Result objects indefinitely. The heap quickly completely fills up with char[] for the String logText member of the Result class. It also seems to be keeping a lot of Strings for JobEntryResult type.

      Kettle can detect cycles in job definitions. Is there a way to configure something so only the latest Result is kept? Or is there a configuration to disable job metric collection?

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jbleuel Jens Bleuel
              Reporter:
              jjarvis Jonathan Jarvis
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: