Details

    • Type: Bug
    • Status: Closed
    • Severity: Medium
    • Resolution: Cannot Reproduce
    • Affects Version/s: 3.1.0 GA
    • Fix Version/s: 3.1.0 GA
    • Component/s: Kitchen
    • Labels:
      None
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Operating System/s:
      RedHat Enterprise Linux 4

      Description

      The attached job was running against a large data set (6.5 billion records in 4.5 * 30 * 24 files) and it eventually bombed out around three fourths of the way through after outputting 4760533 rows (it was conducting a 0.001% sampling of the source set).

      The only thing that should have been retained in memory was the list of files to process and I would expect that memory constraint to be constant.
      The execution arguments were:
      export JAVAMAXMEM=4096
      ./kitchen.sh -file 'zip:file:///home/etl/sample_2008.zip!/sample_all_2008.kjb' -level Minimal

      The tail of the console output was:

      INFO 21-05 10:37:12,972 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample]
      INFO 21-05 10:42:28,714 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-03.gz
      INFO 21-05 10:43:37,668 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file]
      INFO 21-05 12:11:07,905 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample]
      INFO 21-05 12:18:00,592 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-04.gz
      INFO 21-05 12:18:30,028 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file]
      INFO 21-05 14:45:46,102 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample]
      INFO 21-05 14:54:09,574 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-05.gz
      INFO 21-05 14:56:07,936 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file]
      INFO 21-05 20:51:43,483 (LogWriter.java:println:439) -calc_rows_to_sample - Dispatching started for transformation [calc_rows_to_sample]
      INFO 21-05 21:01:34,576 (LogWriter.java:println:439) -Calculate rows_to_sample.0 - access_2008-04-01-06.gz
      INFO 21-05 21:02:43,637 (LogWriter.java:println:439) -sample_one_file - Dispatching started for transformation [sample_one_file]
      ERROR 21-05 21:23:07,790 (LogWriter.java:println:436) -CSV file input.0 - UnexpectedError: java.lang.OutOfMemoryError: Java heap space
      ERROR 21-05 21:26:05,630 (LogWriter.java:println:436) -CSV file input.0 - java.lang.OutOfMemoryError: Java heap space
      at org.pentaho.di.trans.steps.csvinput.CsvInputData.resizeByteBuffer(CsvInputData.java:89)
      at org.pentaho.di.trans.steps.csvinput.CsvInput.readOneRow(CsvInput.java:465)
      at org.pentaho.di.trans.steps.csvinput.CsvInput.processRow(CsvInput.java:111)
      at org.pentaho.di.trans.step.BaseStep.runStepThread(BaseStep.java:2444)
      at org.pentaho.di.trans.steps.csvinput.CsvInput.run(CsvInput.java:641)

      INFO 21-05 21:37:59,000 (LogWriter.java:println:439) -sample_one_file - Transformation detected one or more steps with errors.
      ERROR 21-05 21:38:38,997 (LogWriter.java:println:436) -sample_one_file - Errors detected!
      INFO 21-05 21:39:18,442 (LogWriter.java:println:439) -sample_one_file - Transformation is killing the other steps!
      ERROR 21-05 21:40:37,335 (LogWriter.java:println:436) -sample_one_file - Errors detected!
      INFO 21-05 21:41:58,001 (LogWriter.java:println:439) -Kitchen - Finished!
      ERROR 21-05 21:41:58,002 (LogWriter.java:println:436) -Kitchen - Finished with errors
      INFO 21-05 21:41:58,002 (LogWriter.java:println:439) -Kitchen - Start=2008/05/19 17:10:47.428, Stop=2008/05/21 21:41:58.002
      INFO 21-05 21:41:58,002 (LogWriter.java:println:439) -Kitchen - Processing ended after 189070 seconds.

        Activity

        Hide
        deinspanjer Daniel Einspanjer (Inactive) added a comment -

        The transformation requires the Reservoir Sampling plug-in.

        Show
        deinspanjer Daniel Einspanjer (Inactive) added a comment - The transformation requires the Reservoir Sampling plug-in.
        Hide
        deinspanjer Daniel Einspanjer (Inactive) added a comment -

        This is a different version of the file that just continuously reads the same file /tmp/test.log.gz
        Unfortunately, I wasn't able to reproduce the problem on my local machine using this version.

        The only thing I can guess is that the fact that the job was running over the course of a few days caused some job result status tracking to fill up gradually.

        ::sigh::

        Show
        deinspanjer Daniel Einspanjer (Inactive) added a comment - This is a different version of the file that just continuously reads the same file /tmp/test.log.gz Unfortunately, I wasn't able to reproduce the problem on my local machine using this version. The only thing I can guess is that the fact that the job was running over the course of a few days caused some job result status tracking to fill up gradually. ::sigh::
        Hide
        deinspanjer Daniel Einspanjer (Inactive) added a comment -

        Can't reproduce outside the original environment. :/

        Show
        deinspanjer Daniel Einspanjer (Inactive) added a comment - Can't reproduce outside the original environment. :/

          People

          • Assignee:
            project admin Triage
            Reporter:
            deinspanjer Daniel Einspanjer (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: