Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-18294

Limitation with Excel Writer step - appending data to an existing XLSX document does not work when "Stream XLSX data" is enabled.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Severity: High
    • Resolution: Fixed
    • Affects Version/s: 8.2.0.7 GA
    • Fix Version/s: 9.2.0 GA
    • Component/s: Step
    • Labels:
    • Environment:
      Data Integration (Spoon) 8.2.0.7
    • Story Points:
      0
    • PDI Sub-component:
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Tatooine (Maint)
    • Steps to Reproduce:
      Hide

      Please see ESR-6843 for reproduction path

      Show
      Please see ESR-6843 for reproduction path

      Description

      Pentaho Product: Data Integration
      Version: 8.2.0.7
      Operating System: Windows 10
      Memory Available/Used: 4Gb/32Gb
      Clustered/No: Not clustered
      Virtualized?: Not virtualized
      Database for repository: N/A
      Database for data: N/A
      Browser type and version (If Applicable): N/A
      Did you attach logs? Yes - see ESR-6843
      Did you attach sample demonstrating the problem? Yes - see ESR-6843

      With "Stream XLSX data" enabled in the Excel Writer step, the following exception is recorded when appending data to an existing xlsx spreadsheet:

      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - ERROR (version 8.2.0.7-719, build 8.2.0.7-719 from 2019-06-24 02.06.35 by buildguy) : Error writing line :java.lang.IllegalArgumentException: Attempting to write a row[0] in the range [0,1000] that is already written to disk.
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - ERROR (version 8.2.0.7-719, build 8.2.0.7-719 from 2019-06-24 02.06.35 by buildguy) : Unexpected error
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - ERROR (version 8.2.0.7-719, build 8.2.0.7-719 from 2019-06-24 02.06.35 by buildguy) : org.pentaho.di.core.exception.KettleException: 
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - java.lang.IllegalArgumentException: Attempting to write a row[0] in the range [0,1000] that is already written to disk.
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - Attempting to write a row[0] in the range [0,1000] that is already written to disk.
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - 
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - at org.pentaho.di.trans.steps.excelwriter.ExcelWriterStep.writeNextLine(ExcelWriterStep.java:332)
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - at org.pentaho.di.trans.steps.excelwriter.ExcelWriterStep.processRow(ExcelWriterStep.java:195)
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - at java.lang.Thread.run(Thread.java:748)
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - Caused by: java.lang.IllegalArgumentException: Attempting to write a row[0] in the range [0,1000] that is already written to disk.
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - at org.apache.poi.xssf.streaming.SXSSFSheet.createRow(SXSSFSheet.java:136)
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - at org.apache.poi.xssf.streaming.SXSSFSheet.createRow(SXSSFSheet.java:65)
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - at org.pentaho.di.trans.steps.excelwriter.ExcelWriterStep.writeNextLine(ExcelWriterStep.java:300)
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - ... 3 more
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - child index = 1, logging object : org.pentaho.di.core.logging.LoggingObject@1e7e6a42 parent=144923d3-2e3b-4254-8829-26d700454d55
      2019/08/06 10:38:06 - source_data tab Formatted Report Month.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)
      2019/08/06 10:38:06 - my_excel_writer_support_test_formatted - searching for annotations
      2019/08/06 10:38:06 - my_excel_writer_support_test_formatted - no annotations found
      2019/08/06 10:38:06 - my_excel_writer_support_test_formatted - ERROR (version 8.2.0.7-719, build 8.2.0.7-719 from 2019-06-24 02.06.35 by buildguy) : Errors detected!
      2019/08/06 10:38:06 - Spoon - The transformation has finished!!2019/08/06 10:38:06 - my_excel_writer_support_test_formatted - ERROR (version 8.2.0.7-719, build 8.2.0.7-719 from 2019-06-24 02.06.35 by buildguy) : Errors detected!
      2019/08/06 10:38:06 - my_excel_writer_support_test_formatted - ERROR (version 8.2.0.7-719, build 8.2.0.7-719 from 2019-06-24 02.06.35 by buildguy) : Errors detected!
      2019/08/06 10:38:06 - my_excel_writer_support_test_formatted - Transformation detected one or more steps with errors.
      

      This error doesn't occur when "Stream XLSX data" is disabled.

      After investigation in ESR-6843, this is a limitation on the step. The following changes should be made so the user is aware of this:

      1 Disable options in the UI that are not possible when streaming is enabled
      2 Error out with a meaningful error message when invalid options are used (such as "This option is not valid when 'Stream XLSX data' is enabled")

      A sample transformation and spreadsheet to reproduce this problem is attached to ESR-6843

        PractiTest Integration




          Attachments

            Issue Links

              Activity

                People

                Assignee:
                ana.mendes Ana Rita Clemente Mendes
                Reporter:
                nmidson Nick Midson
                Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved: