Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-17391

Calculator step fails with csv input and lazy conversion active (Function: Create a copy of field A)

    XMLWordPrintable

    Details

    • Story Points:
      0
    • PDI Sub-component:
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Tatooine (Maint)
    • Operating System/s:
      Windows 10
    • Steps to Reproduce:
      Hide

      1) Save the attached ktr and csv.
      2) Run the ktr and note the KettleValueException (see stacktrace in desc)
      3) Modify the csv step, turning off lazy conversion.
      4) Rerun, should be no error.

      Show
      1) Save the attached ktr and csv. 2) Run the ktr and note the KettleValueException (see stacktrace in desc) 3) Modify the csv step, turning off lazy conversion. 4) Rerun, should be no error.

      Description

      Reproduction steps above.

      This error can occur whenever fields from a CSV step using lazy conversion are passed to a Calculator step that performs a metadata conversion. The error below will occur.

      This appears to happen because the result storage type is created with the default (native) storage type even if the input field is using the binary string storage type.

      2018/06/27 11:13:59 - Calculator.0 - ERROR (version 8.1.0.0-365, build 8.1.0.0-365 from 2018-04-30 09.42.24 by buildguy) : Unexpected error
      2018/06/27 11:13:59 - Calculator.0 - ERROR (version 8.1.0.0-365, build 8.1.0.0-365 from 2018-04-30 09.42.24 by buildguy) : org.pentaho.di.core.exception.KettleStepException:
      2018/06/27 11:13:59 - Calculator.0 - Error while running the step
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 - resultType: 3; targetMeta: 2
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 - result Date : There was a data type error: the data type of [B object [[B@442a68f1] does not correspond to value meta [Date]
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 -    at org.pentaho.di.trans.steps.calculator.Calculator.processRow(Calculator.java:179)
      2018/06/27 11:13:59 - Calculator.0 -    at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
      2018/06/27 11:13:59 - Calculator.0 -    at java.lang.Thread.run(Thread.java:748)
      2018/06/27 11:13:59 - Calculator.0 - Caused by: org.pentaho.di.core.exception.KettleValueException:
      2018/06/27 11:13:59 - Calculator.0 - resultType: 3; targetMeta: 2
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 - result Date : There was a data type error: the data type of [B object [[B@442a68f1] does not correspond to value meta [Date]
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 -    at org.pentaho.di.trans.steps.calculator.Calculator.calcFields(Calculator.java:631)
      2018/06/27 11:13:59 - Calculator.0 -    at org.pentaho.di.trans.steps.calculator.Calculator.processRow(Calculator.java:163)
      2018/06/27 11:13:59 - Calculator.0 -    ... 2 more
      2018/06/27 11:13:59 - Calculator.0 - Caused by: org.pentaho.di.core.exception.KettleValueException:
      2018/06/27 11:13:59 - Calculator.0 - result Date : There was a data type error: the data type of [B object [[B@442a68f1] does not correspond to value meta [Date]
      2018/06/27 11:13:59 - Calculator.0 -
      2018/06/27 11:13:59 - Calculator.0 -    at org.pentaho.di.core.row.value.ValueMetaBase.getString(ValueMetaBase.java:1896)
      2018/06/27 11:13:59 - Calculator.0 -    at org.pentaho.di.core.row.value.ValueMetaBase.convertData(ValueMetaBase.java:3807)
      2018/06/27 11:13:59 - Calculator.0 -    at org.pentaho.di.trans.steps.calculator.Calculator.calcFields(Calculator.java:628)
      2018/06/27 11:13:59 - Calculator.0 -    ... 3 more
      2018/06/27 11:13:59 - Calculator.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)
      

        Attachments

        1. data.csv
          0.0 kB
        2. date_csv.ktr
          14 kB

          Issue Links

            Activity

              People

              Assignee:
              lcarvalho Laura Carvalho
              Reporter:
              amartins Andre Martins
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: