Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-16017

Denormaliser step: Minimum Aggregation with multiple columns carries data across columns

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Severity: Urgent
    • Resolution: Fixed
    • Affects Version/s: 7.0.0.1 GA
    • Fix Version/s: 7.1.0 GA
    • Component/s: Step
    • Labels:
    • Story Points:
      0
    • PDI Sub-component:
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Tatooine (Maint)
    • Steps to Reproduce:
      Hide

      Denormaliser step: Minimum Aggregation
      1. Open the sample "Denormaliser - data conversion and aggregation"
      2. Edit the row denormaliser step:
      3. - Change the aggregation of all columns from Sum to Minimum
      4. - Change the type of all columns to string to work around PDI-16015
      5. Save transformation
      6. Run transformation
      7. Preview output of Row denormaliser step

      Results: In the first row, month01, month02, and month03 are all 1111.
      In the second row, month01 and month03 are 1234, month02 is 1111.
      Results for month03 in first row and month01 in second row may vary depending on row order.

      Expected results: In the first row, month01 is 1111 and month02 and month03 are null.
      In the second row, month01 is null, month03 is 1234, and month02 is 1111.

      Show
      Denormaliser step: Minimum Aggregation 1. Open the sample "Denormaliser - data conversion and aggregation" 2. Edit the row denormaliser step: 3. - Change the aggregation of all columns from Sum to Minimum 4. - Change the type of all columns to string to work around PDI-16015 5. Save transformation 6. Run transformation 7. Preview output of Row denormaliser step Results: In the first row, month01, month02, and month03 are all 1111. In the second row, month01 and month03 are 1234, month02 is 1111. Results for month03 in first row and month01 in second row may vary depending on row order. Expected results: In the first row, month01 is 1111 and month02 and month03 are null. In the second row, month01 is null, month03 is 1234, and month02 is 1111.

      Description

      When using minimum aggregation with multiple target fields, the value from the first row of the group may carry over to other columns. Both the sum and maximum aggregations handle this correctly.

      It appears to be initializing the minimum value for all target fields to the value of the value field in the first row of the group. If the all the values for a different target field are greater than that, or no values are present for that target field, it will output the value from a row with the wrong key value.

        Attachments

        1. AverageValuesAfter.PNG
          AverageValuesAfter.PNG
          8 kB
        2. AverageValuesBefore.PNG
          AverageValuesBefore.PNG
          9 kB
        3. ConcatenateValuesAfter.PNG
          ConcatenateValuesAfter.PNG
          15 kB
        4. ConcatenateValuesBefore.PNG
          ConcatenateValuesBefore.PNG
          15 kB
        5. MaxValueAfter.PNG
          MaxValueAfter.PNG
          8 kB
        6. MaxValueBefore.PNG
          MaxValueBefore.PNG
          8 kB
        7. MinimumValuesAfterNoStringConvert.PNG
          MinimumValuesAfterNoStringConvert.PNG
          9 kB
        8. MinimumValuesBefore - No string convert.PNG
          MinimumValuesBefore - No string convert.PNG
          7 kB
        9. MinimumValuesBefore - With String convert.PNG
          MinimumValuesBefore - With String convert.PNG
          8 kB
        10. NumberOfValuesAfter.PNG
          NumberOfValuesAfter.PNG
          8 kB
        11. NumberOfValuesBefore.PNG
          NumberOfValuesBefore.PNG
          8 kB
        12. SumValuesAfter.PNG
          SumValuesAfter.PNG
          8 kB
        13. SumValuesBefore.PNG
          SumValuesBefore.PNG
          9 kB

          Issue Links

            Activity

              People

              Assignee:
              upihin Uladzimir Pihin (Inactive)
              Reporter:
              kcaswick Kevin Caswick
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: