Pentaho Data Integration - Kettle
  1. Pentaho Data Integration - Kettle
  2. PDI-1788

Performance issue against PostgreSQL using Kettle 3.1 GA

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Unknown Unknown
    • Resolution: Fixed
    • Affects Version/s: 3.1.0 GA
    • Fix Version/s: 3.1.1, 3.2.0 GA
    • Component/s: Step
    • Labels:
      None
    • Environment:
      Kettle 3.1 GA (kettle memory Xmx256m)
      ADM 64 bit single core machine
      3 GB physical memory
    • Operating System/s:
      Windows XP
    • Customer Case:

      Description

      Performance issue against PostgreSQL using Kettle 3.1 GA

      Dimension lookup/update step takes unusually long time to populate the records when running against PostgreSQL databases.. This issue is specific to PostgreSQL only

      Please find the "Dimension lookup performance Results.xls" document attachment for performance test results.

      This result are based on the he the execution of the DimPerfTest.ktr transformation.



       
      1. Dimension lookup performance Results.xls
        17 kB
        Gangadhara Boranna
      2. DimPerfTest.ktr
        10 kB
        Gangadhara Boranna

        Activity

        Hide
        Matt Casters added a comment -
        Reverted changes, uncommented autoinc and nullable for version 3.1.1 in revision 9472
        Show
        Matt Casters added a comment - Reverted changes, uncommented autoinc and nullable for version 3.1.1 in revision 9472
        Hide
        Matt Casters added a comment -
        also ported the fix to trunk in revision 9473 so that we can start with a clean slate.
        Show
        Matt Casters added a comment - also ported the fix to trunk in revision 9473 so that we can start with a clean slate.
        Hide
        Jens Bleuel added a comment -
        Due to the performance gains we could additionally get by refactoring the database class, I created PDI-1854 for a follow-up on this in a later release.
        Show
        Jens Bleuel added a comment - Due to the performance gains we could additionally get by refactoring the database class, I created PDI-1854 for a follow-up on this in a later release.
        Hide
        Markus Kaemmerer added a comment -
        After you reverted the change, Kettle 3.1 works very well again (with postgres) and is as fast as Kettle 3.0
        Show
        Markus Kaemmerer added a comment - After you reverted the change, Kettle 3.1 works very well again (with postgres) and is as fast as Kettle 3.0
        Hide
        Kurtis Cruzada added a comment -
        Bulk change to close issue for QA Validation.
        Show
        Kurtis Cruzada added a comment - Bulk change to close issue for QA Validation.

          People

          • Assignee:
            Golda David
            Reporter:
            Gangadhara Boranna
          • Votes:
            2 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: