Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-10704

"Get rows from result" step does not pass rows when transformation is executed in cluster mode

    XMLWordPrintable

    Details

    • Story Points:
      5
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Pervach
    • Operating System/s:
      Windows 7

      Description

      Run the following job with a cluster configured with three carte servers. You'll see that the second transformation (which runs in cluster mode) does not receive any rows from the previous transformation. When the second transformation is not configured to run in clustered mode, everything works fine and rows are passed with no issue.

      In the Master server, you'll see this in the logs indicating the clustered transformation is receiving the rows:

      2013/10/01 17:51:31 - TEST_Main - Start of job execution
      2013/10/01 17:51:31 - TEST_Main - Starting entry [TEST_GenRows]
      2013/10/01 17:51:31 - TEST_GenRows - Loading transformation from XML file [C:\Users\CMorehouse\Desktop\37187\cluster\cluster\TEST_GenRows.ktr]
      2013/10/01 17:51:31 - TEST_GenRows - Dispatching started for transformation [TEST_GenRows]
      2013/10/01 17:51:31 - Generate Hello.0 - Finished processing (I=0, O=0, R=0, W=1, U=0, E=0)
      2013/10/01 17:51:31 - Generate Bonjour.0 - Finished processing (I=0, O=0, R=0, W=1, U=0, E=0)
      2013/10/01 17:51:31 - Alert Rows.0 - Optimization level not specified. Using default of 9.
      2013/10/01 17:51:31 - Copy rows to result.0 - Finished processing (I=0, O=0, R=2, W=2, U=0, E=0)
      2013/10/01 17:51:31 - Alert Rows.0 - Finished processing (I=0, O=0, R=2, W=2, U=0, E=0)
      2013/10/01 17:51:31 - TEST_Main - Starting entry [TEST_AlertRows]
      2013/10/01 17:51:31 - TEST_AlertRows - Loading transformation from XML file [C:\Users\CMorehouse\Desktop\37187\cluster\cluster\TEST_AlertRows.ktr]
      2013/10/01 17:51:34 - TEST_AlertRows - All transformations in the cluster have finished.
      2013/10/01 17:51:36 - TEST_AlertRows - All transformations in the cluster have finished.
      2013/10/01 17:51:36 - TEST_Main - Starting entry [Success 1]
      2013/10/01 17:51:36 - TEST_Main - Finished job entry [Success 1] (result=[true])
      2013/10/01 17:51:36 - TEST_Main - Finished job entry [TEST_AlertRows] (result=[true])
      2013/10/01 17:51:36 - TEST_Main - Finished job entry [TEST_GenRows] (result=[true])
      2013/10/01 17:51:36 - TEST_Main - Job execution finished

      But in the slave servers, you only see this:

      2013/10/01 17:51:34 - TEST_AlertRows (Cluster1:Slave1) - Dispatching started for transformation [TEST_AlertRows (Cluster1:Slave1)]

      Also attached is a screenshot of the slave server metrics screen where you see no rows being read.

        Attachments

          Activity

            People

            Assignee:
            ihar_chekan Ihar Chekan (Inactive)
            Reporter:
            cmorehouse Christopher Morehouse
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: