Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-14901

Clustered Transformation does not release sockets after error

    XMLWordPrintable

    Details

    • Story Points:
      0
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Steps to Reproduce:
      Hide
      1. Set up a Carte cluster
      2. Configure a PDI transformation to use Clustering
      3. Modify the transformation so that it will fail when executed
      4. From Spoon, use "Clustered Execution" to run the transformation
      5. Modify the transformation so that it will succeed when executed
      6. From Spoon, use "Clustered Execution" to run the transformation again
        Expected Results: The transformation runs successfully
        Actual Results: The transformation fails due to a socket not being available
      Show
      Set up a Carte cluster Configure a PDI transformation to use Clustering Modify the transformation so that it will fail when executed From Spoon, use "Clustered Execution" to run the transformation Modify the transformation so that it will succeed when executed From Spoon, use "Clustered Execution" to run the transformation again Expected Results: The transformation runs successfully Actual Results: The transformation fails due to a socket not being available

      Description

      When a transformation is executed on a Carte Cluster, but fails for any reason, the Sockets created by the clustering do not get released. This prevents a transformation from being re-executed until the affected Carte servers are restarted.

      An error occurred while preparing the execution of the master transformation: Initialisation of transformation failed: 
      2016/01/29 12:13:43 - General - Logging plugin type found with ID: CheckpointLogTable
      2016/01/29 12:13:47 - Carte - Installing timer to purge stale objects after 1440 minutes.
      2016/01/29 12:13:47 - Carte - Created listener for webserver @ address : localhost:9081
      2016/01/29 12:14:19 - kettleClusteringTimestampBug (master) - Dispatching started for transformation [kettleClusteringTimestampBug (master)]
      2016/01/29 12:14:19 - Writer : 40001.0 - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : Error creating server socket: java.io.IOException: Server socket on port 40001 is already in use by [kettleClusteringTimestampBug (master) - Writer : 40001.0]
      2016/01/29 12:14:19 - Writer : 40001.0 - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : java.io.IOException: Server socket on port 40001 is already in use by [kettleClusteringTimestampBug (master) - Writer : 40001.0]
      2016/01/29 12:14:19 - Writer : 40001.0 - 	at org.pentaho.di.www.SocketRepository.openServerSocket(SocketRepository.java:113)
      2016/01/29 12:14:19 - Writer : 40001.0 - 	at org.pentaho.di.trans.steps.socketwriter.SocketWriter.init(SocketWriter.java:150)
      2016/01/29 12:14:19 - Writer : 40001.0 - 	at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:69)
      2016/01/29 12:14:19 - Writer : 40001.0 - 	at java.lang.Thread.run(Thread.java:745)
      2016/01/29 12:14:19 - Writer : 40002.0 - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : Error creating server socket: java.io.IOException: Server socket on port 40002 is already in use by [kettleClusteringTimestampBug (master) - Writer : 40002.0]
      2016/01/29 12:14:19 - Writer : 40001.0 - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : Error initializing step [Writer : 40001]
      2016/01/29 12:14:19 - Writer : 40002.0 - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : java.io.IOException: Server socket on port 40002 is already in use by [kettleClusteringTimestampBug (master) - Writer : 40002.0]
      2016/01/29 12:14:19 - Writer : 40002.0 - 	at org.pentaho.di.www.SocketRepository.openServerSocket(SocketRepository.java:113)
      2016/01/29 12:14:19 - Writer : 40002.0 - 	at org.pentaho.di.trans.steps.socketwriter.SocketWriter.init(SocketWriter.java:150)
      2016/01/29 12:14:19 - Writer : 40002.0 - 	at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:69)
      2016/01/29 12:14:19 - Writer : 40002.0 - 	at java.lang.Thread.run(Thread.java:745)
      2016/01/29 12:14:19 - Writer : 40002.0 - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : Error initializing step [Writer : 40002]
      2016/01/29 12:14:19 - kettleClusteringTimestampBug (master) - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : Step [Writer : 40001.0] failed to initialize!
      2016/01/29 12:14:19 - kettleClusteringTimestampBug (master) - ERROR (version 5.4.0.4-149, build 1 from 2015-09-17 09.58.29 by buildguy) : Step [Writer : 40002.0] failed to initialize!
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              eteixeira Emanuel Teixeira (Inactive)
              Reporter:
              mtucker Matt Tucker
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: