Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-19374

Multiple errors happening with existing Kinesis plugin

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Severity: Urgent
    • Resolution: Unresolved
    • Affects Version/s: 8.3.0 GA, 9.1.0 GA, 9.2.0 GA
    • Fix Version/s: Backlog
    • Component/s: Step
    • Labels:
      None
    • Story Points:
      8
    • PDI Sub-component:
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Ackbar

      Description

      Customer has found the following issues with the Kinesis Plugin

      Issue 1: Duplicate message process are being recorded on DynamoDB
      Issue 2: ERROR [ShardConsumer] Cancelling subscription, and restarting
      https://github.com/awslabs/amazon-kinesis-client/issues/448
      https://github.com/awslabs/amazon-kinesis-client/issues/463
      https://stackoverflow.com/questions/61481263/kcl-stops-processing-data-after-throwing-error-cancelling-subscription-and-rest

      Errors shown:

      15:19:54,661 ERROR [ShardConsumer] shardId-000000000000: Last request was dispatched at 2021-09-29T15:19:19.657Z, but no response as of 2021-09-29T15:19:54.661Z (PT35.004S).  Cancelling subscription, and restarting.
      15:20:29,664 ERROR [ShardConsumer] shardId-000000000000: Last request was dispatched at 2021-09-29T15:19:54.661Z, but no response as of 2021-09-29T15:20:29.664Z (PT35.003S).  Cancelling subscription, and restarting.
      15:21:04,668 ERROR [ShardConsumer] shardId-000000000000: Last request was dispatched at 2021-09-29T15:20:29.665Z, but no response as of 2021-09-29T15:21:04.668Z (PT35.003S).  Cancelling subscription, and restarting.
      15:21:39,672 ERROR [ShardConsumer] shardId-000000000000: Last request was dispatched at 2021-09-29T15:21:04.668Z, but no response as of 2021-09-29T15:21:39.672Z (PT35.004S).  Cancelling subscription, and restarting.
      

      Searching suggests MAX_TIME_BETWEEN_REQUEST_RESPONSE is hard coded to 35 seconds as shown on the timestamp on the logs
      Solution: Upgrading to KCL

      Issue 3: TimeoutException

      19:31:03,790 INFO  [Scheduler] No activities assigned
      19:31:03,790 INFO  [Scheduler] Sleeping ...
      19:31:08,744 ERROR [RunnableRequest] Failed to create connection to https://monitoring.us-east-1.amazonaws.com
      java.util.concurrent.TimeoutException: Acquire operation took longer then configured maximum time
              at software.amazon.awssdk.http.nio.netty.internal.utils.BetterFixedChannelPool.<init>(...)(Unknown Source)
      19:31:18,744 ERROR [RunnableRequest] Failed to create connection to https://monitoring.us-east-1.amazonaws.com
      java.util.concurrent.TimeoutException: Acquire operation took longer then configured maximum time
              at software.amazon.awssdk.http.nio.netty.internal.utils.BetterFixedChannelPool.<init>(...)(Unknown Source)
      19:31:28,744 ERROR [RunnableRequest] Failed to create connection to https://monitoring.us-east-1.amazonaws.com
      java.util.concurrent.TimeoutException: Acquire operation took longer then configured maximum time
              at software.amazon.awssdk.http.nio.netty.internal.utils.BetterFixedChannelPool.<init>(...)(Unknown Source)
      19:31:38,744 ERROR [RunnableRequest] Failed to create connection to https://monitoring.us-east-1.amazonaws.com
      java.util.concurrent.TimeoutException: Acquire operation took longer then configured maximum time
              at software.amazon.awssdk.http.nio.netty.internal.utils.BetterFixedChannelPool.<init>(...)(Unknown Source)
      19:31:48,744 ERROR [RunnableRequest] Failed to create connection to https://monitoring.us-east-1.amazonaws.com
      java.util.concurrent.TimeoutException: Acquire operation took longer then configured maximum time
              at software.amazon.awssdk.http.nio.netty.internal.utils.BetterFixedChannelPool.<init>(...)(Unknown Source)
      19:31:58,744 ERROR [RunnableRequest] Failed to create connection to https://monitoring.us-east-1.amazonaws.com
      java.util.concurrent.TimeoutException: Acquire operation took longer then configured maximum time
              at software.amazon.awssdk.http.nio.netty.internal.utils.BetterFixedChannelPool.<init>(...)(Unknown Source)
      

      Related searches:
      https://github.com/awslabs/amazon-kinesis-client/issues/399
      https://github.com/awslabs/amazon-kinesis-client/issues/651

      Solution: Upgrading KCL and Kinesis libraries

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            clopez Carlos Lopez
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: