Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-19130

Process Files: "Move" operation fails when moving a file between S3 buckets.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Severity: Medium
    • Resolution: Unresolved
    • Affects Version/s: 9.1.0 GA, 9.1.0.4 GA, 8.3.0.19 GA
    • Fix Version/s: Backlog
    • Component/s: Step
    • Labels:
      None
    • Story Points:
      0
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Steps to Reproduce:
      Hide
      • Make sure you have the valid credentials file the access_keys to connect to S3
      • Open the attached transformation in Spoon and change the source and destination parameters values as per your environment and execute.
      Show
      Make sure you have the valid credentials file the access_keys to connect to S3 Open the attached transformation in Spoon and change the source and destination parameters values as per your environment and execute.

      Description

      When moving the file from one S3 bucket to another using the Process Files step, we get the error as below

      2021/03/26 23:03:11 - S3 job move test - Dispatching started for transformation [S3 job move test]
      2021/03/26 23:03:11 - Get file and destination.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
      2021/03/26 23:03:25 - Move file.0 - ERROR (version 9.1.0.5-473, build 9.1.0.5-473 from 2021-02-02 10.14.44 by buildguy) : Because of an error, this step can't continue: Invalid copy source bucket name (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: TZ1JH1K81ZQY2JWC; S3 Extended Request ID: EjsdczNK9zlL5fjPYZBO1xRp2f5s0FwKA0rc7VvAXGRQaQFRDIulZ0aYqWuAc68nNankl8nDBQk=)
      2021/03/26 23:03:25 - Move file.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)
      2021/03/26 23:03:25 - S3 job move test - ERROR (version 9.1.0.5-473, build 9.1.0.5-473 from 2021-02-02 10.14.44 by buildguy) : Errors detected!
      2021/03/26 23:03:25 - Spoon - The transformation has finished!!
      

      This issue happens only with the "Move" operation and the transformation works as expected for "Copy" and "Delete"

      Attaching the sample transformation and log.

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            gdev Gurudev
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated: