Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-15501

SFTP VFS provider does not get/attempt to perform public key authentication

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Severity: Urgent
    • Resolution: Fixed
    • Affects Version/s: 6.0.1 GA, 6.1.0.1 GA
    • Fix Version/s: 7.0.0 GA
    • Labels:
    • Environment:
    • Story Points:
      0
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Maintenance
    • Steps to Reproduce:
      Hide

      A. Create two user accounts on Linux system.
      B. Create a .ssh directory in each users home folder, permission 700.
      C. Create private and public key pair using ssh-keygen.
      •You can set up passphrase, but it is not needed.
      D. Copy the users public key into authorized_keys.
      E. Copy the private keys to mac. These can be stored anywhere
      e.g /tmp/keys/user1_key and /tmp/keys/user2_key
      F. Setup .ssh directory on mac for user who will be running spoon, permission 700
      G. Edit vfs.ktr. Set hostname property to linux host
      H. Create test file xls file for transformation and modify columns in read/write steps
      I. Copy test file to each users home directory.

      Test process
      1. Bring up vfs.ktr in spoon. Edit properties username, vfs.sftp.identity, &
      vfs.sftp.authkeypassphrase for user1
      2. Run transformation. This should fail.
      3. Copy the private keys file for user1 to ~/.ssh/id_rsa
      4. Run transformation. This should pass
      5. Alter transformation for user2 (See Step 1)
      6. Run transformation. This should fail.
      7. Copy the private keys file for user2 to ~/.ssh/id_rsa
      8. Run transformation. This should pass.

      Show
      A. Create two user accounts on Linux system. B. Create a .ssh directory in each users home folder, permission 700. C. Create private and public key pair using ssh-keygen. •You can set up passphrase, but it is not needed. D. Copy the users public key into authorized_keys. E. Copy the private keys to mac. These can be stored anywhere e.g /tmp/keys/user1_key and /tmp/keys/user2_key F. Setup .ssh directory on mac for user who will be running spoon, permission 700 G. Edit vfs.ktr. Set hostname property to linux host H. Create test file xls file for transformation and modify columns in read/write steps I. Copy test file to each users home directory. Test process 1. Bring up vfs.ktr in spoon. Edit properties username, vfs.sftp.identity, & vfs.sftp.authkeypassphrase for user1 2. Run transformation. This should fail. 3. Copy the private keys file for user1 to ~/.ssh/id_rsa 4. Run transformation. This should pass 5. Alter transformation for user2 (See Step 1) 6. Run transformation. This should fail. 7. Copy the private keys file for user2 to ~/.ssh/id_rsa 8. Run transformation. This should pass.

      Description

      The connection that is created by the SFTP VFS provider does not get/attempt to perform public key authentication.

      Internal validation of setup and test instructions (see case number in internal comments) found that the connection that is created by the SFTP VFS provider does not get/attempt to perform public key authentication, possibly because the FileSystemOption key we use is "identities", but it appears to be a new constant in Commons VFS 2.1.


      Here is a bit more background:

      The customer is attempting to access files using the VFS and an SFTP connection. The customer is able to get this work with one user, but when they run with another user, the second user cannot access the files. It doesn't seem to be a permissions issue because if they switch the order in which users are attempting to access the files, they have the same problem: the first user is able to access, but the second user can not. This appears to be a caching issue, but it also looks like the value of vfs.sftp.authkeypassphrase is not working consistently (it is either always used or is ignored.)

      When you run a ktr and use a id_rsa file that has an associated passphrase, the transformation can Read the files, but it cannot write. If you use a id_rsa file that was generated without a passphrase, it works correctly.

      A few other things:

      The JSON Input step does not work with id_rsa file with a passphrase. It generates an error that the file can't be found. Using a id_rsa file without passphrase works perfectly.

      The Property Input step doesn't seem to work with VFS SFTP (similar error to JSON Input).

      Ktr for testing is attached to the internal case (see internal comments for more information.)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              upihin Uladzimir Pihin (Inactive)
              Reporter:
              cbrathwaite Chantel Brathwaite (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: