Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-4010

Error posting transformation on remote carte server

    Details

    • Type: Bug
    • Status: Closed
    • Severity: High
    • Resolution: Fixed
    • Affects Version/s: 4.0.0 GA
    • Fix Version/s: 4.1.2
    • Component/s: None
    • Labels:
      None
    • Environment:
      enterprise repository, windows 7, java 1.5
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Browser:
      Mozilla Firefox 3.5
    • Operating System/s:
      Windows 7

      Description

      Trying to validate PDI-3895 from steps found in spira testcase TC479, got an error while trying to execute job with a simple transformation on a remote carte server.

      stack trace...

      org.pentaho.di.core.exception.KettleException:
      org.pentaho.di.core.exception.KettleException:
      There was an error posting the transformation on the remote server:
      org.pentaho.di.core.exception.KettleException:
      Unable to connect to the repository with name 'MyTest1'

      java.lang.AssertionError: java.security.NoSuchAlgorithmException: PBEWithMD5AndDES SecretKeyFactory not available
      java.security.NoSuchAlgorithmException: PBEWithMD5AndDES SecretKeyFactory not available

      org.pentaho.di.trans.TransExecutionConfiguration.<init>(TransExecutionConfiguration.java:656)
      org.pentaho.di.trans.TransConfiguration.<init>(TransConfiguration.java:57)
      org.pentaho.di.trans.TransConfiguration.fromXML(TransConfiguration.java:66)
      org.pentaho.di.www.AddTransServlet.doGet(AddTransServlet.java:85)
      org.pentaho.di.www.BaseHttpServlet.doPost(BaseHttpServlet.java:88)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
      org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
      org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
      org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      org.mortbay.jetty.Server.handle(Server.java:326)
      org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:536)
      org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:930)
      org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
      org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
      org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405)
      org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
      org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

      There was an error posting the transformation on the remote server:
      org.pentaho.di.core.exception.KettleException:
      Unable to connect to the repository with name 'MyTest1'

      java.lang.AssertionError: java.security.NoSuchAlgorithmException: PBEWithMD5AndDES SecretKeyFactory not available
      java.security.NoSuchAlgorithmException: PBEWithMD5AndDES SecretKeyFactory not available

      org.pentaho.di.trans.TransExecutionConfiguration.<init>(TransExecutionConfiguration.java:656)
      org.pentaho.di.trans.TransConfiguration.<init>(TransConfiguration.java:57)
      org.pentaho.di.trans.TransConfiguration.fromXML(TransConfiguration.java:66)
      org.pentaho.di.www.AddTransServlet.doGet(AddTransServlet.java:85)
      org.pentaho.di.www.BaseHttpServlet.doPost(BaseHttpServlet.java:88)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
      org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
      org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
      org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      org.mortbay.jetty.Server.handle(Server.java:326)
      org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:536)
      org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:930)
      org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
      org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
      org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405)
      org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
      org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

      at org.pentaho.di.trans.Trans.sendToSlaveServer(Trans.java:3017)
      at org.pentaho.di.ui.spoon.delegates.SpoonTransformationDelegate.executeTransformation(SpoonTransformationDelegate.java:919)
      at org.pentaho.di.ui.spoon.Spoon$31$1.run(Spoon.java:7135)
      at org.eclipse.swt.widgets.RunnableLock.run(Unknown Source)
      at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Unknown Source)
      at org.eclipse.swt.widgets.Display.runAsyncMessages(Unknown Source)
      at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
      at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1147)
      at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:6589)
      at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:544)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at org.pentaho.commons.launcher.Launcher.main(Launcher.java:110)
      Caused by: org.pentaho.di.core.exception.KettleException:
      There was an error posting the transformation on the remote server:
      org.pentaho.di.core.exception.KettleException:
      Unable to connect to the repository with name 'MyTest1'

      java.lang.AssertionError: java.security.NoSuchAlgorithmException: PBEWithMD5AndDES SecretKeyFactory not available
      java.security.NoSuchAlgorithmException: PBEWithMD5AndDES SecretKeyFactory not available

      org.pentaho.di.trans.TransExecutionConfiguration.<init>(TransExecutionConfiguration.java:656)
      org.pentaho.di.trans.TransConfiguration.<init>(TransConfiguration.java:57)
      org.pentaho.di.trans.TransConfiguration.fromXML(TransConfiguration.java:66)
      org.pentaho.di.www.AddTransServlet.doGet(AddTransServlet.java:85)
      org.pentaho.di.www.BaseHttpServlet.doPost(BaseHttpServlet.java:88)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
      org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
      org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
      org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      org.mortbay.jetty.Server.handle(Server.java:326)
      org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:536)
      org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:930)
      org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
      org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
      org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405)
      org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
      org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

      at org.pentaho.di.trans.Trans.sendToSlaveServer(Trans.java:2989)
      ... 14 more

        Issue Links

          Activity

          rfellows Rob Fellows created issue -
          kcruzada Kurtis Cruzada made changes -
          Field Original Value New Value
          Status Open [ 1 ] Open [ 1 ]
          Affects Version/s 4.0.0 GA [ 10900 ]
          Affects Version/s 4.1.0 GA [ 10866 ]
          Hide
          kcruzada Kurtis Cruzada added a comment -

          Please work with Rob on this.

          Show
          kcruzada Kurtis Cruzada added a comment - Please work with Rob on this.
          kcruzada Kurtis Cruzada made changes -
          Status Open [ 1 ] Open [ 1 ]
          Severity Unknown [ 7 ] Blocker [ 1 ]
          Assignee Triage [ project admin ] Mat Lowery [ mlowery ]
          Fix Version/s 4.0.0 GA [ 10900 ]
          mlowery Mat Lowery (Inactive) made changes -
          Attachment pdi-4010.jar [ 20580 ]
          Hide
          mlowery Mat Lowery (Inactive) added a comment - - edited

          Attached JAR to find out if required algorithm is available. To use:

          java -jar pdi-4010.jar

          Make sure you can run this without exception.

          Show
          mlowery Mat Lowery (Inactive) added a comment - - edited Attached JAR to find out if required algorithm is available. To use: java -jar pdi-4010.jar Make sure you can run this without exception.
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Back to Rob to run attached JAR.

          Show
          mlowery Mat Lowery (Inactive) added a comment - Back to Rob to run attached JAR.
          mlowery Mat Lowery (Inactive) made changes -
          Assignee Mat Lowery [ mlowery ] Rob Fellows [ rfellows ]
          Hide
          rfellows Rob Fellows added a comment -

          ok, I ran the attached jar. Once with java 6 (default java executable on PATH), then once with java 5.

          First, java 6
          ----------------------------
          rfellows@6FYV0M1 /cygdrive/c/Users/rfellows/Desktop
          $ java -jar pdi-4010.jar
          You have the required algorithm.

          then, java 5
          ----------------------------
          rfellows@6FYV0M1 /cygdrive/c/Users/rfellows/Desktop
          $ /cygdrive/c/Program\ Files/Java/jdk1.5.0_21/bin/java -jar pdi-4010.jar
          Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file
          at java.lang.ClassLoader.defineClass1(Native Method)
          at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
          at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
          at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
          at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
          at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
          at java.security.AccessController.doPrivileged(Native Method)
          at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
          at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
          at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

          Show
          rfellows Rob Fellows added a comment - ok, I ran the attached jar. Once with java 6 (default java executable on PATH), then once with java 5. First, java 6 ---------------------------- rfellows@6FYV0M1 /cygdrive/c/Users/rfellows/Desktop $ java -jar pdi-4010.jar You have the required algorithm. then, java 5 ---------------------------- rfellows@6FYV0M1 /cygdrive/c/Users/rfellows/Desktop $ /cygdrive/c/Program\ Files/Java/jdk1.5.0_21/bin/java -jar pdi-4010.jar Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:620) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124) at java.net.URLClassLoader.defineClass(URLClassLoader.java:260) at java.net.URLClassLoader.access$100(URLClassLoader.java:56) at java.net.URLClassLoader$1.run(URLClassLoader.java:195) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
          Hide
          rfellows Rob Fellows added a comment -

          Looks like this attached jar may have been compiled with java 6? that might explain the above error "java.lang.UnsupportedClassVersionError: Bad version number in .class file "

          Show
          rfellows Rob Fellows added a comment - Looks like this attached jar may have been compiled with java 6? that might explain the above error "java.lang.UnsupportedClassVersionError: Bad version number in .class file "
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Attached Java 5 version of JAR.

          Show
          mlowery Mat Lowery (Inactive) added a comment - Attached Java 5 version of JAR.
          mlowery Mat Lowery (Inactive) made changes -
          Attachment pdi-4010.jar [ 20589 ]
          mlowery Mat Lowery (Inactive) made changes -
          Attachment pdi-4010.jar [ 20580 ]
          Hide
          rfellows Rob Fellows added a comment -

          repro steps....

          Setup :
          Follow the instructions below if you need to schedule a job that you want to run on a remote Carte server.
          Without making these configuration changes, you will be unable to remotely execute scheduled jobs. ( keep the ip address of your machine ready >ipconfig )
          1. Stop the DI Server and remote Carte server.
          2. Open the /pentaho/server/data-integration-server/tomcat/webapps/pentaho-di/
          WEB-INF/web.xml file with a text editor.
          3. Find the Proxy Trusting Filter filter section, and add your Carte server's IP address to the paramvalue
          element.
          <filter>
          <filter-name>Proxy Trusting Filter</filter-name>
          <filterclass>
          org.pentaho.platform.web.http.filters.ProxyTrustingFilter</filterclass>
          <init-param>
          <param-name>TrustedIpAddrs</param-name>
          <param-value>127.0.0.1,192.168.0.1</param-value>
          <description>Comma separated list of IP addresses of a trusted
          hosts.</description>
          </init-param>
          <init-param>
          <param-name>NewSessionPerRequest</param-name>
          <param-value>true</param-value>
          <description>true to never re-use an existing IPentahoSession
          in the HTTP session; needs to be true to work around code put in for
          BISERVER-2639</description>
          </init-param>
          </filter>
          4. Uncomment the proxy trusting filter-mappings between the <Unable to render embedded object: File (-- begin trust --> and <) not found.-- end trust -->
          markers.
          <!-- begin trust -->
          <filter-mapping>
          <filter-name>Proxy Trusting Filter</filter-name>
          <url-pattern>/webservices/authorizationPolicy</url-pattern>
          </filter-mapping>
          <filter-mapping>
          <filter-name>Proxy Trusting Filter</filter-name>
          <url-pattern>/webservices/roleBindingDao</url-pattern>
          </filter-mapping>
          <filter-mapping>
          <filter-name>Proxy Trusting Filter</filter-name>
          <url-pattern>/webservices/userRoleListService</url-pattern>
          </filter-mapping>
          <filter-mapping>
          <filter-name>Proxy Trusting Filter</filter-name>
          <url-pattern>/webservices/unifiedRepository</url-pattern>
          </filter-mapping>
          <filter-mapping>
          <filter-name>Proxy Trusting Filter</filter-name>
          <url-pattern>/webservices/userRoleService</url-pattern>
          </filter-mapping>
          <filter-mapping>
          <filter-name>Proxy Trusting Filter</filter-name>
          <url-pattern>/webservices/Scheduler</url-pattern>
          </filter-mapping>
          <filter-mapping>
          <filter-name>Proxy Trusting Filter</filter-name>
          <url-pattern>/webservices/repositorySync</url-pattern>
          </filter-mapping>
          <!-- end trust -->
          5. Save and close the file, then edit the carte.sh or Carte.bat startup script on the machine that runs your
          Carte server.
          6. Add -Dpentaho.repository.client.attemptTrust=true to the java line at the bottom of the file.
          java $OPT -Dpentaho.repository.client.attemptTrust=true
          org.pentaho.di.www.Carte "$

          {1+$@}

          " (all case sensitive)
          7. Save and close the file.
          8. Start your Carte and DI Server
          You can now schedule a job to run on a remote Carte instance.

          Test Steps to do:

          1. Execute start-servers.bat and spoon.bat and carte.bat localhost 7089(port no.can be anything)

          1. 2. Create a simple transformation, and a job to execute it.Create a slave server to point to the carte server using carte.bat localhost 7089( use previous test steps for carte.bat)

          2. 2.Create a job that executes the transformation, setting the remote slave to a vanilla Carte (Carte.bat startup)
          3. Add the remote execution for the transformation,by clicking Advanced tab of the Execute Transformation step. and select the carte server using the list box.
          3. Schedule this job on DIS. Using the menu Action->Schedule

          4. It should work fine. You can verify this by viewing the logs in the console window for carte and the tables in http://localhost:9080/pentaho-di/kettle/status

          Show
          rfellows Rob Fellows added a comment - repro steps.... Setup : Follow the instructions below if you need to schedule a job that you want to run on a remote Carte server. Without making these configuration changes, you will be unable to remotely execute scheduled jobs. ( keep the ip address of your machine ready >ipconfig ) 1. Stop the DI Server and remote Carte server. 2. Open the /pentaho/server/data-integration-server/tomcat/webapps/pentaho-di/ WEB-INF/web.xml file with a text editor. 3. Find the Proxy Trusting Filter filter section, and add your Carte server's IP address to the paramvalue element. <filter> <filter-name>Proxy Trusting Filter</filter-name> <filterclass> org.pentaho.platform.web.http.filters.ProxyTrustingFilter</filterclass> <init-param> <param-name>TrustedIpAddrs</param-name> <param-value>127.0.0.1,192.168.0.1</param-value> <description>Comma separated list of IP addresses of a trusted hosts.</description> </init-param> <init-param> <param-name>NewSessionPerRequest</param-name> <param-value>true</param-value> <description>true to never re-use an existing IPentahoSession in the HTTP session; needs to be true to work around code put in for BISERVER-2639 </description> </init-param> </filter> 4. Uncomment the proxy trusting filter-mappings between the < Unable to render embedded object: File (-- begin trust --> and <) not found. -- end trust --> markers. <!-- begin trust --> <filter-mapping> <filter-name>Proxy Trusting Filter</filter-name> <url-pattern>/webservices/authorizationPolicy</url-pattern> </filter-mapping> <filter-mapping> <filter-name>Proxy Trusting Filter</filter-name> <url-pattern>/webservices/roleBindingDao</url-pattern> </filter-mapping> <filter-mapping> <filter-name>Proxy Trusting Filter</filter-name> <url-pattern>/webservices/userRoleListService</url-pattern> </filter-mapping> <filter-mapping> <filter-name>Proxy Trusting Filter</filter-name> <url-pattern>/webservices/unifiedRepository</url-pattern> </filter-mapping> <filter-mapping> <filter-name>Proxy Trusting Filter</filter-name> <url-pattern>/webservices/userRoleService</url-pattern> </filter-mapping> <filter-mapping> <filter-name>Proxy Trusting Filter</filter-name> <url-pattern>/webservices/Scheduler</url-pattern> </filter-mapping> <filter-mapping> <filter-name>Proxy Trusting Filter</filter-name> <url-pattern>/webservices/repositorySync</url-pattern> </filter-mapping> <!-- end trust --> 5. Save and close the file, then edit the carte.sh or Carte.bat startup script on the machine that runs your Carte server. 6. Add -Dpentaho.repository.client.attemptTrust=true to the java line at the bottom of the file. java $OPT -Dpentaho.repository.client.attemptTrust=true org.pentaho.di.www.Carte "$ {1+$@} " (all case sensitive) 7. Save and close the file. 8. Start your Carte and DI Server You can now schedule a job to run on a remote Carte instance. Test Steps to do: 1. Execute start-servers.bat and spoon.bat and carte.bat localhost 7089(port no.can be anything) 1. 2. Create a simple transformation, and a job to execute it.Create a slave server to point to the carte server using carte.bat localhost 7089( use previous test steps for carte.bat) 2. 2.Create a job that executes the transformation, setting the remote slave to a vanilla Carte (Carte.bat startup) 3. Add the remote execution for the transformation,by clicking Advanced tab of the Execute Transformation step. and select the carte server using the list box. 3. Schedule this job on DIS. Using the menu Action->Schedule 4. It should work fine. You can verify this by viewing the logs in the console window for carte and the tables in http://localhost:9080/pentaho-di/kettle/status
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Got NoSuchAlgorithmException on Vista 64-bit.

          Show
          mlowery Mat Lowery (Inactive) added a comment - Got NoSuchAlgorithmException on Vista 64-bit.
          mlowery Mat Lowery (Inactive) made changes -
          Assignee Rob Fellows [ rfellows ] Mat Lowery [ mlowery ]
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Removing the embedded (32-bit) JRE yields a different exception for Rob:

          com.sun.xml.ws.client.ClientTransportException: The server sent HTTP status code 401: Unauthorized

          Show
          mlowery Mat Lowery (Inactive) added a comment - Removing the embedded (32-bit) JRE yields a different exception for Rob: com.sun.xml.ws.client.ClientTransportException: The server sent HTTP status code 401: Unauthorized
          Hide
          mlowery Mat Lowery (Inactive) added a comment - - edited

          I was successful (no errors of any kind) on Vista 64-bit with 64-bit JDK--not the one embedded. I removed that one.

          There was some pain in getting bat files working.

          • Carte.bat assumes java is on your PATH. Maybe it should use PENTAHO_JAVA_HOME like all other Pentaho bats.
          • Spoon.bat uses javaw. javaw did nothing on my Vista 64-bit machine.
          Show
          mlowery Mat Lowery (Inactive) added a comment - - edited I was successful (no errors of any kind) on Vista 64-bit with 64-bit JDK--not the one embedded. I removed that one. There was some pain in getting bat files working. Carte.bat assumes java is on your PATH. Maybe it should use PENTAHO_JAVA_HOME like all other Pentaho bats. Spoon.bat uses javaw. javaw did nothing on my Vista 64-bit machine.
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Back to Rob for another attempt.

          Show
          mlowery Mat Lowery (Inactive) added a comment - Back to Rob for another attempt.
          mlowery Mat Lowery (Inactive) made changes -
          Assignee Mat Lowery [ mlowery ] Rob Fellows [ rfellows ]
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Rob is still having problems.

          Show
          mlowery Mat Lowery (Inactive) added a comment - Rob is still having problems.
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Linked to new issue that aims to use set-pentaho-java in all PDI scripts.

          Show
          mlowery Mat Lowery (Inactive) added a comment - Linked to new issue that aims to use set-pentaho-java in all PDI scripts.
          mlowery Mat Lowery (Inactive) made changes -
          Link This issue relates to PDI-4037 [ PDI-4037 ]
          Hide
          rfellows Rob Fellows added a comment -

          The 401 Unauthorized error was a mistake on my part. the Carte.bat file i was running was not configured with the proper parameter set...
          -Dpentaho.repository.client.attemptTrust=true

          after setting this and running again, I am back to the original issue.

          plan now is to start fresh and try this again, hope is that the environment just got too dirty from all of the hacking in attempts to fix or find the issue.

          Show
          rfellows Rob Fellows added a comment - The 401 Unauthorized error was a mistake on my part. the Carte.bat file i was running was not configured with the proper parameter set... -Dpentaho.repository.client.attemptTrust=true after setting this and running again, I am back to the original issue. plan now is to start fresh and try this again, hope is that the environment just got too dirty from all of the hacking in attempts to fix or find the issue.
          Hide
          rfellows Rob Fellows added a comment -

          Marc just duplicated the issue on Windows 7 64bit with both java 1.5.0_19 and java 1.6.0_17.

          Show
          rfellows Rob Fellows added a comment - Marc just duplicated the issue on Windows 7 64bit with both java 1.5.0_19 and java 1.6.0_17.
          Hide
          rfellows Rob Fellows added a comment -

          duplicated by marc on windows 7.

          Show
          rfellows Rob Fellows added a comment - duplicated by marc on windows 7.
          rfellows Rob Fellows made changes -
          Assignee Rob Fellows [ rfellows ] Mat Lowery [ mlowery ]
          Hide
          mbatchelor Marc Batchelor added a comment -

          I was able to work-around / fix by making sure that all the java's were the same and on the same drive with the code:

          1- Moved data-integration-server\jre up one level - D:\ServerSoftware\pdi-ee\jre

          2- Modified my Windows path to have D:\ServerSoftware\pdi-ee\jre\bin as first entry

          3- Modified JAVA_HOME and PENTAHO_JAVA_HOME to point to D:\ServerSoftware\pdi-ee\jre\bin

          I was able to run the test without problems.

          Next attempt - will copy the JRE we ship with to C:\jre and D:\jre - then, I'll run the test both ways.

          Show
          mbatchelor Marc Batchelor added a comment - I was able to work-around / fix by making sure that all the java's were the same and on the same drive with the code: 1- Moved data-integration-server\jre up one level - D:\ServerSoftware\pdi-ee\jre 2- Modified my Windows path to have D:\ServerSoftware\pdi-ee\jre\bin as first entry 3- Modified JAVA_HOME and PENTAHO_JAVA_HOME to point to D:\ServerSoftware\pdi-ee\jre\bin I was able to run the test without problems. Next attempt - will copy the JRE we ship with to C:\jre and D:\jre - then, I'll run the test both ways.
          Hide
          mbatchelor Marc Batchelor added a comment -

          Result:

          Having my JRE in C:\jre and d:\jre works fine.

          Next tests:

          a- Try a folder for Java with a space in it: "C:\My Java"

          b- Try a symbolic linked folder - C:\symjre

          c- Lastly, the only other thing that strikes me about my configuration (64-bit Windows) is that I have two "Program Files" folders - "C:\Program Files" (for 64bit programs) and "C:\Program Files (x86)" (for 32bit). Additionally, I have a java folder in each place. In Windows 7, it's possible that objects running in some compatibility mode foisted upon it gets confused about which folder to get the encryption out of.

          Show
          mbatchelor Marc Batchelor added a comment - Result: Having my JRE in C:\jre and d:\jre works fine. Next tests: a- Try a folder for Java with a space in it: "C:\My Java" b- Try a symbolic linked folder - C:\symjre c- Lastly, the only other thing that strikes me about my configuration (64-bit Windows) is that I have two "Program Files" folders - "C:\Program Files" (for 64bit programs) and "C:\Program Files (x86)" (for 32bit). Additionally, I have a java folder in each place. In Windows 7, it's possible that objects running in some compatibility mode foisted upon it gets confused about which folder to get the encryption out of.
          Hide
          mbatchelor Marc Batchelor added a comment -

          I was able to reproduce the error as follows:

          a- Rename C:\jre to "C:\My Jre"

          b- Update my windows Path, JAVA_HOME and PENTAHO_JAVA_HOME variables to all point to C:\My Jre

          • Ka-Boom *
          Show
          mbatchelor Marc Batchelor added a comment - I was able to reproduce the error as follows: a- Rename C:\jre to "C:\My Jre" b- Update my windows Path, JAVA_HOME and PENTAHO_JAVA_HOME variables to all point to C:\My Jre Ka-Boom *
          Hide
          mbatchelor Marc Batchelor added a comment -

          Next test - using dir/x, I you'll see this:

          05/28/2010 12:34 PM <DIR> MYJRE~1 My Jre

          Changing my path and all variables

          From:
          C:\My Jre

          To:
          C:\MYJRE~1

          Allows it all to work. So, the problem is definitely with spaces in the path to the JRE. Only a couple of things left to test:

          a- Quoting the path
          b- Using '/' instead of '\' in the two variables

          Show
          mbatchelor Marc Batchelor added a comment - Next test - using dir/x, I you'll see this: 05/28/2010 12:34 PM <DIR> MYJRE~1 My Jre Changing my path and all variables From: C:\My Jre To: C:\MYJRE~1 Allows it all to work. So, the problem is definitely with spaces in the path to the JRE. Only a couple of things left to test: a- Quoting the path b- Using '/' instead of '\' in the two variables
          Hide
          mbatchelor Marc Batchelor added a comment -

          Using '/' didn't change the result - still fails

          Show
          mbatchelor Marc Batchelor added a comment - Using '/' didn't change the result - still fails
          Hide
          mbatchelor Marc Batchelor added a comment -

          Quoting the environment variable values fails to allow anything to launch

          Show
          mbatchelor Marc Batchelor added a comment - Quoting the environment variable values fails to allow anything to launch
          Hide
          rfellows Rob Fellows added a comment -

          2 more workarounds found...

          1) if the path to the jdk/jre contains a space (c:\program files\java\jdk1.5.0_21 for example) you can create a symlink to this path that has no spaces...
          on windows 7 this can be done on the command line:

          mkdir c:\java
          mklink /J jdk1.5.0_21 "c:\program files\java\jdk1.5.0_21"

          then set your env variables JAVA_HOME & PENTAHO_JAVA_HOME to the sym link path
          then restart carte and di servers

          or 2) use 8.3 pathing in env variables JAVA_HOME & PENTAHO_JAVA_HOME. this eliminates the spaces in the env variables and preserves the original location of your jre.

          a quick way to generate the full path in 8.3 format is to open a cmd prompt, navigate to you java installation (c:\program files\java\jdk1.5.0_21) then just execute "command". this will use the old command prompt and will display the path in 8.3 format. in this case it would be C:\PROGRA~1\JAVA\JDK15~1.0_2

          Show
          rfellows Rob Fellows added a comment - 2 more workarounds found... 1) if the path to the jdk/jre contains a space (c:\program files\java\jdk1.5.0_21 for example) you can create a symlink to this path that has no spaces... on windows 7 this can be done on the command line: mkdir c:\java mklink /J jdk1.5.0_21 "c:\program files\java\jdk1.5.0_21" then set your env variables JAVA_HOME & PENTAHO_JAVA_HOME to the sym link path then restart carte and di servers or 2) use 8.3 pathing in env variables JAVA_HOME & PENTAHO_JAVA_HOME. this eliminates the spaces in the env variables and preserves the original location of your jre. a quick way to generate the full path in 8.3 format is to open a cmd prompt, navigate to you java installation (c:\program files\java\jdk1.5.0_21) then just execute "command". this will use the old command prompt and will display the path in 8.3 format. in this case it would be C:\PROGRA~1\JAVA\JDK15~1.0_2
          Hide
          rfellows Rob Fellows added a comment -

          There are known workarounds for this issue. The basic issue is having spaces in the path to the jre configured in 2 environment variables (PENTAHO_JAVA_HOME & JAVA_HOME). The physical path is not the issue, it is the fact that they have spaces in the ENV variables.

          This should be documented as a known issue with the various workarounds...

          Show
          rfellows Rob Fellows added a comment - There are known workarounds for this issue. The basic issue is having spaces in the path to the jre configured in 2 environment variables (PENTAHO_JAVA_HOME & JAVA_HOME). The physical path is not the issue, it is the fact that they have spaces in the ENV variables. This should be documented as a known issue with the various workarounds...
          rfellows Rob Fellows made changes -
          Status Open [ 1 ] Closed [ 6 ]
          QA Validation Status Not Yet Validated
          Resolution Fixed [ 1 ]
          mbatchelor Marc Batchelor made changes -
          Workflow Pentaho Engineering 7.0 Workflow [ 511672 ] Pentaho Engineering 8.0 Workflow [ 527619 ]
          kcruzada Kurtis Cruzada made changes -
          Link This issue is duplicated by PDI-4753 [ PDI-4753 ]
          wgorman Will Gorman (Inactive) made changes -
          Link This issue is duplicated by PDI-4912 [ PDI-4912 ]
          Hide
          cfauvet Cedric Fauvet (Inactive) added a comment -

          Hi guys,

          I've meet this bug in the last version of BI suite "pentaho-bi-suite-3.7.0-GA-x64.exe" (Windows 7 64bits).

          Same issue same solution :
          if the path to the jdk/jre contains a space (c:\program files\java\jdk1.5.0_21 for example) you can create a symlink to this path that has no spaces...
          on windows 7 this can be done on the command line:

          mkdir c:\java
          mklink /J jdk1.5.0_21 "c:\program files\java\jdk1.5.0_21"

          then set your env variables JAVA_HOME & PENTAHO_JAVA_HOME to the sym link path
          then restart carte and di servers

          Cédric

          Show
          cfauvet Cedric Fauvet (Inactive) added a comment - Hi guys, I've meet this bug in the last version of BI suite "pentaho-bi-suite-3.7.0-GA-x64.exe" (Windows 7 64bits). Same issue same solution : if the path to the jdk/jre contains a space (c:\program files\java\jdk1.5.0_21 for example) you can create a symlink to this path that has no spaces... on windows 7 this can be done on the command line: mkdir c:\java mklink /J jdk1.5.0_21 "c:\program files\java\jdk1.5.0_21" then set your env variables JAVA_HOME & PENTAHO_JAVA_HOME to the sym link path then restart carte and di servers Cédric
          Hide
          cfauvet Cedric Fauvet (Inactive) added a comment -

          I've meet this bug in the last version of BI suite.
          Installer : "pentaho-bi-suite-3.7.0-GA-x64.exe" (Windows 7 64bits).

          Same issue same solution :
          if the path to the jdk/jre contains a space (c:\program files\java\jdk1.5.0_21 for example) you can create a symlink to this path that has no spaces...
          on windows 7 this can be done on the command line:

          mkdir c:\java
          mklink /J jdk1.5.0_21 "c:\program files\java\jdk1.5.0_21"

          then set your env variables JAVA_HOME & PENTAHO_JAVA_HOME to the sym link path
          then restart carte and di servers

          Show
          cfauvet Cedric Fauvet (Inactive) added a comment - I've meet this bug in the last version of BI suite. Installer : "pentaho-bi-suite-3.7.0-GA-x64.exe" (Windows 7 64bits). Same issue same solution : if the path to the jdk/jre contains a space (c:\program files\java\jdk1.5.0_21 for example) you can create a symlink to this path that has no spaces... on windows 7 this can be done on the command line: mkdir c:\java mklink /J jdk1.5.0_21 "c:\program files\java\jdk1.5.0_21" then set your env variables JAVA_HOME & PENTAHO_JAVA_HOME to the sym link path then restart carte and di servers
          cfauvet Cedric Fauvet (Inactive) made changes -
          Resolution Fixed [ 1 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          CI Build Pentaho Data Integration Engine 4.1.0.GA
          mlowery Mat Lowery (Inactive) made changes -
          Assignee Mat Lowery [ mlowery ] Triage [ project admin ]
          jtcornelius Jake Cornelius made changes -
          Fix Version/s 4.1.1 [ 11057 ]
          Fix Version/s 4.0.0 GA [ 10900 ]
          mlowery Mat Lowery (Inactive) made changes -
          Link This issue is duplicated by PDI-4725 [ PDI-4725 ]
          wgorman Will Gorman (Inactive) made changes -
          Assignee Triage [ project admin ] Mat Lowery [ mlowery ]
          Fix Version/s 4.2.0 GA [ 11033 ]
          Fix Version/s 4.1.1 [ 11057 ]
          jtcornelius Jake Cornelius made changes -
          Fix Version/s 4.2.0 GA [ 11097 ]
          Fix Version/s 4.3.0 GA [ 11033 ]
          Hide
          mlowery Mat Lowery (Inactive) added a comment - - edited

          The issue is our use of java.ext.dirs in kitchen, pan, and carte (possibly others) startup scripts. When you set java.ext.dirs but don't include the default extensions dir (i.e. <java.home>/lib/ext) something happens where it tries to be appended behind-the-scenes but spaces in the default extensions dir path aren't handled correctly.

          Possible solutions:
          1. Stop using java.ext.dirs (my personal favorite--why doesn't regular classpath suffice?)
          2. Add the (properly quoted) default extensions dir to java.ext.dirs. Do this by fixing the following lines in the bat files since they do not work with spaces:

          if exist %_PENTAHO_JAVA_HOME%\jre\lib\ext\nul set JAVA_EXT_DIRS=%_PENTAHO_JAVA_HOME%\jre\lib\ext;%JAVA_EXT_DIRS%
          if exist %_PENTAHO_JAVA_HOME%\lib\ext\nul set JAVA_EXT_DIRS=%_PENTAHO_JAVA_HOME%\lib\ext;%JAVA_EXT_DIRS%

          Show
          mlowery Mat Lowery (Inactive) added a comment - - edited The issue is our use of java.ext.dirs in kitchen, pan, and carte (possibly others) startup scripts. When you set java.ext.dirs but don't include the default extensions dir (i.e. <java.home>/lib/ext) something happens where it tries to be appended behind-the-scenes but spaces in the default extensions dir path aren't handled correctly. Possible solutions: 1. Stop using java.ext.dirs (my personal favorite--why doesn't regular classpath suffice?) 2. Add the (properly quoted) default extensions dir to java.ext.dirs. Do this by fixing the following lines in the bat files since they do not work with spaces: if exist %_PENTAHO_JAVA_HOME%\jre\lib\ext\nul set JAVA_EXT_DIRS=%_PENTAHO_JAVA_HOME%\jre\lib\ext;%JAVA_EXT_DIRS% if exist %_PENTAHO_JAVA_HOME%\lib\ext\nul set JAVA_EXT_DIRS=%_PENTAHO_JAVA_HOME%\lib\ext;%JAVA_EXT_DIRS%
          wgorman Will Gorman (Inactive) made changes -
          Fix Version/s 4.1.2 [ 11071 ]
          Fix Version/s 4.2.0 GA [ 11097 ]
          mlowery Mat Lowery (Inactive) made changes -
          Status Reopened [ 4 ] In Progress [ 3 ]
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Per Matt Casters, moving to use of standard classpath.

          Show
          mlowery Mat Lowery (Inactive) added a comment - Per Matt Casters, moving to use of standard classpath.
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          Files to change:
          Carte.bat
          Encr.bat
          Kitchen.bat
          Pan.bat

          Show
          mlowery Mat Lowery (Inactive) added a comment - Files to change: Carte.bat Encr.bat Kitchen.bat Pan.bat
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          How to validate:

          On a Windows machine with JAVA_HOME containing a space in the path, execute
          Pan.bat -rep ent -user joe -pass password -dir public -trans t1

          where ent is an enterprise repository. (Create t1 in /public of course.)

          Show
          mlowery Mat Lowery (Inactive) added a comment - How to validate: On a Windows machine with JAVA_HOME containing a space in the path, execute Pan.bat -rep ent -user joe -pass password -dir public -trans t1 where ent is an enterprise repository. (Create t1 in /public of course.)
          mlowery Mat Lowery (Inactive) made changes -
          Status In Progress [ 3 ] Ready For Test [ 10004 ]
          CI Build Pentaho Data Integration Engine 4.1.0.GA pdi-ee-assembly #5994
          Assignee Mat Lowery [ mlowery ] Golda David [ gdavid ]
          mlowery Mat Lowery (Inactive) made changes -
          Link This issue is related to PDI-5240 [ PDI-5240 ]
          Hide
          mlowery Mat Lowery (Inactive) added a comment -

          This affects 32-bit or 64-bit Windows. Not sure if the problem is exclusive to Windows 7; I doubt it.

          Attached workaround.zip. This zip has only been tested to patch 4.1.1. To install:

          1. Backup Carte.bat, Encr.bat, Kitchen.bat, and Pan.bat in data-integration (aka Spoon or Kettle) folder.
          2. Unzip attached zip.
          3. Run Carte or Pan or Kitchen.

          Show
          mlowery Mat Lowery (Inactive) added a comment - This affects 32-bit or 64-bit Windows. Not sure if the problem is exclusive to Windows 7; I doubt it. Attached workaround.zip. This zip has only been tested to patch 4.1.1. To install: 1. Backup Carte.bat, Encr.bat, Kitchen.bat, and Pan.bat in data-integration (aka Spoon or Kettle) folder. 2. Unzip attached zip. 3. Run Carte or Pan or Kitchen.
          mlowery Mat Lowery (Inactive) made changes -
          Attachment workaround.zip [ 24232 ]
          Hide
          gdavid Golda Thomas added a comment -

          Tested this issue using the bat files from the PDI 4.1.2 GA build and even if the JAVA HOME folder has spaces in between.
          The transformation runs successfully.

          Show
          gdavid Golda Thomas added a comment - Tested this issue using the bat files from the PDI 4.1.2 GA build and even if the JAVA HOME folder has spaces in between. The transformation runs successfully.
          gdavid Golda Thomas made changes -
          Status Ready For Test [ 10004 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          gdavid Golda Thomas added a comment -

          Validated using the PDI 4.1.2 GA build from Jan-20-2011

          It works fine.

          Show
          gdavid Golda Thomas added a comment - Validated using the PDI 4.1.2 GA build from Jan-20-2011 It works fine.
          gdavid Golda Thomas made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          QA Validation Status Validated by QA
          mlowery Mat Lowery (Inactive) made changes -
          Link This issue is related to PDI-4455 [ PDI-4455 ]
          mbatchelor Marc Batchelor made changes -
          Workflow Pentaho Engineering 8.0 Workflow [ 527619 ] Pentaho Engineering 9.0 Workflow [ 572524 ]
          jiraservice Jira Service Acct made changes -
          Workflow Pentaho Engineering 9.0 Workflow [ 572524 ] Pentaho Bug 1.0 Workflow [ 634893 ]
          jiraservice Jira Service Acct made changes -
          Workflow Pentaho Bug 1.0 Workflow [ 634893 ] Pentaho Engineering 9.0 Workflow [ 656929 ]
          jiraservice Jira Service Acct made changes -
          Workflow Pentaho Engineering 9.0 Workflow [ 656929 ] Pentaho Engineering 9.1 Workflow [ 753328 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Open Open
          18h 53m 2 Kurtis Cruzada 27/May/10 9:08 AM
          Open Open Closed Closed
          1d 3h 25m 1 Rob Fellows 28/May/10 12:33 PM
          Closed Closed Reopened Reopened
          228d 21h 49m 1 Cedric Fauvet (Inactive) 12/Jan/11 9:22 AM
          Reopened Reopened In Progress In Progress
          4d 22h 38m 1 Mat Lowery (Inactive) 17/Jan/11 8:01 AM
          In Progress In Progress Ready For Test Ready For Test
          1d 39m 1 Mat Lowery (Inactive) 18/Jan/11 8:41 AM
          Ready For Test Ready For Test Resolved Resolved
          2d 4h 3m 1 Golda Thomas 20/Jan/11 12:44 PM
          Resolved Resolved Closed Closed
          3m 57s 1 Golda Thomas 20/Jan/11 12:48 PM

            People

            • Assignee:
              gdavid Golda Thomas
              Reporter:
              rfellows Rob Fellows
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: