Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-15574

Karaf parameter "pentaho.karaf.root.copy.dest.folder" generates multiple unstable executions

    XMLWordPrintable

    Details

    • Story Points:
      0
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Steps to Reproduce:
      Hide

      Add the property *"pentaho.karaf.root.copy.dest.folder"* as described in
      https://help.pentaho.com/Documentation/6.1/0P0/100/Karaf_Performance_Tuning

      To make each Folder unique, we added a TMP folder assigned to each execution based on the PID

      CONCURRENT_PID=$$
      
      mkdir -p /tmp/pdi6.1.0.3/$CONCURRENT_PID/
      
      echo $CONCURRENT_PID > /tmp/pdi/$CONCURRENT_PID/$CONCURRENT_PID.pid
      OPT="$OPT -Dpentaho.karaf.root.copy.dest.folder=/tmp/pdi/$CONCURRENT_PID/karaf"
      
      
      Show
      Add the property * "pentaho.karaf.root.copy.dest.folder" * as described in https://help.pentaho.com/Documentation/6.1/0P0/100/Karaf_Performance_Tuning To make each Folder unique , we added a TMP folder assigned to each execution based on the PID CONCURRENT_PID=$$ mkdir -p /tmp/pdi6.1.0.3/$CONCURRENT_PID/ echo $CONCURRENT_PID > /tmp/pdi/$CONCURRENT_PID/$CONCURRENT_PID.pid OPT= "$OPT -Dpentaho.karaf.root.copy.dest.folder=/tmp/pdi/$CONCURRENT_PID/karaf"

      Description

      After experiencing issues with Pentaho 6.1.0.3 with Karaf, started to test parameters as used to work as workaround in 6.0.1.3 with karaf.data environment variable.

      Parameter USED is described in the pentaho documentation here:
      https://help.pentaho.com/Documentation/6.1/0P0/100/Karaf_Performance_Tuning

      parameter used: pentaho.karaf.root.copy.dest.folder=<<random location in tmp folder >>

      Result was 90% of the executions failed (using SAMPLES) : attached image

      Errors like

      DEBUG: KETTLE_HOME=/opt/pentaho
      DEBUG: PENTAHO_DI_JAVA_OPTIONS=-Xms512m -Xmx1024m -XX:MaxPermSize=256m
      DEBUG: INITIALDIR=/var/lib/jenkins/workspace/Pentaho-6.1-SP3-Samples/PDI_LOCATION/pdi-ee-client-6.1.0.3-223/TRANSFORM/samples/Analytic Query - Lead One Example.ktr
      DEBUG: BASEDIR=/opt/pentaho/pdi-ee-client-6.1.0.3-223
      DEBUG: CURRENTDIR=.
      DEBUG: Karaf tmp folder in use
      /var/lib/jenkins/workspace/Pentaho-6.1-SP3-Samples/PDI_LOCATION/pdi-ee-client-6.1.0.3-223/TRANSFORM/samples/Analytic Query - Lead One Example.ktr
      WARNING: Using java from path
      DEBUG: _PENTAHO_JAVA_HOME=
      DEBUG: _PENTAHO_JAVA=java
      DEBUG: PENTAHO_INSTALLED_LICENSE_PATH=
      Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
      16:02:29,967 INFO  [KarafInstance] 
      *******************************************************************************
      *** Karaf Instance Number: 1 at /tmp/pdi/10579/karaf/caches/pan/data-1      ***
      *** Karaf Port:8802                                                         ***
      *** OSGI Service Port:9051                                                  ***
      *******************************************************************************
      16:02:29,968 INFO  [KarafBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled
      Jul 15, 2016 4:02:31 PM org.apache.karaf.main.Main$KarafLockCallback lockAquired
      INFO: Lock acquired. Setting startlevel to 100
      Updating configuration from org.ops4j.pax.logging.cfg
      Creating configuration from org.apache.activemq.server-default.cfg
      Creating configuration from org.pentaho.caching-default.cfg
      Updating configuration from org.ops4j.pax.url.mvn.cfg
      Updating configuration from org.apache.karaf.shell.cfg
      Creating configuration from org.apache.felix.fileinstall-deploy.cfg
      2016/07/15 16:02:33 - General - Logging plugin type found with ID: CheckpointLogTable
      Refreshing GOE props...
      2016-07-15 16:02:35 weka.gui.GenericObjectEditor registerEditors
      INFO: ---Registering Weka Editors---
      16:02:35,628 ERROR [BootFeaturesInstaller] Error Boot feature pentaho-yarn-obf not found
      16:02:35,628 ERROR [BootFeaturesInstaller] Error Boot feature pdi-dataservice not found
      16:02:35,632 ERROR [BootFeaturesInstaller] Error Boot feature pentaho-metaverse not found
      16:02:35,633 ERROR [BootFeaturesInstaller] Error Boot feature pdi-data-refinery not found
      16:02:35,633 ERROR [BootFeaturesInstaller] Error Boot feature pentaho-client-ee not found
      16:02:35,634 ERROR [BootFeaturesInstaller] Error Boot feature pentaho-base not found
      Trying to add database driver (JDBC): RmiJdbc.RJDriver - Warning, not in CLASSPATH?
      Trying to add database driver (JDBC): jdbc.idbDriver - Warning, not in CLASSPATH?
      Trying to add database driver (JDBC): org.gjt.mm.mysql.Driver - Warning, not in CLASSPATH?
      Trying to add database driver (JDBC): com.mckoi.JDBCDriver - Warning, not in CLASSPATH?
      *ERROR* [org.osgi.service.cm.ManagedService, id=241, bundle=48/mvn:org.apache.aries.transaction/org.apache.aries.transaction.manager/1.1.1]: Updating configuration org.apache.aries.transaction caused a problem: null
      org.osgi.service.cm.ConfigurationException: null : null
          at org.apache.aries.transaction.internal.TransactionManagerService.<init>(TransactionManagerService.java:136)
          at org.apache.aries.transaction.internal.Activator.updated(Activator.java:63)
          at org.apache.felix.cm.impl.helper.ManagedServiceTracker.updateService(ManagedServiceTracker.java:148)
          at org.apache.felix.cm.impl.helper.ManagedServiceTracker.provideConfiguration(ManagedServiceTracker.java:81)
          at org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceUpdate.provide(ConfigurationManager.java:1448)
          at org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceUpdate.run(ConfigurationManager.java:1404)
          at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:103)
          at java.lang.Thread.run(Thread.java:745)
      Caused by: org.objectweb.howl.log.LogConfigurationException: Unable to obtain lock on /opt/pentaho/pdi-ee-client-6.1.0.3-223/system/karaf/caches/pan/data-1/txlog/transaction_1.log
          at org.objectweb.howl.log.LogFile.open(LogFile.java:191)
          at org.objectweb.howl.log.LogFileManager.open(LogFileManager.java:784)
          at org.objectweb.howl.log.Logger.open(Logger.java:304)
          at org.objectweb.howl.log.xa.XALogger.open(XALogger.java:893)
          at org.apache.aries.transaction.internal.HOWLLog.doStart(HOWLLog.java:233)
          at org.apache.aries.transaction.internal.TransactionManagerService.<init>(TransactionManagerService.java:133)
          ... 7 more
      [KnowledgeFlow] Loading properties and plugins...
      [KnowledgeFlow] Initializing KF...
      Refreshing GOE props...
      .
      .
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                aschurman Alexander Schurman
              • Votes:
                1 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: