Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-8757

Create a release job for our fork of Apache Hive JDBC project

    Details

    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.

      Description

      We need to build a new version of the Apache Hive JDBC driver for the 4.8 release. The source code now resides at https://github.com/pentaho/hive and a CI job has been building this here: http://ci.pentaho.com/job/apache-hive/

      The new published artifacts have the following metadata:

      groupId: org.apache.hive
      artifactId: hive-jdbc

      Proposed version for this release: 0.7.0-pentaho-1.0.2 as the last was 0.7.0-pentaho-1.0.1.

        Issue Links

          Activity

          Hide
          jganoff Jordan Ganoff (Inactive) added a comment -

          Are you using the Pentaho build script in the jdbc dir?

          Show
          jganoff Jordan Ganoff (Inactive) added a comment - Are you using the Pentaho build script in the jdbc dir?
          Hide
          lgrill Larry Grill added a comment -

          Belay that. I had pulled the forked apache hive project instead of the hive-0.7.0 project. Structure and build are quite different. Cleaned up the push i did there and will switch to hive-0.7.0.

          Show
          lgrill Larry Grill added a comment - Belay that. I had pulled the forked apache hive project instead of the hive-0.7.0 project. Structure and build are quite different. Cleaned up the push i did there and will switch to hive-0.7.0.
          Hide
          lgrill Larry Grill added a comment -

          Finally got this sorted out and built. Modified the existing job on the release machine:
          http://release.pentaho.com:8081/job/apache-hive-0.7.0/

          Looks like the dependencies in the big data shims project have already been fixed for PDI-8764 (b8568b20d86266a82f712066ccc38eda6ce837e1).

          Show
          lgrill Larry Grill added a comment - Finally got this sorted out and built. Modified the existing job on the release machine: http://release.pentaho.com:8081/job/apache-hive-0.7.0/ Looks like the dependencies in the big data shims project have already been fixed for PDI-8764 (b8568b20d86266a82f712066ccc38eda6ce837e1).
          Hide
          lgrill Larry Grill added a comment -

          Oops. Turns out that the ivy files were updated for the fixed org/groupId, but the build.properties needed to be updated to 0.7.0-pentaho-1.0.2. Done.

          Show
          lgrill Larry Grill added a comment - Oops. Turns out that the ivy files were updated for the fixed org/groupId, but the build.properties needed to be updated to 0.7.0-pentaho-1.0.2. Done.
          Hide
          lgrill Larry Grill added a comment -

          Reopened issue due to some questions about what version of hive-exec, hive-service, and hive-metastore should be in big-data.

          I only built and deployed a 0.7.0-pentaho-1.0.2 version of hive-jdbc for this ticket. Did not update or try building these other projects, but since the big data ivy.xml files used the same property for the version of all four of these artifacts (dependency.hive-jdbc.revision) I ran into resolve failures when doing the big data release build.

          I changed the three other projects to use a different version property (dependency.hadoop-hive.revision) in the ivy.xml file. Build succeeded, but I wonder if these projects should be kept in sync.

          https://github.com/pentaho/big-data-plugin/commit/3aa704b8ae16171547269814f47b60dab38e53b8

          Show
          lgrill Larry Grill added a comment - Reopened issue due to some questions about what version of hive-exec, hive-service, and hive-metastore should be in big-data. I only built and deployed a 0.7.0-pentaho-1.0.2 version of hive-jdbc for this ticket. Did not update or try building these other projects, but since the big data ivy.xml files used the same property for the version of all four of these artifacts (dependency.hive-jdbc.revision) I ran into resolve failures when doing the big data release build. I changed the three other projects to use a different version property (dependency.hadoop-hive.revision) in the ivy.xml file. Build succeeded, but I wonder if these projects should be kept in sync. https://github.com/pentaho/big-data-plugin/commit/3aa704b8ae16171547269814f47b60dab38e53b8
          Hide
          jganoff Jordan Ganoff (Inactive) added a comment -

          I believe your changes will work fine. I'll work with Carter to determine if we can omit the packaging of all Hive jars that are not the JDBC driver for the MapR Hadoop Configuration.

          Show
          jganoff Jordan Ganoff (Inactive) added a comment - I believe your changes will work fine. I'll work with Carter to determine if we can omit the packaging of all Hive jars that are not the JDBC driver for the MapR Hadoop Configuration.
          Hide
          jganoff Jordan Ganoff (Inactive) added a comment -

          Just to clarify - our modifications are solely contained with the Hive JDBC driver jar. The other jars are dependencies and should match the Hive installation as closely as possible. For this reason I have avoided building them ourselves. We should depend on the Hive jars that match the Hadoop configuration where possible.

          Show
          jganoff Jordan Ganoff (Inactive) added a comment - Just to clarify - our modifications are solely contained with the Hive JDBC driver jar. The other jars are dependencies and should match the Hive installation as closely as possible. For this reason I have avoided building them ourselves. We should depend on the Hive jars that match the Hadoop configuration where possible.
          Hide
          jganoff Jordan Ganoff (Inactive) added a comment -

          Looks like we're okay with the MapR jars. Assigning to Carter so he can confirm.

          Show
          jganoff Jordan Ganoff (Inactive) added a comment - Looks like we're okay with the MapR jars. Assigning to Carter so he can confirm.
          Hide
          ceverett Carter Everett (Inactive) added a comment -

          confirmed. Hive JDBC works on MapR with new build.

          Show
          ceverett Carter Everett (Inactive) added a comment - confirmed. Hive JDBC works on MapR with new build.

            People

            • Assignee:
              Unassigned
              Reporter:
              jganoff Jordan Ganoff (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: