Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-18955

HTTP Post no longer handles Basic Authentication being passed to other servers

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Severity: Urgent
    • Resolution: Fixed
    • Affects Version/s: 8.0.0 GA, 8.1.0 GA, 8.2.0. GA, 8.3.0 GA, 9.0.0 GA
    • Fix Version/s: 9.2.0 GA
    • Component/s: Step
    • Labels:
    • Story Points:
      0
    • PDI Sub-component:
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Sprint Team:
      Tatooine (Maint)
    • Steps to Reproduce:
      Hide
      1. Install PDI 7.1
      2. Edit the following file:  ...\data-integration\classes\log4j.xml   with the following entry: 
        <appender name="HTTPLOG" class="org.apache.log4j.FileAppender">
        <param name="File" value="logs/http.log"/>
        <param name="Append" value="false"/>
        <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %m%n"/>
        </layout>
        </appender>
        <logger name="httpclient.wire">
        <level value="DEBUG"/>
        <appender-ref ref="HTTPLOG"/>
        </logger>
      1. Start Spoon 7.1
      2. Start any Pentaho Server
      3. Connect Spoon to the Server repository
      4. Download http_post_test_repo_export and import this into your repository
      5. Open job_create_test_file.kjb and http-post-step.ktr
      6. Run job_create_test_file.kjb on the Pentaho Server (schedule the job to run now)
      7. Run http-post-step.ktr locally while the job is running
      8. In your data-integration/logs/http.log notice you will have a similar entry:  "Authorization: Basic YWRtaW46cGFzc3dvcmQ=[\r][\n]"
      9. **Install PDI 8.3
      10. Edit the following file:  ...\data-integration\classes\log4j.xml   with the following entry: 
        <appender name="HTTPLOG" class="org.apache.log4j.rolling.RollingFileAppender">
        <rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
        <param name="ActiveFileName"  value="logs/http.log" />
        <param name="FileNamePattern" value="logs/http.%d{yyyy-MM-dd}.log" />
        </rollingPolicy>
        <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p &lt;%t&gt; %m%n"/>
        </layout>
        </appender>
        
        <category name="org.apache.http">
        <priority value="DEBUG" />
        <appender-ref ref="HTTPLOG"/>
        </category>
      1. Start PDI 8.3
      2. Connect Spoon to the Server repository
      3. Open job_create_test_file.kjb and http-post-step.ktr
      4. Run job_create_test_file.kjb on the Pentaho Server (schedule the job to run now)
      5. Run http-post-step.ktr locally while the job is running
      6. Notice in the data-integration/logs/http.log you do not see basic authorization.

      Actual Result:  HTTP Post is not handling basic authentication.

      Expected Result:  You should see the request using the basic authorization.

      Show
      Install PDI 7.1 Edit the following file:  ...\data-integration\classes\log4j.xml   with the following entry:  <appender name="HTTPLOG" class="org.apache.log4j.FileAppender"> <param name="File" value="logs/http.log"/> <param name="Append" value="false"/> <layout class="org.apache.log4j.PatternLayout"> <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %m%n"/> </layout> </appender> <logger name="httpclient.wire"> <level value="DEBUG"/> <appender-ref ref="HTTPLOG"/> </logger> Start Spoon 7.1 Start any Pentaho Server Connect Spoon to the Server repository Download  http_post_test_repo_export and import this into your repository Open job_create_test_file.kjb and http-post-step.ktr Run job_create_test_file.kjb on the Pentaho Server (schedule the job to run now) Run http-post-step.ktr locally while the job is running In your data-integration/logs/http.log notice you will have a similar entry:   "Authorization: Basic YWRtaW46cGFzc3dvcmQ= [\r] [\n] " **Install PDI 8.3 Edit the following file:  ...\data-integration\classes\log4j.xml   with the following entry:  <appender name="HTTPLOG" class="org.apache.log4j.rolling.RollingFileAppender"> <rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy"> <param name="ActiveFileName" value="logs/http.log" /> <param name="FileNamePattern" value="logs/http.%d{yyyy-MM-dd}.log" /> </rollingPolicy> <layout class="org.apache.log4j.PatternLayout"> <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p &lt;%t&gt; %m%n"/> </layout> </appender> <category name="org.apache.http"> <priority value="DEBUG" /> <appender-ref ref="HTTPLOG"/> </category> Start PDI 8.3 Connect Spoon to the Server repository Open job_create_test_file.kjb and http-post-step.ktr Run job_create_test_file.kjb on the Pentaho Server (schedule the job to run now) Run http-post-step.ktr locally while the job is running Notice in the data-integration/logs/http.log you do not see basic authorization. Actual Result:  HTTP Post is not handling basic authentication. Expected Result:  You should see the request using the basic authorization.

      Description

      HTTP Post is no longer handing Basic Authentication that is being passed to other servers.  This is causing HTTP Post requests to fail when attempting to authorize with another web service.

      This appears to have broken starting in 8.0.

      In the 7.1 log: 71_HTTP_POST_http.log

      2020-09-18 18:15:53.801 >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
      2020-09-18 18:15:53.801 >> "Authorization: Basic YWRtaW46cGFzc3dvcmQ=[\r][\n]"
      2020-09-18 18:15:53.801 >> "Host: localhost:8080[\r][\n]"
      2020-09-18 18:15:53.801 >> "Cookie: $Version=0; JSESSIONID=C76E4A3FF2DED38E10DEE948B0960F5A; $Path=/pentaho[\r][\n]"
      2020-09-18 18:15:53.801 >> "[\r][\n]"
      2020-09-18 18:15:53.868 << "HTTP/1.1 200 [\r][\n]"
      2020-09-18 18:15:53.868 << "HTTP/1.1 200 [\r][\n]"
      2020-09-18 18:15:53.868 << "Set-Cookie: session-expiry=1600474553802; Path=/[\r][\n]"
      2020-09-18 18:15:53.868 << "Set-Cookie: server-time=1600467353802; Path=/[\r][\n]"
      2020-09-18 18:15:53.868 << "Set-Cookie: JSESSIONID=030218A4862C7217AF0E826D1AF7EE73; Path=/pentaho; HttpOnly[\r][\n]"
      2020-09-18 18:15:53.868 << "Content-Type: text/plain[\r][\n]"
      2020-09-18 18:15:53.868 << "Transfer-Encoding: chunked[\r][\n]"
      2020-09-18 18:15:53.868 << "Date: Fri, 18 Sep 2020 22:15:53 GMT[\r][\n]"

      In the 8.3 log: 83_HTTP_POST_http.log

      2020-09-18 18:18:59.107 DEBUG <Thread-114> http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_201)[\r][\n]"
      2020-09-18 18:18:59.107 DEBUG <Thread-114> http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
      2020-09-18 18:18:59.107 DEBUG <Thread-114> http-outgoing-0 >> "[\r][\n]"
      2020-09-18 18:18:59.369 DEBUG <Thread-114> http-outgoing-0 << "HTTP/1.1 200 OK[\r][\n]"
      2020-09-18 18:18:59.369 DEBUG <Thread-114> http-outgoing-0 << "Content-Type: application/xml[\r][\n]"
      2020-09-18 18:18:59.369 DEBUG <Thread-114> http-outgoing-0 << "Content-Length: 95[\r][\n]"
      2020-09-18 18:18:59.369 DEBUG <Thread-114> http-outgoing-0 << "Connection: keep-alive[\r][\n]"
      2020-09-18 18:18:59.369 DEBUG <Thread-114> http-outgoing-0 << "Date: Fri, 18 Sep 2020 22:18:59 GMT[\r][\n]"
      2020-09-18 18:18:59.370 DEBUG <Thread-114> http-outgoing-0 << "Server: Apache/2.4.46 () PHP/7.1.33[\r][\n]"
      2020-09-18 18:18:59.370 DEBUG <Thread-114> http-outgoing-0 << "X-Frame-Options: SAMEORIGIN[\r][\n]"
      2020-09-18 18:18:59.370 DEBUG <Thread-114> http-outgoing-0 << "Referrer-Policy: same-origin[\r][\n]"
      2020-09-18 18:18:59.370 DEBUG <Thread-114> http-outgoing-0 << "X-XSS-Protection: 1; mode=block[\r][\n]"
      2020-09-18 18:18:59.370 DEBUG <Thread-114> http-outgoing-0 << "X-Content-Type-Options: nosniff[\r][\n]"
      2020-09-18 18:18:59.371 DEBUG <Thread-114> http-outgoing-0 << "Strict-Transport-Security: max-age=5; includeSubdomains[\r][\n]"
      2020-09-18 18:18:59.371 DEBUG <Thread-114> http-outgoing-0 << "Vary: X-Forwarded-Proto[\r][\n]"
      2020-09-18 18:18:59.371 DEBUG <Thread-114> http-outgoing-0 << "X-Powered-By: PHP/7.1.33[\r][\n]"
      2020-09-18 18:18:59.371 DEBUG <Thread-114> http-outgoing-0 << "X-Cache: Miss from cloudfront[\r][\n]"
      2020-09-18 18:18:59.371 DEBUG <Thread-114> http-outgoing-0 << "Via: 1.1 5175c0b4bfaddbfcca703b6ef1dc6bad.cloudfront.net (CloudFront)[\r][\n]"
      2020-09-18 18:18:59.372 DEBUG <Thread-114> http-outgoing-0 << "X-Amz-Cf-Pop: ORD52-C1[\r][\n]"
      2020-09-18 18:18:59.372 DEBUG <Thread-114> http-outgoing-0 << "X-Amz-Cf-Id: 4bomJw2QHJ3NGSl6NPkSh6WarSKK6B-TwtGAfUPJm8Cjw8PqegWtAw==[\r][\n]"
      2020-09-18 18:18:59.372 DEBUG <Thread-114> http-outgoing-0 << "[\r][\n]"
      2020-09-18 18:18:59.372 DEBUG <Thread-114> http-outgoing-0 << "<?xml version="1.0" encoding="iso-8859-1"?><vercheck protocol="1.0" date="20200918"></vercheck>"
      2020-09-18 18:18:59.379 DEBUG <Thread-114> http-outgoing-0 << HTTP/1.1 200 OK
      2020-09-18 18:18:59.380 DEBUG <Thread-114> http-outgoing-0 << Content-Type: application/xml
      2020-09-18 18:18:59.380 DEBUG <Thread-114> http-outgoing-0 << Content-Length: 95
      2020-09-18 18:18:59.380 DEBUG <Thread-114> http-outgoing-0 << Connection: keep-alive
      2020-09-18 18:18:59.380 DEBUG <Thread-114> http-outgoing-0 << Date: Fri, 18 Sep 2020 22:18:59 GMT

      See ESR-7194 for further details

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              vasco.marques Vasco Marques
              Reporter:
              bowens Brittany Owens
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: