Uploaded image for project: 'Pentaho Data Integration - Kettle'
  1. Pentaho Data Integration - Kettle
  2. PDI-18993

Text File Input not processing escape characters properly ("""" as " instead of "")

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Severity: High
    • Resolution: Unresolved
    • Affects Version/s: 8.3.0 GA, 9.1.0 GA
    • Fix Version/s: Backlog
    • Component/s: Step
    • Labels:
      None
    • Story Points:
      0
    • PDI Sub-component:
    • Notice:
      When an issue is open, the "Fix Version/s" field conveys a target, not necessarily a commitment. When an issue is closed, the "Fix Version/s" field conveys the version that the issue was fixed in.
    • Steps to Reproduce:
      Hide

      Unzip attached zip file. Open exfile.ktr in Spoon and run the transformation. You will notice that:

      "

      {""Example1"":"""",""Example"":""Test""}

      "

      Gets interpreted as

      { "Example1":","Example":"Test"}

      by Text File Input

      instead of

      { "Example1":"","Example":"Test"}

      by CSV Input

      Show
      Unzip attached zip file. Open exfile.ktr in Spoon and run the transformation. You will notice that: " {""Example1"":"""",""Example"":""Test""} " Gets interpreted as { "Example1":","Example":"Test"} by Text File Input instead of { "Example1":"","Example":"Test"} by CSV Input

      Description

      I have a CSV file with embedded JSON. One of the Json fields is blank, so the value in JSON is:

      { "Example1":"", ...}

      When encoded in the CSV column, it becomes:

      "

      {""Example1"":"""", ...}

      "

      However, Text File Input (with no escape, or " as the escape) interprets the value as:

      {"Example1":", ...}

      having just a single quote, where there should be two. (see picture)

      Note that the CSV File Input step interprets this correctly. (see picture)

        Attachments

        1. CSV_Working.png
          CSV_Working.png
          12 kB
        2. exfile.zip
          3 kB
        3. TFI_Bug.png
          TFI_Bug.png
          10 kB

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            jjarvis Jonathan Jarvis
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated: