Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Medium Medium
    • Resolution: Unresolved
    • Affects Version/s: 4.3.0 GA (4.5.0 GA Suite Release)
    • Fix Version/s: Not Planned
    • Component/s: None
    • Labels:
      None
    • QA Validation Status:
      Not Yet Validated

      Description

      I'd like to use executed process STDOUT as data input / STDIN as data ouput

      I have legacy software that I can't change.
      This levacy softtware decodes binary data to either STDOUT or to file (up to hundreds of megabytes)

      Have a possibility to use process standard output as input for my transformation, with same functional behaviour as text file input
      or process standard input as output for my transformation (same functional behaviour as text file output),
      would facilitate my life (and ensure integration of 3rd party processes)

      I know that existing "Execute a process" can catch STDOUT to a field, the problem is that for large output of process, it fills memory and crash JVM.


        Activity

        Hide
        Slawomir Chodnicki added a comment -
        Can you try using a unix pipe for reading from STDOUT and writing to another tool's STDIN?

        Here's a blog post that details the technique:

        http://type-exit.org/adventures-with-open-source-bi/2012/06/implementing-universal-bulk-loading-in-pdi/
        Show
        Slawomir Chodnicki added a comment - Can you try using a unix pipe for reading from STDOUT and writing to another tool's STDIN? Here's a blog post that details the technique: http://type-exit.org/adventures-with-open-source-bi/2012/06/implementing-universal-bulk-loading-in-pdi/
        Hide
        Thierry Accart added a comment -
        Thanks for this interesting article using unix pipes. It could be a workaround.
        I was thinking about something more generic, that would run on any OS, not only unix like ones.

        Anyway, you're pointing an intersting element : instead of managing stdin and stdout, maybe should it address generic pipes.
        Show
        Thierry Accart added a comment - Thanks for this interesting article using unix pipes. It could be a workaround. I was thinking about something more generic, that would run on any OS, not only unix like ones. Anyway, you're pointing an intersting element : instead of managing stdin and stdout, maybe should it address generic pipes.
        Hide
        Dan Keeley (codek) added a comment -
        See PDI-2290 too
        Show
        Dan Keeley (codek) added a comment - See PDI-2290 too

          People

          • Assignee:
            Unassigned User
            Reporter:
            Thierry Accart
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated: