Pentaho Analysis - Mondrian
  1. Pentaho Analysis - Mondrian
  2. MONDRIAN-994

Mapped Diagnostic Context (MDC) lost in RolapResultShepherd

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Severe Severe
    • Resolution: Fixed
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • QA Validation Status:
      Validated by QA

      Description

      What is MDC?

      A Mapped Diagnostic Context (MDC) is an instrument for distinguishing interleaved log output from different sources.
      Log output is typically interleaved when a server handles multiple clients near-simultaneously.
      The MDC is managed on a per thread basis. A child thread automatically inherits a copy of the mapped diagnostic context of its parent
      In short, it allows log consumers to have meta data about threads in a multi-threaded environment.
      It is a Hashtable with key value pairs providing the meta data about the thread.
      Without it, for example, we cannot be sure which logs statements belong to which MDX query being run if multiple MDX queries are run simultaneously
      -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

      Whats the issue?

      When the RolapResultShepherd is created, it automatically gains the MDC of its parent thread.
      The Executions that are run by the RolapResultShepherd end up with the MDC of the RolapResultShepherd, which is never updated after it is first created.
      As a result, the logging statements for these executions will contain the MDC of the RolapResultShepherd, instead of the original thread that created the Execution.
      All log statements end up with the same MDC

      -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

      How to resolve this?

      The MDC is updated from the MDC of the original thread whenever logging is going to be done

      Before the Execution is passed to the RolapResultShepherd, the MDC is saved as a Map in the Execution.
      Later when the Execution is actually processed, the Map is placed back into the MDC.
      1. mdc.patch
        3 kB
        Dan Dosch
      2. mdc-patch.diff
        2 kB
        Dan Dosch

        Activity

        Hide
        Dan Dosch added a comment -
        A diff for a proposed patch that would resolve the issue for logging statements coming from the MDX_LOGGER and SQL_LOGGERS [mondrian.mdx] and [mondrian.sql] categories
        Show
        Dan Dosch added a comment - A diff for a proposed patch that would resolve the issue for logging statements coming from the MDX_LOGGER and SQL_LOGGERS [mondrian.mdx] and [mondrian.sql] categories
        Hide
        Sarah Gerweck added a comment -
        We at Yahoo had a call with Julian about this issue, including some thoughts on threading practices in Mondrian. I think this is an illuminating case as Mondrian starts to work more with thread pools. There is an email floating around with some of our thoughts and I'd be happy to elaborate here if anybody would like.

        This is an important issue for us at Yahoo. We can do the patch ourselves in the short term, but long term we need to make sure that the MDC is maintained on all logging messages.
        Show
        Sarah Gerweck added a comment - We at Yahoo had a call with Julian about this issue, including some thoughts on threading practices in Mondrian. I think this is an illuminating case as Mondrian starts to work more with thread pools. There is an email floating around with some of our thoughts and I'd be happy to elaborate here if anybody would like. This is an important issue for us at Yahoo. We can do the patch ourselves in the short term, but long term we need to make sure that the MDC is maintained on all logging messages.
        Hide
        Dan Dosch added a comment -
        This patch makes sure the MDC sticks around in the places we are concerned with - namely the mondrian.mdx and mondrian.sql log statements. It doesn't necessarily ensure they do not disappear elsewhere in the application
        Show
        Dan Dosch added a comment - This patch makes sure the MDC sticks around in the places we are concerned with - namely the mondrian.mdx and mondrian.sql log statements. It doesn't necessarily ensure they do not disappear elsewhere in the application
        Hide
        Luc Boudreau added a comment -
        Fixed in revision 14955 in //open/mondrian.
        Show
        Luc Boudreau added a comment - Fixed in revision 14955 in //open/mondrian.
        Hide
        Julian Hyde added a comment -
        Thanks for checking in the patch, Luc.

        However, we need a test case to make sure that this issue doesn't recur. Re-opening. I can help create a test case.
        Show
        Julian Hyde added a comment - Thanks for checking in the patch, Luc. However, we need a test case to make sure that this issue doesn't recur. Re-opening. I can help create a test case.
        Hide
        Dan Dosch added a comment -
        Hi Luc,
         I did a checkout today. I see the MDC patches to mondrian.server.Execution and mondrian.rolap.RolapConnection but not mondrian.rolap.FastBatchingCellReader. Wondering if maybe you got the old version of the patch (the one I uploaded originally with this ticket) rather than the one I uploaded on 2/14.

        Thanks,
        Dan
        Show
        Dan Dosch added a comment - Hi Luc,  I did a checkout today. I see the MDC patches to mondrian.server.Execution and mondrian.rolap.RolapConnection but not mondrian.rolap.FastBatchingCellReader. Wondering if maybe you got the old version of the patch (the one I uploaded originally with this ticket) rather than the one I uploaded on 2/14. Thanks, Dan
        Hide
        Luc Boudreau added a comment -
        I used the old patch when fixing that issue. I've integrated the latest patch now. My bad. Fixed in revision 14962 in //open/mondrian.
        Show
        Luc Boudreau added a comment - I used the old patch when fixing that issue. I've integrated the latest patch now. My bad. Fixed in revision 14962 in //open/mondrian.
        Hide
        Luc Boudreau added a comment -
        I've written a unit test for this case.

        Commit: https://github.com/pentaho/mondrian/commit/3db18064c862afaa4e606af3065c819d44b1f012
        Test: UdfTest.testMdc()
        Show
        Luc Boudreau added a comment - I've written a unit test for this case. Commit: https://github.com/pentaho/mondrian/commit/3db18064c862afaa4e606af3065c819d44b1f012 Test: UdfTest.testMdc()
        Hide
        Will Gorman added a comment -
        Validated via unit test.
        Show
        Will Gorman added a comment - Validated via unit test.

          People

          • Assignee:
            Will Gorman
            Reporter:
            Dan Dosch
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: