Pentaho Analysis - Mondrian
  1. Pentaho Analysis - Mondrian
  2. MONDRIAN-163

VirtualCube SegmentArrayQuerySpec.addMeasure assert

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Severe Severe
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      If one creates the virtual cube:


      <VirtualCube name="Sales vs Warehouse">
      <!--
        <VirtualCubeDimension name="Time"/>
      -->
        <VirtualCubeDimension name="Product"/>
        <VirtualCubeMeasure cubeName="Warehouse"
      name="[Measures].[Warehouse Sales]"/>
        <VirtualCubeMeasure cubeName="Sales"
      name="[Measures].[Unit Sales]"/>
      </VirtualCube>

      (not Time commented out) and then runs:

      select
      {
      [Measures].[Warehouse Sales],
      [Measures].[Unit Sales]
      }
      ON COLUMNS,
      {[Product].[All Products]}
      ON ROWS
      from [Sales vs Warehouse];

      one gets the assert exception:
      Caused by: mondrian.olap.MondrianException: Mondrian
      Error:Internal error: assert failed
              at
      mondrian.resource.MondrianResource$_Def0.ex(MondrianResource.java:1027)
              at mondrian.olap.Util.newInternal(Util.java:886)
              at mondrian.olap.Util.assertTrue(Util.java:867)
              at
      mondrian.rolap.agg.SegmentArrayQuerySpec.addMeasure(SegmentArrayQuerySpec.java:110)
              at
      mondrian.rolap.agg.AbstractQuerySpec.nonDistinctGenerateSQL(AbstractQuerySpec.java:90)
              at
      mondrian.rolap.agg.SegmentArrayQuerySpec.generateSqlQuery(SegmentArrayQuerySpec.java:84)
              at
      mondrian.rolap.agg.AggregationManager.generateSql(AggregationManager.java:198)
              at
      mondrian.rolap.agg.Segment.load(Segment.java:251)
              at
      mondrian.rolap.agg.Aggregation.load(Aggregation.java:130)
              at
      mondrian.rolap.agg.AggregationManager.loadAggregation(AggregationManager.java:71)
              at
      mondrian.rolap.FastBatchingCellReader$Batch.loadAggregation(FastBatchingCellReader.java:284)
              at
      mondrian.rolap.FastBatchingCellReader.loadAggregations(FastBatchingCellReader.java:154)
              at
      mondrian.rolap.RolapResult.executeBody(RolapResult.java:239)
              at
      mondrian.rolap.RolapResult.<init>(RolapResult.java:157)
              at
      mondrian.rolap.RolapConnection.execute(RolapConnection.java:330)
              ... 6 more

      On the other hand if the Time dimension is not
      commented out, then there is not assert error
      but "Unit Sales" is zero.

      The junit test file VirtualCubeTest.java can be
      used to see this.

        Activity

        Hide
        Mondrian Importer User added a comment -
        {avix}, 10/30/2006: status_id, 1 |
        {avix}, 10/30/2006: close_date, 0
        Show
        Mondrian Importer User added a comment - {avix}, 10/30/2006: status_id, 1 | {avix}, 10/30/2006: close_date, 0
        Hide
        Mondrian Importer User added a comment -
        {pschoepf}, 04/08/2006: Logged In: YES
        user_id=1493994

        This bug make the virtualcube somehow useless in certain
        situations. Any quick fix tips?
        Show
        Mondrian Importer User added a comment - {pschoepf}, 04/08/2006: Logged In: YES user_id=1493994 This bug make the virtualcube somehow useless in certain situations. Any quick fix tips?
        Hide
        Mondrian Importer User added a comment -
        {tilrzn}, 04/28/2006: Logged In: YES
        user_id=1509041

        The same bug can be reproduced on FoodMart by removing
        aggregations from Sales cube.
        We mentioned that any cube (referenced from vurtual cube)
        without aggregation tables fails pretty simple query, e.g.:
        select {[Measures].[Store Sqft],[Measures].[Units Ordered]}
        ON COLUMNS
        from [Warehouse and Store]
        This vurtual cube based on cubes from FoodMart:
        <VirtualCube name="Warehouse and Store">
          <VirtualCubeDimension name="Store"/>
          <VirtualCubeMeasure cubeName="Store"
        name="[Measures].[Store Sqft]"/>
          <VirtualCubeMeasure cubeName="Warehouse"
        name="[Measures].[Units Ordered]"/>
        </VirtualCube>

        Suppose that in case of aggregation tables code works in a
        different way.

        So, temporary solution for those who experiencing this
        problem could be to add in schema fake aggregation table?.. ;)
        Show
        Mondrian Importer User added a comment - {tilrzn}, 04/28/2006: Logged In: YES user_id=1509041 The same bug can be reproduced on FoodMart by removing aggregations from Sales cube. We mentioned that any cube (referenced from vurtual cube) without aggregation tables fails pretty simple query, e.g.: select {[Measures].[Store Sqft],[Measures].[Units Ordered]} ON COLUMNS from [Warehouse and Store] This vurtual cube based on cubes from FoodMart: <VirtualCube name="Warehouse and Store">   <VirtualCubeDimension name="Store"/>   <VirtualCubeMeasure cubeName="Store" name="[Measures].[Store Sqft]"/>   <VirtualCubeMeasure cubeName="Warehouse" name="[Measures].[Units Ordered]"/> </VirtualCube> Suppose that in case of aggregation tables code works in a different way. So, temporary solution for those who experiencing this problem could be to add in schema fake aggregation table?.. ;)
        Hide
        Mondrian Importer User added a comment -
        {tilrzn}, 05/02/2006: Logged In: YES
        user_id=1509041

        Hi folks!
        I've meditated during whole half of day. I was debugging
        Mondrian under JBoss (with this issue on topic) and looking
        up for zen in source code.
        Well, I can't say I found zen there. Of course, not because
        there is no zen there, but because no one could find zen in
        source code. So,
        I can't say exactly why this assertion fails. Well, in some
        kind of sense we all know why: fact table differs from star
        table. I didn't lookup for answer why.
        I ask myself: "What was going wrong, when this assertion
        failed?" And further there is what I get (instead of zen).

        In a stack trace (you can find it in bug details) one can
        see method call:

           mondrian.rolap.RolapResult.executeBody(RolapResult.java:239)

        As we see, method fails at line 239 on loadAggregation() call.
        But some lines above this call in method there is another
        method call: executeStripe(). Let's see it in details.

           private void executeStripe(int axisOrdinal,
        RolapEvaluator evaluator);

        There is a comment around this block:

           // Compute the cells several times. The first time, use a
        dummy
           // evaluator which collects requests.

        So, I can't say 'correctly' what is made there, I do not
        know all the terminology. I would say as I understood it.
        Here we use FastBatchingCellReader class to get list of
        batches which we should execute to obtain values (from DB).
        And now I run ahead to make it more interesting for you.
        It seems to be THE PROBLEM with these batches.
        In case of 'working' virtual cube [Warehouse and Sales]
        there are 2 batches, but in case of 'non-working' virtual
        cube [Warehouse and Store] there is just one. And I think I
        got it why. And now let's get back to the subject at hand.

        In both cases ([Warehouse and Sales] and [Warehouse and
        Store]) for the first call of this method axisOrdinal=0, so
        we go into 'else' statement, and there is some kind of
        recurse. We call it again and going to the another branch in
        this 'if' statement one time for each member (from query),
        hence, 2 times in my case.
        RolapResult.executeStripe, line 335:

           o = evaluator.evaluateCurrent();

        RolapEvaluator.evaluateCurrent(), line 304:

           Object o = cellReader.get(this);

        FastBatchingCellReader.get(), line 80-81:

           CellRequest request =
                          
        RolapAggregationManager.makeRequest(currentMembers,
           false, false);

        In makeRequest() method we create this request (could be a
        synonym of 'batch'). I'll get back to this method later.
        Then, at line 99 in FastBatchingCellReader.get() we record it:

           recordCellRequest(request);

        This is the way of how these 'batches' are putting into list.
        And where is mistake? WHY there is just one 'batch' in case
        of [Warehouse and Store] cube?
        The problem is in
        FastBatchingCellReader.recordCellRequest(), line 113:

           Batch batch = (Batch) batches.get(key);

        We got it. First (before creating the 'batch') we are
        searching it in list (if it already exist).
        The second batch (for the second member in query) in case of
        [Warehouse and Store] cube has the SAME KEY (zero-value).
        Hence, we OVERWRITE one batch with other, that's why there
        is just one batch for that query.

        Well, and WHY in case of [Warehouse and Sales] everything is
        ok? It's because of aggregations.
        By default, every query for [Warehouse and Sales] cube has
        one more column: 'time_by_day'.'the_year'.
        For this column we make a constraint in request (year=1997).
        It is in RolapAggregationManager.makeRequest(), lines 90-92:

           boolean needToReturnNull =
                                  
        level.getLevelReader().constrainRequest(
                                           member, mapLevelToColumn,
        request);


        In case of this column it is used RegularLevelReader, which
        MODIFIES THE KEY.
        In other cases AllLevelReaderImpl is used, which has empty
        method constrainRequest():

           // We don't need to apply any constraints.

        Hence, default key for such batch (small zero-value) never
        being modified.

        SUMMARY:
        1) the problem is one batch in request instead of two;
        2) main cause is duplicated (default) value of batch key;
        3) and everything is working in case of [Warehouse and
        Sales] cube because there are no duplicate keys.

        I can't offer some solution now, because I'm not familiar in
        semantics of code (commonly what these bits in key means and
        where it is used).
        Could anyone solve it or give an advice how to do so?

        Hope my work will help a little.
        Show
        Mondrian Importer User added a comment - {tilrzn}, 05/02/2006: Logged In: YES user_id=1509041 Hi folks! I've meditated during whole half of day. I was debugging Mondrian under JBoss (with this issue on topic) and looking up for zen in source code. Well, I can't say I found zen there. Of course, not because there is no zen there, but because no one could find zen in source code. So, I can't say exactly why this assertion fails. Well, in some kind of sense we all know why: fact table differs from star table. I didn't lookup for answer why. I ask myself: "What was going wrong, when this assertion failed?" And further there is what I get (instead of zen). In a stack trace (you can find it in bug details) one can see method call:    mondrian.rolap.RolapResult.executeBody(RolapResult.java:239) As we see, method fails at line 239 on loadAggregation() call. But some lines above this call in method there is another method call: executeStripe(). Let's see it in details.    private void executeStripe(int axisOrdinal, RolapEvaluator evaluator); There is a comment around this block:    // Compute the cells several times. The first time, use a dummy    // evaluator which collects requests. So, I can't say 'correctly' what is made there, I do not know all the terminology. I would say as I understood it. Here we use FastBatchingCellReader class to get list of batches which we should execute to obtain values (from DB). And now I run ahead to make it more interesting for you. It seems to be THE PROBLEM with these batches. In case of 'working' virtual cube [Warehouse and Sales] there are 2 batches, but in case of 'non-working' virtual cube [Warehouse and Store] there is just one. And I think I got it why. And now let's get back to the subject at hand. In both cases ([Warehouse and Sales] and [Warehouse and Store]) for the first call of this method axisOrdinal=0, so we go into 'else' statement, and there is some kind of recurse. We call it again and going to the another branch in this 'if' statement one time for each member (from query), hence, 2 times in my case. RolapResult.executeStripe, line 335:    o = evaluator.evaluateCurrent(); RolapEvaluator.evaluateCurrent(), line 304:    Object o = cellReader.get(this); FastBatchingCellReader.get(), line 80-81:    CellRequest request =                    RolapAggregationManager.makeRequest(currentMembers,    false, false); In makeRequest() method we create this request (could be a synonym of 'batch'). I'll get back to this method later. Then, at line 99 in FastBatchingCellReader.get() we record it:    recordCellRequest(request); This is the way of how these 'batches' are putting into list. And where is mistake? WHY there is just one 'batch' in case of [Warehouse and Store] cube? The problem is in FastBatchingCellReader.recordCellRequest(), line 113:    Batch batch = (Batch) batches.get(key); We got it. First (before creating the 'batch') we are searching it in list (if it already exist). The second batch (for the second member in query) in case of [Warehouse and Store] cube has the SAME KEY (zero-value). Hence, we OVERWRITE one batch with other, that's why there is just one batch for that query. Well, and WHY in case of [Warehouse and Sales] everything is ok? It's because of aggregations. By default, every query for [Warehouse and Sales] cube has one more column: 'time_by_day'.'the_year'. For this column we make a constraint in request (year=1997). It is in RolapAggregationManager.makeRequest(), lines 90-92:    boolean needToReturnNull =                            level.getLevelReader().constrainRequest(                                    member, mapLevelToColumn, request); In case of this column it is used RegularLevelReader, which MODIFIES THE KEY. In other cases AllLevelReaderImpl is used, which has empty method constrainRequest():    // We don't need to apply any constraints. Hence, default key for such batch (small zero-value) never being modified. SUMMARY: 1) the problem is one batch in request instead of two; 2) main cause is duplicated (default) value of batch key; 3) and everything is working in case of [Warehouse and Sales] cube because there are no duplicate keys. I can't offer some solution now, because I'm not familiar in semantics of code (commonly what these bits in key means and where it is used). Could anyone solve it or give an advice how to do so? Hope my work will help a little.
        Hide
        Mondrian Importer User added a comment -
        {tilrzn}, 05/03/2006: Logged In: YES
        user_id=1509041

        I think there is some concept confusion.
        'Batches' which I've mentioned in previous letter use this
        bitKey as a bit mask and as an identifier.
        And everything is ok when all members belong to one Star (in
        case of cubes and measures of virtual cube when they are
        from one cube).
        When members belong to different stars, we got this
        assertion failure. It's because there is no 'star
        identifier' in bitKey of 'batch', hence no different batches
        for different measures.

        Well, no one wants to go into this Bits Hell and fix that
        bug, but could we, end-users, get some reaction, some
        progress, any opinion of developers?
        It's a great bug. Virtual cubes are not working. Any ideas?
        Any chances to be fixed?

        I give you my excuses for terrible English in previous
        letter. I think letter on my native language yesterday would
        be not much better (I was wacked by Mondrian's code). :)
        And I verified it once again: problem is with bitKeys.
        You can make any of your virtual cubes work by providing
        constraints.
        For example, this query works fine:

           SELECT {[Measures].[Store Sqft], [Measures].[Units Ordered]}
           DIMENSION PROPERTIES PARENT_UNIQUE_NAME ON COLUMNS
        FROM [Warehouse and Store]
           WHERE ([Store].[All Stores].[Canada])

        And this fails:

           SELECT {[Measures].[Store Sqft], [Measures].[Units Ordered]}
           DIMENSION PROPERTIES PARENT_UNIQUE_NAME ON COLUMNS
        FROM [Warehouse and Store]
           WHERE ([Store].[All Stores])

        No doubts, IT'S BECAUSE OF CONSTRAINTS. They make our
        batches differ from each other (by the keys).
        In first case:

           2006-05-03 10:59:50,093 DEBUG
           [mondrian.rolap.FastBatchingCellReader]
        FastBatchingCellReader:
          
        bitkey=0x0000000000000000000000000000000000000000000000000000000000001000
           2006-05-03 10:59:50,093 DEBUG
           [mondrian.rolap.FastBatchingCellReader]
        FastBatchingCellReader:
          
        bitkey=0x0000000000000000000000000000000000000000000000000000000010000000
           2006-05-03 10:59:50,093 DEBUG
           [mondrian.rolap.agg.AggregationManager] NO MATCH:
           inventory_fact_1997
        foreign=0x0000000000000000000000000000000000000000000000000000000010000000
               
        measure=0x0000000000000000000000000000000000000000000000000000000000100000

        In second:

           2006-05-03 11:00:30,906 DEBUG
           [mondrian.rolap.FastBatchingCellReader]
        FastBatchingCellReader:
          
        bitkey=0x0000000000000000000000000000000000000000000000000000000000000000
           2006-05-03 11:00:30,906 DEBUG
           [mondrian.rolap.agg.AggregationManager] NO MATCH: store
          
        foreign=0x0000000000000000000000000000000000000000000000000000000000000000
               
        measure=0x0000000000000000000000000000000000000000000000000000000000100001

        So, WE NEED TO MAKE OUR BATCHES DIFFER BY DIFFERENT STARS.
        How to do so? Maybe include 'star identifier' in bitKey,
        maybe something else.
        What developer would say?
        Show
        Mondrian Importer User added a comment - {tilrzn}, 05/03/2006: Logged In: YES user_id=1509041 I think there is some concept confusion. 'Batches' which I've mentioned in previous letter use this bitKey as a bit mask and as an identifier. And everything is ok when all members belong to one Star (in case of cubes and measures of virtual cube when they are from one cube). When members belong to different stars, we got this assertion failure. It's because there is no 'star identifier' in bitKey of 'batch', hence no different batches for different measures. Well, no one wants to go into this Bits Hell and fix that bug, but could we, end-users, get some reaction, some progress, any opinion of developers? It's a great bug. Virtual cubes are not working. Any ideas? Any chances to be fixed? I give you my excuses for terrible English in previous letter. I think letter on my native language yesterday would be not much better (I was wacked by Mondrian's code). :) And I verified it once again: problem is with bitKeys. You can make any of your virtual cubes work by providing constraints. For example, this query works fine:    SELECT {[Measures].[Store Sqft], [Measures].[Units Ordered]}    DIMENSION PROPERTIES PARENT_UNIQUE_NAME ON COLUMNS FROM [Warehouse and Store]    WHERE ([Store].[All Stores].[Canada]) And this fails:    SELECT {[Measures].[Store Sqft], [Measures].[Units Ordered]}    DIMENSION PROPERTIES PARENT_UNIQUE_NAME ON COLUMNS FROM [Warehouse and Store]    WHERE ([Store].[All Stores]) No doubts, IT'S BECAUSE OF CONSTRAINTS. They make our batches differ from each other (by the keys). In first case:    2006-05-03 10:59:50,093 DEBUG    [mondrian.rolap.FastBatchingCellReader] FastBatchingCellReader:    bitkey=0x0000000000000000000000000000000000000000000000000000000000001000    2006-05-03 10:59:50,093 DEBUG    [mondrian.rolap.FastBatchingCellReader] FastBatchingCellReader:    bitkey=0x0000000000000000000000000000000000000000000000000000000010000000    2006-05-03 10:59:50,093 DEBUG    [mondrian.rolap.agg.AggregationManager] NO MATCH:    inventory_fact_1997 foreign=0x0000000000000000000000000000000000000000000000000000000010000000         measure=0x0000000000000000000000000000000000000000000000000000000000100000 In second:    2006-05-03 11:00:30,906 DEBUG    [mondrian.rolap.FastBatchingCellReader] FastBatchingCellReader:    bitkey=0x0000000000000000000000000000000000000000000000000000000000000000    2006-05-03 11:00:30,906 DEBUG    [mondrian.rolap.agg.AggregationManager] NO MATCH: store    foreign=0x0000000000000000000000000000000000000000000000000000000000000000         measure=0x0000000000000000000000000000000000000000000000000000000000100001 So, WE NEED TO MAKE OUR BATCHES DIFFER BY DIFFERENT STARS. How to do so? Maybe include 'star identifier' in bitKey, maybe something else. What developer would say?
        Hide
        Mondrian Importer User added a comment -
        {avix}, 10/30/2006: Logged In: YES
        user_id=36405

        This seems to be fixed in 2.2 - I had no problems running
        the example

        <VirtualCube name="Sales vs Warehouse">
        <!--
        <VirtualCubeDimension name="Time"/>
        -->
        <VirtualCubeDimension name="Product"/>
        <VirtualCubeMeasure cubeName="Warehouse"
        name="[Measures].[Warehouse Sales]"/>
        <VirtualCubeMeasure cubeName="Sales"
        name="[Measures].[Unit Sales]"/>
        </VirtualCube>

        (not Time commented out) and then runs:

        select
        {
        [Measures].[Warehouse Sales],
        [Measures].[Unit Sales]
        }
        ON COLUMNS,
        {[Product].[All Products]}
        ON ROWS
        from [Sales vs Warehouse]
        Show
        Mondrian Importer User added a comment - {avix}, 10/30/2006: Logged In: YES user_id=36405 This seems to be fixed in 2.2 - I had no problems running the example <VirtualCube name="Sales vs Warehouse"> <!-- <VirtualCubeDimension name="Time"/> --> <VirtualCubeDimension name="Product"/> <VirtualCubeMeasure cubeName="Warehouse" name="[Measures].[Warehouse Sales]"/> <VirtualCubeMeasure cubeName="Sales" name="[Measures].[Unit Sales]"/> </VirtualCube> (not Time commented out) and then runs: select { [Measures].[Warehouse Sales], [Measures].[Unit Sales] } ON COLUMNS, {[Product].[All Products]} ON ROWS from [Sales vs Warehouse]
        Hide
        Gretchen Moran added a comment -
        Temporary open to allow setting the resolution field.
        Show
        Gretchen Moran added a comment - Temporary open to allow setting the resolution field.
        Hide
        Gretchen Moran added a comment -
        Bulk changed the resolution field from NONE to FIXED in order for JIRA to report properly.
        Show
        Gretchen Moran added a comment - Bulk changed the resolution field from NONE to FIXED in order for JIRA to report properly.

          People

          • Assignee:
            Unassigned User
            Reporter:
            emberson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: