Out of Memory Exception in Active Analysis

Posted by: murali-424 on 3 August 2017, 7:03 am EST

  • Posted 3 August 2017, 7:03 am EST

    Hi,

    We purchased Olap grid recently and later realized it has limitations with loading more than 100K records. So based on your suggestions, we decided to use Active Analysis but even this control also throws same out-of-memory exception.

    - I've used UnBoundDatasource to load a DataTable of just 4000 records to PivotTable.
    - In the initial load, I've loaded all Dimension fields to Rows shelf
    - When I try to drag & drop a Measure field to Data shelf, it is throwing memory exception.

    Can you give us some suggestions? Please see the attached snapshot for more details.

    Similar issue i've posted in Olap grid forum:

    http://our.componentone.com/groups/topic/out-of-memory-exception-in-olap-grid/#post-609247

  • Replied 3 August 2017, 7:03 am EST

    Hi,

    Sometimes, i'm getting memory exception with just 5 records also. This happens only if more fields are selected in the grid - like around 10-15 Attributes, 4 Measures.

    Out of memory exception occurred in below scenarios:
    - When more Attribute fields are selected in the grid and when I try to drag/drop measure field to Data shelf.
    - In case I already saved layout view definition and try to apply it to PivotView (as you see in the below code, pivotview1.read....)
    - When I try to pre-select all fields in the grid. Error is thrown when I commit the layout changes.

    Below is the code I'm using:


    if (isConfigSaved)
    pivotView1.Read(myXml, GrapeCity.ActiveAnalysis.PersistSettings.Layout);
    else
    {
    //Use the ILayoutActions interface
    GrapeCity.ActiveAnalysis.Layout.ILayoutActions myLayoutActions = pivotView1.BeginLayoutUpdate("init");

    //Set the Marking type to Circle.
    myLayoutActions.SetMarkingType(GrapeCity.ActiveAnalysis.Layout.MarkingType.Text);

    foreach (DataColumn column in dt.Columns)
    {
    if (column.DataType == typeof(string))
    {
    myLayoutActions.AppendField("[Dimension].[" + column.ColumnName + "]", GrapeCity.ActiveAnalysis.Layout.ShelfKind.RowShelf);
    }
    else
    {
    myLayoutActions.AppendField("[Measures].[" + column.ColumnName + "]", GrapeCity.ActiveAnalysis.Layout.ShelfKind.DataShelf);
    }
    }

    myLayoutActions.AppendField("[Measure Names]", GrapeCity.ActiveAnalysis.Layout.ShelfKind.ColumnShelf);

    //Commit the layout actions to the viewer.
    myLayoutActions.Commit();
    }


    Please let me know if I'm doing something wrong.
  • Replied 3 August 2017, 7:03 am EST

    Anyone there? Please respond!
  • Replied 3 August 2017, 7:03 am EST

    Thanks for looking into this.

    I came to know that this issue has been escalated to your development team to fix this bug (FB id: 173403).

    Do you know when can I expect the hot-fix? And where I can check the status of this bug?
  • Replied 3 August 2017, 7:03 am EST

    Hello Murali,

    Thanks for the information. I have escalated this issue i.e. ActiveAnalysis becomes unresponsive when more than 12 attributes are placed in Row shelf to the development team for review and would update you as soon as I get anything from them.

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Thanks for your reply!

    This is actually our business requirement:

    - To be able to select any number of columns (attributes/measures) that are available on the screen and generate a report that user wants to review. So based on the selections made on the screen, our backend returns a DataTable that we load to (AA) grid.

    - To be able to load huge data (more than 500K records)

    - Ability to see all columns when many (like 20) attributes are loaded to grid. In Olap Grid, horizontal scrollbar was enabled only for measures so if more columns are selected, we were unable to see all the data.

    Please let me know if you need any other information.
  • Replied 3 August 2017, 7:03 am EST

    Hello Murali,

    Sorry for the delay on this thread. I tried placing more than 12 attributes in the Row shelf and could observe that PivotView becomes unresponsive. However, I would like to know the usecase so that I can forward it to the development team for review. Can you kindly provide more information on the exact requirement for placing so many attributes for analyzing the data ?

    This would help the development team and thereby, they can provide a workaround, if possible.

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Someone please respond! We want to know when this gets fixed so that we can decide whether to use this component in our Project or not.

    And by the way, it is not just happening when I select more Attribute fields in the grid but also, when I try to load huge data (with > 100K records) and select some of the fields in the initial load like below:


    //Use the ILayoutActions interface
    GrapeCity.ActiveAnalysis.Layout.ILayoutActions myLayoutActions = pivotView1.BeginLayoutUpdate("init");

    //Set the Marking type to Circle.
    myLayoutActions.SetMarkingType(GrapeCity.ActiveAnalysis.Layout.MarkingType.Text);

    foreach (DataColumn column in dt.Columns)
    {
    if (column.DataType == typeof(string))
    {
    myLayoutActions.AppendField("[Dimension].[" + column.ColumnName + "]", GrapeCity.ActiveAnalysis.Layout.ShelfKind.RowShelf);
    }
    else
    {
    myLayoutActions.AppendField("[Measures].[" + column.ColumnName + "]", GrapeCity.ActiveAnalysis.Layout.ShelfKind.DataShelf);
    }
    }

    myLayoutActions.AppendField("[Measure Names]", GrapeCity.ActiveAnalysis.Layout.ShelfKind.ColumnShelf);

    //Commit the layout actions to the viewer.
    myLayoutActions.Commit();
  • Replied 3 August 2017, 7:03 am EST

    It’s not yet fixed! I’ve tested with just 5 records and grid is taking forever to load the data.

    I tried to load 20 attributes and 4 measures in the initial load. Waited for almost 40 mins but still not getting loaded.

    Please see the attached snapshot.

  • Replied 3 August 2017, 7:03 am EST

    Hello Murali,

    The development team has made some changes related to your issue and dropped a temporary build. Could you kindly check with this build and let me know if it resolves the issue or not ?

    The build is available at this link:
    ftp://ftp.componentone.com/SupportFTP/Public/Ashish/analysis2.1.958.0.zip

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Hello Murali,

    Can you kindly provide the sample data which you are using so that development team can check and can suggest accordingly ?

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Hello Adrain,

    Are you loading all the attributes on the initial load ? If so, did you tried using the 958 build mentioned in one of my post above?

    If not, then you can use ActiveAnalysis and can refer to the following link describing how you can optimize the performance:
    http://helpcentral.componentone.com/nethelp/ActiveAnalysis2WebHelp/webframe.html#OptimizeDataDynamicsAnalysis.html

    Regards
  • Replied 3 August 2017, 7:03 am EST

    I'm trying that version now and will let you know. One other thing on the our of memory error, is there any way to trap this error so that it can be handled properly. I don't have an issue with running out of memory as such, my system may load anywhere from a few hundred records up to several hundred thousand. If it runs out of memory that's ok but I MUST be able to trap and handle the error cleanly.
  • Replied 3 August 2017, 7:03 am EST

    Hi there,
    I've installed the 958 build and I can report that the out of memory errors don't seem to occur as early as they did however It still seems to be throwing the error when there is still a reasonable amount of standby memory available. Ad there is still the issue of not being able to catch error when it's thrown.

    It seems to me that the power of a component like this is it's ability to analyse a large set of data. My software for instance is looking at capacity and utilisation of a large number of windows servers, say around 5000 this could be several hundred thousand records. The 958 build has alleviated the issue a little but it still doesn't go far enough. Is it possible to look again at the memory usage of AA, perhaps with a view to caching some data to disk to reduce the memory requirements. And as I mentioned in the pervious post, provide the ability to catch the error or raise an event when memory allocation changes so that we can monitor it and gracefully deal with the issue.

    Cheers...

    Adrian.
  • Replied 3 August 2017, 7:03 am EST

    To give you an idea of the scale of my issue. I'm running on windows 8.1 with 8GB memory. The attached image shows the attributes and measures and there is about 1.4M rows in the underlying data.

  • Replied 3 August 2017, 7:03 am EST

    Is there any update on this one, I'm just starting a project and intend to use AA but not if it still suffers from these issues.

    Adrian.
  • Replied 3 August 2017, 7:03 am EST

    Hello Murali,

    Thanks for the information. I have forwarded your comments to the development team and would update you as soon as I get anything from them.

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Hello,

    Thanks for the information. I would update you when I hear anything on enhancement request from the development team.

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Hello,

    Regarding the performance, I would suggest you to try cubes as the same are suggested in case of large data.

    For second, I have posted an enhancement request for adding support for trapping OutOfMemory exception(172386).

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Hello,

    You can copy paste the below url to download the new build:
    http://cdn.componentone.com/files/gcaa/gcaa-2.1.979.0.zip

    Regards
  • Replied 3 August 2017, 7:03 am EST

    Hi Ashish

    That link doesn't seem to work.

    Dom
  • Replied 3 August 2017, 7:03 am EST

    Hello,

    A new event MemoryOverflowed is added in the latest version 979 to catch the error and the build is available at this link:
    http://cdn.componentone.com/files/gcaa/gcaa-2.1.979.0.zip​

    Regards
    Ashish
  • Replied 3 August 2017, 7:03 am EST

    Thanks Ashish,
    I have played a little with SQL Server Analysis Services putting my data into a cube and this does seem to resolve the memory and performance issue but I don't want to have my clients install SSAS when my analysis is quite simple. Anyway, I have a workaround (sortof), I get the client to set some filters before loading the schema etc. and this gets through without memory problems. It's not very satisfactory though as it cuts down the analysis capabilities. A trap for out of memory condition will help a lot.

    Cheers..
  • Replied 3 August 2017, 7:03 am EST

    Ashish, is there a list of changes in version .979?
    Also, any chance the development team will add memory management to support large data sets?

    Dom, can you confirm if the new version indeed improved the behavior?
  • Replied 3 August 2017, 7:03 am EST

    Hello,

    This build didn't contain any new features, hence, the notes were not available. However, the following customer issues were fixed in this version:
    173796: 'System.NullReferenceException' when using the LayoutActions functionality
    172386: Request to add support for trapping OutOfMemory exception
    175261: ExportTo button does not hide whenVisible property is set to false
    175815: Request to add notification about low memory while saving data in AA
    176325: AA, Underlying Data viewer missing records

    Regards
  • Marked as Answer

    Replied 3 August 2017, 7:03 am EST

    Thanks, Ashish!
  • Replied 4 March 2018, 10:27 pm EST

    Hi
    I have the same OutOfMemoryException. It seems occurs when our program LgcBusinessIntelligence, a simple windows form program that use your Windows.Forms.PivotView, reach 1,5 gb of memory, even if the PC where it runs has a lot of free memory. See attached screenshots.

    I have updated to version 2.1.979.0 but nothing has changed.

    Can you help me?

    Thanks

    Alessandro Strazzari
    R & D manager
    mobile: +39 338 6686431
    e-mail: astrazzari@logicasistemi.com


    GrapeCity.ActAna.OutOfMemEx.zip
  • Replied 5 March 2018, 1:05 am EST

    I read better and now I understand that in the new build 2.1.979.0 you have only added support for trapping the OutOfMemory exception, not solved the Out of memory problem.
    Why does your component thrown an Out of memory exception when it reaches 1,5 gb even if there is still many free memory available in the pc?

    Thanks
Need extra support?

Upgrade your support plan and get personal unlimited phone support with our customer engagement team

Learn More

Forum Channels