SaveExcel() throws an OutOfMemoryException

Posted by: danny-meier on 8 September 2017, 5:24 am EST

  • Posted 8 September 2017, 5:24 am EST

    Hi,

    We use FarPoint Spread in version 4.0.3501.2008.

    This exception is only thrown on large worksheets. We're talking about 50'000 - 60'000 rows and 90 columns.

    First step is filling an object[,] array and afterwards calling the method sheet.SetArray() to copy the content from the array to the worksheet. This is working very well and is quite fast. After that, we call SaveExcel(). The memory is increasing continously up to the limit of 920MB. Then the OutOfMemoryException will be thrown.

    The same thing we did with the newest release 4.0.3522.2008. The same thing happens. Only the exception message is a bit different

    =========== 22.07.2010 / 09:12:15 ===========

    Error saving Excel file

       at System.Collections.ArrayList.set_Capacity(Int32 value)

       at System.Collections.ArrayList.EnsureCapacity(Int32 min)

       at System.Collections.ArrayList.Add(Object value)

       at ci.a(Int16 A_0, ArrayList A_1, ArrayList A_2, ArrayList A_3, ArrayList A_4)

       at as.a(Int16 A_0, MemoryStream A_1, ArrayList A_2, ArrayList A_3, ExcelSaveFlags A_4)

       at as.a(Int16 A_0, MemoryStream A_1, ExcelSaveFlags A_2)

       at as.a(Stream A_0, ExcelSaveFlags A_1)

       at FarPoint.Excel.ExcelFileHandler.a(String A_0, ExcelSaveFlags A_1)

       at FarPoint.Excel.ExcelFileHandler.Save(String fileName, ExcelSaveFlags saveFlags)

       at ci.a(String A_0, ExcelSaveFlags A_1)

       at c9.a(String A_0, Stream A_1, ExcelSaveFlags A_2, ExcelWarningList A_3)

    ===========  end of inner exception  ===========

       at c9.a(String A_0, Stream A_1, ExcelSaveFlags A_2, ExcelWarningList A_3)

       at FarPoint.Win.Spread.FpSpread.SaveExcel(String fileName, ExcelSaveFlags saveFlags, ExcelWarningList warningList)

       at FarPoint.Win.Spread.FpSpread.SaveExcel(String fileName)

       at ALM.BL.Excel.ExportSpreadToExcel(IEnumerable`1 sheetViews, String path, Boolean adjustColumnWidth, Boolean openAfterExport)

       at ILIAS.UI.EVData.Plausibility.PlausibilityExport.ExportPlausibility(String path)

       at ILIAS.UI.EVData.Plausibility.CollectionTables.UcCollectionPivot.PivotExport(String path)

       at ILIAS.UI.EVData.Plausibility.CollectionTables.UcCollectionPivot.BtnExport_Click(Object sender, EventArgs e)

       at System.Windows.Forms.Control.OnClick(EventArgs e)

       at System.Windows.Forms.Button.OnClick(EventArgs e)

       at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)

       at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)

       at System.Windows.Forms.Control.WndProc(Message& m)

       at System.Windows.Forms.ButtonBase.WndProc(Message& m)

       at System.Windows.Forms.Button.WndProc(Message& m)

       at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)

       at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)

       at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

    Kind regards,

    Danny Meier

  • Posted 8 September 2017, 5:24 am EST

    Dear Support,

    I wanted to test if this issue still exists in release 5. But I have tried release some month ago and my trial period has expired.

    Is there any chance to get a serial key to extend my trial period for few days further?

    Kind regards,

    Danny Meier

  • Posted 8 September 2017, 5:24 am EST

    Danny,

    These are the people to talk to about that.

    800-645-5913

    powersales@grapecity.com

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    Just wanted to know if you managed to resolve this issue.

    I have release 5 and also am getting OutOfMemoryException on exports of +- 90K records (15 fields)

    I am using the SaveExcel method and have tried using the method on the spread object itself or with Memory/FileStream objects. Still get the same error.

    Any Luck?

     

    Thanks,

    Shimei

    Dear Support,

    I wanted to test if this issue still exists in release 5. But I have tried release some month ago and my trial period has expired.

    Is there any chance to get a serial key to extend my trial period for few days further?

    Kind regards,

    Danny Meier

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    Can you send us the xml file of the spread data that you are saving in Excel? You should be able to call the Save method to save the spreadsheet to XML to send to us to debug the issue you are seeing.

    Thanks.

  • Posted 8 September 2017, 5:24 am EST

    Hi,

    Thanks for your response.

    I outlined the 2 methods below. As mentioned, they both yield an OutOfMemoryException.

    The sheet is generated in code by copying datatable row values to the new sheet using the SetValue method.

    Attached is the sheet xml output.

     

    Common:

    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bF =

    new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();

    System.IO.MemoryStream mS = new MemoryStream();

    bF.Serialize(mS, sheet);

    mS.Seek(0, System.IO.SeekOrigin.Begin);



    FarPoint.Win.Spread.SheetView newSheet = (FarPoint.Win.Spread.SheetView)bF.Deserialize(mS);

    mS.Close();

    FarPoint.Win.Spread.FpSpread spread = new FarPoint.Win.Spread.FpSpread();

    spread.Sheets.Add(newSheet);

    FarPoint.Excel.ExcelWarningList wl = new FarPoint.Excel.ExcelWarningList();

    Method 1:



    spread.SaveExcel(DialogSave.FileName,

    FarPoint.Excel.ExcelSaveFlags.UseOOXMLFormat | FarPoint.Excel.ExcelSaveFlags.SaveCustomColumnHeaders, wl);

    Method 2:

    System.IO.FileStream fs = new System.IO.FileStream(fs, IO.FileMode.Open, IO.FileAccess.ReadWrite);

    spread.SaveExcel(fs,

    FarPoint.Excel.ExcelSaveFlags.UseOOXMLFormat | FarPoint.Excel.ExcelSaveFlags.SaveCustomColumnHeaders, wl);

    fs.Close();

     

    Thanks,

    Shimei

     

    2011/03/sheet_output.zip
  • Posted 8 September 2017, 5:24 am EST

    Hello,

    I am unable to replicate the issue at my end using the Spread vesion 5.0.3518.2008 wherein the imported xml file is exported using the code given by you without any exception at my end.

    Can you please check the version at your end and let us know the same. Thanks.

     

  • Posted 8 September 2017, 5:24 am EST

    Hi

    I sent a shortened version of the sheet (only 1000 records) since I couldn't send a zipped version of the full 170K records 370mb file ( I thought that you wanted to check the field definitions, encoding and such and wouldn't need the actual full file).

    I do however notice that the current version of Spread which I am using is 5.0.3514.2008.

    Are there any known issues with this version?

    Thanks,

    Shimei

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    There are no reported issues with the same. However, can you once try upgrading your application to the latest build and check it once with the complete 90K records you have in your spread and then export it to Excel. Please share your observations on the same

    Here is the link to get the latest build-

     ftp://ftp.fpoint.com/SpreadWinForms5/spwin5_net35-3518.zip
    

    Thanks.

  • Posted 8 September 2017, 5:24 am EST

    I downloaded and installed the new 5.0.3518.2008 version but I'm still getting an out of memory exception.

     

    Is there anything else I can try?

    Can you please provide an email address so that I may try to send you the RARed xml file ?

     

    Please note that this is becoming an urgent issue for our client.

     

    Below is the inner exception stack trace output:

     

     

    at System.IO.MemoryStream.set_Capacity(Int32 value)

    at System.IO.MemoryStream.EnsureCapacity(Int32 value)

    at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)

    at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)

    at System.IO.StreamWriter.Write(String value)

    at System.Xml.XmlTextEncoder.Write(String text)

    at System.Xml.XmlTextWriter.WriteString(String text)

    at System.Xml.XmlWriter.WriteAttributeString(String localName, String ns, String value)

    at System.Xml.Serialization.XmlSerializationWriter.WriteAttribute(String localName, String ns, String value)

    at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriterCT_Worksheet.a(String A_0, String A_1, CT_Row A_2, Boolean A_3, Boolean A_4)

    at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriterCT_Worksheet.a(String A_0, String A_1, CT_Worksheet A_2, Boolean A_3, Boolean A_4)

    at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriterCT_Worksheet.Write129_worksheet(Object o)

    at Microsoft.Xml.Serialization.GeneratedAssembly.CT_WorksheetSerializer.Serialize(Object objectToSerialize, XmlSerializationWriter writer)

    at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o, XmlSerializerNamespaces namespaces, String encodingStyle, String id)

    at System.Xml.Serialization.XmlSerializer.Serialize(Stream stream, Object o, XmlSerializerNamespaces namespaces)

    at System.Xml.Serialization.XmlSerializer.Serialize(Stream stream, Object o)

    at et.a(XmlSerializer A_0, Stream A_1, Object A_2)

    at et.a(Stream A_0, Object A_1)

    at FarPoint.Excel.ExcelXMLHelpers.CreateStreamFromObject(Object obj, Type type)

    at fy.a(IExcelWrite A_0, fe A_1, a A_2, CT_Workbook A_3, Int16 A_4, String A_5, CT_Stylesheet A_6, Int32 A_7, ExcelSaveFlags A_8)

    at fy.a(IExcelWrite A_0, fe A_1, ExcelSaveFlags A_2, a A_3)

    at fg.a(IExcelWrite A_0, String A_1, Stream A_2, ExcelSaveFlags A_3)

    at FarPoint.Excel.ExcelFileHandler.b(String A_0, ExcelSaveFlags A_1)

    at FarPoint.Excel.ExcelFileHandler.Save(String fileName, ExcelSaveFlags saveFlags, String password)

    at ci.a(String A_0, ExcelSaveFlags A_1, String A_2)

    at c9.a(String A_0, Stream A_1, ExcelSaveFlags A_2, ExcelWarningList A_3, String A_4)

     

    Thanks,

     

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    Your issue is important to us. I would request you to please send us the complete XML file at powersupport@grapecity.com with forum post id in subject. I would try to replicate the issue at my end and would assist you further thereon.

    Thanks.

  • Posted 8 September 2017, 5:24 am EST

    Has anything further been found on this topic?

    One of my customers has reported the same Out of Memory exception when saving their large data set to Excel.

    They have no problem saving it to Excel using an old version of our app (which uses Spread [COM] 7) but they get the Out Of Memory exception if they try to use the latest version of the app (which uses Spread [.Net] 4.0.2011.2005)

    Needless to say they are not happy that the version they just upgraded to can not do something that the old version has no problem with.

    In their case we have 35,000 rows and 207 columns.

    They only sent me the first 5000 rows of their data so i can not duplicate it myself.

    (Even as a tab delimited file this is over 5MB so the full file would be 35MB ... probably 50MB as XML and over 100MB as Excel)

     My stack trace is different - referencing MemorySteam rather than the trace above:

    System.ApplicationException: Error saving Excel file ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.

       at System.Collections.ArrayList.set_Capacity(Int32 value)

       at System.Collections.ArrayList.EnsureCapacity(Int32 min)

       at System.Collections.ArrayList.Add(Object value)

       at ci.a(Int16 A_0, ArrayList A_1, ArrayList A_2, ArrayList A_3, ArrayList A_4)

       at as.a(Int16 A_0, MemoryStream A_1, ArrayList A_2, ArrayList A_3, ExcelSaveFlags A_4)

       at as.a(Int16 A_0, MemoryStream A_1, ExcelSaveFlags A_2)

       at as.a(Stream A_0, ExcelSaveFlags A_1)

       at FarPoint.Excel.ExcelFileHandler.a(String A_0, ExcelSaveFlags A_1)

       at FarPoint.Excel.ExcelFileHandler.Save(String fileName, ExcelSaveFlags saveFlags)

       at ci.a(String A_0, ExcelSaveFlags A_1)

       at c9.a(String A_0, Stream A_1, ExcelSaveFlags A_2, ExcelWarningList A_3)

       --- End of inner exception stack trace ---

       at c9.a(String A_0, Stream A_1, ExcelSaveFlags A_2, ExcelWarningList A_3)

       at FarPoint.Win.Spread.FpSpread.SaveExcel(String fileName, ExcelSaveFlags saveFlags, ExcelWarningList warningList)

       at FarPoint.Win.Spread.FpSpread.SaveExcel(String fileName, ExcelSaveFlags saveFlags)

       at Teradata.SQLA.Answer.SaveAsExcel(ExcelSaveFlags flags) in F:\ttu1310_efix_snap\tdcli\qman\sqla\Answer.vb:line 80

       at Teradata.SQLA.SpreadEx.SaveGrid() in F:\ttu1310_efix_snap\tdcli\qman\sqla\SpreadEx.vb:line 413

    Files of this kind of size can not generally be emailed.

    Since you have a sample of the test data for the original post could you not simply duplicate that 'n' times to create a data set that is the size of his original data set. Then hopefully you can replicate the problem.

    I saw another post that indicated a similar problem was caused by setting each cell in the spreadsheet to Locked.

    If you are using a memory stream rather than building the entire image in memory I don't see why that would matter (you only deal with one cell, or maybe row, at a time when writing to a stream). In my case all attributes are set at either the sheet or column level so we should not be hitting anything like that anyway.

    Mike  

  • Posted 8 September 2017, 5:24 am EST

    Hi Mike,

    At the time I was in touch with Grapecity and was informed that they had reported a bug in Spread in exporting large Execl files. I was told that it will be addressed in some coming version..

    I have since bypassed the problem by exporting my data collection (datatable in my case) as a csv file and then use Excel automation (Office.Interop.Excel.dll) to import the file into a spreadsheet (formatting the Excel sheet can be a real pain since you need to use the somewhat obscure InvokeMember method for every cell property changes you wish to make).

    This solution, I find works the fastest and has not (yet) failed even when exporting large amounts (>300K rows X 20 cols).

    Hope this helps.

    Shimei

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    we have exactly the same problem, is there any solution now? We use Version 5.0.3522.2008.

    With this small program you can see the trouble. We fill a Spread with numbers and with a button click, it should be exported to excel. With a process explorer we can see, that the used memory grows rapidly and when about 2 GB is in use, than there comes the exception (there is still 2 GB left for free!!!). We have a 2 Kernel Processor and we use XP and Windows 7 and all PCs have 4 GB memory, on every PC we have the same problem. The customers don't like the solution to export the data to csv file. 

     Private Sub Form1_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

            Dim ii As Integer = 0

            Dim i As Integer = 0

            Dim NumberCellType1 As FarPoint.Win.Spread.CellType.NumberCellType = New FarPoint.Win.Spread.CellType.NumberCellType



            FpSpread1_Sheet1.RowCount = 50000

            FpSpread1_Sheet1.ColumnCount = 100



            For i = 0 To 99

                FpSpread1_Sheet1.ColumnHeader.Cells(0, i).Text = Str(i)

                FpSpread1_Sheet1.Columns(0).Width = 90

                FpSpread1_Sheet1.Columns(0).CellType = NumberCellType1

            Next



            For ii = 0 To 49999

                For i = 0 To 99

                    FpSpread1_Sheet1.Cells(ii, i).Value = ii

                Next

            Next

        End Sub



        Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

            Dim ret As Boolean

            

            ret = FpSpread1.SaveExcel("D:\temp\testexcel.xls")

                  

        End Sub 

     

     

     

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    I tested the issue with the latest version of Spread i.e. Spread.Net 6 v6.0.xx05 and I was not able to replicate the issue at my end. I would request you to test the issue with the latest version of Spread.Net 6 i.e. v 6.0.x05 and share your observations further. You can download the latest version of Spread.Net 6, v 6.0.xx05 from the following link:

                       http://www.gcpowertools.com/DownloadLatestVersion

    Hope it will help you. Please let me know if you have any queries further.

    Thanks,

    Manpreet Kaur

  • Posted 8 September 2017, 5:24 am EST

    Hello Manpreet Kaur,

    thanks for the quick answer. We have made a quick test with the latest version. It is a little bit better, but the used memory is very high and when we raise the Rowcount of about 160000, we have the same error. At start we have 2,8 GB free, after filling the spread there are 2,5 GB left and when we start the export to excel, we became after a minute the "out of Memory" exception. Our Processor Explorer shows that 1,4 GB is free and the program itself uses 1,5 GB. Do we anything wrong?

    Here is our Code, which we used for testing

     Private Sub Button1_Click_1(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

            Dim ret As Boolean





            ret = FpSpread1.SaveExcel("D:\temp\testexcel.xls", FarPoint.Excel.ExcelSaveFlags.UseOOXMLFormat)



         

        End Sub



        Private Sub Form1_Load1(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

            Dim ii As Integer

            Dim i As Integer

            Dim NumberCellType1 As FarPoint.Win.Spread.CellType.NumberCellType = New FarPoint.Win.Spread.CellType.NumberCellType



            FpSpread1_Sheet1.RowCount = 160000

            FpSpread1_Sheet1.ColumnCount = 100



            For i = 0 To 99

                FpSpread1_Sheet1.ColumnHeader.Cells(0, i).Text = Str(i)

                FpSpread1_Sheet1.Columns(0).Width = 90

                FpSpread1_Sheet1.Columns(0).CellType = NumberCellType1

            Next



            For ii = 0 To 159999

                For i = 0 To 99

                    FpSpread1_Sheet1.Cells(ii, i).Value = ii

                Next

            Next

        End Sub 

     

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    Yes, you are right the issue is replicated if there are more number of rows, it seems to be a bug. I have reported the issue to the development team and the bug number for the same is #99926237. It would be fixed in the next maintenance release.

    Thanks,

    Manpreet Kaur

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    can everyone tell me, how long we have to wait for a fix. Our workaround (export to CSV File) isn't really accepted by our costumers.

     thanks 

     

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    I would like you to know that the next maintenance release of Spread is expected to be released by the end of this year. I would let you know as soon as I get any information further in this regard.

    Thanks,

    Manpreet Kaur

  • Posted 8 September 2017, 5:24 am EST

    Thank you for the quick answer :-). “Next Release by the end of the year” is a  great news for us!

  • Posted 8 September 2017, 5:24 am EST

    Manpreet Kaur:

    Hello,

    I would like you to know that the next maintenance release of Spread is expected to be released by the end of this year. I would let you know as soon as I get any information further in this regard.

    Thanks,

    Manpreet Kaur

     So there is new version of Spread.NET 6.0.2006 and 6.0.3506 and 6.0.4006, but #99926237 is not part of the release notes. Any progress?

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    I would like you to know that the bug is kept on hold and the development team is still working on the same as they would need some time to research and find an optimized solution for the same.  I would let you know as soon as I get any information further in this regard.

    Thanks,

    Manpreet Kaur

  • Posted 8 September 2017, 5:24 am EST

    Hello Manpreet Kaur,

    are there new informations about the bug 99926237? We are still waiting for the solution.

    Thanks

     

     

  • Posted 8 September 2017, 5:24 am EST

    Hello,

    I have escalated the issue to the concerned team. I would let you know as soon as I get any information in this regard.

    Thanks,

    Manpreet Kaur
  • Posted 8 September 2017, 5:24 am EST

    Hello,

    This would be fixed in the future builds of Spread but unfortunately there is no an ETA for this issue currently. I apologize for the inconvenience caused.

    Thanks,

    Manpreet Kaur
  • Posted 25 October 2018, 1:57 am EST

    We have been using Spread for nearly 10 years and this particular memory issue remains our only critical problem. We have a Job Server that we use to save client data to an Excel file in a batch-type fashion. This allows the client to submit a job to save an Excel file from the front-end and continue working in our app.

    For some of our larger clients, the resulting file has roughly 100,000 rows, 80 columns, and a file size around 50 megabytes. Unfortunately, we can never complete a job of this size because the memory requirements far exceed the allotted memory on our Job Server machine. The following command in our code is the culprit:

    fpSpread.SaveExcel(exportFilePath + “.xlsx”, ExcelSaveFlags.SaveCustomColumnHeaders | ExcelSaveFlags.UseOOXMLFormat | ExcelSaveFlags.UseCustomPaletteForColorApproximations | ExcelSaveFlags.UseDefaultColorPalette);

    Prior to running this command, we consume a little over 1 gigabytes binding the dataset to the fpSpread control. Once the command above executes, memory usage gradually increases from 1 gigabyte to well over 6 gigabytes—at which point our Job Monitor terminates the process as we can only allot 6 gigabytes per job.

    I have discussed increasing the memory threshold and upgrading the Job Server machine as well but I still need to reduce the memory consumption for this process, if possible. We have some clients who occasionally need to build spreadsheets twice as large as this and I don’t see any way we can accommodate them if we do not find a way to make this process more efficient.

    We are currently using version 11.40.20178.0 of the Farpoint.Web.Spread control. I have looked in the available documentation as well as the release notes but I am not seeing anything in regard to this particular error being resolved by the latter versions of Farpoint.

    Do you have any recommendations as to how we can reduce our memory map on this particular kind of export? Do any of the options above consume more memory than others? I am just trying to find a way to use the Spread control to perform this export in a more efficient manner. Any direction you can provide would be greatly appreciated.

Need extra support?

Upgrade your support plan and get personal unlimited phone support with our customer engagement team

Learn More

Forum Channels