Importing large data

Posted by: chander.pal on 14 January 2024, 9:00 pm EST

    • Post Options:
    • Link

    Posted 14 January 2024, 9:00 pm EST

    Hi ,

    I am trying to import a excel sheet with many formulas , and approximately 5000 rows . While doing so the chrome shows me unresponsive dialog. But when i open console in chrome then it works fine.

    I tried to import on https://developer.mescius.com/spreadjs/demos/features/spreadjs-file-format/overview/purejs the same excel it works there.

    Any advice that will be helpful.

  • Posted 15 January 2024, 7:58 pm EST

    Hi,

    This demo, https://developer.mescius.com/spreadjs/demos/features/spreadjs-file-format/overview/purejs, always uses the latest version, which is currently 17.0.1. Since your Excel file imported without any issues with this latest version, I suggest you upgrade to the latest version for performance advantages if your application is currently using a lower version. Additionally, I have attached the latest version of the designer for you to verify with designer too.

    Designer hosted link: https://developer.mescius.com/spreadjs/designer/index.html

    Designer download file: http://cdn.mescius.com/spreadjs/17.0.1/Files/SpreadJS.Release.17.0.1.zip

    Also, I attempted to replicate the issue but was unsuccessful. To better assist you, could you please share the Excel file and specify the version of SpreadJS you are using in your application? This additional information will help me investigate the problem more thoroughly. Additionally, if possible, providing a GIF or video illustrating the issue would be beneficial.

    If your Excel file contains any sensitive or private data, I recommend submitting a support ticket on our private portal by visiting: https://developer.mescius.com/my-account/my-support. This will allow you to securely share the necessary information with the support team to address the issue.

    Best regards,

    Ankit

  • Posted 17 January 2024, 5:49 pm EST

    Hi,

    Sharing you the Excel . And we are using spreadjs 16.2.

    Can you please share me the incremental/Lazy load code (Open Mode) that is mentioned in this example

    https://developer.mescius.com/spreadjs/demos/features/spreadjs-file-format/overview/purejs

    If anything found please let us know that will be helpful.EXCEL.zip

  • Posted 18 January 2024, 9:16 pm EST

    Hi,

    Thank you for providing a excel file.

    I attempted to replicate the issue with the available information but was unsuccessful; the Excel file loaded without any problems on my end. Please refer to the attached video and sample.

    Could you verify if the issue arises with the provided sample on your end? If the problem persists, I recommend upgrading to the latest version, as the issue does not occur in that version on your end. Alternatively, please provide the following details to help identify the root cause:

    1. A sample along with the steps to replicate the issue, or modify the existing sample to demonstrate the problem. Additionally, it would be helpful if you could provide a GIF or video illustrating the issue.

    2. Provide your system configuration and environment details. I tested with the following system configuration:

    • Browser: Chrome Version 120.0.6099.225

    • Operating System: Windows 10 Pro (Version: 10.0.19045)

    • SpreadJS: Version 16.2.6

    Sample: https://jscodemine.grapecity.com/share/mc-7cCSovkavsv-oMOmTIA/?IsEmbed=false&Theme=Unset&PreviewDirection=0&IsEditorShow=true&IsExplorerShow=true&IsPreviewShow=true&IsConsoleShow=true&IsRunBTNShow=false&IsResetBTNShow=false&IsOpenInCodemineBTNShow=false&PanelWidth=20&PanelWidth=50&PanelWidth=30&defaultOpen={"OpenedFileName"%3A["%2Findex.html"%2C"%2Fsrc%2Fapp.js"]%2C"ActiveFile"%3A"%2Fsrc%2Fapp.js"}

    And for the incremental/Lazy (Open Mode), while loading sjs with open method “spread.open( sjsFile, successCallback, errorCallback, openOptions)” there is openOptions object where you can set open mode. Refer snippet and sample below.

    spread.open( file, function(){}, function(){}, {openMode: GC.Spread.Sheets.OpenMode.incremental});

    References:

    open: https://developer.mescius.com/spreadjs/api/classes/GC.Spread.Sheets.Workbook#open

    openOptions: https://developer.mescius.com/spreadjs/api/modules/GC.Spread.Sheets#openoptions

    Best regards,

    Regards,

    Ankit

    Sample:

    openMode.zip

    Video: excelLoaded__1_.zip

  • Posted 6 February 2024, 6:32 pm EST - Updated 6 February 2024, 6:37 pm EST

    Hi,

    we have updated to latest version 17 .i am trying to use the above code snippet not able to use open mode showing me error.

    spread.open(file, (json) => {

    activeSheet = json.sheets[activeSheetName];

    //resolve(activeSheet);

    }, (error) => {

    resolve(“Not Found”);

    },{openMode: GC.Spread.Sheets.OpenMode.incremental});

    let me know how can we achieve this.

  • Posted 7 February 2024, 8:22 pm EST - Updated 7 February 2024, 8:50 pm EST

    Hi,

    I attempted to reproduce the issue but was unsuccessful. On my end, I was able to utilize the openMode options correctly. Below are snippets and samples demonstrating this. I have provided two samples: one in vanilla JavaScript and another in TypeScript.

    To run TypeScript sample, use commands “npm i” → “npm run build” → “npm run serve”.

    spread.open(file, () => {
          console.log("successfully loaded");
        }, (e: any) => {
          console.log(e);
        }, {openMode:GC.Spread.Sheets.OpenMode.incremental})

    Please refer to below some suggestions:

    1. Make sure to add the package “@grapecity/spread-sheets-io” to your application
    2. If you are using typescript, you might need to use type “any”.

    Refer to the below snippet.

    (spread as any).open(file, () => {
          console.log("successfully loaded");
        }, (e: any) => {
          console.log(e);
        }, {openMode:GC.Spread.Sheets.OpenMode.incremental})

    And in the provided code snippet, you have passed json in success callback but success callback do not receive json.

    It is important to note that spread.open() and spread.save() methods are used to open and save the SJS (.sjs) files. These can not be used for importing/export excel(.xlsx) files. To import/export .xlsx files, kindly use spread.import() and spread.export() methods as discussed in our previous reply. Please refer to the demo at https://developer.mescius.com/spreadjs/demos/features/spreadjs-file-format/overview/purejs and below sample.

    sample: https://jscodemine.grapecity.com/share/W3Vr_ei0Ek61r1g9ltkhUw/?IsEmbed=false&Theme=Unset&PreviewDirection=0&IsEditorShow=true&IsExplorerShow=true&IsPreviewShow=true&IsConsoleShow=true&IsRunBTNShow=false&IsResetBTNShow=false&IsOpenInCodemineBTNShow=false&PanelWidth=20&PanelWidth=50&PanelWidth=30&defaultOpen={"OpenedFileName"%3A["%2Findex.html"%2C"%2Fsrc%2Fapp.js"]%2C"ActiveFile"%3A"%2Fsrc%2Fapp.js"}

    If you encounter any issues despite the above suggestions, it would be helpful if you could provide a sample along with the steps necessary to replicate the behavior. Alternatively, modifying an existing sample to reproduce the issue would also be helpful. This additional information will allow for a more thorough investigation of the problem. Additionally, providing a GIF or video illustrating the issue would further aid in diagnosing the problem.

    Regards

    Sample.zip

  • Posted 13 February 2024, 10:54 pm EST - Updated 14 February 2024, 3:36 pm EST

    Hi,

    While analyzing our angular code, I found out that when I imported Excel, after that, we performed some operations on the data. While placing the data again on the active sheet, the Chrome page went unresponsive and took too much time.

    Scenerio:

    We have a sheet already opened, so we import the Excel file that we have. After some time (approximately 40–50 seconds), that unresponsive dialog comes. It has 5000 rows, and we are looping. We are replacing cells whose values are different or don’t exist in the cell, so we have to go in every row and every cell.

    Findings: we have used the following code

    let inputCell = this.activeSheet.getCell(row, mapping.InputColumnIndex);

    inputCell.value(“”); // it is causing the issue if i comment this then it is very fast.

    I shared my piece of code and excel . Any help will be good code.zip

  • Posted 13 February 2024, 10:55 pm EST - Updated 14 February 2024, 4:23 pm EST

    Actually, what we want to achieve is that when we import the document, we have to check each row and its cell values with the existing open sheet and the current importing sheet, which will overwrite the existing one. If any value needs to be changed according to the condition, we have to update the sheet in the respective cell.

    Video attached how the message is coming and taking too much time.video.zip

  • Posted 15 February 2024, 12:14 am EST

    Hi,

    We are still investigating the issue at our end. We will let you know about our findings as soon as possible.

    Regards

  • Posted 15 February 2024, 10:18 pm EST

    Hi,

    I was able to replicate the issue in my end with provided information.

    Whenever a modification is made to the worksheet ( in your case by “getCell(row, col).value()”), SpreadJS undergoes an automatic refresh, triggering the CellChanged event and recalculating to accommodate the changes. However, this auto-calculation can negatively affect performance. I suggested adopting best practices, similar to those implemented in the provided sample, which resulted in improved performance.

    Refer below snippet.

    private Import(importType: any): Promise<any> {
        return new Promise((resolve) => {
    
          let endRow: number;
    
          let beginRow = 5;
    
          endRow = 5206;
    
          this.spreadService.spread.suspendPaint();
          this.spreadService.spread.suspendEvent();
          this.spreadService.spread.suspendCalcService(true);
    
          this.activeSheet = this.spreadService.spread.getActiveSheet();
          for (let currentRow = beginRow; currentRow <= endRow; currentRow++) {
    
            this.HandleDuplicateCombinations(currentRow,endRow);
          }
    
          this.spreadService.spread.resumeCalcService(false);
          this.spreadService.spread.resumeEvent();
          this.spreadService.spread.resumePaint();
    
          resolve({beginRow });
        });
      }

    In sample, try import excel file and then click “Import data” button to see results.

    Sample: https://jscodemine.grapecity.com/share/rs_uHl6Ec0akbysYkoztMQ/?IsEmbed=false&Theme=Unset&PreviewDirection=0&IsEditorShow=true&IsExplorerShow=true&IsPreviewShow=true&IsConsoleShow=true&IsRunBTNShow=false&IsResetBTNShow=false&IsOpenInCodemineBTNShow=false&PanelWidth=20&PanelWidth=50&PanelWidth=30&defaultOpen={"OpenedFileName"%3A["%2Findex.html"%2C"%2Fsrc%2Fapp.component.ts"]%2C"ActiveFile"%3A"%2Fsrc%2Fapp.component.ts"}

    References:

    Best practice: https://developer.mescius.com/spreadjs/docs/BestPractices

    Best regards,

    Priyam

  • Posted 19 February 2024, 8:42 pm EST - Updated 19 February 2024, 11:01 pm EST

    Hi,

    I tried this code and it works fine , thank you so much .

  • Posted 21 February 2024, 3:39 am EST - Updated 21 February 2024, 2:27 pm EST

    Hi ,

    There is one situation that after importing the data if we update the data as we you also done in the example the cells becomes dirty. Now when i try to SAVE we have to validate cells value too. but in case of large number of cells/rows it gives same unresponsive message because of the large number of data . We use same way to get value of cells as previous getCell(row, col).value() and sheet.getValue() .

    What we do is we get all dirty cells and validate them if in numeric field string is entered we show validation.

    Any advice or suggestion please

  • Posted 22 February 2024, 12:10 am EST

    Hi,

    We are still investigating the issue at our end. We will let you know about our findings as soon as possible.

    Regards

  • Posted 22 February 2024, 5:37 pm EST

    Hi,

    Instead of looping all the cells, just check for the range that contains the data. You could use the getUsedRange() method to get the used range (pass the usedRangeType as “data”).

    Refer to the following API on getUsedRange: https://developer.mescius.com/spreadjs/api/v15/classes/GC.Spread.Sheets.Worksheet#getusedrange

    Also, refer to the following for UsedRangeType Enumeration: https://developer.mescius.com/spreadjs/api/v15/enums/GC.Spread.Sheets.UsedRangeType

    Also, for getting the values of the cells, use the “getArray()” method to get the values of the range. Use the range that you get using the “usedRangeType()” method.

    getArray() method: https://developer.mescius.com/spreadjs/api/v15/classes/GC.Spread.Sheets.Worksheet#getarray

    If you still face the performance issue with the above suggestion and the suggestions earlier, kindly do share us a working sample replicating the issue so that we could also investigate the issue at our end and could assist you accordingly.

    You may also fork the above sample and share it with us. Also, kindly share your implementation how exactly you are validating the numeric fields also in your sample.

    Regards,

    Ankit

  • Posted 25 February 2024, 1:10 pm EST

    Hi,

    I have shared the code that we are you using while doing save . We are doing validations before save. showing the same error of unresponsive. afterSavedirtyValidationsCode.zip

  • Posted 26 February 2024, 10:58 pm EST

    Hi,

    I attempted to create a sample to replicate the issue using the provided code and information, but I was unsuccessful. The information provided was insufficient to create a sample. Refer below sample.

    Sample: https://jscodemine.grapecity.com/share/cfqJgWOK1UKIrbottxKX9g/?defaultOpen={"OpenedFileName"%3A["%2Findex.html"%2C"%2Fsrc%2Fapp.component.ts"%2C"%2Fsrc%2Fapp.main.ts"%2C"%2Fsrc%2Fapp.module.ts"%2C"%2Fsrc%2Fapp.component.html"%2C"%2Fpackage.json"]%2C"ActiveFile"%3A"%2Fsrc%2Fapp.component.ts"}

    To better assist you, could you share a sample along with the steps to replicate the behavior you have observed, or modify the existing sample to replicate the behavior? This will enable me to investigate the problem more thoroughly. And It would be helpful if you could provide a GIF or video illustrating the issue.

    Regards,

    Priyam

  • Posted 28 February 2024, 11:38 pm EST

    Hi,

    As i was trying to replicate the scenerio found that the dirtycells are empty if i change the value too. sheet.getdirtycell() sending you two files please have a look. I am getting 38000 cells dirty in actual code.files.zip

  • Posted 29 February 2024, 8:43 pm EST

    Hi,

    I tried to reproduce the issue by creating a sample based on the provided files and information, but I was unable to do so. The information provided was insufficient as there are many functions and variables that were not defined, and the use case of those variables and functions is not clear. In the attached sample, I have added comments “// not defined” to all those functions and variables that were not defined.

    Sample: https://jscodemine.mescius.io/share/dvbyQvfUZkSYTmy8-RupJQ/?defaultOpen={"OpenedFileName"%3A["%2Findex.html"%2C"%2Fsrc%2Fapp.component.ts"]%2C"ActiveFile"%3A"%2Fsrc%2Fapp.component.ts"}

    Providing a working sample along with steps to replicate the issue would greatly help in debugging the root cause. Just examining the code may not reveal the underlying problem. Without a sample, it would be challenging to identify the issue accurately. Could you share a sample along with the steps to replicate behavior or You may fork the provided sample and make modifications to replicate the behavior, then share it with us. This will enable us to investigate the problem more thoroughly. Additionally, providing a GIF or video illustrating the issue would be helpful for better understanding.

    Regards,

    Priyam

Need extra support?

Upgrade your support plan and get personal unlimited phone support with our customer engagement team

Learn More

Forum Channels