Posted 29 January 2018, 4:59 pm EST
I have approx 95 Data integration interfaces that are extracting and loading data from an SAP ECC environment. Each dataset “Lands” in Teradata in a single table and is subsequently processed into a more structured and Keyed table for downstreaam consumption. This takes care of dupes and delta changes, etc.
My problem is the initial Landing table which gets truncated each time a new load comes in. If a Landed dataset fails to process forward it will be truncated and a new dataset will replace that in Landing as the next load comes in. I did not design or develop any of this, but it causes a lot of grief as well as remedy (bug) tickets.
I want to propose a solution and wanted to know if it is possible to not truncate the initial Landing table but instead create a new Partition for each Landed dataset as a new load came in. In that fashion we would avoid missing the dataset altogether. Is this possible in Teradata?
I wanted to add that we are using Informatica to connect [url=https://tekslate.com/sap-bw-training/]SAP BW to Teradata as that has some bearing on the degree of difficulty we are facing. Not a huge Fan of Informatica as it is used in this case.