We have recently upgraded our SCM system from SCM7.02 to SCM7.13. We have also migrated from Windows to Linux. Since the upgrade / migration we have had issues with reading in flat files from our interfaces directory. We have many BW process chains and Z programs that read in flat files.
We have found that on numerous occasions, data goes missing while the files are being read in. The files are being read into a Linux based SCM system from Windows share. This is really easy to see when it happens in one of our BW process chains. When I look at the PSA, I can see that one of the rows from the flat file stopped populating to the correct PSA fields in the middle of the record. Also, the first couple of values of what should have been the next record are populated in the record that is missing data. Enough of the next record is populating in the errored record to fill all of the fields in that row of the PSA. For example, if each record of the file was made up of fields A;B;C;D;E;F;G;H the data in the PSA would look like the rows below; however, the data in the flat file is correct:
A1;B1;C1;D1;E1;F1;G1;H1
A2;B2;C2;D2;E2;F2;G2;H2
A3;B3;C3;D3;E3;F3;G3;H3
A4;B4;C4;D4;A5;B5;C5;D5;
A6;B6;C6;D6;E6;F6;G6;H6
When we have these issues, it is usually the first time the flat file is read in. It is also usually the first time the system has connected to the interfaces directory in some time. 100% of the time, rerunning the job or process chain from the top corrects the issue. The source flat file never changes in between runs. We are out of ideas on what to look at? Anyone out there have a similar issue or have any ideas what would case the data to go missing midstream. Right now we think it most likely has something to do with a Linux system picking up a flat file from a windows directory. We had turned off DFS for connecting to the windows share and the issue went away for about 2 weeks. Not it is back.
Any help is greatly appreciated. A screen shot of the issue is below.
