Again about leaky abstractions (or about unpredictable environments)

So, quite a simple part of the program under Windows. There is a file containing several entries. And they need to be filtered in a certain way.

The solution is quite simple - open the file, read the records one by one, we write the necessary ones to a temporary file. Close the file. Remove it. Rename temporary to original. Everything is so simple that I will not even give the code. Is this a sufficient reason for the article?

While everything is working, there is no reason to write about it, and the truth is not. But then suddenly one day “everything falls”, because renaming does not occur due to an “Access denied” error. This happens very rarely, but still much more often to suspect cosmic rays.

We start the excavation. The first clue found: in the process of digging, this alarmed quote from Microsoft's documentation was alarmed:

Function for deletion on close. Therefore, it is not true that the file is deleted. ERROR_ACCESS_DENIED.

In terms of symptoms, it’s very similar, but where do these “other handlers” come from, if, apart from us, nobody does and shouldn’t do anything with this file? And we have no other threads with threads that would do something with this file?
')
The culprit was found thanks to SysInternals and their ProcessMonitor. We launch it, install the filter on our long-suffering file and long and aggressively try to reproduce it. Reproducible. We look. And what do we see there?

01. 2: 25: 28.3162097 PM our_prog.exe 1288 CreateFile our.file SUCCESS Desired Access: Generic Read / Write
02. 2: 25: 28.3164513 PM our_prog.exe 1288 WriteFile our.file SUCCESS Offset: 0, Length: 898, Priority: Normal
...
34. 2: 25: 28.3173405 PM our_prog.exe 1288 WriteFile our.file SUCCESS Offset: 35,290, Length: 1,113
35. 2: 25: 28.3173493 PM our_prog.exe 1288 WriteFile our.file SUCCESS Offset: 36,403, Length: 1,128
36. 2: 25: 28.3173736 PM our_prog.exe 1288 FlushBuffersFile our.file SUCCESS
37. 2: 25: 28.3174212 PM our_prog.exe 1288 WriteFile our.file SUCCESS Offset: 0, Length: 40,960,
38. 2: 25: 28.3175927 PM Explorer.EXE 1884 QueryBasicInformationFile our.file SUCCESS
39. 2: 25: 28.3176144 PM Explorer.EXE 1884 CloseFile our.file SUCCESS
40. 2: 25: 28.3263642 PM Explorer.EXE 1884 CreateFile our.file SUCCESS Desired Access: Read Attributes,
41. 2: 25: 28.3294990 PM our_prog.exe 1288 CloseFile our.file SUCCESS
42. 2: 25: 28.3351356 PM our_prog.exe 1288 CreateFile our.file SUCCESS Desired Access: Read Attributes, Delete,
43. 2: 25: 28.3351856 PM our_prog.exe 1288 QueryAttributeTagFile our.file SUCCESS Attributes: A, ReparseTag: 0x0
44. 2: 25: 28.3352020 PM our_prog.exe 1288 SetDispositionInformationFile our.file SUCCESS Delete: True
45. 2: 25: 28.3352218 PM our_prog.exe 1288 CloseFile our.file SUCCESS
46. ​​2: 25: 28.3358275 PM our_prog.exe 1288 CreateFile our.file DELETE PENDING Desired Access: Generic Read / Write,
47. 2: 25: 28.3362207 PM our_prog.exe 1288 CreateFile our.file DELETE PENDING Desired Access: Generic Read / Write,
48. 2: 25: 28.3367696 PM Explorer.EXE 1884 QueryBasicInformationFile our.file SUCCESS
49. 2: 25: 28.4279152 PM Explorer.EXE 1884 CloseFile our.file SUCCESS
50. 2: 25: 28.4282859 PM Explorer.EXE 1884 CreateFile our.file NAME NOT FOUND Desired Access: Read Attributes,
...
83. 2: 25: 29.3497760 PM our_prog.exe 1288 CreateFile our.file SUCCESS Desired Access: Generic Read / Write,

And we see the following there (extra data is deleted so as not to clutter up). Lines 1 to 36 - we create a file, write to it, make flush. The most interesting begins in lines 38-40. Explorer.exe appears in them and starts reading our file.

In line 41 we close our file. In line 42 - delete. And since explorer.exe still reads it, the file is not deleted. What we can see in lines 46 and 47 when we try to rename our temporary file to the main one (DELETE PENDING status instead of SUCCESS).

Explorer.exe finishes reading only in line 49. Only at this moment the file is physically deleted (which is indirectly indicated by line 50, where our persistent explorer.exe again tries to open the file for reading, but it fails because the file no longer).

What follows from this? Thanks to Microsoft Windows, even the simple operation of deleting a file now needs to be done in the style of paranoid programming. They called the delete function, made sure that it returned OK, and entered the wait cycle "until the file has been physically deleted yet." Well, yes, without knowing that “she has a neon inside her”, practically nothing can be done ...

But now I doubt that such an approach is a common practice. So that during any work with files in Windows, to keep in mind that the OS will at any time decide something to do with files without your knowledge, and write code that is resistant to it. In this connection, the survey is lower.