Work with error messages - git

Sometimes applications give us very strange error messages and have to pretty much break your head to understand exactly what the problem is and how to fix it.

We decided that such cases are worthy of a series of small articles under the general title “Working with Error Messages”.

And today we have a gift from everyone's favorite git program.
Once again, having made git pull to the test server, the client saw the following message:

fatal: write failure on 'stdout': No space left on device

And he asked us to find out why he could not write on stdout

Disk space and free inode was enough. Cloning the same repository to the same server in a different directory took place without problems.
')
Googling for this phrase did not lead to anything.

The fact that cloning takes place aggressively drives us to the conclusion that a local copy of the test server has broken. We started searching for tools to fix the git repository, but then we noticed a strange thing:

df -h Filesystem Size Used Avail Use% Mounted on /dev/simfs 535G 364G 144G 72% / none 2.9G 2.9G 0 100% /dev none 583M 1.1M 582M 1% /run none 5.0M 0 5.0M 0% /run/lock none 1.2G 0 1.2G 0% /run/shm none 100M 0 100M 0% /run/user 

The / dev section takes up 100% of the space. All 2.9G is weird.
Go to / dev and see the file / dev / null the size of just this 2.9 gigabyte.

Apparently, someone once deleted the device / dev / null, and the services continued to write to / dev / null, creating a regular file and slowly filling the space.

They deleted the file, created the device, restarted the services that wrote to / dev / null - the space on / dev was freed up and git pull worked without problems and errors.