The File That Exists Twice

During my Code the Dream practicum, our team ran into a Git problem that kept coming back.

Merges kept conflicting over two test files. One was learnPage.test.jsx. The other was LearnPage.test.jsx.

Same name. One capital letter.

We deleted the files and reset the repository. The conflict came back the next time we merged code. And the time after that.

We called it the endless bug squashing convention.

In the end it was never truly resolved. I honestly dislike that we let it happen, but what could I even do? I meant to bring it up at our team demo and never got the chance.

So this is the demo I did not give.

In this article, we are going to rebuild that problem on Windows and take it apart. By the end, you will be able to see the extra name, remove it without losing the file, rename a file without making a new one, and run a check that prints nothing when every name is unique:

git ls-files | sort -f | uniq -di

One file in the folder, two names in Git#

Windows treats Cat.txt and cat.txt as one file. Git treats them as two.

The file systems behind that (NTFS on Windows, APFS on macOS) remember the capital letters in a name, but by default they ignore them when they look a file up. To them, learnPage.test.jsx and LearnPage.test.jsx are one file. Linux is case-sensitive: there, they are two.

Git does not go by the folder. It keeps its own list of tracked names, called the index, and in that list a capital letter makes a different name.

Your folder (Windows):  learnPage.test.jsx
Git's list:             LearnPage.test.jsx
                        learnPage.test.jsx

So a repository can track two names while your folder can hold only one. We will call the extra name the twin.

Someone on Linux can commit both names without noticing anything. Later we will see how Windows can do it too. We do not need to find out who did it. We will make the repository that way on purpose.

mkdir twin-lab && cd twin-lab
git init --bare -b main origin.git
git clone origin.git alice
cd alice
echo one > learnPage.test.jsx
git add . && git commit -m "add test"
blob=$(echo two | git hash-object -w --stdin)
git update-index --add --cacheinfo 100644,$blob,LearnPage.test.jsx
git commit -m "add the capital-T twin"
git push origin main
cd ..
git clone origin.git bob
cd bob

What Git shows you#

Cloning is where Git first complains. This is the end of the git clone origin.git bob output:

warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:

  'LearnPage.test.jsx'
  'learnPage.test.jsx'

Inside bob, the folder holds one file:

$ ls
learnPage.test.jsx

Now ask Git how it feels about that.

$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   LearnPage.test.jsx

no changes added to commit (use "git add" and/or "git commit -a")

Nobody edited that file. Git is comparing the capital-T entry with the only file in the folder, and they do not match. git diff shows how:

$ git diff
diff --git a/LearnPage.test.jsx b/LearnPage.test.jsx
index f719efd..5626abf 100644
--- a/LearnPage.test.jsx
+++ b/LearnPage.test.jsx
@@ -1 +1 @@
-two
+one

The capital-T name holds two. The file in the folder holds one, the lowercase name's text. Keep that in mind; it matters when we pick a name to keep.

git ls-files lists every name Git tracks:

$ git ls-files
LearnPage.test.jsx
learnPage.test.jsx

And core.ignorecase is the setting that tells Git the file system ignores case:

$ git config core.ignorecase
true

Git set it to true when it noticed this file system ignores case. With it on, Git matches files in the folder to names in its list without caring about capital letters. That helps day to day, but it is not a guard. It does not stop two such names from being tracked. In our rebuild, both are.

Why deleting and resetting does not help#

Both names point at the same file in the folder, so Git can only ever make one of them look right.

$ git restore LearnPage.test.jsx
$ git status --short
 M learnPage.test.jsx

The capital-T name is fixed. The lowercase one now looks modified.

git reset --hard does the same thing from the other side. It prints HEAD is now at ..., and git status --short still shows one of the two names as modified.

And when a teammate pushes a change, git pull can refuse to run:

$ git pull
error: Your local changes to the following files would be overwritten by merge:
	LearnPage.test.jsx
Please commit your changes or stash them before you merge.
Updating c831c25..95e73f7
Aborting

Git thinks you have edits the pull would overwrite. You do not. Whether it refuses depends on which name Git currently thinks is modified, which is part of why this bug feels random. We will see it happen to alice in a moment.

I cannot tell you our team's repository looked exactly like this one. We saw it as merges that kept conflicting. But it explains why our deleting and resetting never held.

The problem is not in the folder. It is in the names that the commits track.

Remove one name, not the file#

We have to take one name out of Git's list and leave the file alone.

First, decide what to keep. We keep the lowercase name, and the folder already holds its text, one, as git diff showed. If the text you need is in the name you are removing, copy it into the folder first with git show HEAD:LearnPage.test.jsx > learnPage.test.jsx, and git add learnPage.test.jsx before you commit.

Plain git rm is the wrong tool:

$ git rm LearnPage.test.jsx
error: the following file has local modifications:
    LearnPage.test.jsx
(use --cached to keep the file, or -f to force removal)

Git refuses, and with -f it would delete the one file in your folder, which is also the other name. Use --cached instead, so only the index changes:

$ git rm --cached LearnPage.test.jsx
rm 'LearnPage.test.jsx'
$ git commit -m "Keep one name"
[main 95e73f7] Keep one name
 1 file changed, 1 deletion(-)
 delete mode 100644 LearnPage.test.jsx
$ git status --short
$ git ls-files
learnPage.test.jsx

The status prints nothing. For the first time since the clone, the folder and Git agree. If you see M learnPage.test.jsx instead, the folder holds the twin's text: run git restore learnPage.test.jsx to get the kept name's text back, or git add and commit it if that is the text you want.

Push it as usual with git push origin main.

Rename without making a new twin#

Say the team decides component files are PascalCase, with a capital first letter on each word, like LearnPage. So header.jsx becomes Header.jsx. Back in bob, make a file to rename and commit it:

echo hello > header.jsx
git add header.jsx
git commit -m "Add header"

Plain mv renames the file in your folder, and Git notices nothing:

$ mv header.jsx Header.jsx
$ git status --short
$ git ls-files
header.jsx
learnPage.test.jsx

The status is empty because core.ignorecase lets Git match Header.jsx in the folder to header.jsx in its list. The rename never reaches the repository, and a teammate on Linux still gets header.jsx.

Leave the file as it is and use git mv, so Git records the rename. Then commit it:

$ git mv header.jsx Header.jsx
$ git status --short
R  header.jsx -> Header.jsx
$ git commit -m "Rename header to Header"
[main da0304e] Rename header to Header
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename header.jsx => Header.jsx (100%)

That worked in one step here. If your Git answers fatal: destination exists, it thinks the new name is already taken. There are two ways out:

git mv -f header.jsx Header.jsx

or two moves through a name that differs by more than case:

git mv header.jsx header-tmp.jsx
git mv header-tmp.jsx Header.jsx

Both end the same way: R header.jsx -> Header.jsx.

How Windows makes a twin#

A common tip for a rename that "does not work" is git config core.ignorecase false. Do not take it. With the setting off, Git sees a new file and a missing one, and if you add only the new name, the commit keeps both.

Try it in alice, which we can treat as scratch from here on:

$ echo bye > footer.jsx
$ git add footer.jsx
$ git commit -m "Add footer"
$ git config core.ignorecase false
$ mv footer.jsx Footer.jsx
$ git status --short
 D footer.jsx
?? Footer.jsx
$ git add Footer.jsx
$ git commit -m "Rename footer"
$ git config core.ignorecase true
$ git ls-files
Footer.jsx
footer.jsx
learnPage.test.jsx

That is a twin, made on Windows, in a few ordinary commands. No Linux machine needed. Set core.ignorecase back to true as we did, and the twin stays anyway: it is in the commit now.

Stop it coming back#

First, a naming rule that fits in one sentence. For example, components are PascalCase and everything else is lowercase. Write it in the README. The rule does not prevent anything by itself. It gives a reviewer something to point at.

Second, a check. This lists any tracked names that differ only by case, and prints nothing when there are none. In bob it prints nothing. In alice, with the footer twin, it prints:

$ git ls-files | sort -f | uniq -di
Footer.jsx

sort -f ignores case, so a twin lands right beside its pair. uniq -di prints names that repeat, ignoring case, but only one name from each pair. That is enough to know something is wrong. The hook below uses -D, which prints both names. In PowerShell, the same idea reads:

git ls-files | Group-Object { $_.ToLower() } | Where-Object Count -gt 1 | ForEach-Object { $_.Group }

To make Git run the check for you, create a file named exactly pre-commit, with no extension, inside the hidden .git/hooks folder of the repository. From Git Bash, cat > .git/hooks/pre-commit works: paste the lines below, then press Ctrl+D (twice, if the cursor is not at the start of an empty line). An editor works too.

#!/bin/sh
dupes=$(git ls-files | sort -f | uniq -Di)
if [ -n "$dupes" ]; then
  echo "These names differ only by case:"
  echo "$dupes"
  exit 1
fi

On Linux and macOS the file must also be executable, so run chmod +x .git/hooks/pre-commit. Git Bash on Windows ran it for us without that.

Install it in alice and try a commit. --allow-empty lets you commit with nothing staged, just to see the hook run:

$ git commit --allow-empty -m "test the hook"
These names differ only by case:
Footer.jsx
footer.jsx

The commit stops. Hooks live inside .git, so a clone does not bring them along. Each person installs it, or the same lines can run as a step in your CI.

Try it#

Here is the whole fix in six commands. Type them one at a time.

If you get stuck, the terminal will give you a hint. And in step 4, try the long way through a temporary name.

What we learned#

• Windows and macOS folders treat names that differ only by case as one file. Git's index does not.
• A repository can track both names, and then no state of the folder looks clean.
• git ls-files shows what Git tracks, and git diff shows which text each name holds.
• git rm --cached removes a name from Git and leaves the file.
• git mv, with -f or a temporary name if needed, renames without creating a twin. Turning core.ignorecase off is how you make one.
• git ls-files | sort -f | uniq -di prints nothing when every name is unique.

Our bug was never truly resolved, and I never gave that demo. If I had the five minutes, it would have gone something like this: it is not your computer, and nobody broke the test. Git has two names for one file. Take one out.

This is the version I can give now.