We are used to imagining things as being objective measures. One inch is one inch in all cases. One mile is one mile. One kilogram is one kilogram.
However, not all units of measure need to be objective, and hit points are an example of a relative measurement. One hit point does not necessarily mean the same thing for one character that it does for another. For character A, 1 hp damage may well be a mortal wound. For Character B, 1 hp damage might be a notable wound. For character C, it might be the equivalent of a paper cut.
And, of course, the first 1 hp damage Gygax's example fighter takes is negligible, while the last 1 hp they take (after all other hit points are gone) is a mortal blow. Hit points are relative even for a given character themselves.
If you stop trying to figure out how much a "hit point" is objectively, and accept that hit points are always a relative measure, this problem goes away.
So what, then, does it mean if you have to heal hit points? A high-level warrior has lost 20 hp, which is relatively insignificant, but requires rest or magic to heal. Why?
Because the warrior isn't at their best. They are not all beaten up, and they are still better than many others at their best, but they could be better. Conan with nicks, cuts, and abrasions is still formidable, but not quite as formidable as Conan without.