STARING. It's a problem with a lot of painters, including myself.

When you spend too much time focusing on one area, your eyes get accustomed to that "exposure reading" and you see subtleties ever more nuanced.

When you walk away, your eyes focus on other things. As your eyes move around the real world they make a more generalized "exposure reading" for the whole area, not settling on any one thing for very long.

The solution is to stop focusing on ONLY the figure in your painting. You have to CONSTANTLY be comparing the figure to its surroundings. Don't stare into the shadows . . . shift your eyes back and forth from shadow to light so you are painting the relationship between the two and not just the one at the expense of the other.

Also, it helps to have someone more knowledgeable to point out what that flesh color might really be. I had a good teacher at the art students league of new york who stopped me at the last session of a 60-hour figure painting. I asked what amounted to the same question you have put forth here. His answer was to ask me if he could paint on my model. He just made a few marks, but they were the right color as I looked from my painting to the model, and all the work I had done on my painting was very pale in comparison. When someone shows you the color they see in the distant hills and you believe them, you begin to look for it everywhere and find it more and more. When they show you the color of flesh in light and you believe them, you begin to look for it everywhere and find it more and more.