Assuming a total loss at Dunkirk, which I think is unlikely, there's going to be a long-term hit to the British army that shouldn't be underestimated in terms of the loss of professional NCOs and officers. New recruits can be trained. New soldiers with 10 years' experience, not so much, by definition.
That said, the British army on the continent was not large. There should be a hint in this for how we look at Sea Lion. I wonder why the British didn't bother to invest in a large army. It's almost like they didn't consider their army their main line of defence...
The bigger question is what might happen politically in the event of the total loss. Maybe if that happens and France falls rapidly, you set up a political crisis in Britain that leads to a new government that opens peace talks with Germany. Before I get shouted down, I don't think that's the most likely outcome, but at least there's a plausible alternative pathway there. There isn't by September in our timeline.
And the Germans can't invade "immediately after the fall of France" for the simple reason that they don't have the sealift capacity organized. Hence the need for improvisation, hence Sea Lion.
If you rewind back and have the Germans spend years developing a credible amphibious force -- and we've gone down this road before -- then the British will respond by increasing their naval assets in the Channel, increasing their fixed fortifications in England, and standing up and keeping at home a much larger army for defensive purposes. Then, the loss of the expeditionary force in France still won't help the Germans.