Yes , I would strongly suggest you do you should.
Languages which get compiled and linked to other binary artefacts (such as C++, but even Python and/or Python modules with native parts) will contain all sorts of references to Foxy-specific versions of those artefacts (ie: libraries).
Locations of other resources will also end up in your nodes and other (non-)executable pieces.
I would be surprised if a workspace compiled with Foxy would actually work with Galactic, or vice-versa.
Edit: and to make it even stronger: you are actually required to rebuild your workspace every time you update your ROS installation. Meaning: every time you sudo apt update
and ROS packages get updated, you should rebuild your workspace.
We've discussed this quite a few times here on ROS Answers, but to summarise: there is no ABI compatibility guarantee between (re)builds of ROS packages at the moment (neither ROS 1, nor ROS 2). So after updating ROS packages, there is no guarantee your workspace binary artefacts are compatible with their dependencies any more.
In a way, going from Foxy -> Galactic could be considered "updating your packages", so the same applies in that situation.