You should not need to do a clean after refreshing from git or changing libraries, that kind of stuff is make’s day job.
If it does not do the right thing, then that is a bug and needs to be fixed by changes to the makefile(s).
I can see maybe using clean after you switch branches within git, but only to ensure that the first build after switching is fresh.