It is recommended that you wash your face mask after each wear. This means that even if you remove your mask for only moment, you should not put the same mask on and instead use a clean one.
Once you take off your mask, you should wash it straight away. Alternatively, if you are not home, you should put your mask in a zip lock bag immediately after taking it off and keep it sealed until you are able to wash it.
Check with your local Department of Health for more information as recommendations regarding this may differ depending on the country you are from and the severity of COVID-19 in your area.