This sounds like a logic problem. Hard to make any suggestions without seeing the code.
Does the code that is to remove the entries from the HashMap get called when you want it to be called?