hi, i was wondering if someone could give me an efficient way of checking and removing duplicates.

basically i have this giant file which parses for IDs and Groups Num.

The condition is that I...