I am not sure if I understand you but it sounds like you want the app to act either as a server or a client, but not both.
I think you can put these two in separate threads. While Socket.accept gets...