Search:

In your first code, you have invoked connectSocket() as first thing but accept, beeing blocking, is the problem.
In the second updated code, yes you have created a new thread but it's not very...

But the accept() is "blocking" ...

Sorry, but you have not yet understood the concept.

<start of your application>
Does the "lock" file exist?

Yes ----> Quit immediately (eventually, tell the user that there is another instance...

But ... did you understand what is a "shutdown hook"? It's a piece of code (the code in run() of Runnable instance encapsulated in Thread, passed to addShutdownHook) that gets executed only when the...

Two things:
1) your shutdown hook creates the file. But conceptually, the application should create the file at start and then, at shutdown, remove the file.
2) (this is most important) your code...

You don't need to "accept" incoming connections! You just need to verify if the constructor of ServerSocket fails (with exception, that in the particular case is java.net.BindException) or not.