You need to set the focus to the JPanel,
Make it focusable first and then request the focus.

You may also want to make it so the player stops moving when the keys are released. Unless your...