Can't you just add the KeyListener to the JFrame? Or use KeyBindings instead?