You could combine it with BoxLayout and utilize horizontal and vertical glues for the alignment. In your case, you could use this layout inside the GridLayout, perhaps create a new type of JPanel

...