In case anyone else is looking there is an introduction to How to Use HTML in Swing Components in Oracle's Tutorial.

Bear in mind that JLabel != web browser: everything has its limits! Marked up...