You sometimes hear the term "Java Core API" or "Java SE", SE = Standard Edition, to describe the libraries or packages that come included with Java's SDK. If you look at the main API page
(the link happens to be to Java 7), you can scroll through all of the included packages in the upper left portion of the page, select one (like javax.swing for example), and then see the classes included in that package in the bottom left portion of the page, including JFrame (which you listed incorrectly - remember that Java is VERY case sensitive). If you scroll through the whole package list, you'll see they all start with either java, javax, or org.
I doubt you will have to download any 3rd party libraries to do the work in the book you have, although it would be a good lesson to teach you someday.