A programming language is a tool. Ideally, you should pick the most suitable tool for the task at hand. In practice, there are a lot of other factors, including usability, technical complexity, availability of tools and libraries, learning curve, educational value; then there's approval & support (e.g. in the work environment), and popularity & future (for future employment possibilities), and so-on.
All these factors change in importance with time as your experience and goals change. Before deciding which language to concentrate on, you need to assess your current position, where you want to get to in the short term, and where you'd like to be in the longer term.
For example, are you just starting out? is this going to be a hobby or a career? is your interest in gaming, or web apps, or low-level OS stuff, or finance, or research, or graphics or what?
C++ is technically complex, hugely flexible, and interfaces well with most operating systems; it lets you code 'down to the metal', but with that power you need care and respect. Java is technically simpler, less linguistically flexible, higher-level (abstracted from the hardware and OS), but has a multitude of libraries and frameworks for GUI development and especially web-based development. It's a more forgiving environment, but more restricted.
As a (very) broad-brush generalisation, operating systems, servers, and low-level tools and utilities are more likely to be written in C++, and the applications that run on them are more likely to be written in Java. Both languages are showing their age, and in the business world, the Microsoft .NET language equivalents, and other scripting languages, are serious alternatives for higher-level stuff. In general, the more languages you become familiar with, the more options you'll have.
I think it helps to have an understanding of the low-level basis of programming and programming language design & structure, so a familiarity with C++ and other languages is very useful, but if I were starting out today, I'd be looking primarily at Java and web development (scripting languages and web frameworks) with C++, etc., as an educational adjunct.