First, let's review what values and references are.
A value is the actual data of bits and bytes which contains useful information (such as the amount of money you have in your bank account). These values are passed as copies, so any changes made to one copy don't effect the other copies. So, for example take a bank statement. The bank had to print out a copy with how much money I have and how much I owe them. I can make any changes to this bank statement (say I want to add a few zeros to the end of how much I have), but at the end of the day these changes don't affect how much I actually have.
A reference is analogous to an address. The values which are useful to us have to reside somewhere in memory, and the reference tells us where in the memory this object is. An example is a house address. If someone gets a hold of my address and decides that my door should be red, they could paint it and it would physically affect what the color of my door is.
So how is this used in Java?
In Java, all primitive values are modified by value. All objects are modified by reference. The reasons why objects are passed by reference is because some objects are actually quite large, and making a copy of these objects would take a lot longer than passing a simple 4-byte reference (or 8-byte if you have a 64-bit OS), and it may even be impossible if the object is large enough.
Now let's examine some of the consequences of these design choices:
1. When you pass a primitive, you can't change the original variable's value. This means that you must either re-design to pass an object so the value can be modified, or use a return value and then re-assign the value of the primitive.
2. When you pass an object, you can change the original value. This means that the method could potentially make changes to the object that you don't want. The obvious fix to this is to manually create a copy of the object you have and pass that instead so your original object doesn't get messed up (you'll need to make a deep copy in order to ensure that everything about the original object remains the same).
Some things to be careful about when using references: