The brackets represent a code body. An opening bracket must have a closing bracket of course. So the first set would be the "body" of the class MyFirstJavaProgram. The second being the body of the main method.
The public modifier is used so that other classes accessing this class would be able to use this particular method. If you desire to learn more about this, do a search on Inheritance.
Now the static modifier (I seem to always even confuse myself on this one so correct me if I'm wrong) essentially tells the program that there is only one instance of whatever it is modifying. This is why you can't have non-static variables in a static body.
The void is the return type. This can be anything, but for this particular method, it returns void; meaning it doesn't return anything.
The following bit is the method's name and parameters. Any program which you want to run must have a main method, which takes an array of Strings as the argument.
Hopefully this helps. I'll be glad to try and clarify any of this if possible.