Have you looked at Apache Math? It has classes and solvers for linear systems.
Commons Math: The Apache Commons Mathematics Library
The Commons Math User Guide - Linear Algebra

--- Update ---
...