Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 5 of 5

Thread: My program keeps lagging when dealing with large arrays.

  1. #1
    Junior Member
    Join Date
    Oct 2011
    Posts
    5
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question My program keeps lagging when dealing with large arrays.

    Hi,

    I'm not posting specific code for this problem, because the problem keeps happening with lots of variations I've thrown at it, and even completely different programs I've written!

    The general problem is that I'm trying to make an array of all prime numbers under 10000000. (Why? Blame projecteuler.net .) I had to make it in parts, save it to a file, and run the program again with the next set of numbers to test for primality dozens of times, because after about 300,000 numbers it would simply freeze up. The first 100,000 or so would go by at ridiculous speed. There was nothing wrong with the code, it all worked, I just had to only give it 200,000 numbers at a time, then start it again! I thought maybe it was just that primes really are that much trouble, but after I finally got the array built I wrote a program to simply read the array and print it all to the screen one number at a time. The same thing happened! After about 300,000 (though it would sometimes get as far as 500,000) the program would simply freeze up and refuse to move again! The array's all there - I can ask it to start at a later point, and it will read about 300,000 from there - it just keeps freezing. I should mention that it starts slowing down and jumping before freezing up completely.

    To be completely clear, I was having the program read the array with a for loop. Something like:
    for (int i = 0; i < primes.length; i++){
       System.out.println("" + primes[i]);
    }
    So, my question: Could the problem be that my array's simply too big? To big even to read?? It that a problem with java, or is my computer just getting slow? Any ideas on how I can fix this? It starts running extremely quickly, then it seems to just start tripping over it's own feet.
    Last edited by helloworld922; October 21st, 2011 at 09:16 AM.


  2. #2
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,267
    Thanks
    176
    Thanked 821 Times in 764 Posts
    Blog Entries
    5

    Default Re: My program keeps lagging when dealing with large arrays.

    Quote Originally Posted by chrynelson View Post
    So, my question: Could the problem be that my array's simply too big? To big even to read??
    No

    It that a problem with java, or is my computer just getting slow? Any ideas on how I can fix this? It starts running extremely quickly, then it seems to just start tripping over it's own feet.
    You don't post enough code to know what could be the cause.

  3. #3
    Super Moderator helloworld922's Avatar
    Join Date
    Jun 2009
    Posts
    2,896
    Thanks
    23
    Thanked 619 Times in 561 Posts
    Blog Entries
    18

    Default Re: My program keeps lagging when dealing with large arrays.

    The problem sounds like you're using a poor algorithm. A 10,000,000 int array takes ~38MB of memory, easily handled by most modern systems (even fairly old systems can handle 38MB).

    From your description it sounds like you're using the naive algorithm for checking for primality. If you need to generate all the primes less than a certain number I would suggest using the Sieve of Eratosthenes instead.

    Which problem on Project Euler are you trying to solve? There's a chance that maybe there's a different way to approach this problem all-together.
    Last edited by helloworld922; October 21st, 2011 at 09:24 AM.

  4. #4
    Junior Member
    Join Date
    Oct 2011
    Posts
    5
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: My program keeps lagging when dealing with large arrays.

    Well, I'm pleased to hear that it's not just that my computer is slow!

    The thing is, I got the same problems in a variety of different programs. I finally did get my array of all primes under 10000000 made, and then wrote a simple program to read them out again:

    import tools.SaveLoad;
     
    public class Tester{
      public static void main(String[] args){
        int primes[] = SaveLoad.loadIntArray("primes10000000");  //This just gets my saved array.
                                                                 //The data is saved as an array, not a bunch of prime numbers.
                                                                 //That is to say, if you open primes10000000 in notepad, you see a bunch of nonsense, not prime numbers.
     
        for (int i = 0; i < primes.length; i++){  //This loop just reads out the numbers in the array one at a time.
          System.out.println("" + primes[i]);
        }
     
        System.out.println("Done!");
      }
    }

    And the SaveLoad:

    package tools;
     
    import java.io.*;
     
    public class SaveLoad{
      public static void saveIntArray(String filename, int[] output_veld) {
        try {
          FileOutputStream fos = new FileOutputStream(filename);
          ObjectOutputStream out = new ObjectOutputStream(fos);
          out.writeObject(output_veld);
          out.flush();
          out.close();
        }
        catch (IOException e) {
          System.out.println(e);
        }
      }
     
      public static int[] loadIntArray(String filename) {
        try {
          FileInputStream fis = new FileInputStream(filename);
          ObjectInputStream in = new ObjectInputStream(fis);
          int[] gelezen_veld = (int[])in.readObject();
          in.close();
          return gelezen_veld;
        }
        catch (Exception e) {
          System.out.println(e);
        }
        return null;
      }
    }

    I'm a little worried the problem might be in that part, because I didn't write that part myself, I just got it off another forum. I'm a little hazy on i/o with files in Java.

    In any case, I get the same slowing and jumping and freezing with a program like this, which does nothing but read the array out, as I do with the program to find the primes in the first place. It starts out going fast, then stutters and stops!

    So, is there a better way to save an array, perhaps?


    P.S. I was working on the Project Euler problems 69 and 70, which actually deal with totient numbers. I wanted to make a list of all totients numbers under 10000000 that I can save and reuse later, and thought I'd start with prime numbers, just to get the saving a reusing part figured out, and since so many Project Euler problems needs primes.

  5. #5
    Junior Member
    Join Date
    Oct 2011
    Posts
    5
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: My program keeps lagging when dealing with large arrays.

    As a side note: Wow! Thanks for that Sieve of Eratosthenes link! I never thought of approaching the problem by removing non-primes!

Similar Threads

  1. diplay a large text in JTextArea (>2GB)
    By NeoDesWay in forum AWT / Java Swing
    Replies: 4
    Last Post: August 18th, 2013, 07:20 AM
  2. [SOLVED] Writing a program with arrays and class methods... PLEASE HELP?
    By syang in forum What's Wrong With My Code?
    Replies: 17
    Last Post: August 8th, 2011, 07:02 AM
  3. Digital Watch program and Sorting arrays
    By c.P.u1 in forum What's Wrong With My Code?
    Replies: 9
    Last Post: February 5th, 2011, 04:21 PM
  4. [SOLVED] Using Arrays to make a scoring program
    By woodcutterni in forum What's Wrong With My Code?
    Replies: 13
    Last Post: January 7th, 2011, 06:02 PM
  5. exception while Read very large file > 300 MB
    By ps.ganesh in forum File I/O & Other I/O Streams
    Replies: 2
    Last Post: June 11th, 2009, 11:39 PM

Tags for this Thread