My program keeps lagging when dealing with large arrays.
Hi,
I'm not posting specific code for this problem, because the problem keeps happening with lots of variations I've thrown at it, and even completely different programs I've written!
The general problem is that I'm trying to make an array of all prime numbers under 10000000. (Why? Blame projecteuler.net .) I had to make it in parts, save it to a file, and run the program again with the next set of numbers to test for primality dozens of times, because after about 300,000 numbers it would simply freeze up. The first 100,000 or so would go by at ridiculous speed. There was nothing wrong with the code, it all worked, I just had to only give it 200,000 numbers at a time, then start it again! I thought maybe it was just that primes really are that much trouble, but after I finally got the array built I wrote a program to simply read the array and print it all to the screen one number at a time. The same thing happened! After about 300,000 (though it would sometimes get as far as 500,000) the program would simply freeze up and refuse to move again! The array's all there - I can ask it to start at a later point, and it will read about 300,000 from there - it just keeps freezing. I should mention that it starts slowing down and jumping before freezing up completely.
To be completely clear, I was having the program read the array with a for loop. Something like:
Code Java:
for (int i = 0; i < primes.length; i++){
System.out.println("" + primes[i]);
}
So, my question: Could the problem be that my array's simply too big? To big even to read?? It that a problem with java, or is my computer just getting slow? Any ideas on how I can fix this? It starts running extremely quickly, then it seems to just start tripping over it's own feet.
Re: My program keeps lagging when dealing with large arrays.
Quote:
Originally Posted by
chrynelson
So, my question: Could the problem be that my array's simply too big? To big even to read??
No
Quote:
It that a problem with java, or is my computer just getting slow? Any ideas on how I can fix this? It starts running extremely quickly, then it seems to just start tripping over it's own feet.
You don't post enough code to know what could be the cause.
Re: My program keeps lagging when dealing with large arrays.
The problem sounds like you're using a poor algorithm. A 10,000,000 int array takes ~38MB of memory, easily handled by most modern systems (even fairly old systems can handle 38MB).
From your description it sounds like you're using the naive algorithm for checking for primality. If you need to generate all the primes less than a certain number I would suggest using the Sieve of Eratosthenes instead.
Which problem on Project Euler are you trying to solve? There's a chance that maybe there's a different way to approach this problem all-together.
Re: My program keeps lagging when dealing with large arrays.
Well, I'm pleased to hear that it's not just that my computer is slow!
The thing is, I got the same problems in a variety of different programs. I finally did get my array of all primes under 10000000 made, and then wrote a simple program to read them out again:
Code :
import tools.SaveLoad;
public class Tester{
public static void main(String[] args){
int primes[] = SaveLoad.loadIntArray("primes10000000"); //This just gets my saved array.
//The data is saved as an array, not a bunch of prime numbers.
//That is to say, if you open primes10000000 in notepad, you see a bunch of nonsense, not prime numbers.
for (int i = 0; i < primes.length; i++){ //This loop just reads out the numbers in the array one at a time.
System.out.println("" + primes[i]);
}
System.out.println("Done!");
}
}
And the SaveLoad:
Code :
package tools;
import java.io.*;
public class SaveLoad{
public static void saveIntArray(String filename, int[] output_veld) {
try {
FileOutputStream fos = new FileOutputStream(filename);
ObjectOutputStream out = new ObjectOutputStream(fos);
out.writeObject(output_veld);
out.flush();
out.close();
}
catch (IOException e) {
System.out.println(e);
}
}
public static int[] loadIntArray(String filename) {
try {
FileInputStream fis = new FileInputStream(filename);
ObjectInputStream in = new ObjectInputStream(fis);
int[] gelezen_veld = (int[])in.readObject();
in.close();
return gelezen_veld;
}
catch (Exception e) {
System.out.println(e);
}
return null;
}
}
I'm a little worried the problem might be in that part, because I didn't write that part myself, I just got it off another forum. I'm a little hazy on i/o with files in Java.
In any case, I get the same slowing and jumping and freezing with a program like this, which does nothing but read the array out, as I do with the program to find the primes in the first place. It starts out going fast, then stutters and stops!
So, is there a better way to save an array, perhaps?
P.S. I was working on the Project Euler problems 69 and 70, which actually deal with totient numbers. I wanted to make a list of all totients numbers under 10000000 that I can save and reuse later, and thought I'd start with prime numbers, just to get the saving a reusing part figured out, and since so many Project Euler problems needs primes.
Re: My program keeps lagging when dealing with large arrays.
As a side note: Wow! Thanks for that Sieve of Eratosthenes link! I never thought of approaching the problem by removing non-primes!
