Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 14 of 14

Thread: Faster Run Time - Scanner(file) and Arrays

  1. #1
    Junior Member
    Join Date
    Mar 2011
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Faster Run Time - Scanner(file) and Arrays

    I have a working code right now, it does what I want it to do, which is loading text from a txt file and loading it into an arrayList, line by line. Three of them, to be exact. This is remake of the Pokemon games, Nintendo's ideas, not mine. I just wanted to write it in Java to test myself.

    The code runs fine but it takes about three minutes to compile. It isn't my computer, as far as I can tell. Is there a faster way to write my code?

    for (int count = 0; count < (pokemonsTypeCheck.size()); count++){
    	File typeList1 = new File("C:\\Users\\Jacob\\Desktop\\types1.txt");
    	File typeList2 = new File("C:\\Users\\Jacob\\Desktop\\types2.txt");
    	final Scanner types1 = new Scanner(typeList1);
    	final Scanner types2 = new Scanner(typeList2);
    	String currentLine;
    	int spareCounter=0;
    	while (types1.hasNext()){
    		typesList1.ensureCapacity(spareCounter);				
    		currentLine = types1.nextLine();
    		typesList1.add(spareCounter, currentLine);
    		spareCounter++;
    	}
    	spareCounter=0;
    	while (types2.hasNext()){
    		typesList2.ensureCapacity(spareCounter);
    		currentLine = types2.nextLine();
    		typesList2.add(spareCounter, currentLine);
    		spareCounter++;
    	}
    }
    System.out.print(pokemonsTypeCheck.get(pokemonNumber)+" is a "+typesList1.get(pokemonNumber));
    if (typesList2.get(pokemonNumber).equals(" ")){
    	System.out.println(" type.");
    }
    else{
    	System.out.println("/"+typesList2.get(pokemonNumber)+" type.");
    }


  2. #2
    Member vanDarg's Avatar
    Join Date
    Jan 2011
    Location
    Chicago
    Posts
    65
    My Mood
    Mellow
    Thanks
    1
    Thanked 7 Times in 7 Posts

    Default Re: Faster Run Time - Scanner(file) and Arrays

    If this the only code you have, and it takes three minutes to compile, then there is likely something wrong with your computer or some configuration. On the other hand, if it is taking three minutes to run it is a different story. If this is the case, look into the BufferedReader class that java provides, this should speed up the loading of the data.
    "Everything should be made as simple as possible, but not simpler."
    Asking Questions for Dummies | The Java Tutorials | Java Coding Styling Guide

  3. #3
    Junior Member
    Join Date
    Mar 2011
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Faster Run Time - Scanner(file) and Arrays

    Sorry, yeah, the run time is what's killing me. I don't see how BufferedReader putting the data in the array would help speed up the process. It would still have the same looping and runtime as Scanner(file) putting the data into an array, wouldn't it?

  4. #4
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,320
    Thanks
    181
    Thanked 833 Times in 772 Posts
    Blog Entries
    5

    Default Re: Faster Run Time - Scanner(file) and Arrays

    Quote Originally Posted by Dogeatdog6 View Post
    Sorry, yeah, the run time is what's killing me. I don't see how BufferedReader putting the data in the array would help speed up the process. It would still have the same looping and runtime as Scanner(file) putting the data into an array, wouldn't it?
    Reading files requires calls to the underlying system which are thought to be relatively expensive. Using a BufferedReader or reading in a buffered way reduces the number of these calls as it does so in larger chunks than the typical call to a next* method. Not fully sure how Scanner works underneath the hood, but I don't think it has the buffering capacity - you could pass a BufferedReader to Scanner should you wish to still use a Scanner. How big are these files? Three minutes to read two files sees excessive regardless of how you read them (unless you are working on a slow computer).

  5. #5
    Junior Member
    Join Date
    Mar 2011
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Faster Run Time - Scanner(file) and Arrays

    My problem still hasn't been helped. :/ I tried using a BufferedReader, and a FileReader to gain access to the txt file. The run time is still about the same length. Is there any other possibility that would help me out?

    I have the files loaded up this way:
    public static void main(String[]args){
    	Battler battle = new Battler();
    	try {
    		battle.loadPokemon("pokemons.txt", "types1.txt", "types2.txt");
    	} catch (NoSuchElementException e) {
    		e.printStackTrace();
    	} catch (FileNotFoundException e){
    		e.printStackTrace();
    	}
    	battle.runner();
    }
    ...
    public void loadPokemon(String pokemonList, String typeList1, String typeList2) throws NoSuchElementException, FileNotFoundException{
    for (int count = 0; count < (pokemonsTypeCheck.size()); count++){
    	//File typeList1 = new File("types1.txt");
    	//File typeList2 = new File("types2.txt");
    	//final Scanner types1 = new Scanner(typeList1);
    	//final Scanner types2 = new Scanner(typeList2);
    	BufferedReader types1 = new BufferedReader(new FileReader(typeList1));
    	BufferedReader types2 = new BufferedReader(new FileReader(typeList2));
    	int spareCounter=0;
    	try {
    		while ((typesCheck1=types1.readLine()) !=null){
    			typesList1.ensureCapacity(spareCounter);				
    			typesList1.add(spareCounter, typesCheck1);
    			spareCounter++;
    		}
    	} catch (IOException e1) {
    		// TODO Auto-generated catch block
    		e1.printStackTrace();
    	}
    	spareCounter=0;
    	try {
    		while ((typesCheck2=types2.readLine()) !=null){
    			typesList1.ensureCapacity(spareCounter);				
    			typesList1.add(spareCounter, typesCheck2);
    			spareCounter++;
    		}
    	} catch (IOException e) {
    		// TODO Auto-generated catch block
    		e.printStackTrace();
    	}
    }
     
    System.out.print(pokemonsTypeCheck.get(pokemonNumber)+" is a "+typesList1.get(pokemonNumber));
    if (typesList2.get(pokemonNumber).equals(" ")){
    	System.out.println(" type.");
    }
    else{
    	System.out.println("/"+typesList2.get(pokemonNumber)+" type.");
    }

    Any help would be appreciated. Right now, the run time is about three to five minutes. I suppose I could always go in and hard code the ArrayLists, but that seems extremely problematic, as all three of the ArrayLists are 600+ strings long.

  6. #6
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,320
    Thanks
    181
    Thanked 833 Times in 772 Posts
    Blog Entries
    5

    Default Re: Faster Run Time - Scanner(file) and Arrays

    Again, how large are the files? The bottleneck could simply be the file size, that being said 3-5 minutes is quite a long time to read files several hundred MB's in size, unless your computer is excruciatingly slow. I personally don't see any red flags in the code above that would cause this. Are you sure this code portion is the problem? Have you flanked this code with some println's to print out the times before and after to really profile the code?

  7. #7
    Junior Member
    Join Date
    Mar 2011
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Faster Run Time - Scanner(file) and Arrays

    It's just three text files. 3kb, 5kb, and 6kb. Each have 646 lines with one(or just a space) word on each line.

    The idea was using the first text file, comparing that to the user entered string to find a match in the ArrayList, then using the index of the matching string to compare to the other two ArrayLists to get the two types of the Pokemon that the user entered.

    i.e. User enters: Blastoise
    Output: You have chosen Blastoise!
    Blastoise is a water type pokemon!

    The type comes from the second and third ArrayLists, because some pokemon have two types.

  8. #8
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,320
    Thanks
    181
    Thanked 833 Times in 772 Posts
    Blog Entries
    5

    Default Re: Faster Run Time - Scanner(file) and Arrays

    Did you profile the code? Add some println's in there to time how long it takes for each step in your process (you can use System.currentTimeMillis(), subtracting a start value from the current value to get the duration). Until you've done this, you will not know where the bottleneck lies (in this case I doubt its the file reading)

  9. #9
    Member vanDarg's Avatar
    Join Date
    Jan 2011
    Location
    Chicago
    Posts
    65
    My Mood
    Mellow
    Thanks
    1
    Thanked 7 Times in 7 Posts

    Default Re: Faster Run Time - Scanner(file) and Arrays

    I can't tell by your code, but I'm not sure if or how you are storing the data. You can input the data into arrayLists, and from there search through those arrayLists. I'm not exactly sure if this will fix your problem, but searching through an arrayList seems to be easier than searching through a text file.
    "Everything should be made as simple as possible, but not simpler."
    Asking Questions for Dummies | The Java Tutorials | Java Coding Styling Guide

  10. #10
    Super Moderator helloworld922's Avatar
    Join Date
    Jun 2009
    Posts
    2,896
    Thanks
    23
    Thanked 619 Times in 561 Posts
    Blog Entries
    18

    Default Re: Faster Run Time - Scanner(file) and Arrays

    while ((typesCheck2=types2.readLine()) !=null){
    			typesList1.ensureCapacity(spareCounter);
    			//...
    This is rather worrying (it's in a few other places, too). You shouldn't manually ensure the capacity each time through the loop (especially if you're only ensuring the capacity to be 1 larger).

    If you have an idea of what the ball-park size should be, call ensureCapacity once with this value. Otherwise, let the ArrayList automatically expand the capacity as it sees fit (or use a combination of the two).

  11. #11
    Junior Member
    Join Date
    Mar 2011
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Faster Run Time - Scanner(file) and Arrays

    Quote Originally Posted by helloworld922 View Post
    This is rather worrying (it's in a few other places, too). You shouldn't manually ensure the capacity each time through the loop (especially if you're only ensuring the capacity to be 1 larger).

    If you have an idea of what the ball-park size should be, call ensureCapacity once with this value. Otherwise, let the ArrayList automatically expand the capacity as it sees fit (or use a combination of the two).
    I took out the EnsureCapacities and put them at the end of the looping setup. So now it adds them automatically as it sees fit, and then the number of Strings in each array is hardcoded in, because I know the exact number that exists. I got an error, rangeCheck or something.

    Output[the initial "Blastoise" is user entered]:
    Name of Player 1's Pokemon: Blastoise
    You have chosen Blastoise!
    Blastoise is a Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 8, Size: 0
    at java.util.ArrayList.RangeCheck(Unknown Source)
    at java.util.ArrayList.get(Unknown Source)
    at Battler.loadPokemon(Battler.java:157)
    at Battler.main(Battler.java:448)

  12. #12
    Super Moderator helloworld922's Avatar
    Join Date
    Jun 2009
    Posts
    2,896
    Thanks
    23
    Thanked 619 Times in 561 Posts
    Blog Entries
    18

    Default Re: Faster Run Time - Scanner(file) and Arrays

    You need to pay attention to the documentation. Calling ensureCapacity should come before the loops. It's used to ensure that the underlying array is at least that capacity, so when you add elements there's no need to re-allocate a new array until at least you fill that capacity. If you call it after you've added everything, it's kind of useless.

    I don't know exactly what's causing the ArrayIndexOutOfBoundsException, but looking more deeply at your reading code just doesn't seem right to me.

    What is pokemonsTypeCheck? Also, why are you looping through pokemonsTypeCheck.size() and read the typeList in two times each time through the loop? You should read the files into memory and leave it there, then if you need to do anything with pokemonsTypeCheck, use a separate loop. Did your code before work? If so, what else did you change? Removing ensureCapacity calls should never cause ArrayIndexOutOfBoundsExceptions.

  13. The Following User Says Thank You to helloworld922 For This Useful Post:

    Dogeatdog6 (April 7th, 2011)

  14. #13
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,320
    Thanks
    181
    Thanked 833 Times in 772 Posts
    Blog Entries
    5

    Default Re: Faster Run Time - Scanner(file) and Arrays

    I will again say: time the code - print out how long it takes for each section of code and you will know where the bottleneck lies. helloworld's points are spot on, but things like calling ensureCapacity should not cause major delays you are seeing (the code for add indirectly calls the same that ensureCapacity does anyway). Things like looping several times to read a file may be the cause, but until you time the code you will not know, and until you do you may spend hours blaming it on the file reading and debugging code that doesn't need to be debugged (or on the alternative profile the code and spend hours debugging code that needs to be debugged ). My .02
    Last edited by copeg; April 7th, 2011 at 09:14 AM.

  15. The Following User Says Thank You to copeg For This Useful Post:

    Dogeatdog6 (April 7th, 2011)

  16. #14
    Junior Member
    Join Date
    Mar 2011
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Faster Run Time - Scanner(file) and Arrays

    I got it to work. I moved around the EnsureCapacity and messed with it some more. I figured I could do it all in one loop, as far as the last two ArrayLists went, using the size of the first arrayList to ensure the capacity of the others. :3 Thanks for all the help, guys. Sorry for being a bother ;P

    typesList1.ensureCapacity(pokemonsTypeCheck.size());
    		typesList2.ensureCapacity(pokemonsTypeCheck.size());
    		for (int count = 0; count < (pokemonsTypeCheck.size()); count++){
    			//File typeList1 = new File("types1.txt"); //Buffered Reader    		//Fix
    			//File typeList2 = new File("types2.txt");
    			//final Scanner types1 = new Scanner(typeList1);
    			//final Scanner types2 = new Scanner(typeList2);
    			BufferedReader types1 = new BufferedReader(new FileReader(typeList1));
    			BufferedReader types2 = new BufferedReader(new FileReader(typeList2));
     
    			//String currentLine;
    			int spareCounter=0;
    			spareCounter=0;
     
    			while (hnnng){
    				try {
    					typesCheck1 = types1.readLine();
    					typesCheck2 = types2.readLine();
    				} catch (IOException e) {
    					// TODO Auto-generated catch block
    					e.printStackTrace();
    				}
    				typesList1.add(spareCounter, typesCheck1);
    				typesList2.add(spareCounter, typesCheck2);
    				spareCounter++;
    				if ((typesList1.size())==(pokemonsTypeCheck.size())){
    					hnnng=false;
    				}
    			}
    		}
     
     
    		System.out.print(pokemonsTypeCheck.get(pokemonNumber)+" is a "+typesList1.get(pokemonNumber));
    		if (typesList2.get(pokemonNumber).equals(" ")){
    			System.out.println(" type.");
    		}
    		else{
    			System.out.println("/"+typesList2.get(pokemonNumber)+" type.");
    		}

Similar Threads

  1. Basic Java File I/O with Scanner Class Problem
    By miss confused in forum What's Wrong With My Code?
    Replies: 1
    Last Post: July 26th, 2010, 08:04 AM
  2. Read file until EOF from offset each time
    By Pr0ject-Rec0n in forum File I/O & Other I/O Streams
    Replies: 4
    Last Post: April 2nd, 2010, 09:13 PM
  3. Storing data from a socket to a file in real time
    By colossusdub in forum Java Networking
    Replies: 0
    Last Post: March 2nd, 2010, 09:10 AM
  4. using Scanner for 75mb file
    By zort in forum Java SE APIs
    Replies: 5
    Last Post: November 9th, 2009, 04:20 AM
  5. printing output to console & to a text file at the same time...
    By prasanna in forum File I/O & Other I/O Streams
    Replies: 3
    Last Post: August 26th, 2009, 03:43 AM

Tags for this Thread