Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 10 of 10

Thread: Bizarre Issue with Scanner Failing to "notice" rest of file

  1. #1
    Junior Member
    Join Date
    Feb 2011
    Location
    Adelaide, Australia
    Posts
    9
    My Mood
    Busy
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Bizarre Issue with Scanner Failing to "notice" rest of file

    I'm working on a rather irritating problem, and was hoping someone might have gone through something similar.

    I'm trying to convert a massive amount of microsoft excel data into a format easy to read for Flash AS3, with
    some manipulation required. So I figured I'd write a very simple parser in Java taking in a .csv file, make the
    necessary little tweaks on the way and I'd be done.

    What a fool I was! After almost 4 hours of errors I've run into I'm wondering if it wouldn't be faster to do it manually.
    It's essentially working now, thank heavens but I've hit one massive problem that has got me completely perplexed,
    and wondering if there's something strange going on with my data or I'm not realizing how to use the Scanner properly.

    After about cell 40, the scanner completely fails to recognize any more of the many hundreds of cells after that line. I
    thought it might be because it has a limited buffer or something so I cut down the file size but after that line it's as
    though the file is invisible- the scanner opens fine, no IO exception but the second you call readLine command I get
    a NoSuchElementException, informing me that the file I provided must be completely empty. I do this and open up
    the file with thousands more lines to read and scratch my head in confusion.

    I won't worry with a code sample because it really seems more like a data problem- really it's just opening the file
    and reading it using a scanner, which we all know how to do. Has anyone ever tried parsing large files with a scanner
    and hit this problem, and if so can you suggest a solution?

    Thanks a lot for reading, and I'd be very appreciative of a response. There's no mad rush to solve the problem, I can
    use the 40 lines I have now to demonstrate the core of the program is working, but I will need those other lines
    eventually and I really don't want to manipulate them by hand, especially if there's more coming!

    NitrogenFingers


  2. #2
    Think of me.... Mr.777's Avatar
    Join Date
    Mar 2011
    Location
    Pakistan
    Posts
    1,136
    My Mood
    Grumpy
    Thanks
    20
    Thanked 82 Times in 78 Posts
    Blog Entries
    1

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    Well, why don't you try converting your file date to byteArray and then start parsing it?

  3. #3
    Junior Member
    Join Date
    Nov 2011
    Posts
    5
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    For some help I need to see the line with the problem and how do your scanner work? If it is possible...

  4. #4
    Junior Member
    Join Date
    Feb 2011
    Location
    Adelaide, Australia
    Posts
    9
    My Mood
    Busy
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    For your interest RmSc I've included a SSCCE, though the code works fine for a portion of the file:

     
    Scanner reader;
    String filename = "";
    try {
        filename = args[0] + ".csv";
        reader = new Scanner(new File(filename));
    } catch (FileNotFoundException fnfe) {
         System.err.println("File not found : " + filename);
         System.exit(1);
    } catch (IOException ioe) {
         System.err.println("Cannot write- IO exception occurred");
         System.exit(1);
    }
     
    //Clears the first line, the headers in the excel spreadsheet
    reader.nextLine();
    //Error occurs at this line of the code

    Mr 777, thanks for your reply, I had a go at using a FileReader and then a BufferedReader to get
    a ByteBuffer, but once again I just get an empty buffer when calling the reader.read(cb) method.
    Or have I misinterpreted?

  5. #5
    Think of me.... Mr.777's Avatar
    Join Date
    Mar 2011
    Location
    Pakistan
    Posts
    1,136
    My Mood
    Grumpy
    Thanks
    20
    Thanked 82 Times in 78 Posts
    Blog Entries
    1

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    Well, the problem may be the lack of the memory but JVM must throw an exception about OutofMemory.
    Well, JVM is memory limited so i guess there are different memory limits for 32 bit and 64 bit operating system. You are trying to run it on 32 bit OS?
    Well, there can be a soltuion to your problem and that is; parse the file without reading it to the memory.
    You can use Runtime Environment using System commands, that will be efficient, and might solve your problem too.

  6. #6
    Junior Member
    Join Date
    Nov 2011
    Posts
    5
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    I try this with a file .cvs and work well... Maybe I need to see your file .cvs or you can print the line with the problem...

  7. #7
    Junior Member
    Join Date
    Feb 2011
    Location
    Adelaide, Australia
    Posts
    9
    My Mood
    Busy
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    Thanks for your response Mr. 777. I'm running this on a 64 bit machine, and a fairly powerful one at that. Changing memory allocation to stack and heap
    in Java through command line options hasn't changed my results.

    A memory issue was my first suspicion, which is why I reduced the file size to a subset of the data I already had- in this instance the same thing happened,
    suggesting that the particular data I was looking at was somehow less visible to Java than the rest of it. Disappointingly, this makes up most of the data I have.

    Nevertheless I will try a non-buffered read- will post again if that works or I solve the issue some other way, but for now off to bed.

    Thanks again,
    Nitrogen Fingers

  8. #8
    Think of me.... Mr.777's Avatar
    Join Date
    Mar 2011
    Location
    Pakistan
    Posts
    1,136
    My Mood
    Grumpy
    Thanks
    20
    Thanked 82 Times in 78 Posts
    Blog Entries
    1

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    Sure, it doesn't seem to be a memory issue as no such error generated by JVM, well, you might give it a try without buffering into the memory and check the results. Good Luck.

  9. #9
    Member
    Join Date
    Jun 2011
    Posts
    182
    My Mood
    Where
    Thanks
    15
    Thanked 8 Times in 8 Posts

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    Where exactly are you actually READING the lines of the file with the Scanner? I only see a nextLine() call...

    nextLine() returns a String that you aren't capturing. You commented that it was skipping over the header line... ok great. Now where is the reading code?

  10. #10
    Think of me.... Mr.777's Avatar
    Join Date
    Mar 2011
    Location
    Pakistan
    Posts
    1,136
    My Mood
    Grumpy
    Thanks
    20
    Thanked 82 Times in 78 Posts
    Blog Entries
    1

    Default Re: Bizarre Issue with Scanner Failing to "notice" rest of file

    Quote Originally Posted by bgroenks96 View Post
    Where exactly are you actually READING the lines of the file with the Scanner? I only see a nextLine() call...

    nextLine() returns a String that you aren't capturing. You commented that it was skipping over the header line... ok great. Now where is the reading code?
    reader.nextLine();
    Here.... and i hope you read the whole post......

Similar Threads

  1. Replies: 7
    Last Post: August 13th, 2011, 02:22 AM
  2. Scanner.useDelimiter("\n") Problem
    By Noob_Programmer in forum What's Wrong With My Code?
    Replies: 6
    Last Post: July 31st, 2011, 10:28 AM
  3. first itteration of "for" statement acts differntly to all the rest, why?
    By SPACE MONKEY in forum Java Theory & Questions
    Replies: 4
    Last Post: March 1st, 2011, 11:06 AM
  4. Unable to run "Inspect" on my scrabjob.jpage file
    By GunnDawg in forum What's Wrong With My Code?
    Replies: 2
    Last Post: July 4th, 2010, 11:07 PM
  5. "java.lang.NoSuchMethodError: main" and "fatal exception occured."
    By joachim89 in forum What's Wrong With My Code?
    Replies: 2
    Last Post: January 10th, 2010, 08:35 AM