Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 11 of 11

Thread: Memory requirement for Collections,JList and data structure

  1. #1
    Junior Member
    Join Date
    May 2009
    Location
    Mumbai/ Bombay - India
    Posts
    15
    Thanks
    1
    Thanked 1 Time in 1 Post

    Lightbulb Memory requirement for Collections,JList and data structure

    Hello!

    I am new to the Java world - you could say about 1 year experience here and there.

    I have a problem. Its not a code problem, but more of concept and theory.

    I am making an application which contains a list box. Now the contents of this list box can be added, edited and deleted - I have taken care of that. When the application exits the contents are copied to file. When the application starts the contents are read from the file and added to the list box.

    I have made the application first in AWT and then in Swing. For this particular application I feel Swing would be better. Why? -
    - While using AWT's List class I had to manage 2 data structures- one a collection class and the other the List itself.
    - While using Swing I have to create and manage only 1 data structure, which is the object of class javax.swing.DefaultListModel.
    I have termed the above approaches as "More Memory Less FileIO"

    Now I have this THING/FEAR making rounds in my head. What is there are a million entries in the listbox? How much can the data structures provided by Java take (contain) before the application crashes (OutOfMemory)? Should I consider another way around?

    The only other way around I can think of is "Less Memory More FileIO".

    Can anyone help?


  2. #2
    mmm.. coffee JavaPF's Avatar
    Join Date
    May 2008
    Location
    United Kingdom
    Posts
    3,336
    My Mood
    Mellow
    Thanks
    258
    Thanked 294 Times in 227 Posts
    Blog Entries
    4

    Default Re: Collections,JList - Memory requirement

    Hello jacinto and welcome to the Java Programming Forums

    If you have a million entries loading into the listbox, you will run out of memory really, really fast. Do you expect your program to have this amount of entries?

    I doubt you would want all of these entries loaded into the listbox at once so you could always load them in as the user scrolls down. Maybe you could have an ArrayList, fill it in a different thread and then you can use this as a buffer for your listbox?

    Reading/Writing the file will be a problem too. How are you currently reading in the file? I think the most memory efficient way is to read the file in chunks.

    Have you tested your application yet to see what it can currently handle?
    Please use [highlight=Java] code [/highlight] tags when posting your code.
    Forum Tip: Add to peoples reputation by clicking the button on their useful posts.

  3. #3
    Junior Member
    Join Date
    May 2009
    Location
    Mumbai/ Bombay - India
    Posts
    15
    Thanks
    1
    Thanked 1 Time in 1 Post

    Default Re: Collections,JList - Memory requirement

    Nope! I have just created a small apps. Well I guess I will have to display some part of the code if this further explanation is not sufficient.

    - The record contains 4 attributes. Each record is stored on one line in a file (database.txt). In other words all 4 attributes are seperated by a seperator of my choice and stored on line for each record

    - When the application starts, it reads the data from a file database.txt. I use the BufferedReader's readLine() method.

    - As it reads the data it fills up the Data Structure which is an object of javax.swing.DefaultListModel. Now I have chosen this data structure because (I found out that) currently it is the only editable data structure that can be used with JList.

    - The main JFrame displays. It contains the JList. It also has 3 buttons- add, edit, delete.

    - I manipulate the data structure and the contents it contains using these 3 buttons. (2 JDialogs for data entry.)

    - When I am finished I close the window. A method gets called which overwrites the database.txt file with the contents in the JLIst i.e. the data structure.

    So File IO is done only twice during the process's life. No problem.
    But as I said, I fear for the size?
    Is there another implicit(library class) or explicit(algorithm) way around this.

    Sorry for the trouble.
    If you want then next time I shall paste the code. But I dont think it will be of any use.

  4. #4
    mmm.. coffee JavaPF's Avatar
    Join Date
    May 2008
    Location
    United Kingdom
    Posts
    3,336
    My Mood
    Mellow
    Thanks
    258
    Thanked 294 Times in 227 Posts
    Blog Entries
    4

    Default Re: Collections,JList - Memory requirement

    Hello jacinto,

    I am unsure what to suggest here. The real test will be to see if your current application can handle a file of this size then take it from there..
    Please use [highlight=Java] code [/highlight] tags when posting your code.
    Forum Tip: Add to peoples reputation by clicking the button on their useful posts.

  5. #5
    Junior Member
    Join Date
    May 2009
    Location
    Mumbai/ Bombay - India
    Posts
    15
    Thanks
    1
    Thanked 1 Time in 1 Post

    Default Re: Collections,JList - Memory requirement

    Thinking the same thing. Thanks for your help. Atleast you understood my dilemma.

    So I guess I will have to start filling alot of data.

    Thank you.

  6. #6
    mmm.. coffee JavaPF's Avatar
    Join Date
    May 2008
    Location
    United Kingdom
    Posts
    3,336
    My Mood
    Mellow
    Thanks
    258
    Thanked 294 Times in 227 Posts
    Blog Entries
    4

    Default Re: Collections,JList - Memory requirement

    Hey jacinto,

    No idea if this will work without dying but you could use this to write a million lines to a file lol

    import java.io.*;
     
    public class MillionLines {
     
        public static void main(String[] args) throws IOException {
     
            String newLine = System.getProperty("line.separator");
     
            Writer output = null;
            File file = new File("BIGFILE.txt");
            output = new BufferedWriter(new FileWriter(file));
     
            for(int a = 0; a < 1000000; a++){
            output.write("Random data to fill the file BLA BLA!");
            output.write(newLine);
            }
     
            output.close();
     
            System.out.println("File written");   
     
        }
     
    }
    Please use [highlight=Java] code [/highlight] tags when posting your code.
    Forum Tip: Add to peoples reputation by clicking the button on their useful posts.

  7. #7
    Junior Member
    Join Date
    May 2009
    Location
    Mumbai/ Bombay - India
    Posts
    15
    Thanks
    1
    Thanked 1 Time in 1 Post

    Default Re: Collections,JList - Memory requirement

    I shall give it a try and let you know.

  8. #8
    Junior Member
    Join Date
    May 2009
    Location
    Mumbai/ Bombay - India
    Posts
    15
    Thanks
    1
    Thanked 1 Time in 1 Post

    Exclamation Re: Collections,JList - Memory requirement

    POOF! CRASH! BAM!

    I made the necessary changes in the String of output.write. I compiled and executed the class.

    I took the datastore i.e. the ".txt" file and kept in in the place of the old file which already had some useful data.

    First of all, this ".txt" that was generated had a size of 41.8 MB. I tried to open it in notepad, but didn't work.
    This gave me a glimpse of what was to come if I tried to open and get all the contents of the file in my Java application.

    As I said I replaced the present ".txt" file with the one just generated by the program. I then clicked on the jar file of my application.
    I wait. . . . . ?

    I opened the task manager (windows XP). No mention of any java runtime. This time I keep the task manager open and click on the jar file. An entry comes up (in the Processes tab and not in the Applications tab) signifying that the process (javaw.exe) is running. Just as soon as it appears it disappears.

    I click again, this time I keep a watch on the "Memory Usage" column. It starts at around 30,000 KB and keeps on increasing to about 100,000 KB and then POOF!

    I "think" my application just crashed. ?!?!

  9. #9
    mmm.. coffee JavaPF's Avatar
    Join Date
    May 2008
    Location
    United Kingdom
    Posts
    3,336
    My Mood
    Mellow
    Thanks
    258
    Thanked 294 Times in 227 Posts
    Blog Entries
    4

    Default Re: Collections,JList - Memory requirement

    Yes this looks to me like your application can not handle so many lines of text and has indeed died

    When the application loads, why don't you only load part of the data into the list box. As the user scrolls down, you can load in more data.

    This will stop the issues you are having on start up and it will not matter how large the text file is. Just an idea...
    Please use [highlight=Java] code [/highlight] tags when posting your code.
    Forum Tip: Add to peoples reputation by clicking the button on their useful posts.

  10. #10
    Little Star
    Join Date
    May 2009
    Posts
    30
    Thanks
    0
    Thanked 9 Times in 7 Posts

    Default Re: Collections,JList - Memory requirement

    If you need to deal with that many entries I would suggest using database and implement pageview, where you have lets say xx entries per page.

  11. #11
    Junior Member
    Join Date
    May 2009
    Location
    Mumbai/ Bombay - India
    Posts
    15
    Thanks
    1
    Thanked 1 Time in 1 Post

    Default Re: Collections,JList - Memory requirement

    Dalisra! You mentioned about "pageview". Can you explain a little bit. Please!

Similar Threads

  1. [SOLVED] JAVA JList Problem in visual images
    By antitru5t in forum AWT / Java Swing
    Replies: 4
    Last Post: April 15th, 2009, 03:09 PM
  2. Replies: 1
    Last Post: March 31st, 2009, 02:49 PM
  3. Dropping to graphic element dragged from JList
    By tua1 in forum AWT / Java Swing
    Replies: 1
    Last Post: November 29th, 2008, 08:22 AM

Tags for this Thread