Memory requirement for Collections,JList and data structure

Hello!^#(^

I am new to the Java world - you could say about 1 year experience here and there.

I have a problem. Its not a code problem, but more of concept and theory.

I am making an application which contains a list box. Now the contents of this list box can be added, edited and deleted - I have taken care of that. When the application exits the contents are copied to file. When the application starts the contents are read from the file and added to the list box.

I have made the application first in AWT and then in Swing. For this particular application I feel Swing would be better. Why? -
- While using AWT's List class I had to manage 2 data structures- one a collection class and the other the List itself.
- While using Swing I have to create and manage only 1 data structure, which is the object of class javax.swing.DefaultListModel.
I have termed the above approaches as "More Memory Less FileIO"

Now I have this THING/FEAR making rounds in my head. What is there are a million entries in the listbox? How much can the data structures provided by Java take (contain) before the application crashes (OutOfMemory)? Should I consider another way around?

The only other way around I can think of is "Less Memory More FileIO".

Can anyone help?

Hello jacinto and welcome to the Java Programming Forums :)

If you have a million entries loading into the listbox, you will run out of memory really, really fast. Do you expect your program to have this amount of entries?

I doubt you would want all of these entries loaded into the listbox at once so you could always load them in as the user scrolls down. Maybe you could have an ArrayList, fill it in a different thread and then you can use this as a buffer for your listbox?

Reading/Writing the file will be a problem too. How are you currently reading in the file? I think the most memory efficient way is to read the file in chunks.

Have you tested your application yet to see what it can currently handle?

Nope! I have just created a small apps. Well I guess I will have to display some part of the code if this further explanation is not sufficient.

- The record contains 4 attributes. Each record is stored on one line in a file (database.txt). In other words all 4 attributes are seperated by a seperator of my choice and stored on line for each record

- When the application starts, it reads the data from a file database.txt. I use the BufferedReader's readLine() method.

- As it reads the data it fills up the Data Structure which is an object of javax.swing.DefaultListModel. Now I have chosen this data structure because (I found out that) currently it is the only editable data structure that can be used with JList.

- The main JFrame displays. It contains the JList. It also has 3 buttons- add, edit, delete.

- I manipulate the data structure and the contents it contains using these 3 buttons. (2 JDialogs for data entry.)

- When I am finished I close the window. A method gets called which overwrites the database.txt file with the contents in the JLIst i.e. the data structure.

So File IO is done only twice during the process's life. No problem.
But as I said, I fear for the size?
Is there another implicit(library class) or explicit(algorithm) way around this.

Sorry for the trouble.
If you want then next time I shall paste the code. But I dont think it will be of any use.

Hello jacinto,

I am unsure what to suggest here. The real test will be to see if your current application can handle a file of this size then take it from there..

Thinking the same thing. Thanks for your help. Atleast you understood my dilemma. :rolleyes:

So I guess I will have to start filling alot of data.:D

Thank you.

Hey jacinto,

No idea if this will work without dying but you could use this to write a million lines to a file lol

Code :

---

import java.io.*;
 
public class MillionLines {
 
    public static void main(String[] args) throws IOException {
 
        String newLine = System.getProperty("line.separator");
 
        Writer output = null;
        File file = new File("BIGFILE.txt");
        output = new BufferedWriter(new FileWriter(file));
 
        for(int a = 0; a < 1000000; a++){
        output.write("Random data to fill the file BLA BLA!");
        output.write(newLine);
        }
 
        output.close();
 
        System.out.println("File written");   
 
    }
 
}

---

I shall give it a try and let you know.

POOF! CRASH! BAM! :D

I made the necessary changes in the String of output.write. I compiled and executed the class.

I took the datastore i.e. the ".txt" file and kept in in the place of the old file which already had some useful data.

First of all, this ".txt" that was generated had a size of 41.8 MB. I tried to open it in notepad, but didn't work.
This gave me a glimpse of what was to come if I tried to open and get all the contents of the file in my Java application. :D

As I said I replaced the present ".txt" file with the one just generated by the program. I then clicked on the jar file of my application.
I wait. . . . . ? :confused:

I opened the task manager (windows XP). No mention of any java runtime. This time I keep the task manager open and click on the jar file. An entry comes up (in the Processes tab and not in the Applications tab) signifying that the process (javaw.exe) is running. Just as soon as it appears it disappears.

I click again, this time I keep a watch on the "Memory Usage" column. It starts at around 30,000 KB and keeps on increasing to about 100,000 KB and then POOF!

I "think" my application just crashed. ?!?! 8-x

Yes this looks to me like your application can not handle so many lines of text and has indeed died b-(

When the application loads, why don't you only load part of the data into the list box. As the user scrolls down, you can load in more data.

This will stop the issues you are having on start up and it will not matter how large the text file is. Just an idea...

If you need to deal with that many entries I would suggest using database and implement pageview, where you have lets say xx entries per page.

Dalisra! You mentioned about "pageview". Can you explain a little bit. Please!