Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Page 1 of 3 123 LastLast
Results 1 to 25 of 57

Thread: Program to find the number of occurrences of words in the text file

  1. #1
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Program to find the number of occurrences of words in the text file

    Hello i am doing an small project to find the top "n" words of an text file based on their frequency of occurrence but i am having few problems in my code now its printing the number of occurrences in each line not as whole file and my another problem is if the file have the following words
    tall,taller,tallest then the code must find the root word(tall) and print its frequency as 3 but i am confused how to do this so i am posting my code here i request people here to kindly help me to rectify my mistake and help me to learn.
    import java.io.*;
    import java.util.*;
    class Counter
    {
     public static void main(String[] args) throws NullPointerException
      {
      System.out.println("Enter the number of words to find its frequency:");
      Scanner scan = new Scanner(System.in);
      int k=scan.nextInt();//Number of top words to be found is stored here
      Scanner scan1 = new Scanner(System.in);
      System.out.println("Enter the file path to analyse the words:");
      String path=scan1.nextLine();// Directory path will be stored here
      String files;
      File folder = new File(path);
      File[] listOfFiles = folder.listFiles(); 
      BufferedReader br = null;
      String words[];
      String line="";
        for(File f:listOfFiles) {
            if (f.isFile()) {
       		files = f.getName();
                    if (files.endsWith(".txt") || files.endsWith(".TXT")) {
                             try{
    			 br=new BufferedReader(new FileReader(f));
    			       try{
                                      while((line=br.readLine())!=null){
                                       Map<String, Integer> unique = new TreeMap<String, Integer>();
                                       String wordlist[]=line.split("\\s+");
                                       for (int i=0; i<wordlist.length; i++) {
                                          String string=wordlist[i];
                                          unique.put(string,(unique.get(string) == null?1:(unique.get(string)+1)));
     
                                                                              }
     
                                          System.out.println(unique);
     
     
                                                  }
                                         }
                                          catch(IOException e){
                                               System.out.println("I am sorry:"+e);
                                                               }
                                                                               }
                               catch(FileNotFoundException e){
                                 System.out.println("Here i have got"+e.getMessage());
                                                              }
     
                                         }
                           }
                      }
              }
        }
    Last edited by Dinesh Raja; December 9th, 2013 at 12:08 PM. Reason: Formatting problem :)


  2. #2
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    its printing the number of occurrences in each line not as whole file
    The code should wait until the end of the file is found before printing the number.

    find the root word(tall)
    What is the definition of a root word? You will need to get that definition before designing and writing any code.

    The formatting of the code is all messed up. Please edit the post and fix its formatting.
    If you don't understand my answer, don't ignore it, ask a question.

  3. #3
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    What is the definition of a root word? You will need to get that definition before designing and writing any code.
    I meant to say that for example tallest and taller are derived or enhanced from the word "tall" so now the frequency of tall must be 3
    The formatting of the code is all messed up. Please edit the post and fix its formatting.
    I am sorry in a hurry i made the post like that now i have formatted it

  4. #4
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    So how would a program identify a word that is derived from a root?
    Does the word start with, end with or contain the root?

    How does the program get the roots?
    If you don't understand my answer, don't ignore it, ask a question.

  5. #5
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Thats why i am confused how can i achieve it i couldnt find any logic for this but i need to do that in my code

  6. #6
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    You need to define what it is before you can write any code.

    A root is ....
    A word that is derived from a root has ....

    How does the program get a list of root words?
    If you don't understand my answer, don't ignore it, ask a question.

  7. #7
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    But my code is going to find the frequency of words for an random file which the user selects so its not possible to predetermine the root words list and use it for reference :-(

  8. #8
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Then how will the program find root words?
    If you don't understand my answer, don't ignore it, ask a question.

  9. #9
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Can i use substring formation?or is there any other good way for it

  10. #10
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    You are talking about what code to write before you have stated what the definition of a root is.
    Can you explain what a "root" is? Given a definition, then worry about how to write the code to find and use a root.
    If you don't understand my answer, don't ignore it, ask a question.

  11. #11
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    I am defining based on my code requirements if my code reads an text file a.txt it contains 6 words say big,bigger,biggest and long,longest then the root words are big and long so my code needs to count the frequency of big as 3 and long as 2. It needs to strip down the words bigger,biggest,longest . I am sorry if i am bad in my english

  12. #12
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    You still have not said: How will the code decide if a word is a root?
    A root is a word that ....
    What goes where I put the ....?
    If you don't understand my answer, don't ignore it, ask a question.

  13. #13
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Actually i have not coded it i am asking how can i do that and the root word can be in any part of the text like it can start with or endwith or it may contain it.

  14. #14
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Another problem that needs to be solved: What if a root word is not in the input? Is the program supposed to derive a root word from other words? For example if the input has taller and tallest, is the program to determine that the root word is tall?

    These problems need to be solved before any coding is done.
    If you don't understand my answer, don't ignore it, ask a question.

  15. #15
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Word frequency counter

    Hi,I am working on an small assignment to read the text file(s) and count the frequency of words occurring in those files and print the top n words (which user will specify) and print the word and frequency of that word and one constraint i need to satisfy is if the file have words like long,longer,longest then the count of long has to be 3 not 1.so far i have made the code present in the pastie but its not working as required kindly help me to make it working properly please!!
    [Java] Counter.java - Pastebin.com

  16. #16
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Word frequency counter

    What problems are you having? Post the code and the questions you have about the code here on the forum.
    Be sure to wrap the code in code tags.

    Also post the program's output and add some comments saying what the problem is and describe what you want the output to look like.


    Is this a related problem: http://www.javaprogrammingforums.com...text-file.html
    If you don't understand my answer, don't ignore it, ask a question.

  17. #17
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    If you don't understand my answer, don't ignore it, ask a question.

  18. #18
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Yes Norm but i have forgot about this post i am sorry for posting the same problem again but i have updated the code now i will remove my new post and continue it here

    --- Update ---

    I have updated my code but still its not working (i think an try catch error) but my another question is my staff said i must use indexof() method for finding the count of big,bigger,biggest and i want my output as below:
    word:count
    example...
    hi:3
    hello:2
    here is my code kindly help me to correct my code please
    import java.io.*;
    import java.util.*;
    class Counter
    {
     public static void main(String[] args) throws NullPointerException
      {
      System.out.println("Enter the number of words to find its frequency:");
      Scanner scan = new Scanner(System.in);
      int k=scan.nextInt();//Number of top words to be found is stored here
      Scanner scan1 = new Scanner(System.in);
      System.out.println("Enter the file path to analyse the words:");
      String path=scan1.nextLine();// Directory path will be stored here
      File folder = new File(path);
      File[] listOfFiles = folder.listFiles();//To get the list of file-names present in the path 
      BufferedReader br = null;
      String words[]=null;
      String line="";
      String files;
      String word;
      int count=0;
     // List<String> list = new ArrayList<String>();
        for(File f:listOfFiles) {
            if (f.isFile()) {
       		files = f.getName();
                    if (files.endsWith(".txt") || files.endsWith(".TXT")) {
                             try{
    			 br=new BufferedReader(new FileReader(f));
    			       try{
                                      while((line=br.readLine())!=null){ 
                                            line = line.toLowerCase(); // convert to lower case 
                                             words = line.split("\\s+");
                                                                    }
                                           java.util.Arrays.sort(words);
     
     
                                       for(int i=0;i<words.length;i++) {
                                                     word = words[i];
                                       for(int j=0;j<words.length;j++) {
                                          if(words[j+1].equals(word)){
                                                        count++;
                                                      }
     
                                                              }
                                                  }
                                          System.out.println(count);
     
     
     
     
                                         }
                                          catch(IOException e){
                                               System.out.println("I am sorry:"+e);
                                                               }
                                                                               }
                               catch(FileNotFoundException e){
                                 System.out.println("Here i have got"+e.getMessage());
                                                              }
     
                                         }
                           }
                      }
              }
        }

  19. #19
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Word frequency counter

    Quote Originally Posted by Norm View Post
    yes i am sorry i have forgot about that post i will continue from there

  20. #20
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    still its not working
    Please explain what the problem is. Post the program's output and add some comments saying what is wrong.


    The code's formatting needs to be fixed. The positions of the ending } are not properly aligned making the code very hard to read and understand.
    If you don't understand my answer, don't ignore it, ask a question.

  21. #21
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Quote Originally Posted by Norm View Post
    Please explain what the problem is. Post the program's output and add some comments saying what is wrong.


    The code's formatting needs to be fixed. The positions of the ending } are not properly aligned making the code very hard to read and understand.
    import java.io.*;
    import java.util.*;
     
    class Counter {
     
        public static void main(String[] args) throws NullPointerException {
            System.out.println("Enter the number of words to find its frequency:");
            Scanner scan = new Scanner(System.in);
            int k = scan.nextInt();//Number of top words to be found is stored here
            Scanner scan1 = new Scanner(System.in);
            System.out.println("Enter the file path to analyse the words:");
            String path = scan1.nextLine();// Directory path will be stored here
            File folder = new File(path);
            File[] listOfFiles = folder.listFiles();//To get the list of file-names present in the path 
            BufferedReader br = null;
            String words[] = null;
            String line = "";
            String files;
            String word;
            int count = 0;
            // List<String> list = new ArrayList<String>();
            for (File f : listOfFiles) {
                if (f.isFile()) {
                    files = f.getName();
                    if (files.endsWith(".txt") || files.endsWith(".TXT")) {
                        try {
                            br = new BufferedReader(new FileReader(f));
                            try {
                                while ((line = br.readLine()) != null) {
                                    line = line.toLowerCase(); // convert to lower case 
                                    words = line.split("\\s+");
                                }
                                java.util.Arrays.sort(words);
     
                                for (int i = 0; i < words.length; i++) {
                                    word = words[i];
                                    for (int j = 0; j < words.length; j++) {
                                        if (words[j + 1].equals(word)) {
                                            count++;
                                        }
     
                                    }
                                }
                                System.out.println(count);
     
                            } catch (IOException e) {
                                System.out.println("I am sorry:" + e);
                            }
                        } catch (FileNotFoundException e) {
                            System.out.println("Here i have got" + e.getMessage());
                        }
     
                    }
                }
            }
        }
    }
    Sorry if still i am having an poor formatting skill and here is the output which i have got
    Enter the number of words to find its frequency:
    3
    Enter the file path to analyse the words:
    /home/dinesh
    0
    Actually the path /home/dinesh is having an text file with contents but now its not displaying the frequency of words in that file .the output which is expected is follows:
    hi:3
    hello:2
    norm:1
    P.S:The file present in the path have these words "hi hello hi hello hi norm" but still i am getting "0" as output. kindly help me to make my code work correctly guide me if i am wrong please

  22. #22
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Try debugging the code by adding some println statements that print out the values of all the variables as they are assigned values so that you can see what the computer sees when the code executes.
    For example print out:
    1) the value of line after a record is read into it.
    2) the value of the words array after the split. Use the Arrays class's toString() method as follows:
    System.out.println("an ID "+ java.util.Arrays.toString(theArrayName));
    If you don't understand my answer, don't ignore it, ask a question.

  23. #23
    Super Moderator
    Join Date
    Jun 2013
    Location
    So. Maryland, USA
    Posts
    5,520
    My Mood
    Mellow
    Thanks
    215
    Thanked 698 Times in 680 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Don't create a new Scanner object each time you need to get input. The object scan can be used both times.

    I see that this line will most likely cause an error each time it is run:

    if ( words[j + 1].equals( word ) )

    You're not seeing that?

  24. #24
    Member
    Join Date
    Dec 2013
    Posts
    32
    My Mood
    Confused
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Quote Originally Posted by Norm View Post
    Try debugging the code by adding some println statements that print out the values of all the variables as they are assigned values so that you can see what the computer sees when the code executes.
    For example print out:
    1) the value of line after a record is read into it.
    2) the value of the words array after the split. Use the Arrays class's toString() method as follows:
    System.out.println("an ID "+ java.util.Arrays.toString(theArrayName));
    here is the output after adding "System.out.println("1"+ java.util.Arrays.toString(words));"
    Enter the number of words to find its frequency:
    3
    Enter the file path to analyse the words:
    /home/dinesh/Desktop
    1 [hi]
    1 [hello]
    1 [hi]
    1 [hello]
    1 [hi]
    1 [norm]
    Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
    	at Counter.main(Counter.java:40)

  25. #25
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: Program to find the number of occurrences of words in the text file

    Is that output what you expected? All the arrays that were printed only have one word in them.
    If there is only one word, then there is nothing to compare that one word against:
      if (words[j + 1].equals(word))

    Also did you see Greg's post about the error you are getting? The for loop's index: j+1 is past the end of the array.


    You forgot to print out the value of line after a record is read into it.
    If you don't understand my answer, don't ignore it, ask a question.

Page 1 of 3 123 LastLast

Similar Threads

  1. [SOLVED] how to find all occurrences of a number in an array recursivelly
    By mia_tech in forum What's Wrong With My Code?
    Replies: 4
    Last Post: June 13th, 2012, 01:37 PM
  2. Reading text file and counting the number of words, integers, and floats.
    By Jsmooth in forum File I/O & Other I/O Streams
    Replies: 11
    Last Post: April 12th, 2012, 06:39 PM
  3. [SOLVED] Replacing words in a text file
    By sanelko in forum File I/O & Other I/O Streams
    Replies: 5
    Last Post: January 30th, 2012, 01:56 AM
  4. A program that counts the number of punctuation marks in a text file
    By Twoacross in forum What's Wrong With My Code?
    Replies: 1
    Last Post: October 21st, 2011, 04:03 PM
  5. Replies: 4
    Last Post: June 10th, 2009, 01:04 AM

Tags for this Thread