Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 4 of 4

Thread: Need code to get count from huge json file

  1. #1
    Junior Member
    Join Date
    Oct 2013
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Need code to get count from huge json file

    The json file i'm using is around 4GB. Need java code to get no. of entries from json file.
    The "id" must be present multiple times in this huge file, so need to get distinct count.

    The file looks like,
    {"$xmlns":{"mm":"MetadataManagement","abm":"Title" },"startIndex":1,"itemsPerPage":500,"entryCount":2 08,"entries":[
    {"id":"4643245807014,"updated":1373162401000,"titl e":"Life is Elsewhere","added":1351040528000},
    {"id":"4643245807015,"updated":1373162401001,"titl e":"Titani","added":1851040528000},
    {"id":"4643245807015,"updated":1473162401001,"titl e":"Titanic","added":1851040528000}
    ]}


  2. #2
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,264
    Thanks
    176
    Thanked 821 Times in 764 Posts
    Blog Entries
    5

    Default Re: Need code to get count from huge json file


  3. #3
    Junior Member
    Join Date
    Oct 2013
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Need code to get count from huge json file

    The code i used works for a small file but for huge size file it throws out of memory error.
    I'm a beginner in java, please help me.

    JSONObject jsonObject = (JSONObject) parser.parse(new FileReader("C:\\Kishore\\2013-09-29.17_30_00.full.json"));
    JSONArray jsonMainArr = (JSONArray) jsonObject.get("entries");

    int count = jsonMainArr.size();
    System.out.println("No. of elements = " + count);

  4. #4
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,264
    Thanks
    176
    Thanked 821 Times in 764 Posts
    Blog Entries
    5

    Default Re: Need code to get count from huge json file

    Quote Originally Posted by kishorekumar87 View Post
    The code i used works for a small file but for huge size file it throws out of memory error.
    I'm a beginner in java, please help me
    Increase the heap size of you program, or parse the JSON object manually, counting as you parse such that the data is not kept in memory

Similar Threads

  1. Java XML parsing for HUGE size file and no root tags
    By peekuabc in forum Java Theory & Questions
    Replies: 5
    Last Post: June 28th, 2013, 10:53 AM
  2. loading json file into jsp page
    By swapnareddy in forum What's Wrong With My Code?
    Replies: 0
    Last Post: April 20th, 2013, 05:00 AM
  3. word count in a in a file
    By raj4322 in forum What's Wrong With My Code?
    Replies: 3
    Last Post: November 20th, 2012, 08:54 AM
  4. Open JSON file with Jfilechooser
    By nautilusz in forum File I/O & Other I/O Streams
    Replies: 0
    Last Post: May 14th, 2012, 10:38 PM
  5. Replies: 1
    Last Post: April 7th, 2011, 01:32 PM