Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 2 of 2

Thread: How to append batches of stream data to one CSV file using Java NIO Package

  1. #1
    Junior Member
    Join Date
    Feb 2022
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question How to append batches of stream data to one CSV file using Java NIO Package

    I am reading data from API response in batches of bytes which is of Content-Type = text/CSV and using Java's NIO package to transfer bytes between two Channels. I want to write API responses in batches to the same File (APPEND). With below code append doesn't seem to work correctly, it's more of overriding results.
    And once all the data is written then I also want to print the number of total lines in CSV.

     
    private void downloadFile_NIO(String encoded_token) throws Exception 
    {
          long start_Range = 0;
        long end_Range = 5000;
        long batch_size = 5000;
        long totalsize = 1080612174;
        long counter = 0;
        //Path path = Paths.get(FILE_NAME);
        //long lines = 0;
     
        while (start_Range <= totalsize)
        {
           URL url = new URL(FILE_URL);
           HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
           connection.setRequestProperty("Authorization", "Bearer " + encoded_token );
           connection.setDoOutput(true);
           connection.setRequestMethod("GET"); 
     
           ReadableByteChannel readableByteChannel = Channels.newChannel(connection.getInputStream());
            FileOutputStream fileOutputStream=new FileOutputStream(new File(FILE_NAME),true);
           fileOutputStream.getChannel().transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
           // lines = Files.lines(path).count();
          // System.out.println("lines->" + lines);
          System.out.println();
     
     
             fileOutputStream.close();
             readableByteChannel.close();
     
            counter = counter + 1;
            if (counter < 2)
            {
               start_Range = start_Range + batch_size + 1;
               end_Range = end_Range + batch_size;
            }
            else
            {
                start_Range = start_Range + batch_size;
                   end_Range = end_Range + batch_size;
            }
     
     
     
        }
     
    }

  2. #2
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,042
    Thanks
    63
    Thanked 2,708 Times in 2,658 Posts

    Default Re: How to append batches of stream data to one CSV file using Java NIO Package

    Can you provide a small complete program that compiles and executes for testing?

    --- Update ---

    Also posted here: https://coderanch.com/t/749839/java/...tream-data-CSV
    If you don't understand my answer, don't ignore it, ask a question.

Similar Threads

  1. Replies: 0
    Last Post: October 4th, 2017, 12:24 PM
  2. How can I append an existing file in Java?
    By diyaots in forum File I/O & Other I/O Streams
    Replies: 2
    Last Post: February 8th, 2012, 11:26 AM
  3. Append data from user input into array
    By brncao in forum Collections and Generics
    Replies: 1
    Last Post: October 11th, 2011, 04:37 AM
  4. Replies: 1
    Last Post: April 7th, 2011, 01:32 PM
  5. How to append data to an already existing file?
    By siteregsam in forum File I/O & Other I/O Streams
    Replies: 1
    Last Post: May 3rd, 2010, 02:31 PM

Tags for this Thread