Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 2 of 2

Thread: java.net.HttpURLConnection:large file to upload

  1. #1
    Junior Member
    Join Date
    Oct 2009
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default java.net.HttpURLConnection:large file to upload

    hello,
    I want to upload large file to my sever,idea:set a maxmalChunkSize ,split file in chunks ,e.g. 5M file split in 2M,2M,1M and write into outputstream, server side received the chunks of a file,and write into a empty file in order to merge all of the chunks.
    but I met some problem. during the uploding, the server respons only the last chunk of the file,that is: only the last chunk of the file is uploaded,the rest chunks are not.i dont know why. Is anybody could help me,that would be nice.thanks a lot!


    code:
    public boolean openConnection(String numberOfFiles,String type) {
     
     
           try{
                this.numberOfFiles = numberOfFiles;
                //this.connect = new URL("http://127.0.0.1/bdb/bdb_create.php");
                this.connection=(HttpURLConnection)(this.connect.openConnection());
     
                this.connection.setDoInput(true);
                this.connection.setDoOutput(true);
                this.connection.setUseCaches(false);
     
                this.connection.setRequestMethod("POST");
                this.connection.setRequestProperty("Connection", "Keep-Alive");
     
                System.out.println("#########################OPEN CONNECTION...connected...################################");  
                System.out.println(type+"; boundary=******");
     
                this.connection.setRequestProperty("Content-Type",type+";boundary=******");
     
                this.fileOut = new DataOutputStream(this.connection.getOutputStream());
     
            }
            catch(MalformedURLException ie){
                return false;
            }//ie.printStackTrace();}
     
            catch (ProtocolException ex) {
                System.out.println(ex);
            }
     
            catch (IOException ioex) {
                System.out.println(ioex);
            }
     
            return true;
        }
     
    private final static int BUFLEN = 4096;
     
     
        private final byte readBuffer[] = new byte[BUFLEN];
     
        public boolean submitChunks(File fileName){
            try{
                this.uploadRemainingLength = this.getUploadLength(fileName);
                this.fileSize = this.getUploadLength(fileName);
     
                System.out.println("=>Original filesize= "+this.fileSize);
     
                boolean bLastChunk = false;
                int chunkPart = 0;
                long contentLength = 0;
                long thisChunkSize = 0;
                //int chunkCount = 0;
     
                //bChunkEnabled=1
                this.fileOut.writeBytes("--" + "******" + "\r\n");
                this.fileOut.writeBytes("Content-Disposition: form-data;name=\"bChunkEnabled\""+"\r\n");
                this.fileOut.writeBytes("\r\n");
                this.fileOut.writeBytes("1");
                this.fileOut.writeBytes("\r\n");
     
                System.out.println("--" + "******" + "\r\n");
                System.out.println("Content-Disposition: form-data;name=\"bChunkEnabled\""+"\r\n");
                System.out.println("\r\n");
                System.out.println("1");
                System.out.println("\r\n");
     
                while(!bLastChunk){
                    chunkPart = chunkPart +1;
                    bLastChunk = (contentLength > this.getRemainingLength(fileName));
                    if(bLastChunk){
                        thisChunkSize = this.getRemainingLength(fileName);
     
                        System.out.println("@=>this is the last chunk part "+chunkPart+", " +
                                "chunkpart size= "+thisChunkSize+" uploading...");
                        System.out.println("insgesamt "+chunkPart+" chunks");
     
                    }else{
                        thisChunkSize = this.maxChunkSize;
     
                        System.out.println("@=>this is the chunkpart "+chunkPart+", chunkpart size= "+maxChunkSize+" uploading...");
                    }
                    contentLength = thisChunkSize;
     
                    //chunkPart
                    this.fileOut.writeBytes("--" + "******" + "\r\n");
                    this.fileOut.writeBytes("Content-Disposition: form-data;name=\"chunkPart\"" + "\r\n");
                    this.fileOut.writeBytes("\r\n");
                    this.fileOut.writeBytes(Integer.toString(chunkPart));
                    this.fileOut.writeBytes("\r\n");
     
                    System.out.println("--" + "******" + "\r\n");
                    System.out.println("Content-Disposition: form-data;name=\"chunkPart\"" + "\r\n");
                    System.out.println("\r\n");
                    System.out.println(Integer.toString(chunkPart));
                    System.out.println("\r\n");
     
                    //bLastChunk
                    this.fileOut.writeBytes("--" + "******" + "\r\n");
                    this.fileOut.writeBytes("Content-Disposition: form-data;name=\"bLastChunk\"" + "\r\n");
                    this.fileOut.writeBytes("\r\n");
                    this.fileOut.writeBytes(String.valueOf(bLastChunk));
                    this.fileOut.writeBytes("\r\n");
     
                    System.out.println("--" + "******" + "\r\n");
                    System.out.println("Content-Disposition: form-data;name=\"bLastChunk\"" + "\r\n");
                    System.out.println("\r\n");
                    System.out.println(String.valueOf(bLastChunk));
                    System.out.println("\r\n");
     
                    //file data
                    this.fileOut.writeBytes("--" + "******" + "\r\n");
                    this.fileOut.writeBytes("Content-Disposition: form-data;name=\""+this.numberOfFiles+"\";"
                      + " filename=\"" + fileName.toString() +"\"" + "\r\n");
                    this.fileOut.writeBytes("\r\n");
     
                    System.out.println("--" + "******" + "\r\n");
                    System.out.println("Content-Disposition: form-data;name=\""+this.numberOfFiles+"\";"
                              + " filename=\"" + fileName.toString() +"\"" + "\r\n");
                    System.out.println("\r\n");
     
     
                    this.fileIn = new FileInputStream(fileName);
     
                    int amount = (int)(thisChunkSize);
     
     
                    while(amount > 0){
                        int toread = (amount > BUFLEN) ? BUFLEN : (int) amount;
                        int towrite = 0;
     
                        try{
                            towrite = this.fileIn.read(this.readBuffer,0,toread);
                        }catch (IOException e) {
                            e.printStackTrace();
                         }
     
                        if(towrite > 0){
                            try{
                                fileOut.write(this.readBuffer,0,towrite);
                                amount = amount - towrite;
                                this.uploadRemainingLength = this.uploadRemainingLength - towrite;
     
                                System.out.println("->write into outputstream: "+towrite
                                            +" || the left to be uploading: "+amount);
                            }catch (IOException ioe) {
                                    ioe.printStackTrace();
                             }
     
     
                        }
     
                    }
     
                    this.fileOut.writeBytes("\r\n");
                    this.fileOut.writeBytes("--" + "******" +"--"+"\r\n");
     
                    System.out.println("\r\n");
                    System.out.println("--" + "******" +"--"+"\r\n");
     
                    this.fileOut.flush();
                    this.fileIn.close();
     
     
     
                }//end while
     
     
            } catch (IOException ex) {ex.getStackTrace();}
     
            BufferedReader input;
            //DataInputStream input;
            try {
                input = new BufferedReader(new InputStreamReader(this.connection.getInputStream()));
                //input = new DataInputStream(this.connection.getInputStream());
                String out="";
     
                while(out!=null) {
     
                    out = input.readLine();
                    System.out.println("response:"+out);
                }
     
                input.close();
            } catch (IOException e) {e.printStackTrace();}
     
     
            return true;
        }
    debug output:
    #########################OPEN CONNECTION...connected...################################
    multipart/form-data; boundary=******
    A.filesize bigger than maxchunksize,we use chunk upload
    =>Original filesize= 36192
    --******
     
    Content-Disposition: form-data;name="bChunkEnabled"
     
     
     
    1
     
     
    @=>this is the chunkpart 1, chunkpart size= 20000 uploading...
    --******
     
    Content-Disposition: form-data;name="chunkPart"
     
     
     
    1
     
     
    --******
     
    Content-Disposition: form-data;name="bLastChunk"
     
     
     
    false
     
     
    --******
     
    Content-Disposition: form-data;name="10"; filename="/home/i6stud/sijitang/Desktop/demo/export_bilder_10/export_bilder/5.jpg"
     
     
     
    ->write into outputstream: 4096 || the left to be uploading: 15904
    ->write into outputstream: 4096 || the left to be uploading: 11808
    ->write into outputstream: 4096 || the left to be uploading: 7712
    ->write into outputstream: 4096 || the left to be uploading: 3616
    ->write into outputstream: 3616 || the left to be uploading: 0
     
     
    --******--
     
    @=>this is the last chunk part 2, chunkpart size= 16192 uploading...
    insgesamt 2 chunks
    --******
     
    Content-Disposition: form-data;name="chunkPart"
     
     
     
    2
     
     
    --******
     
    Content-Disposition: form-data;name="bLastChunk"
     
     
     
    true
     
     
    --******
     
    Content-Disposition: form-data;name="10"; filename="/home/i6stud/sijitang/Desktop/demo/export_bilder_10/export_bilder/5.jpg"
     
     
     
    ->write into outputstream: 4096 || the left to be uploading: 12096
    ->write into outputstream: 4096 || the left to be uploading: 8000
    ->write into outputstream: 4096 || the left to be uploading: 3904
    ->write into outputstream: 3904 || the left to be uploading: 0
     
     
    --******--
     
    response:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    response:    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    response:<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="de" lang="de">
    response:  <head>
    response:    <title>FAU-Bilddatenbank::FAU-Bilddatenbank</title>
    response:	<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
    response:	<meta name="Language" content="Deutsch" />
    response:...
    ...
    response:</html>
    response:null
    #############################DISCONNECTED##################################### 0
    Last edited by tommy_725; October 15th, 2009 at 05:43 PM.


  2. #2
    Junior Member
    Join Date
    Oct 2009
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Exclamation Re: java.net.HttpURLConnection:large file to upload

    I have tried the way you mentioned to call the servlet from the client side.

    But the connection is not getting established between the client and the server.

    Even I have a similar requirement .... I want to upload a file greater than 10GB, where the user need to login to upload the file. He can even pause the download for days and then resume the same. And in that even the upload should start from the point paused.

    We have tried with Apache Commons and the URL Method that u have mentioned. In the first case the problem that we have faced is skipping the InputStream receieved in the server. For example a file of size 10GB, has been paused after uploading of 9.5GB, now when the upload is resumed we have to skip 9.5GB and then start dumping the content of the file in the desired location.The time taken to skip the InputStream is same as uploading it back again, It is taking huge time to reach the place from where to begin the dumping is there any way we can skip the same in faster way.

    Secondly, the way that u have mentioned is ur post can ease the work, but the servlet is not getting called at all if you can share some code which specifies how you are accessing the stream or is their any tweak why the servlet is not getting invoked then it would be great help for us.

    thanx in advance

Similar Threads

  1. How to rename a file in your computer in Java?
    By JavaPF in forum File Input/Output Tutorials
    Replies: 1
    Last Post: December 24th, 2011, 12:05 PM
  2. Writing in a file using Java
    By JavaPF in forum File Input/Output Tutorials
    Replies: 4
    Last Post: December 17th, 2011, 04:33 PM
  3. exception while Read very large file > 300 MB
    By ps.ganesh in forum File I/O & Other I/O Streams
    Replies: 2
    Last Post: June 11th, 2009, 11:39 PM
  4. How to upload a file by clicking a link instead of button?
    By raghuprasad in forum Java Theory & Questions
    Replies: 2
    Last Post: May 3rd, 2009, 05:21 AM