Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 3 of 3

Thread: Merge 20+ xml files with 7000+nodes into one using java

  1. #1
    Junior Member
    Join Date
    Jul 2012
    Posts
    26
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default Merge 20+ xml files with 7000+nodes into one using java

    im trying to merge more than 20 xml files into one, any of them contains more than 7000 nodes code that im using for that is like this

    private void createFile(String month, String year) throws Exception {
     
        Properties prop = new Properties();
        prop.load(new FileInputStream("Config.properties"));
        String edb = prop.getProperty("EDB");
        String bbbCode = prop.getProperty("BBB");
        String InFolder = prop.getProperty("IN");
        String OutFolder = prop.getProperty("OUT");
        String Particija = prop.getProperty("DISK");
        DateFormat dateFormat = new SimpleDateFormat("ddMMyyyy");
        Date date = new Date();
        String ImeFile = "O3" + dateFormat.format(date) + "0000001.UJP." + bbbCode + ".xml";
        String MesecGodina = month + "/" + year;
        DateFormat dateTimeFormat = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
        Date dateTime = new Date();
        String DatPratka = dateTimeFormat.format(dateTime);
        try {
     
            DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
            DocumentBuilder docBuilder = docFactory.newDocumentBuilder();
            // root elements
            Document doc = docBuilder.newDocument();
            Element rootElement = doc.createElement("UplatiIsplatiMesecIzvestaj");
            doc.appendChild(rootElement);
            // PrireduvacVodecki elements
            Element PrireduvacVodecki = doc.createElement("PrireduvacVodecki");
            rootElement.appendChild(PrireduvacVodecki);
            // edbElement elements
            Element edbElement = doc.createElement("O3_Edb");
            edbElement.appendChild(doc.createTextNode(edb));
            PrireduvacVodecki.appendChild(edbElement);
            // imeFileElement elements
            Element imeFileElement = doc.createElement("O3_ImeFile");
            imeFileElement.appendChild(doc.createTextNode(ImeFile));
            PrireduvacVodecki.appendChild(imeFileElement);
            // mesecGodinaElement elements
            Element mesecGodinaElement = doc.createElement("O3_MesecGodina");
            mesecGodinaElement.appendChild(doc.createTextNode(MesecGodina));
            PrireduvacVodecki.appendChild(mesecGodinaElement);
            // datPratkaElement elements
            Element datPratkaElement = doc.createElement("O3_DatPratka");
            datPratkaElement.appendChild(doc.createTextNode(DatPratka));
            PrireduvacVodecki.appendChild(datPratkaElement);
            // rbrPratkaElement elements
            Element rbrPratkaElement = doc.createElement("O3_RbrPratka");
            rbrPratkaElement.appendChild(doc.createTextNode("0000001"));
            PrireduvacVodecki.appendChild(rbrPratkaElement);
     
            File dir = new File(Particija + ":\\" + InFolder + "\\");
            File[] rootFiles = dir.listFiles();
            int stavki = 0;
            for (File rootFile : rootFiles) {
                display.setText(display.getText() + rootFile + "\n");
                InputStream in = new FileInputStream(rootFile);
                DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
                DocumentBuilder db = dbf.newDocumentBuilder();
                Document docc = db.parse(new InputSource(in));
                docc.getDocumentElement().normalize();
                int y = docc.getElementsByTagName("UplataIsplata").getLength();
                display.setText(display.getText() + Integer.toString(y) + "\n");
                stavki = stavki + y;
                for (int i = 0; i < y; i++) {
                    // UplataIsplata elements
                    Element UplataIsplata = doc.createElement("UplataIsplata");
                    rootElement.appendChild(UplataIsplata);
     
                    Element idElement = doc.createElement("O3_Id");
                    idElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Id").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(idElement);
     
                    Element sifraUplMestoElement = doc.createElement("O3_Sifra_uplmesto");
                    sifraUplMestoElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Sifra_uplmesto").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(sifraUplMestoElement);
     
                    Element sifraKompElement = doc.createElement("O3_Sifra_komp");
                    sifraKompElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Sifra_komp").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(sifraKompElement);
     
                    Element serNumTicketElement = doc.createElement("O3_Serial_num_ticket");
                    serNumTicketElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Serial_num_ticket").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(serNumTicketElement);
     
                    Element codeTicketElement = doc.createElement("O3_Code_ticket");
                    codeTicketElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Code_ticket").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(codeTicketElement);
     
                    Element vremeUplataElement = doc.createElement("O3_Vreme_uplata");
                    vremeUplataElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Vreme_uplata").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(vremeUplataElement);
     
                    Element iznosUplataElement = doc.createElement("O3_Iznos_uplata");
                    iznosUplataElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Iznos_uplata").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(iznosUplataElement);
     
                    Element iznosMoznaDobivkaElement = doc.createElement("O3_Iznos_mozna_dobivka");
                    iznosMoznaDobivkaElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Iznos_mozna_dobivka").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(iznosMoznaDobivkaElement);
     
                    Element iznosIsplatenElement = doc.createElement("O3_Iznos_isplaten");
                    iznosIsplatenElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Iznos_isplaten").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(iznosIsplatenElement);
     
                    Element brojKombinaciiElement = doc.createElement("O3_Broj_kombinacii");
                    brojKombinaciiElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Broj_kombinacii").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(brojKombinaciiElement);
     
                    Element vkupenKoefElement = doc.createElement("O3_Vkupen_koef");
                    vkupenKoefElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Vkupen_koef").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(vkupenKoefElement);
     
                    Element vidTransakcijaElement = doc.createElement("O3_Vid_transakcija");
                    vidTransakcijaElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Vid_transakcija").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(vidTransakcijaElement);
     
                    Element statusTicketElement = doc.createElement("O3_Status_ticket");
                    statusTicketElement.appendChild(doc.createTextNode(docc.getElementsByTagName("O2_Status_ticket").item(i).getChildNodes().item(0).getNodeValue()));
                    UplataIsplata.appendChild(statusTicketElement);
     
                }
            }
     
            // vkBrStavkiElement elements
            Element vkBrStavkiElement = doc.createElement("O3_VkBrStavki");
            vkBrStavkiElement.appendChild(doc.createTextNode(Integer.toString(stavki)));
            PrireduvacVodecki.appendChild(vkBrStavkiElement);
            // write the content into xml file
            TransformerFactory transformerFactory = TransformerFactory.newInstance();
            Transformer transformer = transformerFactory.newTransformer();
            DOMSource source = new DOMSource(doc);
            StreamResult result = new StreamResult(new File(Particija + ":\\" + OutFolder + "\\" + ImeFile));
            transformer.setOutputProperty(OutputKeys.ENCODING, "WINDOWS-1251");
            transformer.transform(source, result);
     
            JOptionPane.showMessageDialog(mainPanel, "File Saved!", "XML Merge Information", JOptionPane.INFORMATION_MESSAGE);
            jButton1.setEnabled(true);
            mainPanel.setCursor(java.awt.Cursor.getPredefinedCursor(java.awt.Cursor.DEFAULT_CURSOR));
     
        } catch (ParserConfigurationException pce) {
            pce.printStackTrace();
        } catch (TransformerException tfe) {
            tfe.printStackTrace();
        }
    }

    but after some time of creating file program stops working.
    i got this exception Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space –
    how to change this code to write to file part by part not from memory.


  2. #2
    Member
    Join Date
    Mar 2011
    Posts
    198
    My Mood
    Daring
    Thanks
    7
    Thanked 4 Times in 4 Posts

    Default Re: Merge 20+ xml files with 7000+nodes into one using java

    I would actually create a for loop, First detect how many files there are... Next you want to read the file one by one, Store each line using a scanner or some other file reader method to store line by line into an array, You could also try using a vector so you wouldnt have to specify the size. Add the lines to the array, using a while next line != null loop. Wen the file is finished reading close that stream, then the for loop starts again for the next file opening a new stream.

    Once you have stored all the lines into the array, vector, arraylist etc.. you can then just create a new xml file and overright another xml file if you are wanting to overright the xml file... or create a new file and write the lines in the array using another for loop to detect the array size.. currentLine < array.length etc.

    And write line by line to the new file.

  3. #3
    Senior Member PhHein's Avatar
    Join Date
    Mar 2013
    Location
    Germany
    Posts
    646
    My Mood
    Sleepy
    Thanks
    10
    Thanked 95 Times in 88 Posts

    Default Re: Merge 20+ xml files with 7000+nodes into one using java

    Ugh, Vector is soo 1990s

    The OP will probably run into the same problems. You're right about not using documents here. I'd just use a reader and a writer. Read a line and write it away immediately without storing all the stuff.

Similar Threads

  1. JDBC with XML files.
    By alkic1 in forum JDBC & Databases
    Replies: 2
    Last Post: August 29th, 2012, 06:18 PM
  2. Eclipse Merge Two Files
    By aussiemcgr in forum Java IDEs
    Replies: 0
    Last Post: June 8th, 2012, 02:36 PM
  3. Replies: 1
    Last Post: March 18th, 2012, 01:16 PM
  4. Replies: 1
    Last Post: February 10th, 2011, 08:57 AM
  5. Merge 2 Sorted Text Files into a Third
    By Epyllion in forum What's Wrong With My Code?
    Replies: 0
    Last Post: February 10th, 2010, 08:24 PM

Tags for this Thread