Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 3 of 3

Thread: Multithreading Problem

  1. #1
    Member Staticity's Avatar
    Join Date
    Jul 2011
    Location
    Texas
    Posts
    105
    My Mood
    Inspired
    Thanks
    3
    Thanked 5 Times in 5 Posts

    Lightbulb Multithreading Problem

    The idea of multithreading is new to me, and in order to practice I have decided to make my Web Crawler a bit faster.

    After implementing the new process using a thread pool with ExecutorService, and an ArrayBlockingQueue to organize the links to crawl through, the process still works at the same rate as the previous program (10 threads compared to a single thread).

    Again, I'm new and am not aware of all of the pitfalls that come with multithreading. Any help would be great.

    Here's my shortened code to provide an example.

     
    // For reference (not actual code)
    ArrayBlockingQueue<String> links = new ArrayBlockingQueue<String>(10000);
    ExecutorService executor = Executors.newFixedThreadPool(10);
    TreeSet<String> foundLinks = new TreeSet<String>();
     
    // Actual code
    public void actionPerformed(ActionEvent event) {
     
    				links.clear();
    				foundLinks.clear();
     
    				links.add(addressField.getText());
    				foundLinks.add(addressField.getText());
     
    				int activeThreads = 0;
    				while (!links.isEmpty()
    						|| (activeThreads = ((ThreadPoolExecutor) executor)
    								.getActiveCount()) != 0) {
    					System.out.println(activeThreads);
    					try {
    						executor.submit(new LinkSearch(links.take()));
    					} catch (InterruptedException e) {
    						e.printStackTrace();
    					}
    				}
    			}

    Here is what a LinkSearch is:
    class LinkSearch implements Runnable {
     
    		String link;
     
    		public LinkSearch(String link) {
    			this.link = link;
    		}
     
    		public void run() {
    			searchLinks(link);
    		}
    	}
     
    // The method serachLinks(String link) is below:
     
    public void searchLinks(String link) {
    		try {
    			URL url = new URL(link);
    			String src = getSource(url);
    			src = src.toLowerCase();
    			int index = 0;
    			while ((index = src.indexOf("<a", index)) != -1) {
    				if ((index = src.indexOf("href", index)) == -1)
    					break;
    				if ((index = src.indexOf("=", index)) == -1)
    					break;
    				index++;
     
    				String remaining = src.substring(index);
    				StringTokenizer st = new StringTokenizer(remaining,
    						"\t\n\r\">#");
    				String strLink = st.nextToken();
     
    				URL urlLink;
    				try {
    					urlLink = new URL(url, strLink);
    					strLink = urlLink.toString();
    				} catch (MalformedURLException e) {
    					continue;
    				}
     
    				if (urlLink.getProtocol().compareTo("http") != 0)
    					break;
     
    				if (isHTML(strLink)) {
    					if (strLink.contains(addressField.getText())
    							&& linkNotFound(strLink) && exists(strLink)) {
    						System.out.println(strLink);
    						links.add(strLink);
    						addValidLink(strLink);
    						pageOrganizer.add(new ButtonLink(strLink));
    						addSource(src);
    						repaint();
    					}
    				}
    			}
    		} catch (MalformedURLException e) {
    			e.printStackTrace();
    			return;
    		}
    	}

    Hopefully it is all understandable/readable.
    Last edited by Staticity; August 2nd, 2012 at 06:42 PM.
    Simplicity calls for Complexity. Think about it.


  2. #2
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,332
    Thanks
    181
    Thanked 829 Times in 772 Posts
    Blog Entries
    5

    Default Re: Multithreading Problem

    More threads does not always equate to better performance - performance relates to how many parallel processes your computer can perform (eg processors), and once the computing power approaches maxing out overhead can begin to have a negative impact. While I don't know how many processors your computer has, 10 threads seems like a lot. I'd suggest first working with 2, profile the code, then report back as to whether two threads speeds up your previous single threaded process.

  3. #3
    Member Staticity's Avatar
    Join Date
    Jul 2011
    Location
    Texas
    Posts
    105
    My Mood
    Inspired
    Thanks
    3
    Thanked 5 Times in 5 Posts

    Default Re: Multithreading Problem

    I reduced the number to 2, and the speed was the same. If you look at the actionPerformed method I provided.. The problem is with the condition. The while loop finishes for some reason after the first submit call, which only allows a single thread to run.

    I thought that maybe changing the loop to a "do while" loop would fix the issue (since it would be required to run once, and there must be an active thread), but nothing changed. The thread that took the submit call continues to run, and doesn't call on any other threads. It is really quite weird. There's something wrong within my code.
    Simplicity calls for Complexity. Think about it.

Similar Threads

  1. Multithreading / Multiple Connections
    By Spicyfish in forum Java Networking
    Replies: 21
    Last Post: August 11th, 2012, 01:21 AM
  2. Java Multithreading Example - Issues
    By vijayinani in forum What's Wrong With My Code?
    Replies: 3
    Last Post: May 30th, 2012, 10:46 AM
  3. Idle thread in multithreading program.
    By Endian in forum Threads
    Replies: 5
    Last Post: May 15th, 2012, 02:02 PM
  4. Multithreading in java
    By skillet in forum Java Theory & Questions
    Replies: 1
    Last Post: March 17th, 2012, 07:32 AM
  5. Multithreading and its importance in java
    By jessie143143 in forum The Cafe
    Replies: 0
    Last Post: October 13th, 2011, 02:09 PM

Tags for this Thread