Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 4 of 4

Thread: Writing to database (concurrency)

  1. #1
    Junior Member
    Join Date
    Feb 2014
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Writing to database (concurrency)

    Hello guys,

    recently I discovered power of Java, I'm coming from Perl, Python world so I'm at very beginning.
    I have firm understanding of OOP and I'm ready for new challenges.

    I have question regarding writing to database performance and theory.

    Basically as my personal project I'm trying to write some small program or addon which will enable me to write concurrently to database.
    Let's say that I have some software and that has FIFO output to database in single stream, but since the output interface is not fast enough congestion can occur if the input to the system is high.
    I would like to extend that software with addon which will take that one output stream and create 2,3,4 concurrent input streams towards database. API from that software is provided so I can redirect output to third-party extensions and apps.

    Since I'm new I don't even know where to start, connection pooling, connection concurrency,... Can someone explain me what is even needed to write such kind of program and where to gain materials to learn something about this kind of things?
    I know that google is my best friend, but it has tons of info and it's easy to lost your head. I know there will be a lot of theory on that before I do something meaningful.

    Thank you very much


  2. #2
    Senior Member PhHein's Avatar
    Join Date
    Mar 2013
    Location
    Germany
    Posts
    609
    My Mood
    Sleepy
    Thanks
    10
    Thanked 93 Times in 86 Posts

    Default Re: Writing to database (concurrency)

    Hi there,
    start having a look at the JDBC tutorial for the basics: Trail: JDBC(TM) Database Access (The Java™ Tutorials)

  3. #3
    Member andbin's Avatar
    Join Date
    Dec 2013
    Location
    Italy
    Posts
    443
    Thanks
    4
    Thanked 122 Times in 114 Posts

    Default Re: Writing to database (concurrency)

    Quote Originally Posted by Neven1986 View Post
    Let's say that I have some software and that has FIFO output to database in single stream, but since the output interface is not fast enough congestion can occur if the input to the system is high.
    Please, let me understand first: you have 1...N threads that insert informations into the FIFO and only 1 thread that extracts one information and inserts into the DB? And you are saying that the throughput of the extractor is not sufficient for the incoming informations? Is it correct?

    Before to resort to multiple extractor threads, you should evaluate (and explain here) other things: are you using JDBC directly? Or a library on top of JDBC (e.g. Hibernate)? Are you using the same connection for all inserts or are you getting a new connection for each insert? (this would be a big bottleneck). Then, how much complex is the insert on DB? How the application is connected to the database? (local machine? on a local network?)

    These are just some questions ...
    Andrea, www.andbin.netSCJP 5 (91%) – SCWCD 5 (94%)

    Useful links for Java beginnersMy new project Java Examples on Google Code

  4. #4
    Junior Member
    Join Date
    Feb 2014
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Writing to database (concurrency)

    Quote Originally Posted by andbin View Post
    Please, let me understand first: you have 1...N threads that insert informations into the FIFO and only 1 thread that extracts one information and inserts into the DB? And you are saying that the throughput of the extractor is not sufficient for the incoming informations? Is it correct?

    Before to resort to multiple extractor threads, you should evaluate (and explain here) other things: are you using JDBC directly? Or a library on top of JDBC (e.g. Hibernate)? Are you using the same connection for all inserts or are you getting a new connection for each insert? (this would be a big bottleneck). Then, how much complex is the insert on DB? How the application is connected to the database? (local machine? on a local network?)

    These are just some questions ...
    Thank you for reply.

    All stated questions are valid. The application in front communicates towards various systems via Radius or Diameter. Depending on config there can be multiple interfaces or just one, by each interface one thread. In any case, since this are highly transactional systems which can go to 400 TPS per interface and DB output is limited only to 1 thread or better to say 1 FIFO stream for writing to it.

    I haven't think about communication since this will be just challenge for me. But let's say that DB to which I'm writing is on other server and the inserts are simple. 30 - 50 values per insert, no updates or complicated stored procedures behind it. And I'm planning to use no third party extensions.

    I know this is big peace of cake for me right now, but that is I would like to have systematic path of progress and also to have some pointers in which direction to go.

    Thank you

Similar Threads

  1. concurrency over serialport object in RXTX
    By michelekapp in forum What's Wrong With My Code?
    Replies: 0
    Last Post: February 4th, 2014, 06:09 AM
  2. Java Concurrency
    By aknessy in forum What's Wrong With My Code?
    Replies: 13
    Last Post: November 24th, 2013, 07:18 AM
  3. Replies: 1
    Last Post: February 27th, 2012, 09:16 AM
  4. Concurrency Programming GURUS: Help!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    By Novice85 in forum Object Oriented Programming
    Replies: 7
    Last Post: September 29th, 2011, 05:31 PM
  5. java concurrency problem
    By Nick7 in forum What's Wrong With My Code?
    Replies: 5
    Last Post: August 1st, 2011, 06:02 AM