Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 2 of 2

Thread: Large or small memory size for my App?

  1. #1
    Member
    Join Date
    Nov 2017
    Location
    USA
    Posts
    137
    Thanks
    6
    Thanked 1 Time in 1 Post

    Default Large or small memory size for my App?

    Should I be running my application with few instances (i.e. machines) with large memory size or a lot of instances with small memory size? Which strategy is optimal? This question might be confronted often. After building applications for 2 decades, after building JVM performance engineering/troubleshooting tools (GCeasy, FastThread, HeapHero), I still don’t know the right answer to this question. At the same time, I believe there is no binary answer to this question as well. In this article, I would like to share my observations and experiences on this topic.

    Two multi-billion dollars enterprises story

    Since our JVM performance engineering/troubleshooting tools has been widely used in major enterprises, I had an opportunity to see world-class enterprise applications implementations in action. Recently I had the chance to see two hyper-growth technology companies (If I say their name everyone reading this article will know them). Both companies are headquartered in Silicon Valley. Their business is technology, so they know what they are doing when it comes to engineering. They are wall-street darlings, enjoying great valuations. Their market cap is in the magnitude of several billions of dollars. They are the poster child of modern thriving enterprises. For our conversation let’s call these two enterprises as company-A and company-B.

    It immensely surprises me to see how both enterprises has adopted *two extremes* when it comes to memory size. Company-A has set its heap size (i.e. -Xmx) to be 250gb, whereas company-B has set its heap size to be 2gb. i.e. company-A’s heap size is 125 times larger than Company-B’s heap size. Both enterprises are confident about their memory size settings. As they say: ‘Proof is in the pudding’, both enterprises are scaling and handling billions of business-critical transactions.

    This is a great experience to see both companies who are into the same business, having more or less same revenue/same market cap, located in the same geographical region, at same point in time adopting two extremes when it comes to memory size. Given this real-life experience, what is the right answer? Large size or small size memory? My conclusion is: You can succeed with either strategy if you have a good team in place.

    Large memory size can be expensive

    Large memory size with few instances (i.e. machines) tends to be expensive than with small memory size, a greater number of instances. Here is simple math, based on the cost of an AWS EC2 instances in US East (N. Virginia) region:

    m4.16xlarge – 256GB RAM – Linux on-demand instance cost: $3.2/hour

    T3a small – 2GB RAM – Linux on-demand instance cost: $0.0188/hour

    So, to have capacity of 256GB RAM, we would have to get 128 ‘T3a small’ instance (i.e. 128 instances x 2GB = 256GB).

    128 x T3a small – 2GB RAM – Linux on-demand instance cost: $2.4064/hour (i.e. 128 x $0.0188/hour)

    It means large memory size with few instances costs $0.793/hour (i.e. $3.2 – $2.4064) more than small memory size with a lot of instances. In other words, ‘large memory size with few instances strategy is 33% more expensive.

    Of course, another counter-argument can be made is: you might need fewer engineers, less electricity, less real estate if you have a smaller number of machines. Patching, upgrading servers might be easier to do as well.

    Business Demands

    In some cases, the nature of your business itself dictates the memory size of your application. Here is a real-life incident that we faced: When we built HeapHero (Heap Dump analysis tool), our tool’s memory size had to be larger than heap dump file it parses. Say suppose heap dump file size is 100gb, then HeapHero tool’s memory size must be more than 100gb. There is no choice.

    Say suppose you are caching large amount (say 200gb) of data for maximizing your application’s performance, then your heap size must be more than 200gb. You will not have a choice. Thus, in some cases, the business requirement will dictate your memory size.

    Performance & Troubleshooting

    If your memory size is large, then typically Garbage Collection pause times will also be high. Garbage collection is a process that runs in your application to clean-up unreferenced objects in memory. If your memory size is large, then the amount of garbage in the memory will also be large. Thus, amount of time taken to clean up garbage will also be high. When garbage collection runs, it pauses your application. But there are solutions to this problem:

    • You can use pauseless JVM (like ‘Azul’)
    • Proper GC tuning needs to be done to reduce pause times


    Similarly, if you need to troubleshoot any memory problem, you will have to capture heap dumps from the application. A heap dump is basically a file which contains all information about your application’s memory like what objects were present, what are their references, how much memory each object occupies, …. Heap dumps of large memory size application will also tend to be very large. Analyzing large size heap dumps are difficult as well. Even world’s best heap dump tools like Eclipse MAT, HeapHero have challenges in parsing heap dumps that are more than 100gb. Reproducing these problems in test lab, storing these heap dump files, sharing these heap dump files are all challenges.

    Emotions comes first, Rationale next

    After reading books like ‘How we decide’ written by Jonah Lehrer – I am fairly convinced that your prior experience, emotions play a key role in deciding your application’s memory size. I used to work for a major financial institution. Chief architect of this financial institution was suggesting us to run our JVMs with very large memory size, rationale he gave was: “We used to run mainframes with very large memory size ”.

    Conclusion

    If you are working for very large corporations, then there is a 99.99% chance that you may not have to say on what should be the memory size for your application. Because that decision has already been made by elites/demi-gods who are sitting on the ivory tower. It might be to be hard to reverse or change that decision.

    But if you have choice or option to make that decision, your decision for memory size can be most likely be influenced by your prior experience and emotions . But either way, you can’t go wrong (i.e. going with few instances with large memory size or lot of instances with small memory size), provided you have the right team in place.

    Reference Article: https://blog.gceasy.io/2019/06/21/la...ze-for-my-app/

  2. #2
    Junior Member
    Join Date
    Oct 2021
    Posts
    5
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Large or small memory size for my App?

    Your all the posts are very useful.

Similar Threads

  1. Omitting local large variable to save memory
    By Wolf in forum What's Wrong With My Code?
    Replies: 5
    Last Post: September 30th, 2014, 03:20 PM
  2. Splitting large video files into small chunks and combining them
    By imran.ali@adamm.eu in forum File I/O & Other I/O Streams
    Replies: 16
    Last Post: December 2nd, 2013, 02:35 PM
  3. Size of App
    By aussiemcgr in forum Android Development
    Replies: 6
    Last Post: May 2nd, 2013, 01:59 PM
  4. Building a small app... (GridBagLayout problems)
    By lowfatbeans in forum What's Wrong With My Code?
    Replies: 0
    Last Post: March 14th, 2011, 06:49 PM
  5. Fitting a large primitive into a small reference variable
    By Phobia in forum Java Theory & Questions
    Replies: 15
    Last Post: October 23rd, 2009, 03:10 PM