Search:

Type: Posts; User: Perd1t1on

Search: Search took 0.22 seconds.

  1. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    K, well since you've settled the precision loss problem for float, I'm going to use long for sure now. The reason both x and y have to fit into it is so I can use the smallest data type possible. for...
  2. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    The objective is kind of difficult to explain properly to someone who hasn't seen all of the code but that's not really important. I was originally seeing if it would be possible to use float instead...
  3. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    I'm not sure what you mean by using a long instead but I redid the code using a long instead of a double. I know that double and long also both require 8bytes so I'm not sure if this is more...
  4. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    And here is the undo portion of my code:


    System.out.println("mag: "+mag+", and yx: "+yx+", and y: "+y+", and x: "+x);
    String breaknum=Double.toString(yx);
    short newy;
    short...
  5. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    the range is also a potential problem, it seems using double, I cannot exceed an x value beyond 99999, if I'm using my current method. But that's probably okay, because I can see other ways around...
  6. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    Well I need the precision more so I'll have to use doubles for this. The y and x are not a million right now, I've been working with around 200x100 max just to weed out errors. But it can be expanded...
  7. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    well I'm not super experienced, but the I have already seen the out of memory exception so far. I forget where, it may have been a mess up with a while loop. In terms of a rough estimation of the...
  8. [SOLVED] Re: Why is my float automatically rounding and how do i get it to stop

    Well this program is going to be gigantic so I'm trying to use the smallest data types possible at every opportunity.
  9. [SOLVED] Why is my float automatically rounding and how do i get it to stop

    I don't want to use double if I don't have to because I want to save memory. Here is the code using double:

    short mag;
    int magx;
    short y=154;
    short...
Results 1 to 9 of 9