Comparison of float data types.

eroach99eroach99 Posts: 4
Hi,

Having trouble comparing float data types. If the value is too large, compare comes up un-equal. Is this supposed to work?????

Comments

  • Brian DonahueBrian Donahue Posts: 6,590 New member
    Hello,

    I think this is the same issue we've always had where the results in the result grid are rounded off, but the actual data (in binary form) is still different. Maybe it would be possible to see this if you selected the data from one of the columns and CAST it as bytes.
  • That was it exactly - once I queried the fields and CAST them out further I saw the exact values.
  • Hi,

    I am having the same problem.

    I was wondering if you could give me an example of how to CAST the values. Thanks
  • The compare apparently rounds at 13 decimal places - in our case there were more data past 13 decimal places - we used the following to see this :

    select cast(column as decimal(25,15)) from table
Sign In or Register to comment.