Comparison of float data types.

eroach99

Hi,

Having trouble comparing float data types. If the value is too large, compare comes up un-equal. Is this supposed to work?????

Brian Donahue

Hello,

I think this is the same issue we've always had where the results in the result grid are rounded off, but the actual data (in binary form) is still different. Maybe it would be possible to see this if you selected the data from one of the columns and CAST it as bytes.

eroach99

That was it exactly - once I queried the fields and CAST them out further I saw the exact values.

masterbaker

Hi,

I am having the same problem.

I was wondering if you could give me an example of how to CAST the values. Thanks

eroach99

The compare apparently rounds at 13 decimal places - in our case there were more data past 13 decimal places - we used the following to see this :

select cast(column as decimal(25,15)) from table