Title Image

Don Xml's Grok This

The home of Don Demsak
Welcome to Don Xml's Grok This Sign in | Help
in Search

This Blog

Syndication

Site Sponsors

DonXml's All Things Techie

Problems With Precision On Conversion Of A Double To String

Here’s a gotcha for you.  If you are using a Double in .Net, and it has more than 15 digits of precision (A Double has a max of 17), and you try to convert it to a string (using the ToString() method), by default it will round it up to 15 digits (losing the last 2 digits of precision).   To get the last 2 digits, use a format of “G17” (to always return 17 digits of precision) or “R” (which will return 15 digits if the number can be represented with that precision).  This is buried in the ToString documentation for a Double.

Oh, and if you are getting this value back from a database (via a DataSet), and you are converting the resultset to a CSV file, the DataRow Item property is an object, so you will have to first convert the Item to a Double, and then use the Double’s ToString() method (since the Object.ToString method does not take any formatting options).  The good news is that if you save the Dataset as XML, it does export the Doubles with the proper precision.

Published Wednesday, May 19, 2004 12:35 PM by donxml
Filed under:

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

TrackBack said:

Take Outs for 19 May 2004
May 19, 2004 10:55 PM

Leave a Comment

(required) 
(optional)
(required) 
Submit

About donxml

I’m an independent consultant, specializing in .Net solutions architecture, based out of New Jersey who also doubles as an evangelist for XML, Domain Driven Design, enterprise architecture and .Net. I do not work for Microsoft, the W3C or any other big company that you may know of (at least not yet). I’ve been an indie for over ten years, and although I’ve been tempted a couple times to take a job with companies like Microsoft, I’ve haven’t found something better than my current situation. I work mostly with the large pharmaceuticals that are based here in New Jersey, and usually find myself on long term contracts. Definitely not the prototypical indie consultant, but it lets me dedicate time to my non-income generating activities like the developer community stuff, plus financing open source projects like XPathmania and MVP-XML. If you would like to talk to me about doing some contract work, just contact me via the contact page. My rates vary widely, depending on lots of different variables, but mostly distance from Jersey, and type of work. Plus, I’ve been known to donate some of my code for various projects.
Powered by Community Server, by Telligent Systems