Saturday, 26 December 2020

Shock horror - Excel only stores numbers to fifteen significant figures

This will probably turn out to be one of those things that everybody except me already knew, so I know I am setting myself up for a fall here.

I only noticed today when I was trying to enter a really long number. Excel records the first fifteen digits and replaces everything after than with zeroes. For sure, it was my Tesco Clubcard number, so I could save it as text, but that's not the point. It's an up to date version of Excel, by the way.

Ho hum.

14 comments:

Mark said...

64 bits even.

15 digits, plus some bits left over for +/- and exponent.

James James said...

If you start the entry with an apostrophe ' it will save as text automatically. Useful if you want to start an entry with an equals sign = without it being interpreted as a formula.

Frank said...

Libre Office stores up to 16 digits.

Your Tesco Clubcard number is an identifier, it's more like text than a number. Tesco could have used a series of letters and it would have worked just as well.

Mark Wadsworth said...

M, I'll have to think about that.

JJ, that trick is one that I happen to know and use a lot.

F, what's Libre Office?

Re identifier, agreed. In the end, I saved it as text.

George Carty said...

It does sound though like they're storing (limited-precision) decimal numbers not (for example) double-precision floating-point...

Dinero said...
This comment has been removed by the author.
Dinero said...

So its to do with processor word length.
So how did pocket calculators in the early 7O s do 8 digits long numbers. Work it out or see next comment.

Dinero said...

They used individual circuits rather than a microprocessor and used Binary Coded Decimal.

Mark Wadsworth said...

GC and D, that's above my head. I just use Excel, a basic calculator and the CalcTastic app on my phone (which is excellent) and pen and paper.

Bayard said...

"what's Libre Office?"

The open source version of Excel, almost as good, but not quite.

Mark Wadsworth said...

B, ta.

Frank said...

@Bayard
"...almost as good, but not quite."

But good enough. At least, good enough for what about 99% of people use an Office suite for.

And it's free.

Bayard said...

F, even after almost 20 years of using Libre Office, there are features of Excel that I still miss, in particular its ability to recognise a sequence and follow it when you are drag-copying.

I have always regarded Excel as the only decent piece of software Microsoft has ever produced, especially when compared to the execrable Word.

Mark Wadsworth said...

B: "I have always regarded Excel as the only decent piece of software Microsoft has ever produced, especially when compared to the execrable Word."

Exactly. They keep adding extra features to Excel without buggering up what went before. Word is one step forward, two steps back. The auto-formatting is infuriating.