How Floating Point Numbers Work and When You Should or Shouldn't Use Them
(Hint: Floating-point numbers aren't the best choice for financial applications, but they can be great for graphic-intensive applications.)
When working with numbers, as programmers we often have one important choice to make: Integers or what we often call "floats." But what's the difference and when should we use them? And when should we use something else entirely?
The Short Answer
Floating point numbers are useful when you need a number with a decimal point, such as 3.565. Integers are for whole numbers, typically for counting things and as indexes into arrays and lists. Floating point numbers are implemented inside the CPU, and are therefore very fast. However, floating point numbers have some shortcomings, typically with rounding problems. If you need precise decimal numbers, most languages offer alternatives with type names such as "decimal" or "currency." And a perfect example is indeed a financial application. When building a financial application, don't use floating point numbers unless you can guarantee that you're rounding to the appropriate precision every step of the way. Instead you'll usually be better off with types from decimal libraries.
Pro-tip: Although floating-point numbers have their limitations, in many cases they're totally fine. Games, simulators, graphics apps, etc. will likely not be impacted by the problems described below, as they're so small, they're basically inconsequential. Please don't get the impression I'm saying to never use floating points!
Examples of Problems with Floating Point Numbers
If you subtract these two numbers:
1.1 - 1.0
You'll get exactly 0.1. But try it in python:
>>> 1.1 - 1.0
0.10000000000000009
That's very, very close, but definitely not exactly 0.1. In a lot of cases, such as calculating the strength in a game character, that might be fine. But in financial applications, over time that tiny problem can become huge.
How about this one:
math.sin(math.pi)
1.2246467991473532e-16
The actual sin of pi is 0. (We're not using degrees here, but rather radians. Imagine the radius of a circle curved to fit on the circle itself. That's a radian.)
This number uses scientific notation and is equivalent to:
0.00000000000000012246467991473532
Again, that's very close to zero, but still isn't exact. And the reason for this error is at least two-fold: First, the number pi itself has an infinite number of numbers after the decimal point, meaning right away it's not exact. And second, the sine function uses a method of calculating sine that would require an infinite number of steps to be perfect; again, it can't do it infinitely.
In a car navigation app that calculates distances, that's probably fine. In a graphics app, it's almost certainly fine. Even in a Martian lander, that might be okay. But it might not be either, especially if it's carrying humans to populate the planet.
Alternatives to Floating Point
If you need exact precision, pretty much every programming language today offers a couple levels of floating point numbers, often called "float" and "double." Double uses "double precision" meaning a number takes up twice as much space and therefore offers better precision (but still not perfect, but definitely better; it might be fine for your application).
If you're building a game, most likely the built in floating point types will be exactly what you need; they might not be exact, but they're so close users won't even notice, and they're much faster.
But if you're building a financial app, or a scientific app that requires exact precision, you have some alternatives.
For financial applications, a great choice is the "decimal" type mentioned earlier. This can help overcome the problems.
(But regardless of what industry your app runs in, recognize there are some tradeoffs; your calculations will be slower with these decimal types. Factor that in before automatically switching to a decimal type.)
Pro-tip: If you're interviewing or starting work at a company that builds financial applications, ask them how they work around rounding problems and whether they use a decimal library. This is especially great during a job interview, as it will show you understand one problem they might be faced with.
Here are the decimal or currency types available in different languages:
- Python: Use the Decimal type by importing the "decimal" module.
- JavaScript: Technically, JavaScript uses a revised version of the system other languages uses; it's generally the same, except it also supports integers well, all in a single type called "Number". For decimal types, there are various third-party libraries you can use. The npm package manager includes one called decimal.js.
- C#: The .net framework includes a type called Decimal. To specify numbers in your code, you follow them with a lowercase "m" (which stands for "money").
- Java: The java.math library contains a type called BigDecimal.
- C++: The popular "Boost" library contains a type called multiprecision.
Still Be Careful with Division
Division can cause a lot of problems in calculations, even with decimal libraries, especially when dealing with financial applications.
For example, if you divide 1 by 3, you get the number 0.3333333, where the 3's go on forever. There are ways to store this number (such as fraction libraries) but decimal libraries are still going to run into a problem, since they don't have an infinite amount of space to store the 3's. And things get worse when dividing by numbers such as 7.
(Other calculations, however, are fine, such as dividing most numbers by 2.)
Be Careful with Testing for Equality
Continuing with the theme of division, that's a great place to see where tests for equality can be a big problem. If you're going to stick to the floating point numbers, be very aware that two numbers might be close but not equal – and then fail a test for equality.
If I take these two fractions:
2/7
3/7
And add them together, I'll get
5/7
(Remember, just add the top numbers, 2 + 3 \= 5.)
If I calculate these out in python:
>>> f3 \= 3/7
>>> f2 \= 2/7
>>> f2 + f3 \== 5/7
False
You can see that Python says that when I add the two, I don't get the same number as 5 divided by 7.
Different decimal libraries have different ways around problems like this. For example, the Python Decimal library detects them as being equal:
>>> from decimal import Decimal
>>> f2 \= Decimal(2) / Decimal(7)
>>> f3 \= Decimal(3) / Decimal(7)
>>> f5 \= Decimal(5) / Decimal(7)
>>> f2 + f3 \== f5
True
But you can still run into problems. How about 1/3 times 3? That should take us back to exactly 1. But it doesn't here, because the Decimal type can't hold an infinite number of 3s after the decimal point:
>>> f1 \= Decimal(1) / Decimal(3)
>>> f1 * 3 \== 1
False
What if you need to work with fractions like 1/3? (And remember, fractions are just division: The number 1/3 is the same as dividing 1 by 3.) You have some choices:
- The lack of precision might be fine! You might be good with just 0.3333333333 as opposed to the "exact" number that has an infinite number of 3's. But recognize if you add three of these together, you won't get exactly 1, but close. Again, that might be fine for your app. Consider the tradeoffs.
- Different languages have different libraries for fractions that you could use. Python, for example, offers a Fractions library (which I'll be covering in a future post; later I'll link it here.) Another great library is sympy, which has a "Rational" type. (Rational numbers are those that can be represented by the division of two whole numbers.)
Where did Floating Point come from?
Most languages use a method of storing floating point numbers based on a standard called IEEE 754: Standard for Floating-Point Arithmetic. It was standardized by the Institute of Electrical and Electronics Engineers (IEEE) in 1985.
It was based on techniques developed much earlier for storing floating point numbers. Early on, computer designers were very aware that they needed to store numbers with decimals, and they needed to be able to do the calculations quickly.
Tip: As a software developer, you usually don't need to know the details of the inner workings for floating point numbers. But if you're curious, it uses exponents with a base of either 2 or 10. If you're really curious, Wikipedia has a great writeup here: https://en.wikipedia.org/wiki/IEEE_754#Formats
Conclusion
Floating point numbers are likely fine for most applications, such as graphics and games (especially where speed is important). But when exact results are critical (especially with financial applications, and often with scientific applications), you'll want to use a different type, such as a decimal type or a fraction type.