# Issues when applying a formula

**URL:** <https://discourse.processing.org/t/issues-when-applying-a-formula/45520>\
**Category:** Beginners\
**Created:** [December 30, 2024, 11:17am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520 "2024-12-30T11:17:09Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![GoToLoop](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gotoloop/32/86_2.png) [@GoToLoop](https://discourse.processing.org/u/GoToLoop)\
**Post date:** [December 30, 2024, 11:52am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/2 "2024-12-30T11:52:18Z")

</div>

> [@Proyecto\_Zeus](#):
>
> … and I need the highest precision possible.

Processing uses the 32-bit `float` primitive datatype as its default floating-point precision:

> **[float / Reference](https://processing.org/reference/float.html)**
>
> Data type for floating-point numbers, e.g. numbers that have a decimal point. Floats are not precise, so adding small values (such as 0.0001) may not always increment precisely due to rou…

For 64-bit floating-point precision we use `double` instead:

> **[double / Reference](https://processing.org/reference/double.html)**
>
> Datatype for floating-point numbers larger than those that can be stored in a float. A float is a 32-bit values that can be as large as 3.40282347E+38 and as low as -3.40282347E+38. A

However, it’s not enough to simply declare variables w/ primitive datatype `double`!

We also must suffix any floating-point literal w/ a `d`, so it uses a 64-bit storage: `6356911.94613d`

Without it, `6356911.94613` would be coerced to `6356912.0` due to 32-bit storage constraint!

Plus all Processing’s math functions need to be replaced w/ their corresponding Java’s Math class version:

> **[Math (Java SE 17 & JDK 17)](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Math.html)**
>
> declaration: module: java.base, package: java.lang, class: Math

```auto
final double a = 6378388;
final double b = 6356911.94613d;

final double ab = Math.sqrt(a*a - b*b);

final double e1 = ab / a; 
println("e1 =", e1);

final double e2 = ab / b; 
println("e2 =", e2);

```

---

_[View the full topic](https://discourse.processing.org/t/issues-when-applying-a-formula/45520)._