# Issues when applying a formula

**URL:** <https://discourse.processing.org/t/issues-when-applying-a-formula/45520>\
**Category:** Beginners\
**Created:** [December 30, 2024, 11:17am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520 "2024-12-30T11:17:09Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Proyecto\_Zeus](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/proyecto_zeus/32/20278_2.png) [@Proyecto\_Zeus](https://discourse.processing.org/u/Proyecto_Zeus)\
**Post date:** [December 30, 2024, 11:17am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/1 "2024-12-30T11:17:10Z")

</div>

Hi there, I am trying to apply the next formulas on my Processing code for my project:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/3X/9/1/9193e0ab0a3162ac05601e6012513ddb1f378a6d.jpeg)

In order to do that, I wrote the next code:

```auto
float a = 6378388.0;
float b = 6356911.94613;

float e = sqrt(sq(a) - sq(b))/a; 
println("e =", e);
float e2 = sqrt(sq(a) - sq(b))/b; 
println("e' =", e2);

```

This code prints some values that, even thought they are similar to the expected ones, differ a bit on the last numbers. This could seem unsignificant, but I’m working with coordinates and I need the highest precission possible. Here there is a table with the values given by the example I’m following, the values that Python printed (because I made a test with Python to see if other programming languages could do it well) and the values Processing did:

| Source | e | e’ |
| --- | --- | --- |
| Example | 0.08199189 | 0.08226889 |
| Python | 0.08199188997511561 | 0.08226888960338334 |
| Processing | 0.08199162 | 0.08226862 |

As you can see, they are similar, but Processing’s last numbers make no sense. I don’t know why do this happen, and I couldn’t find any information about it.

This is the Python code I used, if you think it can be useful to solve the problem with Processing:

```auto
import math

a = 6378388.0
b = 6356911.94613

e = math.sqrt(a **2-b** 2)/a
e2 = math.sqrt(a **2-b** 2)/b
print(e, e2)

```

I hope someone can help me, thanks.

---

<div class="post-metadata">

**Author:** ![GoToLoop](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gotoloop/32/86_2.png) [@GoToLoop](https://discourse.processing.org/u/GoToLoop)\
**Post date:** [December 30, 2024, 11:52am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/2 "2024-12-30T11:52:18Z")

</div>

> [@Proyecto\_Zeus](#):
>
> … and I need the highest precision possible.

Processing uses the 32-bit `float` primitive datatype as its default floating-point precision:

> **[float / Reference](https://processing.org/reference/float.html)**
>
> Data type for floating-point numbers, e.g. numbers that have a decimal point. Floats are not precise, so adding small values (such as 0.0001) may not always increment precisely due to rou…

For 64-bit floating-point precision we use `double` instead:

> **[double / Reference](https://processing.org/reference/double.html)**
>
> Datatype for floating-point numbers larger than those that can be stored in a float. A float is a 32-bit values that can be as large as 3.40282347E+38 and as low as -3.40282347E+38. A

However, it’s not enough to simply declare variables w/ primitive datatype `double`!

We also must suffix any floating-point literal w/ a `d`, so it uses a 64-bit storage: `6356911.94613d`

Without it, `6356911.94613` would be coerced to `6356912.0` due to 32-bit storage constraint!

Plus all Processing’s math functions need to be replaced w/ their corresponding Java’s Math class version:

> **[Math (Java SE 17 & JDK 17)](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Math.html)**
>
> declaration: module: java.base, package: java.lang, class: Math

```auto
final double a = 6378388;
final double b = 6356911.94613d;

final double ab = Math.sqrt(a*a - b*b);

final double e1 = ab / a; 
println("e1 =", e1);

final double e2 = ab / b; 
println("e2 =", e2);

```

---

<div class="post-metadata">

**Author:** ![quark](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/quark/32/26_2.png) [@quark](https://discourse.processing.org/u/quark)\
**Post date:** [December 30, 2024, 11:59am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/3 "2024-12-30T11:59:33Z")

</div>

I see @GoToLoop has clearly explained the difference between the float and double data types but I will add my code solution because

1. I created it before I read GoToLoops reply and
2. I have created a function to do the maths for any a,b

😀

```auto
double a = 6378388.0D;
double b = 6356911.94613D;

void setup() {
  println(e(a, b, a));
  println(e(a, b, b));
}

double e(double a, double b, double div) {
  return Math.sqrt(a * a - b * b)/div;
}

```

---

<div class="post-metadata">

**Author:** ![GoToLoop](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gotoloop/32/86_2.png) [@GoToLoop](https://discourse.processing.org/u/GoToLoop)\
**Post date:** [December 30, 2024, 12:02pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/4 "2024-12-30T12:02:26Z")

</div>

> [@quark](#):
>
> `double b = 6356911.94613;`

You’ve forgotten to suffix that high-precision floating-point literal w/ a `d`: 😉  
`double b = 6_356_911.94613D;`

---

<div class="post-metadata">

**Author:** ![quark](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/quark/32/26_2.png) [@quark](https://discourse.processing.org/u/quark)\
**Post date:** [December 30, 2024, 2:10pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/5 "2024-12-30T14:10:44Z")

</div>

> [@GoToLoop](#):
>
> You’ve forgotten to suffix that high-precision floating-point literal

Well spotted and I have corrected the code in my post. Although not needed for `a` I have added it anyway to indicate to the coder that a _literal double_ value is expected here. Useful if the value has to be edited later.

---

<div class="post-metadata">

**Author:** ![Chrisir](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/chrisir/32/45_2.png) [@Chrisir](https://discourse.processing.org/u/Chrisir)\
**Post date:** [December 30, 2024, 7:34pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/6 "2024-12-30T19:34:05Z")

</div>

I had a similar issue with datatype `Long`.

The issue was the internal (invisible) conversion of the number to `String` performed by the `text()` command (or the `println()` command). So even when the program got it right, the output was wrong.

To avoid this, glv suggested `Long.toString(number)` which was great.

Maybe there’s also `Double.toString(number)`

---

<div class="post-metadata">

**Author:** ![GoToLoop](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gotoloop/32/86_2.png) [@GoToLoop](https://discourse.processing.org/u/GoToLoop)\
**Post date:** [December 30, 2024, 8:58pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/7 "2024-12-30T20:58:46Z")

</div>

> [@Chrisir](#):
>
> (or the `println()` command).

The only overloading datatype version missing for both [**println()**](https://processing.github.io/processing-javadocs/core/processing/core/PApplet.html#println--) & [**print()**](https://processing.github.io/processing-javadocs/core/processing/core/PApplet.html#print-byte-) is `short`; which would be auto-coerced to `int` anyways:

Therefore both printing functions should have no problem at all console-logging whatever datatype argument we pass to them; with the notably only exception for `short[]` arrays!

BtW, datatype `short[]` is the only datatype Processing functions can’t properly deal with at all!

On the other hand, method [**text()**](https://processing.github.io/processing-javadocs/core/processing/core/PApplet.html#text-char-float-float-) is indeed more restrictive for the available overloading datatype signatures for its parameters.

Besides the obviously String datatype, it has overloaded signatures for `int`, `float`, `char` and even `char[]` datatypes.

Arguments of primitive datatypes `byte` & `short` would be nicely auto-coerced to `int`.

However, we wouldn’t be able to pass arguments of primitive datatypes `boolean`, `long` and `double`; neither for any non-primitive datatypes apart from String!

All of those non-supported datatypes would need to be converted to String before invoking **text()**, as you have already found out.

But a much simpler `"" + arg` would coerce any of those argument datatypes to String just fine, as long as they’re not arrays.

---

<div class="post-metadata">

**Author:** ![LangdonS](https://avatars.discourse-cdn.com/v4/letter/l/ebca7d/32.png) [@LangdonS](https://discourse.processing.org/u/LangdonS)\
**Post date:** [January 4, 2025, 12:14am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/8 "2025-01-04T00:14:26Z")

</div>

Math should probably be done in pure Java.

Processing seems to be slow at math for no good reason.

According to VisualVM, my own distance method, `fastDist`, uses WAY less CPU than using Processing’s dist() method.

```auto
private static double fastDist(double x1, double y1, double x2, double y2) {
  return Math.sqrt((x1 - x2) * (x1 - x2) + (y1 - y2) * (y1 - y2));
}

```

---

<div class="post-metadata">

**Author:** ![eightohnine](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/eightohnine/32/19793_2.png) [@eightohnine](https://discourse.processing.org/u/eightohnine)\
**Post date:** [January 4, 2025, 7:43am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/9 "2025-01-04T07:43:50Z")

</div>

“…for no good reason.”

Processing consciously trades raw power for approachability and beginner-friendliness.

dist() is a neat and compact way to handle that oft-used calculation. The compromise is that it has overhead.

You can use sqrt() and pow() to write out that formula. I’m sure they perform OK against their respective Math counterparts.

---

<div class="post-metadata">

**Author:** ![quark](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/quark/32/26_2.png) [@quark](https://discourse.processing.org/u/quark)\
**Post date:** [January 4, 2025, 11:37am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/10 "2025-01-04T11:37:10Z")

</div>

In the Java class libraries all internal math calculations are done using the `double` data type but when Processing was first created they decided to use `float` as the default data type for the _Processing language_.

So when the Processing executes the statement  
`float v = sqrt(3.14159);`

it calls the equivalent Java method parameter value is passed to the Java method  
`Math.sqrt(3.14059);`

but this method returns a `double` which has to be cast to a `float`. Using the Processing `sqrt` method involves

1. an extra function call
2. a cast of the Java result back to a `float`

Personally I think using `float` instead of `double` was a mistake, it has 2 adverse consequences

1. Decreases performance. With modern processors and JVM optimisation this should not be a major problem, certainly it has never impacted me but might be an issue for others.
2. Some problems require the use of the `double` data type because they _ **cannot be solved** _ using `float` variables. For me this is the more critical issue and all my Processing libraries use the `double` data type internally because of this.

---

<div class="post-metadata">

**Author:** ![eightohnine](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/eightohnine/32/19793_2.png) [@eightohnine](https://discourse.processing.org/u/eightohnine)\
**Post date:** [January 4, 2025, 2:48pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/11 "2025-01-04T14:48:55Z")

</div>

Always so many detailed insights, love it!

Though I’m intrigued by your “float was a mistake” statement.

In my mind, close to all discussions on limitations and underperformance in Processing can be answered with “it’s a deliberate compromise in service of our core target audience”.

My theory on why `float` made it over `double`; Put yourself in the mind of that coding newbie, starting out with drawing a few basic shapes onto the screen. As if grokking the concept of a variable wasn’t enough, they’re also dealing with a typed programming language. The first variable types they’ll encounter are most likely `int` and `float`. The good thing is, there are number analogues they know already – “whole numbers” and “numbers with a decimal point”. Being able to fall back on these real world analogues in an otherwise alien programming environment is super valuable.  
Imagine now that the next type after `int` is a `double`. “It’s a number with a decimal point, but in fact you need to think of two different types of decimal point numbers, float and double, and you want to use one for tasks X and the other for tasks Y and in Processing we are using the more advanced version”. That’s immediately a bigger hurdle to deal with. And if you can consciously hide away that complexity by way of simplification, then I’m all for it, even if it means to let go of the advantages `double` can offer.

But maybe those arguments are irrelevant to one of the major user groups in Processing? Truth be told, I’ve never written a sketch that required `double` and I’m pretty sure I went 10+ years fuzzing around in Processing before even becoming aware that there’s an “older sibling to `float`”.

All this being said, I’m fully willing to accept that I’m way off with my “core target audience” argument. Possibly I’m sorely underestimating what you can expect a newbie to understand. But I’ve seen so many newcomer questions trip over such banal topics, that I feel the decision to err on the side of _simplicity over power_ is the correct one.

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [January 4, 2025, 3:55pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/12 "2025-01-04T15:55:55Z")

</div>

> [@LangdonS](#):
>
> According to VisualVM, using Math.sqrt() directly uses WAY less CPU than using Processing’s dist() method.

The Processing helper functions can be found here in _ **PApplet.java** _ to see what is going on _[under the hood](https://www.quora.com/What-does-under-the-hood-mean-in-programming)_:

- [processing4/core/src/processing/core/PApplet.java at main · processing/processing4 · GitHub](https://github.com/processing/processing4/blob/main/core/src/processing/core/PApplet.java)

_**sqrt() :**_

- [sqrt() / Reference / Processing.org](https://processing.org/reference/sqrt_.html)

- [processing4/core/src/processing/core/PApplet.java at main · processing/processing4 · GitHub](https://github.com/processing/processing4/blob/main/core/src/processing/core/PApplet.java#L4057)

```auto
  static public final float sqrt(float n) {
    return (float)Math.sqrt(n);
  }

```

_**dist() :**_

- [dist() / Reference / Processing.org](https://processing.org/reference/dist_.html)

- [processing4/core/src/processing/core/PApplet.java at main · processing/processing4 · GitHub](https://github.com/processing/processing4/blob/main/core/src/processing/core/PApplet.java#LL4550)

```auto
  static public final float dist(float x1, float y1, float x2, float y2) {
    return sqrt(sq(x2-x1) + sq(y2-y1));
  }

```

`:)`

---

<div class="post-metadata">

**Author:** ![quark](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/quark/32/26_2.png) [@quark](https://discourse.processing.org/u/quark)\
**Post date:** [January 4, 2025, 4:06pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/13 "2025-01-04T16:06:38Z")

</div>

> [@eightohnine](#):
>
> The first variable types they’ll encounter are most likely `int` and `float`. The good thing is, there are number analogues they know already – “whole numbers” and “numbers with a decimal point”.

Consider metallurgists, doctors, engineers, artists, footballers, actors, builders, plumbers … every specialist discipline has its own ‘vocabulary’. This is essential because it allows specialists to communicate complex details, information and ideas accurately and rapidly; thereby making it easier to _improve_ and _enhance_ their specialism.

As pre-school infants we are exposed to whole numbers but once at school we learn to use fractions followed by decimals. It allowed us to count and share the sweets out equally 😀 Later we would be introduced to new words like _ **integer numbers** _ and _ **floating point numbers** _.

Now consider the newbie to Processing which is basically a wrapper framework built around Java which is a strongly-typed language. The newbie has to learn a new ‘vocabulary’ and my contention is we could teach either

1. `float` is the data type to be used for decimal numbers, or
2. `double` is the data type to be used for decimal numbers

From a teaching and learning perspective I do not see the problem with going with (2). After all the newbie is going to have a lot more complex concepts to master if they want to get on.

Did you know that Processing created the keyword `color` as an alias to `int`

```auto
int c = color(255, 128, 128);
color c = color(255, 128, 128);

```

Both of these statements do the same thing but in the second line `color` represents both a data type and a function name. Personally I think this was their second and last mistake when they created Processing.

Don’t get me wrong, I think Processing is a wonderful way to get started in programming but the newbie must be prepared to learn the vocabulary of the computer programmer.

Finally the creators of Processing must be congratulated on making the object orientated language Java look like a procedural language and thereby making it accessible to all. 👍

---

<div class="post-metadata">

**Author:** ![LangdonS](https://avatars.discourse-cdn.com/v4/letter/l/ebca7d/32.png) [@LangdonS](https://discourse.processing.org/u/LangdonS)\
**Post date:** [January 4, 2025, 6:09pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/14 "2025-01-04T18:09:57Z")

</div>

I don’t see why my own distance method that is higher precision should be faster for the exact same task.

The default should be almost exactly the same speed.

How is this possible?  
I thought stuff can’t be static _and_ final?

```auto
static public final float sqrt(float n) {
  return (float)Math.sqrt(n);
}
```

---

<div class="post-metadata">

**Author:** ![LangdonS](https://avatars.discourse-cdn.com/v4/letter/l/ebca7d/32.png) [@LangdonS](https://discourse.processing.org/u/LangdonS)\
**Post date:** [January 4, 2025, 6:12pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/15 "2025-01-04T18:12:26Z")

</div>

Err… `mag` is different from `dist`.

---

<div class="post-metadata">

**Author:** ![eightohnine](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/eightohnine/32/19793_2.png) [@eightohnine](https://discourse.processing.org/u/eightohnine)\
**Post date:** [January 4, 2025, 6:25pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/16 "2025-01-04T18:25:05Z")

</div>

Hmm… I fell for what the Germans call _Gefährliches Halbwissen_. I assumed that in programming languages `float` was always the “general use” type and double the “speciality case” type. Hence my “Why apply the more advanced type as default?” stance. Turns out this differentation ain’t necessarily so. Goes to show that I need to understand more and better, before digging my heels in.

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [January 4, 2025, 7:02pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/17 "2025-01-04T19:02:16Z")

</div>

> [@Chrisir](#):
>
> To avoid this, glv suggested `Long.toString(number)` which was great.
> 
> Maybe there’s also `Double.toString(number)`

Testing:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/3X/8/c/8c9b7224f5a387e2a3125134f2b366c82e4a540a.png)

`:)`

---

<div class="post-metadata">

**Author:** ![GoToLoop](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gotoloop/32/86_2.png) [@GoToLoop](https://discourse.processing.org/u/GoToLoop)\
**Post date:** [January 4, 2025, 7:02pm UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/18 "2025-01-04T19:02:31Z")

</div>

> [@LangdonS](#):
>
> I thought stuff can’t be `static` _and_ `final`?

[`static`](https://processing.org/reference/static.html) is a Java keyword that is used to declare fields, methods and classes which can be accessed w/o instantiating the class they belong to; meaning they are independent from a particular instance of its class.

So the method PApplet.**sqrt()** can be invoked w/o requiring to instantiate its PApplet class.

The keyword [`final`](https://processing.org/reference/final.html) when used to declare fields, variables & parameters, it disallows them to be re-assigned w/ another value; although mutable objects they refer to can still be changed.

`final` methods can’t be [@override](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Override.html) by subclasses; and `final` classes themselves can’t be subclassed at all.

---

<div class="post-metadata">

**Author:** ![LangdonS](https://avatars.discourse-cdn.com/v4/letter/l/ebca7d/32.png) [@LangdonS](https://discourse.processing.org/u/LangdonS)\
**Post date:** [January 5, 2025, 2:26am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/19 "2025-01-05T02:26:22Z")

</div>

I know.

But IIRC, Intellij IDEA won’t allow me to make a method static and final.

---

<div class="post-metadata">

**Author:** ![GoToLoop](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gotoloop/32/86_2.png) [@GoToLoop](https://discourse.processing.org/u/GoToLoop)\
**Post date:** [January 5, 2025, 11:29am UTC](https://discourse.processing.org/t/issues-when-applying-a-formula/45520/20 "2025-01-05T11:29:04Z")

</div>

I’ve forgotten to explain about methods & fields which are both `static` & `final`: 🤦‍♂️

- When a subclass inherits `static` members from its superclass, we can re-define them within the subclass using the same name.
- For methods, that re-definition isn’t the same as [`@override`](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Override.html); it’s called overshadowing instead.
- The difference is that if we use the `(cast)` operator w/ the superclass name over an instance of the subclass, we end up accessing the original `static` super-member!
- For inherited non-`static` methods, we always get the [`@override`](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Override.html) version of the subclass, even if we use the `(cast)` operator to temporarily change it to its supertype.
- As you can notice, `static` inheritance rules are a mess. And we can even re-define an inherited `static` member as non-`static` and vice-versa!
- In order to stop such madness, we can add the `final` keyword over a `static` member, so it won’t be re-defined in subclasses in any way.

However, I’ve got no idea why IDEs such as Intellij IDEA would “prohibit” using keywords `static` & `final` together! 🤷‍♂️