Error thrown from usubMag() when trying to assign dig[1]=0

[lebecki]

Found now error (?) in usubMag(). I tried to apply changes to make it more similar to uaddMag(). Hopefully I am right.
See: my fork

[bwakell]

I'm not sure it's an issue though. usubMag, subtracts the magnitude of the number, which by definition cannot go negative. So the current number must always >= sub, so a reallocation() to increase length should never be necessary.

…

________________________________ Från: lebecki <[email protected]> Skickat: den 15 april 2020 17:02 Till: bwakell/Huldra <[email protected]> Kopia: Subscribed <[email protected]> Ämne: [bwakell/Huldra] Error thrown from usubMag() when trying to assign dig[1]=0 (#16) Found now error (?) in usubMag(). I tried to apply changes to make it more similar to uaddMag(). Hopefully I am right. See: my fork<lebecki@d5cddc2> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#16>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAKMTEKTTXTTLLDVO5YB3G3RMXSB7ANCNFSM4MIXY24Q>.

[bwakell]

If there is an issue, it must be in one of the methods calling it (i.e. some invariant breaks). So if you have a concrete example of a failing scenario (which is entirely possible ^^), do share.

[lebecki]

Example sent to you. I think in this example the call that is causing the trouble looks like BigInt(3).sub(1) - rather simple.
The reason being the line:
dig[1] = (int)dif;
that is called always, even if len==1

[lebecki]

More precisely speaking, here is the code of a simple failing scenario:
BigInt sum = new BigInt(3);
long one = 1; // Important: long, not int!
sum.sub(one); // Error thrown here:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
at org.huldra.math.BigInt.usubMag(BigInt.java:844)
at org.huldra.math.BigInt.usub(BigInt.java:897)
at org.huldra.math.BigInt.sub(BigInt.java:1097)
at Glowna.main(Glowna.java:78)

[bwakell]

Oh yeah a classic issue of assuming that just because it's a long it takes up more than 32 bits.

Part of the issue is in usub(final long a) where this statement:
|| ah==0&&(dig[0]&mask)>=al

Should instead be in an else if and call usubMag((int)al).