Determining the Double to Long Dimension Multiplier

Performing polygonal operations with integers as the coordinates is better for computation than doubles as coordinates. This is because of issues related to equality and slope finding. This concept is found in Clipper which is used in TVGL. In order to convert the double precision dimensions to integer values, they are multiplied by a large factor to maintain a large number of significant digits. One would probably be fine picking a number like 1 billion, but the larger the value the more susceptible to overflowing the long variable. This is more an issue for operations like dot and cross product which internally multiply coordinates values. Vertex coordinates are probably the result of common fractions like 1/64 of an inch, so perhaps there is a number which successfully multiplies all double-values coordinates to integer value ones without losing precision. Let's explore some common small fractions that may exists in solids.

Value	Description	In other units	Multiply Factor to get to unity	Prime factors	Prime divisors
0.015625	1/64 inch	0.015625	64	2,2,2,2,2,2
1	1 micron in mm (or mm in a meter)	0.001	1000	2,2,2,5,5,5
0.333333333	1/3 micrometer in mm	0.000333333	3000	2,2,2,3,5,5,5
1	1 micrometer in feet	3.28084E-06	304800	2,2,2,2,2,2,3,5,5,127
0.333333333	1/3 inch in m	0.008466667	118.1102362	2,2,2,3,5,5,5,5	127
0.333333333	1/3 of a mm in feet	0.001093613	914.4	2,2,3,3,127	5

The union of the prime factors is: 2,2,2,2,2,2,3,3,5,5,5,5,127 or (2^6)(3^2)(5^4)*127, which is B = 45,720,000. Thankfully, the divisors are already in the prime factors and can be ignored.

So, this factor, B (45720000) is used in TVGL to convert to integer points, which are stored as longs (Int64's). Given the maximum value of a long, this is sufficient for original coordinate values up to 201,736,046,369.2. However, as mentioned at the beginning since there are numerous operations involving multiplications of two coordinate values, we really shouldn't exceed the square root of long.MaxValue. This gives us an uppermost dimension of 66.42; and that's too small. To solve this problem Clipper defines Int128 to store these multiplicative values. This may be overkill. Perhaps an additional int (Int32) will be sufficient - making the calculation Int96's. Let's check.

The max value would be 63+32 bits and taking the square root of that would be 2^47.5. This number divided by B is 4,353,299.75. This seems sufficiently high enough that it is unlikely a solid has real valued coordinates about 500,000. But there is not much benefit to adding just a few bits. Clipper's Int128 is optimized and works well, putting the largest coordinate value at 200 billion.

But in the interest of keeping things quick for SIMD operations, we would like to fit both X and Y into a single 128-bit vector or two longs. As a compromise, let us set B to (2^3)(3)(5^3)(127) = 381,000. This means that the input polygon as real-valued coordinates cannot exceed 7,971.