Parameter value precision issue

[Srwoerner]

It seems that at some point in handling parameters Dronekit is losing some precision with decimal values.

Example: I have a copter running 4.1.1

Looking at the param MOT_SPIN_MIN, the possible values for this param are: 0.0:Low, 0.15:Default, 0.3:High. The reported value however is 0.15000000596046448.

Damn you Java with your floating point issues!

[kellyschrock]

I've noticed this a lot lately. While tuning my copter, I'll do something like param set MOT_SPIN_ARM 0.05, which works. Then I do param show MOT_SPIN_ARM, and it's always some value like 0.05019652786406069. No idea why the AP wants to get all fancy with simple values, but it seems to. Note that in this particular case, I'm in a JavaScript environment, not Java. It seems to be something the AP does.

[Srwoerner]

That's interesting, and doesn't quite make sense to me. How is Mission Planner able to present the accurate value? Are they just rounding off insignificant digits?

[kellyschrock]

Not sure, I guess the MP code would probably provide an insight.

…

On Fri, Jan 21, 2022 at 3:24 PM Stephen ***@***.***> wrote: That's interesting, and doesn't quite make sense to me. How is Mission Planner able to present the accurate value? Are they just rounding off insignificant digits? — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA42DTIEHUKL7K2GBFGJV6TUXHFHLANCNFSM5MNSB6TQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>