[Suggestion] Find incompatibilities w.r.t. CircuitPython implementation

[elpekenin]

Yesterday, a user reported on discord an issue in which their library's CI was failing.

Upon a quick investigation, i found out that it had to do with slight differences between the CircuitPython code they were running on their board and the Blinka code that was later running/failing on CI.

Since we have both

• The implementation (Blinka)
• The typing stubs it should[1] match with (circuitpython-stubs)

...some automated testing could be run comparing them, to have lists of "this is not implemented yet" or "this function doesnt work the same way".

I ran a very quick Google search and seen that MyPy's stubtest command could be used (there might be other/better alternatives out there).

Since im not familiar with the codebase nor said tool, I havent really tested anything but thought it was a good thing to share the idea in case you guys find it useful.

---

[1] Im not sure whether 100% compatibility is desired or even posible on all cases... (not talking about coverage, but about function signatures and whatnot)

[makermelissa]

Yeah, they should be the same where the functionality exists So if a module is present in Blinka, code should run with very little modification (Pin names are ok to differ). So for function signatures, you are correct. We try and match, but occasionally CircuitPython makes changes that I'm not aware of. Do you know which function it was?

Also, I like you idea of a tool that we can use to compare that can be run during CI, so that we can be aware of those changes even without notification.

[elpekenin]

The specific problem was keypad's EventQueue being named _EventQueue instead

Will try and get some testing working locally

[makermelissa]

Ah, gotcha. Looks like it was just fixed by #883. Will leave this open for a more general issue.

[elpekenin]

My idea would be something like this: https://github.com/elpekenin/Adafruit_Blinka/tree/stubtest

[elpekenin]

Some diagnostics after a couple (already-pushed) changes to the script:

Running for: BLINKA_OS_AGNOSTIC
===============================
analogio failed to import. NotImplementedError: analogio not supported for this board.
keypad.Event.__init__ is inconsistent, runtime does not have argument "timestamp"
keypad.Event.key_number is read-only at runtime but not in the stub
keypad.Event.pressed is read-only at runtime but not in the stub
keypad.Event.released is read-only at runtime but not in the stub
keypad.EventQueue.__init__ is inconsistent, stub does not have argument "max_events"
keypad.EventQueue.overflowed is read-only at runtime but not in the stub
keypad.KeyMatrix.__exit__ is inconsistent, stub does not have argument "exception_type"
keypad.KeyMatrix.__exit__ is inconsistent, stub does not have argument "exception_value"
keypad.KeyMatrix.__exit__ is inconsistent, stub does not have argument "traceback"
keypad.KeyMatrix.__init__ is inconsistent, runtime does not have argument "debounce_threshold"
keypad.KeyMatrix.events is read-only at runtime but not in the stub
keypad.KeyMatrix.key_count is read-only at runtime but not in the stub
keypad.KeyMatrix.key_number_to_row_column is not present at runtime
keypad.KeyMatrix.row_column_to_key_number is not present at runtime
keypad.Keys.__exit__ is inconsistent, stub does not have argument "exception_type"
keypad.Keys.__exit__ is inconsistent, stub does not have argument "exception_value"
keypad.Keys.__exit__ is inconsistent, stub does not have argument "traceback"
keypad.Keys.__init__ is inconsistent, runtime does not have argument "debounce_threshold"
keypad.Keys.events is read-only at runtime but not in the stub
keypad.Keys.key_count is read-only at runtime but not in the stub
keypad.ShiftRegisterKeys.__exit__ is inconsistent, stub does not have argument "exception_type"
keypad.ShiftRegisterKeys.__exit__ is inconsistent, stub does not have argument "exception_value"
keypad.ShiftRegisterKeys.__exit__ is inconsistent, stub does not have argument "traceback"
keypad.ShiftRegisterKeys.__init__ is inconsistent, runtime does not have argument "debounce_threshold"
keypad.ShiftRegisterKeys.events is read-only at runtime but not in the stub
keypad.ShiftRegisterKeys.key_count is read-only at runtime but not in the stub
onewireio.OneWire.__enter__ is not present at runtime
onewireio.OneWire.__exit__ is not present at runtime
digitalio.DigitalInOut.__exit__ is inconsistent, stub does not have argument "exc_type"
digitalio.DigitalInOut.__exit__ is inconsistent, stub does not have argument "exc_value"
digitalio.DigitalInOut.__exit__ is inconsistent, stub does not have argument "traceback"
digitalio.Direction.__init__ is inconsistent, stub does not have *args argument "args"
digitalio.DriveMode.__init__ is inconsistent, stub does not have *args argument "args"
digitalio.Pull.__init__ is inconsistent, stub does not have *args argument "args"
usb_hid failed to import. Exception: dwc2 module not present in your kernel. did you insmod it?
bitbangio.I2C.__exit__ is inconsistent, stub does not have argument "exc_type"
bitbangio.I2C.__exit__ is inconsistent, stub does not have argument "exc_value"
bitbangio.I2C.__exit__ is inconsistent, stub does not have argument "traceback"
bitbangio.I2C.__init__ is inconsistent, runtime argument "frequency" is not keyword-only
bitbangio.I2C.__init__ is inconsistent, runtime does not have argument "timeout"
bitbangio.I2C.readfrom_into is inconsistent, runtime argument "start" is not keyword-only
bitbangio.I2C.readfrom_into is inconsistent, runtime argument "end" is not keyword-only
bitbangio.I2C.sys is not present at runtime
bitbangio.I2C.writeto is inconsistent, runtime argument "start" is not keyword-only
bitbangio.I2C.writeto is inconsistent, runtime argument "end" is not keyword-only
bitbangio.I2C.writeto is inconsistent, stub does not have argument "stop"
bitbangio.I2C.writeto_then_readfrom is not present at runtime
bitbangio.SPI.__exit__ is inconsistent, stub does not have argument "exc_type"
bitbangio.SPI.__exit__ is inconsistent, stub does not have argument "exc_value"
bitbangio.SPI.__exit__ is inconsistent, stub does not have argument "traceback"
bitbangio.SPI.configure is inconsistent, runtime argument "baudrate" is not keyword-only
bitbangio.SPI.configure is inconsistent, runtime argument "polarity" is not keyword-only
bitbangio.SPI.configure is inconsistent, runtime argument "phase" is not keyword-only
bitbangio.SPI.configure is inconsistent, runtime argument "bits" is not keyword-only
bitbangio.SPI.readinto is inconsistent, stub argument "buffer" differs from runtime argument "buf"
bitbangio.SPI.readinto is inconsistent, runtime does not have argument "end"
bitbangio.SPI.readinto is inconsistent, runtime does not have argument "start"
bitbangio.SPI.readinto is inconsistent, runtime does not have argument "write_value"
bitbangio.SPI.sys is not present at runtime
bitbangio.SPI.write is inconsistent, runtime does not have argument "end"
bitbangio.SPI.write is inconsistent, runtime does not have argument "start"
bitbangio.SPI.write_readinto is inconsistent, stub argument "out_buffer" differs from runtime argument "buffer_out"
bitbangio.SPI.write_readinto is inconsistent, stub argument "in_buffer" differs from runtime argument "buffer_in"
bitbangio.SPI.write_readinto is inconsistent, runtime does not have argument "in_end"
bitbangio.SPI.write_readinto is inconsistent, runtime does not have argument "in_start"
bitbangio.SPI.write_readinto is inconsistent, runtime does not have argument "out_end"
bitbangio.SPI.write_readinto is inconsistent, runtime does not have argument "out_start"
neopixel_write failed to import. NotImplementedError: Board not supported
pwmio failed to import. NotImplementedError: pwmio not supported for this board.
pulseio failed to import. NotImplementedError: pulseio not supported for this board.
busio.I2C.__exit__ is inconsistent, stub does not have argument "exc_type"
busio.I2C.__exit__ is inconsistent, stub does not have argument "exc_value"
busio.I2C.__exit__ is inconsistent, stub does not have argument "traceback"
busio.I2C.__init__ is inconsistent, runtime argument "frequency" is not keyword-only
busio.I2C.__init__ is inconsistent, runtime does not have argument "timeout"
busio.I2C.readfrom_into is inconsistent, runtime argument "end" has a default value of type None, which is incompatible with stub argument type builtins.int
busio.I2C.sys is not present at runtime
busio.I2C.writeto is inconsistent, runtime argument "end" has a default value of type None, which is incompatible with stub argument type builtins.int
busio.I2C.writeto_then_readfrom is inconsistent, stub argument "out_buffer" differs from runtime argument "buffer_out"
busio.I2C.writeto_then_readfrom is inconsistent, stub argument "in_buffer" differs from runtime argument "buffer_in"
busio.I2C.writeto_then_readfrom is inconsistent, runtime argument "in_end" has a default value of type None, which is incompatible with stub argument type builtins.int
busio.I2C.writeto_then_readfrom is inconsistent, runtime argument "out_end" has a default value of type None, which is incompatible with stub argument type builtins.int
busio.I2C.writeto_then_readfrom is inconsistent, stub does not have argument "stop"
busio.Parity is not present at runtime
busio.SPI.__exit__ is inconsistent, stub does not have argument "exc_type"
busio.SPI.__exit__ is inconsistent, stub does not have argument "exc_value"
busio.SPI.__exit__ is inconsistent, stub does not have argument "traceback"
busio.SPI.__init__ is inconsistent, runtime does not have argument "half_duplex"
busio.SPI.configure is inconsistent, runtime argument "baudrate" is not keyword-only
busio.SPI.configure is inconsistent, runtime argument "polarity" is not keyword-only
busio.SPI.configure is inconsistent, runtime argument "phase" is not keyword-only
busio.SPI.configure is inconsistent, runtime argument "bits" is not keyword-only
busio.SPI.frequency is read-only at runtime but not in the stub
busio.SPI.readinto is inconsistent, stub argument "buffer" differs from runtime argument "buf"
busio.SPI.readinto is inconsistent, runtime argument "start" is not keyword-only
busio.SPI.readinto is inconsistent, runtime argument "end" is not keyword-only
busio.SPI.readinto is inconsistent, runtime argument "write_value" is not keyword-only
busio.SPI.sys is not present at runtime
busio.SPI.write is inconsistent, stub argument "buffer" differs from runtime argument "buf"
busio.SPI.write is inconsistent, runtime argument "start" is not keyword-only
busio.SPI.write is inconsistent, runtime argument "end" is not keyword-only
busio.SPI.write_readinto is inconsistent, stub argument "out_buffer" differs from runtime argument "buffer_out"
busio.SPI.write_readinto is inconsistent, stub argument "in_buffer" differs from runtime argument "buffer_in"
busio.SPI.write_readinto is inconsistent, runtime argument "out_start" is not keyword-only
busio.SPI.write_readinto is inconsistent, runtime argument "out_end" is not keyword-only
busio.SPI.write_readinto is inconsistent, runtime argument "in_start" is not keyword-only
busio.SPI.write_readinto is inconsistent, runtime argument "in_end" is not keyword-only
busio.UART.__exit__ is inconsistent, stub does not have argument "exc_type"
busio.UART.__exit__ is inconsistent, stub does not have argument "exc_value"
busio.UART.__exit__ is inconsistent, stub does not have argument "traceback"
busio.UART.__init__ is inconsistent, stub argument "tx" has a default value but runtime argument does not
busio.UART.__init__ is inconsistent, stub argument "rx" has a default value but runtime argument does not
busio.UART.__init__ is inconsistent, runtime argument "baudrate" is not keyword-only
busio.UART.__init__ is inconsistent, runtime argument "bits" is not keyword-only
busio.UART.__init__ is inconsistent, runtime argument "parity" is not keyword-only
busio.UART.__init__ is inconsistent, runtime argument "stop" is not keyword-only
busio.UART.__init__ is inconsistent, runtime argument "timeout" is not keyword-only
busio.UART.__init__ is inconsistent, runtime argument "receiver_buffer_size" is not keyword-only
busio.UART.__init__ is inconsistent, stub does not have argument "flow"
busio.UART.__init__ is inconsistent, runtime does not have argument "cts"
busio.UART.__init__ is inconsistent, runtime does not have argument "rs485_dir"
busio.UART.__init__ is inconsistent, runtime does not have argument "rs485_invert"
busio.UART.__init__ is inconsistent, runtime does not have argument "rts"
busio.UART.readinto is inconsistent, stub does not have argument "nbytes"
busio.UART.reset_input_buffer is not present at runtime
rainbowio.colorwheel is inconsistent, stub argument "n" differs from runtime argument "color_value"
board failed to import. Maybe install the runtime package or alter PYTHONPATH? ModuleNotFoundError: No module named 'pkg_resources'

[makermelissa]

That's pretty cool. I probably won't have time to work on it this week, but after making it more compatible, we can push a new major version as it may break peoples' code, but that's ok. We want it to be compatible. Having a check like this in place will help with:

1. Keeping up with CircuitPython function parameter changes
2. Preventing users from making changes to Blinka that would stray away from compatibility.