PupilEXT Remote Hardware development

[kheki4]

Hi Everyone!

I have been /am designing a modular hardware for remote pupillometer/eye tracker usage of PupilEXT. This will be a casing, that houses the cameras and necessary circuitry for IR LED illuminators and their driver. The elements will be 3D printable, but for safety reasons, the technology will not be FDM due to the low melting point of its materials. As instead, the models are designed for best compatibility with SLS printing - even though the price of the printer is higher, printing from China is almost the same price for FDM and SLS for small parts, so the latter is very much worthy for its better physical properties.

Modular casing

My main idea is a modular system of elements that encapsulate different functionality and can be assembled together to produce a remote (a.k.a. desk-top) system.
For example:

• a module for containing the camera(s) in straight or angled variations, with or without heatsinks attached around them
• a module for containing the IR LEDs with their heatsink and a fan for active cooling (Passive cooling was also considered, but abandoned for now. A good one would need a very very large heatsink, or possibly a custom-machined metal block, which is not yet in scope. Maybe later)
• a module for IR LED driver circuitry (more on this below)
• a module to carry the microcontroller unit (MCU) that syncs image acquisition of stereo cameras, and also controls the IR illuminator driver (to be switched on/off from software)
• a module for mounting the device in various ways: standing on adjustable angled legs, or screwable on the top to a monitor, monting on GoPro-mount node(s), etc.
• various other modules for possible extended use cases:
  • one containing a kensington anti-theft locking hole,
  • one to house a widely available w5500 module to extend the connectivity of the MCU with an ethernet interface, for controlling the device from long distances,
  • a module for containing an additional small fan, if the device is used in a hot environment, etc.

Here are a few render pictures of the basic concept (These are a bit outdated now, but show the main idea):

These modules could be assembled in different constellations, to be used with PupilEXT software, as its capabilities get extended in the future. For example, one could simply put together a version consisting of 1 camera and 1 IR light source. But if they need better accuracy, they could build a 2 camera version with 1 IR illuminator. And also if someone tries to reach better camera-head distance measurements while not wanting to buy another camera for a stereo setup, they could make e.g. a 1 camera 2 IR illuminator setup... (though this lastly mentioned case would lower the reliability of pupil detection as the chance of disrupting the pupil circumference by a corneal reflection is twice as big as in case of a single-IR version). Also as @sasfog and me are currently working on incorporating gaze tracking to the software, it could be interesting to observe what tradeoffs we see in the better distance approximation vs. worse pupil detection confidence when using 2 IR sources. Actually I am more into improving PupilEXT for pupillometry, but I know gaze-tracking and eye event detection would also be crucial for better artefact rejection, and ultimately would add a huge plus to the current pupillometry use of this software (and also to involve more people to collaborate to the project in the future). But this post is not about the software, so I am getting back to hardware development in the following paragraphs :)

I have already made three different iterations of most modules, and now they are waiting for the finishing moves. However, the IR illumination driver circuitry is still under design (described below). I also came up with the idea that PupilEXT software should be able to switch the illuminator on/off via the microcontroller unit, which functionality we will add to a future release of the software. Furthermore, the MCU could also monitor the temperature of the LEDs and the driver as well, so that PupilEXT could inform the experimenter that the device has reached a thermal equilibrium with its environment and is likely fit for best reliable pupil measurement ("warm-up" done).

Illuminator driver

My father and me are designing a little driver circuit to empower a group of IR (actually NIR) LEDs to provide proper independent lighting for the PupilEXT hardware.

So far we have tried many different ways of driving IR illuminator LEDs:

• 1, constant voltage solution. This proved to be inappropriate due to thermal runaway, which causes the LED light emission to change on the fixed voltage.
• 2, constant voltage but pulse-width modulation (PWM) (actually achieved with a switching-mode power supply (SMPS) circuit) to reach the best driver efficiency, which proved successful. However, these can also exhibit the runaway actually, and also on/off cycles have a little bit of wear effect on the LEDs expected life. Moreover, it is important that if we do not want to stick to ready-made chinese SMPS boards, and also maximize size and shape compatibility with the device casing, we should make an own PCB... though SMPS design is a fairly tricky mastery, which we still did not acquire completely (it needs a bit of high-frequency knowledge, and for best efficiency, would need simulations as well). Our DIY versions failed to reach even the efficiency of some noname cheap ready-made boards, but also were far from that of the maximum possible (described by TI and other SMPS regulator IC manufacturers). Even if we use a cheap pre-made SMPS board for this, they need a little bit of calibration individually, which makes the whole assembly process complicated. So we currently halted our efforts in this direction, though we might get back to it later. (Note that even if PWM is still compatible with EEG or other high-frequency instruments possibly operating in proximity in a typical cognitive science lab setup, as the modulation frequency is in a much higher range so that they could not interfere with those devices.)
• 3, constant current LED drivers. They virtually eliminate thermal runaway, which minimizes system-drift of the pupillometer/eye tracker, i.e. leaving your device on for half an hour for "warm-up" will not have an enourmus effect on measured pupil diameter. But the backside is that these constant-current drivers that we can build, which rely on a transistor/IC are to regulate the current, are not really efficient. This means, they generate a considerable amount of heat if the difference between the input voltage and the current-controlled output voltage is large (but also depending on how much the output current is). They likely need a huge heatsink, but the situation is not so dramatic that active cooling (a fan) would be needed hopefully, as long as the environmental temperature is considered "room temperature", being between 18-35° Celsius. However, we did not perform exact calculations to locate boundaries yet (as we did not choose a standard heatsink and transistor yet, though TI LM338 is our current/latest candidate to be honest).
• 4, a hybrid version that applies an SMPS before the constant-current driver, to minimise the voltage difference, and thus maximise the efficiency of the driving element. However, we ditched this version, as it is too complicated to build (see my previous thoughts on this), also expensive, and takes up a lot more space in the device casing.

... so we ended up choosing the 3rd option: It heats a little bit, but that is still tolerable. And actually no matter what we do, the "weakest" part of the system regarding efficiency and useless heat generation, will always be the IR LED (or LEDs), and not this driver in my opinion. And since we cannot improve the LEDs, and already have to stick to an active cooling solution, the driver is not much of a design concern for further optimization yet.

Note: we did not experiment with camera image acquisition-locked illumination driving, though it could be an efficient method also, worth a look in the future.

The illuminator unit

Regarding the type of LEDs, we have also gone through a journey:
First we tried a huge matrix of cheap low-power, standard 5mm dia. thru-hole SMT packaged LEDs, around ~850nm centroid wavelength, which:
+ have the advantage of better distribution of heat which is easier to dissipate over their large surface, even without an additional heatsink, but...
- inevitably cause a large corneal-reflection trace on the pupil image, which lowers the efficient circumference length used to fit the ellipse for pupil detection, which is unfavourable for us, as our main goal is high quality pupillometry.
- are not really easy to reliably source. Better manufacturers have already drifted towards SMD packaging, and no matter how many manufacturers there are that produce mainstream packaged low-power LEDs, they hugely vary regarding their exact spectral emission profile, radiation angle characteristics, and ultimately how bright and uniformly they can illuminate the eye

... so in the end, these are not the best option.

A smaller matrix of high-power, standard 5mm dia. thru-hole SMT packaged LEDs, around ~850nm centroid wavelength, which:
+ are in most aspects more favorable than their low-power alternatives, regarding corneal reflection trace size, and sourcing reliability, though...
- are slowly phased out by SMD-packaged alternatives
- generate concentrated heat, but this heat cannot be well transferred to a heatsink, due to the through-hole solder joints, and applicable glass-fiber PCB material used

... not be best option either.

A small group of SMD packaged LEDs, namely the Osram SFH 4717, with reliable 850nm centroid, which:
+ can give a pinpoint corneal reflection trace, which is preferred for pupillometry
+ generate concentrated heat, which can be properly transferred to a heatsink, using alimunium PCB material
- are expensive and bound to one manufacturer, though this type may be so widespread in future that equivalent models will possibly be available from other manufacturers

... so we opted for this.

Once we chose the SFH 4717 type LED, we tried different constellations:

• 1 of this LED: simply cannot deliver the needed maximum illumination from 58cm viewing distance, even when driven at maximum applicable current, which is delivered by branded eye-trackers (We measured the LED light output from a standard distance, and compared it to other branded eye-trackers from that same distance, using measurements with an uncalibrated, NIR-sensitive photodiode-based, custom-made device, under environmental parameters kept constant.) Of course, it generated an unsafe amount of heat.
• 2 of this LED: could actually reach the needed maximum needed illumination, though still a heavy amount of heat was generated per LED, and this use would likely shorten the life of the diodes in the long run
• 3 of this LED: is a feasible solution, and heat concentration was tolerable per LED, but still needed active cooling at the LEDs. This could run fine with a 15v power-delivery profile of an USB-C PD capable power-supply, with enough additional voltage that constant-current driving is still possible. Driving these constant-current generates heat on the driving transistor which has to be dissipated using a large heatsink, but not necessarily using a fan.
• 4 of this LED: similar to the 3 LED constellation, but with lower heat per LED, and lower current on the serially connected 4 LEDs, also a likely longer LED life accordingly. This could run fine with a 20v power-delivery profile of an USB-C PD capable power-supply, with enough additional voltage that constant-current driving is still possible. However, driving these constant-current generates much heat on the driving transistor that it has to be dissipated using a large heatsink, but not necessarily using a fan.
  (Also, if one would like to use a first prototype of the device and try the illuminator without a specific driver, the 4 LED illuminator is optimal, as the needed illumination is produced at ~12 volts with it, thus any standard PC power supply can be used to power the LEDs - though there will be a considerable thermal runaway, as described earlier, and just barely running the LEDs with a 12v power source is not advised for real measurements in that case.)

... so we settled at the 4 LED version. (Edit: which is anyway quite similar to what Zandi et al., 2021 show in their PupilEXT debut paper.)

Plans for now

Currently, we are prototyping and adding:

• a custom, smaller board for the MCU, because Nucleo dev/eval boards are rather large, and also a little costly. Even Arduino would also be capable, though I am planning now with the STM32
• USB-C power delivery compatibility (for use with a standard laptop power supply for feeding the illuminator and its driver)
• overheating protection
• overcurrent protection
• ESD safety /transient voltage supression
• ability to switch on/of the illuminator from software (via the STM32 MCU also responsible for stereo camera sync)
• continous illuminator and driver temperature monitoring from software (via the MCU as well)
• (not yet decided) ability to switch between different illumination intensity levels (also via the MCU from software)
• the optimal hardware design for: smallest footprints in the casing, while maintaining best airflow for heatsinks and the PCBs

These developments are on the way. I think we could finish the hardware this fall or until next spring. I will be coming with updates as we proceed.

[kheki4]

Here is a brief update on hardware progress! :)

We started using a version of the device without its own operating IR illuminator, but assembled in the designed housing. It turned out that several fixes were needed for better rigidity and easier setup/use. So I re-iterated on the modules and applied several changes.

• The holder legs were redesigned to implement a basic mechanism, to allow instant angle changes in coarse and fine scales. Still, a contraweight can be fitted in this element to procide a better weight distribution of the enclosure. (Excuse me for the behemoth-sized machine leg on the picture, that as the only one free I found for a pic at that moment, but of course there are a lot more sleek versions of swivel joint angled legs with metric thread.)
• The heatsink element enclosure was changed to allow access to the 1/4" camera mount screws from the bottom.
  Also, as this plastic element is is only fitted to the official Ace 2 camera series heatsinks produced for Basler, I was thinking whether it is possible to be transformed to gain compatibility with the older Ace series Basler cameras. It turned out that it is completely possible: by only making a few small changes by CNCing (or precision drilling) on the Ace 2 version. I will soon make a mechanical drawing for that.
  (Although after all, the Basler Ace 2 heatsink is a simple alu-profile, drilled in a custom constellation, so it is not too hard to fabricate a version for any camera in the future. Its base is a classic heatsink profile, can also be bought from Fischer: https://www.fischerelektronik.de/web_fischer/en_GB/heatsinks/A01/Standard%20extruded%20heatsinks/PG/SK08/index.xhtml)
• I designed and made a preliminary version of the MCU controllable IR driver circuit, with its necessary power management elements. There is still a lot of room for improvement, but this is fair for a start I think. It will get connected to the MCU module using a sort of internal "bus", physically through a simple board-to-board 2.54 pin header block, or using IDC ribbon cables... according to the actual hardware constellation. The basic 1 camera 1 IR variant will definitely just connect by the pin header in a sandwich fashion. The airflow that is richochet off the angled IR LED heatsink will be then passing through the driver, and then the MCU board. The driver will work nicely with a typical 65W USB-C laptop charger as power supply.
• I designed a new version of the angle-adjustable IR LED + heatsink holder enclosure. The newer variant tackles the problem of undesired internal light reflection, that could arise by the non-perpendiculariy of the LEDs and the IR window glass. This newer version will also allow adding C-mount focusing lens instead of the simple IR pass glass window.

The next plans are to optimize the electronics further, and stress-test the whole assembly. The 3D printable elements are almost completely done, I think. However, I am to test a few very little fine tunings in one next iteration, before calling it done. In the meanwhile I am to head for adding the MCU finally, which will need also slight changes to the software, to properly identify and control the device once plugged. I will keep you updated once there is further progress!