How to capture long exposure stills one after each other without delay between captures

[meteorkees]

Hi,
i would like to take long exposure stills with my HQ camera during the night with fixed gain, awb, and exposure-time settings and no preview, without missing any second.
A: is this possible?
B: which mode to use?
C: in libcamera documentation i found the mode Timelapse, but not in the picamera2 documentation?
D: is timelapse the right one?
E: In the terminal I use now this cmd to take one still: Any suggestions to improve/use in take long exposure stills with a timestamp?
libcamera-still --datetime --shutter 20000000 --gain 8 --awbgains 2,1.81 --immediate --denoise cdn_hq

Many thanks in advance?

Kees.

[meawoppl]

Hello hello! This is a use case for us as well (hello fellow sky observer!). This isn't fully possible with picamera2 right now, as there are some less than solid guarantees about the internal request processing event loop. Specifically, you can't ensure that frames aren't dropped from the libcamera request cycle. We @Exclosure are doing some work to bring that all to a better state, but there are some short-term options I can point you at if you are interested.

[meteorkees]

Hello Meawoppl, thx for your reply. Yes i am interested. But keep in mind that i am a 'software user' and not a developer. My plan is to take captures with a allsky cam, day and night. Goal is to capture major events during the day and night, like capturing a fireball. Key is not to miss any second and thus not to miss any 'big' event, if possible.. ??
I want to experiment with the arducam HQ IR-cut camera on a Raspberry compute module 4 IO board. This board supports 4 lane CSI-2 ports.
https://www.arducam.com/product/arducam-high-quality-ir-cut-camera-for-raspberry-pi-12-3mp-1-2-3-inch-imx477-hq-camera-module-with-6mm-cs-lens-for-pi-4b-3b-2b-3a-pi-zero-and-more/

I thought that this could maybe a good feature regarding 'fast saving data' after a capture, and keep taking captures at the same time, or something like that. It is more important to not miss any (milli)second then the quality of the image in MB pixels.

[wardhills]

metorkees, meawoppi, Thanks for the question and reply.

meawoppi, I would be very interested learning more about the short term options to which you refer.

[meteorkees]

I am also curious about the "short-term options" :-)

[davidplowman]

Hi, your libcamera-still command looks fine to me. You could do it as a timelapse, and I expect the results would essentially be the same:

libcamera-still -t 0 -n --shutter 20000000 --gain 8 --awbgains 2,1.81 --timelapse 1 --datetime

I'd expect this to do a capture every (just over) 20 seconds (obviously the minimum given the exposure time). There will be a small gap between each capture when the camera switches between modes, though you could use libcamera-vid to avoid that:

libcamera-vid -t 0 -n --shutter 20000000 --gain 8 --denoise cdn_hq --segment 1 --codec mjpeg --width 4056 --height 3040 -o image%04d.jpg

In this case you'd have to write a small script to rename the files according to their creation dates.

If you wanted to use Picamera2 I would try something like this:

from picamera2 import Picamera2
from datetime import datetime

picam2 = Picamera2()
config = picam2.create_still_configuration(buffer_count=2)
picam2.configure(config)
picam2.set_controls({"ExposureTime": 20000000, "AnalogueGain": 8, "ColourGains": (2, 1.81)})
picam2.start()

while True:
    picam2.capture_file(datetime.now().strftime("%H-%M-%S.jpg"))

Again, I'd expect this to give you a frame every 20 seconds. Picamera2 does have a start_and_capture_files function, but I expect most people would prefer to write the loop for themselves.

[meteorkees]

Thx for this example. I did a quick trial with the Picamera2 script, and I see every x seconds a image getting saved. I did not check yet if any data slips through, but it looks ok to me. I can go ahead with this.
One thing I do not understand is 'buffer_count=2'. Reading the Picamera2 manual did not help for me. Does it make a big difference in performance if the bufer_count is increased for instance to 4?

Many thxs!

[davidplowman]

To explain a bit more...

Using the libcamera-still "timelapse" method I would expect there to be about a second (maybe a bit more) between captures that you would "miss" because of all the mode switching and stopping/starting of the camera. With both the Picamera2 and the libcamera-vid methods, I would expect this gap to be of the order of a millisecond, or even less, because the camera is simply free-running.

The manual does talk a bit about the buffer count, briefly in section 4.2.1.3 and again under the more "Advanced Topics" in section 8.4.

But the summary is that, because the next frame is already being exposed while the previous one is being read out, the camera system may require a buffer to be available to receive the new frame even while you're still using the buffer with the previous frame. If that new buffer isn't available, that frame will get dropped, and your 20s per image will instantly double to 40s. To be honest, it all depends a bit on the precise timings of everything within the system, but I think buffer_count=2 will guarantee that the next buffer is always available.

Using a higher buffer count is at worst harmless (so long as the memory is available). At higher framerates we would recommend larger numbers, but at low framerates I don't think you would see any differences versus 2 buffers. In the end I would recommend trying it out and checking the behaviour that you get!

[fuherb]

Hi, I just find this thread and think this is very interesting. I am using an pi hq camera for astrophotography and EAA. Current I am using a python script I found in the web to control the image capturing. I normally capture 10-15 images in 1 loop, then do a quick check and run another cycle. I am calling out libcamera-still as a process everytime and that is quick inefficient.
I tried the timelapse example from above and it works. Normally I add --immediate option to libcamera-still to suppress the calibration process to shorten the image capturing time. In this timelapse script the --immedate option does not work. It will shut down the action after 1 image. Without the --immediate option, then the cycle time becomes very long. For example to take a 3s image, the cycle time is about 15s. Have I missed anything?

[ScottTheHippy]

Howdy gang, I'm following this helpful guidance and am getting a result that is bending my noodle. My command line is using the libcamera-still method that David suggested, and I've tweaked it a bit for my light levels (here is the specific syntax: libcamera-still -t 0 -n --timelapse 1 --shutter 15000000 --gain 8 --awbgains 1,1 --contrast 4 --datetime) and what happens is I get four or five lovely images, and then not so lovely images thereafter. I've attached examples so you can see what I'm trying to describe. One has more stars shown and is clearer than the other. I thought maybe this was a deal where the delay between shots might be exhausting a buffer, so I've tried --timelapse 2000 per the documentation default and that didn't seem to make any difference. Do you all have any idea what I'm up against? The hardware is RPi 4 4gb and a v3 camera with no IR filter. I'm quite pleases with the good image, and would like to collect those all night if I can.

Thank you thank you thank you!

[davidplowman]

Hi, thanks for the question. I wonder if it may be related to the fact that you're using a v3 camera with autofocus? By default libcamera-still will start in continuous autofocus mode which means the lens is prone to start refocussing spontaneously - and this probably won't work too well with images like this. Maybe add --lens-position 0 to your command to fix the lens at infinity and see if that changes things? Thanks!

[ScottTheHippy]

Howdy David, thank you for the suggestion. I thought I'd give it a test now as it's still dark out here, and alas, clouds have moved in. Grr! I'll give it a try later when I get a clear view and see what that does. Honestly- I'm not sure that I ever would have thought of that as a potential variable. It does kind of seem like a viable answer. I was initially thinking it was more to do with light sensitivity, but as I look at the pictures from earlier, yeah- that could be a focus change that's causing some stars to get lost. Crazy. I'll let you know as soon as I'm able to properly retest, but in the meantime, here's some atomic looking clouds that are passing through. :) And thank you again- seriously. I was properly stumped.

[ScottTheHippy]

Another cloudy night here, but it's not like last night- there is a few stars visible. I think you might be on target with the autofocus being the source of the inconsistency. While this evenings shots aren't worth looking at for content, they are consistent at least. I'll keep at it, but I think you're a super-genius David :) Thanks again!