one cause of frozen scans - using ct_dark_all() plan with fast acquition times - intermittent issue #38

ambarb · 2020-07-31T23:09:52Z

Versions of DAMA software:
- Bluesky : v1.6.3
- Ophyd : v1.5.1
- Databroker : v1.0.6

This problem is old and not consistently occurring. removed all .value in January and that did not help. seems to happen when exposure time is short and "network" robustness.

successful run has the setpoint never change (camonitor) but the readback value does change

[xf23id1@xf23id-ws3 ~/Beamline/Commissioning/2020_07]$ camonitor XF:23ID1-ES{FCCD}FastCCD1:CaptureBgnd_RBV
XF:23ID1-ES{FCCD}FastCCD1:CaptureBgnd_RBV 2020-07-31 18:41:04.261963 Off  ### was the immedate after Enter
XF:23ID1-ES{FCCD}FastCCD1:CaptureBgnd_RBV 2020-07-31 19:06:06.197300 On  ### happened with plan start
XF:23ID1-ES{FCCD}FastCCD1:CaptureBgnd_RBV 2020-07-31 19:06:06.272830 Off

waiting for it to fail again to capture camonitor


In [301]: RE(mv(fccd.exposure, (0.02, .1, 20)))                                                                                                                
Out[301]: ()

In [302]: RE(bpp.pchain(ct_dark_all(20),rel_scan([fccd],nanop.by,-.1,0.1,201, md={'purpose':'ZP zoom 1/8 cut'})))                                              

Starting procedure to acquire darks 36.4Hz or 0.028s.

	Current number of images = 20.

	Setting to 20 images.

  A 'deferred pause' has been requested. The RunEngine will pause at the next checkpoint. To pause immediately, hit Ctrl+C again in the next 10 seconds.       
Deferred pause acknowledged. Continuing to checkpoint.
^Ctrying a second time
Pausing...
---------------------------------------------------------------------------
RunEngineInterrupted                      Traceback (most recent call last)
~/Beamline/Commissioning/2020_07/MonoStab.py in <module>
----> 1 RE(bpp.pchain(ct_dark_all(20),rel_scan([fccd],nanop.by,-.1,0.1,201, md={'purpose':'ZP zoom 1/8 cut'})))

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py in __call__(self, *args, **metadata_kw)
    796 
    797         if self._interrupted:
--> 798             raise RunEngineInterrupted(self.pause_msg) from None
    799 
    800         return tuple(self._run_start_uids)

RunEngineInterrupted: 
Your RunEngine is entering a paused state. These are your options for changing
the state of the RunEngine:

RE.resume()    Resume the plan.
RE.abort()     Perform cleanup, then kill plan. Mark exit_stats='aborted'.
RE.stop()      Perform cleanup, then kill plan. Mark exit_status='success'.
RE.halt()      Emergency Stop: Do not perform cleanup --- just stop.


In [303]: RE.abort()                                                                                                                                           
Aborting: running cleanup and marking exit_status as 'abort'...

Returning to intial conditions (pre-count).
        Total images per trigger are NOW:        20                                                                                                            
	FCCD FCRIC gain value is NOW:		 auto


Run aborted
Traceback (most recent call last):
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py", line 1353, in _run
    msg = self._plan_stack[-1].send(resp)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 1307, in __call__
    return (yield from plan)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 1162, in baseline_wrapper
    return (yield from plan_mutator(plan, insert_baseline))
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 803, in monitor_during_wrapper
    return (yield from plan2)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 861, in fly_during_wrapper
    return (yield from plan2)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 268, in pchain
    rets.append((yield from p))
  File "/epics/xf/23id/ipython/profile_collection/startup/csx1/plans/dark_ff.py", line 190, in ct_dark_all
    yield from bps.mv(fccd.fccd1.capture_bgnd, 1)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/plan_stubs.py", line 258, in mv
    yield Msg('wait', None, group=group)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
    inner_ret = yield msg
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
    inner_ret = yield msg
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
    inner_ret = yield msg
  [Previous line repeated 2 more times]
bluesky.utils.RequestAbort
Out[303]: ()

In [304]: RE(bpp.pchain(ct_dark_all(20),rel_scan([fccd],nanop.by,-.1,0.1,201, md={'purpose':'ZP zoom 1/8 cut'})))                                              

Starting procedure to acquire darks 36.4Hz or 0.028s.

	Current number of images = 20.

	Setting to 20 images.

                                                                                                                                                               
Returning to intial conditions (pre-count).
        Total images per trigger are NOW:        20                                                                                                            
	FCCD FCRIC gain value is NOW:		 auto


Run aborted
Traceback (most recent call last):
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py", line 1353, in _run
    msg = self._plan_stack[-1].send(resp)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 1307, in __call__
    return (yield from plan)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 1162, in baseline_wrapper
    return (yield from plan_mutator(plan, insert_baseline))
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 803, in monitor_during_wrapper
    return (yield from plan2)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 861, in fly_during_wrapper
    return (yield from plan2)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 170, in plan_mutator
    raise ex
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 123, in plan_mutator
    msg = plan_stack[-1].send(ret)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 268, in pchain
    rets.append((yield from p))
  File "/epics/xf/23id/ipython/profile_collection/startup/csx1/plans/dark_ff.py", line 190, in ct_dark_all
    yield from bps.mv(fccd.fccd1.capture_bgnd, 1)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/plan_stubs.py", line 256, in mv
    ret = yield Msg('set', obj, val, group=group, **kwargs)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
    inner_ret = yield msg
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
    inner_ret = yield msg
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py", line 195, in plan_mutator
    inner_ret = yield msg
  [Previous line repeated 2 more times]
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py", line 1413, in _run
    new_response = await coro(msg)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py", line 1909, in _set
    ret = msg.obj.set(*msg.args, **kwargs)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/ophyd/signal.py", line 1670, in set
    return super().set(value, timeout=timeout, settle_time=settle_time)
  File "/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/ophyd/signal.py", line 301, in set
    raise RuntimeError('Another set() call is still in progress '
RuntimeError: Another set() call is still in progress for fccd_fccd1_capture_bgnd
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
~/Beamline/Commissioning/2020_07/MonoStab.py in <module>
----> 1 RE(bpp.pchain(ct_dark_all(20),rel_scan([fccd],nanop.by,-.1,0.1,201, md={'purpose':'ZP zoom 1/8 cut'})))

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py in __call__(self, *args, **metadata_kw)
    793             self._task_fut.add_done_callback(set_blocking_event)
    794 
--> 795         self._resume_task(init_func=_build_task)
    796 
    797         if self._interrupted:

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py in _resume_task(self, init_func)
    917                     if (exc is not None
    918                             and not isinstance(exc, _RunEnginePanic)):
--> 919                         raise exc
    920 
    921     def install_suspender(self, suspender):

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self)
   1486             exit_reason = str(err)
   1487             self.log.exception("Run aborted")
-> 1488             raise err
   1489         finally:
   1490             if not exit_reason:

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self)
   1351                     else:
   1352                         try:
-> 1353                             msg = self._plan_stack[-1].send(resp)
   1354                         # We have exhausted the top generator
   1355                         except StopIteration:

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in __call__(self, plan)
   1305         plan = monitor_during_wrapper(plan, self.monitors)
   1306         plan = baseline_wrapper(plan, self.baseline)
-> 1307         return (yield from plan)
   1308 
   1309 

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name)
   1160         return (yield from plan)
   1161     else:
-> 1162         return (yield from plan_mutator(plan, insert_baseline))
   1163 
   1164 

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals)
    801     plan1 = plan_mutator(plan, insert_after_open)
    802     plan2 = plan_mutator(plan1, insert_before_close)
--> 803     return (yield from plan2)
    804 
    805 

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers)
    859     plan1 = plan_mutator(plan, insert_after_open)
    860     plan2 = plan_mutator(plan1, insert_before_close)
--> 861     return (yield from plan2)
    862 
    863 

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    168                     continue
    169                 else:
--> 170                     raise ex
    171         # if inserting / mutating, put new generator on the stack
    172         # and replace the current msg with the first element from the

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    121             ret = result_stack.pop()
    122             try:
--> 123                 msg = plan_stack[-1].send(ret)
    124             except StopIteration as e:
    125                 # discard the exhausted generator

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in pchain(*args)
    266     rets = deque()
    267     for p in args:
--> 268         rets.append((yield from p))
    269     return tuple(rets)
    270 

/epics/xf/23id/ipython/profile_collection/startup/csx1/plans/dark_ff.py in ct_dark_all(numim, detectors)
    188         # SET TO 1 TO ARM FOR NEXT EVENT so that the FastCCD1 is
    189         # already bkg subt
--> 190         yield from bps.mv(fccd.fccd1.capture_bgnd, 1)
    191 
    192         # take darks

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/plan_stubs.py in mv(group, *args, **kwargs)
    254     (step,) = utils.merge_cycler(cyl)
    255     for obj, val in step.items():
--> 256         ret = yield Msg('set', obj, val, group=group, **kwargs)
    257         status_objects.append(ret)
    258     yield Msg('wait', None, group=group)

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc)
    193         try:
    194             # yield out the 'current message' and collect the return
--> 195             inner_ret = yield msg
    196         except GeneratorExit:
    197             # special case GeneratorExit.  We must clean up all of our plans

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self)
   1411                         # exceptions (coming in via throw) can be
   1412                         # raised
-> 1413                         new_response = await coro(msg)
   1414 
   1415                     # special case `CancelledError` and let the outer

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/bluesky/run_engine.py in _set(self, msg)
   1907         group = kwargs.pop('group', None)
   1908         self._movable_objs_touched.add(msg.obj)
-> 1909         ret = msg.obj.set(*msg.args, **kwargs)
   1910         p_event = asyncio.Event(loop=self.loop)
   1911         pardon_failures = self._pardon_failures

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/ophyd/signal.py in set(self, value, timeout, settle_time)
   1668         '''
   1669         if not self._put_complete:
-> 1670             return super().set(value, timeout=timeout, settle_time=settle_time)
   1671 
   1672         # using put completion:

/opt/conda_envs/collection-2020-2.0rc4/lib/python3.7/site-packages/ophyd/signal.py in set(self, value, timeout, settle_time)
    299 
    300         if self._set_thread is not None:
--> 301             raise RuntimeError('Another set() call is still in progress '
    302                                f'for {self.name}')
    303 

RuntimeError: Another set() call is still in progress for fccd_fccd1_capture_bgnd
n [305]:                                                                                                                                                      

In [305]: fccd.fccd1.capture_bgnd              
                                                                                                                
Out[305]: EpicsSignalWithRBV(read_pv='XF:23ID1-ES{FCCD}FastCCD1:CaptureBgnd_RBV', name='fccd_fccd1_capture_bgnd', parent='fccd_fccd1', value=0, timestamp=1596235264.261963, auto_monitor=False, string=False, write_pv='XF:23ID1-ES{FCCD}FastCCD1:CaptureBgnd', limits=False, put_complete=False)

The text was updated successfully, but these errors were encountered:

ambarb · 2020-12-22T17:01:50Z

related to #25

ambarb · 2020-12-22T19:36:09Z

bluesky/bluesky#1159 other useful discussion.

mrakitin · 2021-03-26T15:59:03Z

It's a bit hard to fully understand the issue without the source code from ~/Beamline/Commissioning/2020_07/MonoStab.py. I think we should develop an approach to keep these files in an accessible place. If there is some private information, there can be a private GitHub repo for these things. Here is a similar repo for the SIX beamline, which made it more convenient to provide support: https://github.com/NSLS-II-SIX/science_commissioning.

ambarb · 2021-03-26T16:03:21Z

@mrakitin I don't understand. the issue I report is using bpp.pchain. It doesn't come from a script or ,py file

but you are correct in that this GitHub goodness is missing form commissioning plans. I've allocated a path on the main workstation but haven't had time to creat the repo. Are there any guidelines form DSSI?

We want the repo to be independent from the profile

mrakitin · 2021-03-26T17:31:40Z

@mrakitin I don't understand. the issue I report is using bpp.pchain. It doesn't come from a script or ,py file

Got it. I was confused by the following lines after a quick look at the error:

RunEngineInterrupted                      Traceback (most recent call last)
~/Beamline/Commissioning/2020_07/MonoStab.py in <module>

but you are correct in that this GitHub goodness is missing form commissioning plans. I've allocated a path on the main workstation but haven't had time to creat the repo. Are there any guidelines form DSSI?

We want the repo to be independent from the profile

I think that can be a repo with the name of your preference. The current Commissioning directory can easily be made a repo with just git init and git add <...> commands. I can assist with that.

ambarb · 2021-05-18T03:47:21Z

Following up. We discussed at the beamline and @mrakitin suggested we use the facility two-button shutter ophyd device because it is heavily tested and more robust.

our two-button shutter is based off of the same ioc, but the strings associated with the state are "In" and "Out" and "Inserted" and "Not Inserted". @mrakitin liked other aspects of the ophyd device too but he was not sure that we should try to "fix" the facility two-button shutter since it could potentially break a lot for a lot of people.

The inout device is used to block the beam for dark image collection. ct_darks_all()

Three options:

integrated facility two-button shutter key points into our two-button shutter
change atleast 4 on CSX from "IN" / "OUT" two-button shutter ioc / eps (CSX has two chassis for itself, so this is why EPS) to match standard facility EPS 2-button shutter
make a new version of a "better" two-button shutter (more generalized) and roll it out with V1 still available.

mrakitin · 2021-05-18T14:43:07Z

@ambarb, thanks for documenting more details about the issue!

ambarb · 2022-01-27T20:29:47Z

will update with a scan number and the full traceback as there is more information now.

ambarb · 2022-07-03T22:01:58Z

looks like we should check also exception handling when inout fails. To be clear, inout is not failing; what happens is bluesky thinks it failed.

generator count_flatfield ['0bb2fe57'] (scan num: 156290)
	Setting to 200 images.

** (2022-07-03 16:13:40) Had to reactuate shutter while Ining                                                                                      
** (2022-07-03 16:13:40) Had to reactuate shutter while Ining
                                                                                                                                                   
Returning to intial conditions (pre-count).
An exception has occurred, use '%tb verbose' to see the full traceback.
FailedStatus: DeviceStatus(device=inout, done=True, success=False)

During handling of the above exception, another exception occurred:
RuntimeError: trying to set while a set is in progress

See /var/log/bluesky/bluesky.log for the full traceback.

Versions of DSSI software:

- Bluesky    : v1.8.3
- Ophyd      : v1.6.4
- Databroker : v2.0.0b2

ambarb · 2023-05-04T19:30:03Z

@johnsinsheimer I think this is pretty in control, right? We can test more by letting making the acquire period as small as possible when running the camera at 0.2s or faster.

This was referenced Dec 22, 2020

[fccd] periodic issue that requires restart of bsui #25

Closed

[CSX] Error during BS script execution (fCCD related?) bluesky/bluesky#1159

Closed

ambarb self-assigned this Jan 13, 2021

ambarb added DAMA-help DAMA-input-not-work labels Jan 13, 2021

ambarb added bug prioritize and removed DAMA-input-not-work labels May 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

one cause of frozen scans - using ct_dark_all() plan with fast acquition times - intermittent issue #38

one cause of frozen scans - using ct_dark_all() plan with fast acquition times - intermittent issue #38

ambarb commented Jul 31, 2020

ambarb commented Dec 22, 2020

ambarb commented Dec 22, 2020

mrakitin commented Mar 26, 2021 •

edited

Loading

ambarb commented Mar 26, 2021 •

edited

Loading

mrakitin commented Mar 26, 2021

ambarb commented May 18, 2021

mrakitin commented May 18, 2021

ambarb commented Jan 27, 2022

ambarb commented Jul 3, 2022 •

edited

Loading

ambarb commented May 4, 2023

one cause of frozen scans - using ct_dark_all() plan with fast acquition times - intermittent issue #38

one cause of frozen scans - using ct_dark_all() plan with fast acquition times - intermittent issue #38

Comments

ambarb commented Jul 31, 2020

ambarb commented Dec 22, 2020

ambarb commented Dec 22, 2020

mrakitin commented Mar 26, 2021 • edited Loading

ambarb commented Mar 26, 2021 • edited Loading

mrakitin commented Mar 26, 2021

ambarb commented May 18, 2021

mrakitin commented May 18, 2021

ambarb commented Jan 27, 2022

ambarb commented Jul 3, 2022 • edited Loading

ambarb commented May 4, 2023

mrakitin commented Mar 26, 2021 •

edited

Loading

ambarb commented Mar 26, 2021 •

edited

Loading

ambarb commented Jul 3, 2022 •

edited

Loading