Focalplane Hardware Model

Overview

The DESI focalplane consists of petals with individual positioners and each positioner is a unique device connected to a fiber. The fibers travel to the spectrographs and are connected to a position on a slitblock. Each positioner has a range of angular motion along two axes (theta and phi).

The canonical focalplane calibration is stored in the ICS database, but we want to copy it into desimodel for offline analysis and to be able to tag exactly what version was used for fiber assignment and data processing runs.

The focalplane model currently uses the CS5 coordinate system for the X/Y locations of devices on the focalplane.

Tracking Changes

Although the generated focalplane models are checked into svn, we need to be able to get the state of the hardware at any time, across the full history of the DESI project. We enable this feature through the use of our file format (see section below on the details of the format). Since most changes are small (a positioner breaks or gets stuck, etc), we keep a running log of these small changes. A completely new model is only generated for large events (e.g. a petal is swapped out). When a focalplane is loaded, the most recent model for a given time is found and the events in this log are replayed up to the requested time. It is worth emphasizing the previous text again. When a focalplane is loaded for a particular timestamp:

  1. Each focalplane model has a starting time, and remains valid until it is superceded by a newer model. The most recent set of 3 focalplane files which comes before the requested timestamp is read from disk. The static focalplane properties are kept as a Table and the exclusion polygons are read into a dictionary.
  2. The state log (one of the 3 files) is parsed line by line. If the timestamp for that line comes before the requested time, then the event in that line is applied. All positioners have an initial state specified in the state log with a timestamp that matches the starting time of the focalplane. Subsequent events are appended to the state log with a timestamp.

The file formats used are text-based (ECSV and YAML). However, these are intended to be modified by the included scripts, which can ensure that the formatting is correct. The risk of typos and subtle errors if hand-editing these files is large. If you find that you frequently need to edit these files, then please open an issue to document your use case.

Inputs

Focalplane models are generated and updated from ICS database dumps at KPNO in /data/focalplane/calibration/*.ecsv.

Additionally, the following DocDB files are downloaded and parsed to get the mapping of fiber focalplane location to slitblock location (i.e. fiber number on the spectrographs).

DocDB Number Purpose
4042 Petal verification for petal ID 02
4043 Petal verification for petal ID 03
4807 Petal verification for petal ID 04
4808 Petal verification for petal ID 05
4809 Petal verification for petal ID 06
4190 Petal verification for petal ID 07
4806 Petal verification for petal ID 08
4810 Petal verification for petal ID 09
4042 Petal verification for petal ID 10
4042 Petal verification for petal ID 11

Generating a New Model

Under normal circumstances focalplane updates are done daily by a KPNO cronjob running etc/desimodel_sync_kpno_cron.sh (update existing model) or etc/desimodel_sync_kpno_force.sh (use --reset to make a new model). The rest of this section documents what those are doing “under the hood”, but it should not be necessary to run by hand.

A new focalplane model is generated with the desi_sync_focalplane script:

usage: desi_sync_focalplane [-h] --calib_file CALIB_FILE [--test] [--reset]
                            [--simulate_good] [--debug_dir DEBUG_DIR]
                            [--commit]

optional arguments:
  -h, --help            show this help message and exit
  --calib_file CALIB_FILE
                        The ECSV database dump file
  --test                Go through the process of updating the focalplane, but
                        do not actually write new files.
  --reset               Create a new focalplane model from the calib file,
                        ignoring all previous state information
  --simulate_good       Create a focalplane model for simulations. Non-broken
                        fibers set to good
  --debug_dir DEBUG_DIR
                        Override the output directory for debugging.
  --commit              Commit updated focalplane model to svn.

Note that the --reset option generates a new focalplane model, while without that option it updates the state ledger of the current focalplane model for only the positioners that changed. In addition to the --calib_file input with the latest ICS database focalplane dump, this script automatically downloads the necessary DocDB entries listed above, which requires you to have DESI DocDB credentials stored in your $HOME/.netrc file.

See desimodel tags 0.16.0 and prior for documentation of an older script desi_generate_focalplane which uses lab-measured focalplane metrology from DESI SVN code/focalplane/fp_settings/pos_settings to create a new focalplane model. This has been removed from newer versions of desimodel in favor of loading the metrology as measured in-situ at KPNO.

Updating the State of a Model

After a focalplane model is created, the state can be updated by rerunning desi_sync_focalplane with a new ICS database dump without using the --reset option. If needed, one can override the database to update the state of an individual positioner with the following command line tool:

usage: desi_update_focalplane_log [-h] [--location LOCATION] [--petal PETAL]
                                  [--device DEVICE] [--state STATE]
                                  [--exclusion EXCLUSION] [--time TIME]

optional arguments:
  -h, --help            show this help message and exit
  --location LOCATION   The device location (petal * 1000 + device loc)
                        modified by this event.
  --petal PETAL         The petal (focalplane location, not petal ID) modified
                        by this event (--device must also be used)
  --device DEVICE       The device location (--petal must also be given)
                        modified by this event.
  --state STATE         The new state to assign to the device.
  --exclusion EXCLUSION
                        The new exclusion polygon to assign to the device
                        (e.g. 'legacy', 'default', etc)
  --time TIME           Optional date/time (default is current date/time) when
                        this event happens. Format is YYYY-MM-DDTHH:mm:ss in
                        UTC time.

Updating the Exclusion Polygons in a Model

After a focalplane model is created, one can update the available exclusion polygons with the following command line tool:

usage: desi_update_focalplane_exclusion [-h]
                                        [--exclusion [EXCLUSION [EXCLUSION ...]]]
                                        [--time TIME]

optional arguments:
  -h, --help            show this help message and exit
  --exclusion [EXCLUSION [EXCLUSION ...]]
                        One or more text config files containing some
                        exclusion polygons to use. For example,
                        '$DESIMODEL/exclusions.conf'. This file should contain
                        a parameter 'NAME' with the name to use for this set
                        of exclusions.
  --time TIME           Optional date/time (default is current date/time) to
                        use when selecting the focalplane.Format is YYYY-MM-
                        DDTHH:mm:ss in UTC time.

Existing exclusion polygons with the same name as any input files will be replaced. New polygons will be appended.

File Format and Loading

A single focalplane model (with a starting valid datetime) consist of 3 files on disk. These files contain matching date / time stamps that correspond the the first valid time for that focalplane model. For example:

desi-focalplane_2019-09-16T00:00:00.ecsv
desi-exclusion_2019-09-16T00:00:00.yaml
desi-state_2019-09-16T00:00:00.ecsv

The first is an ECSV text file containing the static information such as the positioner locations and angle range, the mapping of device locations to fibers, etc.

Column Data Type Description
PETAL int32 Petal location [0-9]
DEVICE int32 Device location on the petal
LOCATION int32 PETAL * 1000 + DEVICE
PETAL_ID int32 The physical petal ID
DEVICE_ID string The physical device ID string
DEVICE_TYPE string The device type (POS, ETC, FIF)
SLITBLOCK int32 The slit block where this fiber goes
BLOCKFIBER int32 The fiber index within the slit block
CABLE int32 The cable ID
CONDUIT string The conduit
FIBER int32 PETAL * 500 + SLITBLOCK * 25 + BLOCKFIBER
FWHM float64 FWHM at f/3.9
FRD float64 FRD Throughput
ABS float64 ABS Throughput
OFFSET_X float64 X location of positioner center
OFFSET_Y float64 Y location of positioner center
OFFSET_T float64 THETA zero point angle
OFFSET_P float64 PHI zero point angle
LENGTH_R1 float64 Length of THETA arm
LENGTH_R2 float64 Length of PHI arm
MAX_T float64 Maximum THETA angle relative to OFFSET_T
MIN_T float64 Minimum THETA angle relative to OFFSET_T
MAX_P float64 Maximum PHI angle relative to OFFSET_P
MIN_P float64 Minimum PHI angle relative to OFFSET_P

The second file is a YAML format file which contains one or more exclusion polygons for the positioners. Each named exclusion entry actually has multiple polygons: for the GFA, petal boundary, theta arm and phi arm. These define the shape of the polygon at the origin, which is then translated and rotated differently for every positioner based on the arm length, etc. Exclusion polygons are specified in terms of lists of circles and line segments.

The third file is another ECSV format file that contains the state log for the focalplane model. This is the running log of events that happen which modify the instantaneous state of the focalplane.

Column Data Type Description
TIME string The timestamp of the event (UTC, ISO format)
PETAL int32 Petal location [0-9]
DEVICE int32 Device location on the petal
LOCATION int32 PETAL * 1000 + DEVICE
STATE uint32 State bit field (good == 0)
EXCLUSION string The exclusion polygon for this device

The file formats above are documented for completeness, but you should not generally read these manually. Instead, one calls the load_focalplane() function:

desimodel.io.load_focalplane(time=None, get_time_range=False)[source]

Load the focalplane state that is valid for the given time.

Parameters:time (datetime) – The time to query with explicit timezone. Default to current time (now()) with timezone UTC.
Returns:A tuple of (FP layout, exclusion polygons, state, time string). The FP layout is a Table. The exclusion polygons are a dictionary indexed by names that are referenced in the state. The state is a Table. The time string is the resulting UTC ISO format time string for the creation date of the FP model.

If get_time_range=True, returns two additional values: time_low and time_high, both datetime objects giving the range of dates over which this description of the focal plane is valid. time_high may be None, indicating that there is no later known hardware state. In particular, these dates refer to the state of the positioners, which are more fine-grained than the fp and exclusion objects.

Return type:tuple

The state table returned by this function contains the instantaneous state of the focalplane at the input time (i.e. the log of events has already been replayed and the state at the requested time is returned)

TO-DO

There are several small features needed: