How to update inputs from DocDB¶

Context: Inputs from DocDB that require reformatting before using are kept in data/inputs/ and then reformatted using scripts in bin/ . Inputs from DocDB that can be used as-is are directly added to their final location under data/.

Basic Setup¶

Make branches of both desimodel GitHub code and svn data:

git checkout -b update_inputs
base=https://desi.lbl.gov/svn/code/desimodel
svn copy $base/trunk $base/branches/update_inputs
svn checkout $base/branches/update_inputs/data
export DESIMODEL=`pwd`

Add entries to $HOME/.netrc to enable downloading DocDB files without having to enter a password every time:

machine desi.lbl.gov
login StephenBailey
password NotMyRealPassword

The code in desimodel.inputs.docdb requires requests (for communicating with DocDB) and xlrd (for reading Microsoft Excel spreadsheets). Both of these are available via Anaconda.

Inputs to update¶

The update functions below belong to desimodel.inputs and take an optional argument testdir to specify an alternate directory where updated outputs should be written. When testdir is not specified, outputs are written to their standard locations under desimodel.io.datadir().

DESI-0530-v13 Excel spreadsheet to .ecsv file for GFA locations¶

This writes the gfa.ecsv file containing the GFA data, which is pulled from the “GFALocation” tab on the DESI-0530-v13 Excel spreadsheet and from rows 16-23 and columns A-I. The function desimodel.inputs.gfa.build_gfa_table() writes the file in the current directory.

import desimodel.inputs.gfa
desimodel.inputs.gfa.build_gfa_table()

Positioner to Fiber Mapping¶

This updates the mapping of device locations on the focal plane to spectrograph fiber numbers using DESI-0530-v14, DESI-2721-v2 and DESI-329-v15.

import desimodel.inputs.fiberpos
desimodel.inputs.fiberpos.update()

To update a DESI-doc versions, edit the corresponding docdb.download(...) call.

Throughput¶

This updates the throughput model from DESI-0347 and DESI-0344 and also copies the top-level desi.yaml from DESI-0347:

import desimodel.inputs.throughput
desimodel.inputs.throughput.update()

To update the version of DESI-347 that is used, change the default value of desi347_version in the update() function. Similiarly for DESI-344.

Only three rows of the throughput spreadsheet from DESI-347 are used, with hard-coded row numbers. There are some simple checks that these are correct, using the specthru_row and thru_row arguments to load_throughput(), but check the outputs carefully if you think the spreadsheet structure might have changed.

Blur and Offsets¶

Use the notebook doc/nb/DESI-0347_Throughput.ipynb to update the following ouputs derivied from DESI-347:

data/inputs/throughput/raytracing.txt

data/throughput/DESI-0347_blur.ecsv

data/throughput/DESI-0347_offset.ecsv

data/throughput/DESI-0347_static_offset_[123].fits

Refer to the instructions in that notebook for details.

Testing¶

After changing any outputs that might break a unit test, update the small test dataset following Testing desimodel and edit DESIMODEL_VERSION in .travis.yml to point to the new version.

Commissioning Instrument corners¶

This updates the CI as-measured corner locations from DESI-4633v11 Corners.txt and reformats them into the GFA-corners format needed by desimodel, writing the results into $$DESIMODEL/data/focalplane/ci-corners.ecsv:

import desimodel.inputs.ci
desimodel.inputs.ci.update()

We don’t anticipate needing to update this again, so this section is just documenting the provenance of that file.

Commit Changes to SVN¶

Once you have finished making updates on the update_inputs svn branch, checkout the trunk and merge your changes using:

svn checkout $base/trunk/data data.trunk
cd data.trunk
svn merge --dry-run $base/branches/update_inputs/data
# Make sure everything looks good. Then do it for real.
svn merge $base/branches/update_inputs/data
svn commit -m "Merge branch update_inputs into trunk"

Finally, remove the update_inputs branch:

svn remove $base/branches/update_inputs -m "Cleanup after updating inputs"

and tag the updated trunk (replace 0.13.0 as needed):

version=0.13.0
svn copy $base/trunk $base/tags/$version -m "Tagging desimodel $version"

To Do¶

Update methodology and document how to update the following:

PSF model from DESI-0334

PSF spots -> PSF for quicksim

Fiber input loss calculations

desimodel/data/focalplane/platescale.txt