DESI simulations

desihub, updated 🕥 2023-02-22 22:30:24

======= desisim =======

|Actions Status| |Coveralls Status| |Documentation Status|

.. |Actions Status| image:: https://github.com/desihub/desisim/workflows/CI/badge.svg :target: https://github.com/desihub/desisim/actions :alt: GitHub Actions CI Status

.. |Coveralls Status| image:: https://coveralls.io/repos/desihub/desisim/badge.svg :target: https://coveralls.io/github/desihub/desisim :alt: Test Coverage Status

.. |Documentation Status| image:: https://readthedocs.org/projects/desisim/badge/?version=latest :target: https://desisim.readthedocs.io/en/latest/ :alt: Documentation Status

Introduction

This package contains scripts and packages for simulating DESI spectra. For full documentation, please visit desisim on Read the Docs_

.. _desisim on Read the Docs: https://desisim.readthedocs.io/en/latest/

License

desisim is free software licensed under a 3-clause BSD-style license. For details see the LICENSE.rst file.

Issues

io.py find_basis_template does not select most recent template file

opened on 2023-01-13 21:01:31 by paulmartini

find_basis_template() is supposed to return the most recent template file in DESI_BASIS_TEMPLATES, yet returns the last element from a glob return value and this is in arbitrary order

``` objfile_wild = os.path.join(indir, objtype.lower()+'templates*.fits') objfiles = glob(objfile_wild)

if len(objfiles) > 0:
    return objfiles[-1]

```

Sorting the list produced by glob will fix it: objfile_wild = os.path.join(indir, objtype.lower()+'_templates_*.fits') objfiles = glob(objfile_wild) objfiles.sort(key=os.path.getmtime) Will submit a PR shortly with this fix.

- Change defaults for some options

opened on 2022-11-10 02:25:14 by alxogm

Changed the defaults for options --zbest, --bbflux and --desi_footprint to be true as defult. Now in order to set them false we have to give --no-zbest, --no-bbflux, and no-desifootprint. @andreufont @HiramHerrera @andreicuceu I think those are the only arguments that make sense to change for now... let me know if there are other suggestions.

I also updated a bit the notebook about reproducibility to work with the latest version of the mocks. I simplified the notebook and included some information about the truth files. This can be improved in a future PR is we want to...

problem interpolating atmosphere in desisim.specsim.get_simulator

opened on 2022-02-06 12:57:50 by moustakas

``` from desisim.simexp import simulate_spectra from desisim.templates import ELG flux, wave, meta, obj = ELG().make_templates(1, seed=1) sim1 = simulate_spectra(wave, flux) INFO:iers.py:82:freeze_iers: Freezing IERS table used by astropy time, coordinates. DEBUG:simexp.py:418:simulate_spectra: loading specsim desi config desi DEBUG:simexp.py:422:simulate_spectra: creating specsim desi simulator

ValueError Traceback (most recent call last) in ----> 1 sim1 = simulate_spectra(wave, flux)

~/code/desihub/desisim/py/desisim/simexp.py in simulate_spectra(wave, flux, fibermap, obsconditions, redshift, dwave_out, seed, psfconvolve, specsim_config_file) 422 log.debug('creating specsim desi simulator') 423 # desi = specsim.simulator.Simulator(config, num_fibers=nspec) --> 424 desi = desisim.specsim.get_simulator(config, num_fibers=nspec, 425 camera_output=psfconvolve) 426

~/code/desihub/desisim/py/desisim/specsim.py in get_simulator(config, num_fibers, camera_output) 50 #- New config; create Simulator object 51 import specsim.simulator ---> 52 qsim = specsim.simulator.Simulator(config, num_fibers, 53 camera_output=camera_output) 54

~/code/desihub/specsim/specsim/simulator.py in init(self, config, num_fibers, camera_output, verbose) 72 73 # Initalize our component models. ---> 74 self.atmosphere = specsim.atmosphere.initialize(config) 75 self.instrument = specsim.instrument.initialize(config, camera_output) 76 self.source = specsim.source.initialize(config)

~/code/desihub/specsim/specsim/atmosphere.py in initialize(config) 709 710 # Load tabulated data. --> 711 surface_brightness_dict = config.load_table( 712 atm_config.sky, 'surface_brightness', as_dict=True) 713 extinction_coefficient = config.load_table(

~/code/desihub/specsim/specsim/config.py in load_table(self, parent, column_names, interpolate, as_dict) 486 kind='linear', copy=False, 487 bounds_error=bounds_error, fill_value=fill_value) --> 488 interpolated_values = interpolator(self.wavelength.value) 489 unit = loaded_columns[column_name].unit 490 if unit:

~/opt/anaconda3/envs/desi/lib/python3.8/site-packages/scipy/interpolate/polyint.py in call(self, x) 76 """ 77 x, x_shape = self._prepare_x(x) ---> 78 y = self._evaluate(x) 79 return self._finish_y(y, x_shape) 80

~/opt/anaconda3/envs/desi/lib/python3.8/site-packages/scipy/interpolate/interpolate.py in _evaluate(self, x_new) 682 y_new = self._call(self, x_new) 683 if not self._extrapolate: --> 684 below_bounds, above_bounds = self._check_bounds(x_new) 685 if len(y_new) > 0: 686 # Note fill_value must be broadcast up to the proper size

~/opt/anaconda3/envs/desi/lib/python3.8/site-packages/scipy/interpolate/interpolate.py in _check_bounds(self, x_new) 714 "range.") 715 if self.bounds_error and above_bounds.any(): --> 716 raise ValueError("A value in x_new is above the interpolation " 717 "range.") 718

ValueError: A value in x_new is above the interpolation range. ```

desisim.simexp.simulate_spectra not varying exposure time

opened on 2022-02-06 01:21:27 by humnaawan

Hi, I have been trying to figure out why my spectra don't change with exposure time, and I've isolated the issue to desisim.simexp.simulate_spectra.

Specifically, say I have a wave array wave_arr and flux array flux_arr, and I run simulator = desisim.simexp.simulate_spectra(wave=wave_arr, flux=flux_arr, ) out0 = simulator.camera_output[0] vs. ``` simulator = desisim.simexp.simulate_spectra(wave=wave_arr, flux=flux_arr, obsconditions=obsconditions_dict )

out1 = simulator.camera_output[0] where obsconditions_dict = {'TILEID': 11147, 'RA': 156.5882658712762, 'DEC': 22.643639342687276, 'PASS': 1, 'EXPID': 344, 'MJD': 58824.50497106333, 'EXPTIME': 4700, 'SNR2FRAC': 0.5060822, 'AIRMASS': 1.0281637, 'SEEING': 0.7948105, 'TRANSP': 0.82818073, 'SKY': 1.0, 'PROGRAM': 'DARK', 'NIGHT': '20191206', 'FLAVOR': 'science', 'MOONFRAC': 0.5, 'MOONALT': -10.0, 'MOONSEP': 90.0} `` Then I'd expectout0andout1` to not be the same -- is that not right?

The calls to simexp seems to suggest that the observation parameters being considered are indeed the ones I pass, as I see the following for the first run DEBUG:simexp.py:418:simulate_spectra: loading specsim desi config desi DEBUG:simexp.py:422:simulate_spectra: creating specsim desi simulator WARNING:simexp.py:428:simulate_spectra: Assuming DARK conditions INFO:simexp.py:444:simulate_spectra: MJD not in obsconditions, using DATE-OBS 2019-12-07T12:07:09.500 DEBUG:simexp.py:448:simulate_spectra: obsconditions SEEING = 1.1 DEBUG:simexp.py:448:simulate_spectra: obsconditions EXPTIME = 1000 DEBUG:simexp.py:448:simulate_spectra: obsconditions AIRMASS = 1.0 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONFRAC = 0.0 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONALT = -60 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONSEP = 180 WARNING:simexp.py:653:get_source_types: (SV1_)DESI_TARGET not in fibermap table; using source_type='star' for everything DEBUG:simexp.py:490:simulate_spectra: running simulation with fastsim fiber loss method DEBUG:simexp.py:496:simulate_spectra: source types: 500 star and this for the second DEBUG:simexp.py:418:simulate_spectra: loading specsim desi config desi DEBUG:simexp.py:422:simulate_spectra: creating specsim desi simulator INFO:simexp.py:442:simulate_spectra: exposure_start 2019-12-07T12:07:09.500 DEBUG:simexp.py:448:simulate_spectra: obsconditions SEEING = 0.7948104739189148 DEBUG:simexp.py:448:simulate_spectra: obsconditions EXPTIME = 4700 DEBUG:simexp.py:448:simulate_spectra: obsconditions AIRMASS = 1.0281636714935303 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONFRAC = 0.5 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONALT = -10.0 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONSEP = 90.0 WARNING:simexp.py:653:get_source_types: (SV1_)DESI_TARGET not in fibermap table; using source_type='star' for everything DEBUG:simexp.py:490:simulate_spectra: running simulation with fastsim fiber loss method DEBUG:simexp.py:496:simulate_spectra: source types: 500 star but then I don't understand why the output spectra are not different. I'd really appreciate some help since I don't know what is going wrong. I thought it was my fibermap (that I was passing to desisim.simexp.simulate_spectra) but that seems to be making no difference to the actual outputs as well, although I do see that the source types are read: DEBUG:simexp.py:418:simulate_spectra: loading specsim desi config desi DEBUG:simexp.py:422:simulate_spectra: creating specsim desi simulator INFO:simexp.py:442:simulate_spectra: exposure_start 2019-12-07T12:07:09.500 DEBUG:simexp.py:448:simulate_spectra: obsconditions SEEING = 0.7948104739189148 DEBUG:simexp.py:448:simulate_spectra: obsconditions EXPTIME = 4700 DEBUG:simexp.py:448:simulate_spectra: obsconditions AIRMASS = 1.0281636714935303 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONFRAC = 0.5 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONALT = -10.0 DEBUG:simexp.py:448:simulate_spectra: obsconditions MOONSEP = 90.0 DEBUG:simexp.py:490:simulate_spectra: running simulation with fastsim fiber loss method DEBUG:simexp.py:496:simulate_spectra: source types: 180 elg 10 star 167 qso 103 lrg 40 sky WARNING:simexp.py:511:simulate_spectra: the half light radii are fixed here for LRGs and ELGs (and not magnitude or redshift dependent)

The only other thing I can think of is the config file but then I don't understand why there's an obsconditions optional parameter at all if I am supposed to be passing a non-default config file (which would be a pain to create for every tile).

Thanks so much!

Test failure at NERSC due to basis template configuration

opened on 2022-01-15 17:04:37 by weaverba137

Nightly tests of desisim are failing due to the following (one!) error: ``` ====================================================================== ERROR: test_qso_options (desisim.test.test_templates.TestTemplates) Test that the QSO keyword arguments work

Traceback (most recent call last): File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desisim/master/py/desisim/test/test_templates.py", line 256, in test_qso_options flux, wave, meta, objmeta = QSO(wave=self.wave, balqso=True).make_templates( File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desisim/master/py/desisim/templates.py", line 2003, in init bal = BAL() File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desisim/master/py/desisim/bal.py", line 27, in init balflux, balwave, balmeta = read_basis_templates(objtype='BAL') File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desisim/master/py/desisim/io.py", line 948, in read_basis_templates infile = find_basis_template(ltype) File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desisim/master/py/desisim/io.py", line 887, in find_basis_template raise IOError('No {} templates found in {}'.format(objtype, objfile_wild)) OSError: No bal templates found in /global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desisim-testdata/master/desi/spectro/templates/basis_templates/v3.0/bal_templates_*.fits

`` I think it needs to look atv3.2rather thanv3.0`, but I'm not sure where that is configured.

RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 80 from C header, got 88 from PyObject

opened on 2022-01-11 17:20:18 by moustakas

Running python py/desisim/test/test_lya.py yields the following non-catastrophic warning: <frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 80 from C header, got 88 from PyObject This is happening on my laptop (a newer MacBook Pro) with python 3.8.5 and numpy 1.20.3 but I'm also seeing it in our Github Actions tests.

Googling suggests that simply reinstalling numpy should fix it, although it didn't work for me: https://stackoverflow.com/questions/66060487/valueerror-numpy-ndarray-size-changed-may-indicate-binary-incompatibility-exp

This isn't a critical issue, I'm just curious whether anyone else has seen this and/or has a suggested fix.

Releases

desisim 0.28.0 2018-07-18 22:10:00

Used by DESI software release 18.6

Integration updates and QSO / LyA updates.

desisim 0.27.0 2018-03-29 23:21:56

Major update especially for pixsim; see doc/changes.rst. Candidate tag for DESI software release 18.3.

desisim 0.21.0 2017-09-29 23:42:40

  • Major refactor of newexp to add connection to upstream mocks, surveysims, and fiber assignment (PR #250).
  • Support latest (>DR4) data model in the templates metadata table and also scale simulated templates by 1e17 erg/s/cm2/Angstrom (PR #252).
  • Add desi_qa_s2n script (PR #254)
  • Refactor desi_qa_zfind script (PR #254)
  • Refactor redshift QA for new data model (PR #254)
  • Refactor shared QA methods to desisim.spec_qa.utils (PR #254)
  • New plots for S/N of spectra for various objects (ELG, LRG, QSO) (PR #254)
  • Add BGS, MWS to z_find QA
  • Miscellaneous polishing in QA (velocity, clip before RMS, extend [OII] flux, S/N per Ang)
  • Bug fix: correctly select both "bright" and "faint" BGS templates by default (PR #257).
  • Updates for newexp/fastframe wrappers for end-to-end sims (PR #258).

desisim 0.18.2 2017-03-28 03:39:20

Fixes documentation and QSO template generation scaling

desisim 0.18.1 2017-03-05 19:20:32

update basis_templates in module file to v2.3

desisim 0.18.0 2017-03-04 22:25:36

Many updates to the template-generating code (e.g., support for white dwarfs), quickcat, and quicksurvey, and several small updates to the desihub code dependencies.

desihub

Public code associated with the Dark Energy Spectroscopic Instrument (DESI).

GitHub Repository