New resolution matrix implementation #181

jdbuhler · 2024-09-02T18:02:37Z

Reimplement enough of desispec.resolution.Resolution to perform matrix-vector operations, which is all we do with it, with a significant speedup via Numba compared to the scipy.sparse implementation.

Retire the old res_emline version of the matrix, which is slower for dot product than either DESI or our new implementation. Allow conversion on the fly from our new impl to the internal rep used by res_emline, which is used way down in the guts of emline fitting but not otherwise. Switching away from res_emline for dot products does result in a small change in the output of emline fitting, which propagates to masking.

In addition, make two small but meaningful changes to least_squares call in continuum fitting: switch from lsmr to exact method for TRF algorithm, which reduces least_squares overhead, and use a looser ftol of 1e-3 instead of 1e-5, which greatly reduces the number of lsq iterations. (These should only be accepted if the output still looks OK!)

In passing, implement several small code cleanups that do not impact the results at all, including more efficient storage of some fields in the data dictionary and avoiding a copy in the _stellar_objective().

which yields a nice speedup. Merge the needed functionality of the emline_fit sparse matrix, so that we don't have to pass two sepearate resolution matrices to different parts of the code.

…ices * don't allocate unused local reference to pixkms_bounds in ContinuumTools * emit a log message when we use an acceelerated FFT library * store frequently used, frozen lists in input data dictionary as fixed-sized tuples or arrays for efficiency

moustakas · 2024-09-03T12:51:12Z

Thanks @jdbuhler. The fitting code looks good but running fastqa crashes with

[snip]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/code/desihub/fastspecfit/bin/fastqa", line 9, in <module>
    fastqa(comm=None)
  File "/global/homes/i/ioannis/code/desihub/fastspecfit/py/fastspecfit/qa.py", line 1431, in fastqa
    _wrap_qa(redrockfile, indx)
  File "/global/homes/i/ioannis/code/desihub/fastspecfit/py/fastspecfit/qa.py", line 1407, in _wrap_qa
    for _ in mp_pool.starmap(desiqa_one, qaargs): pass
  File "/global/homes/i/ioannis/code/desihub/fastspecfit/py/fastspecfit/util.py", line 97, in apply_to_dict
    return f(**argdict)
  File "/global/homes/i/ioannis/code/desihub/fastspecfit/py/fastspecfit/qa.py", line 25, in desiqa_one
    qa_fastspec(data, sc_data.templates, fastfit, metadata,
  File "/global/homes/i/ioannis/code/desihub/fastspecfit/py/fastspecfit/qa.py", line 432, in qa_fastspec
    _desiemlines = EMFit.emlinemodel_bestfit(fastspec, fastspec['Z'], np.hstack(data['wave']), data['res_emline'],
KeyError: 'res_emline'
[snip]

Can you take a look and push a fix?

Also, can you please update the doc/changes.rst file with info regarding this PR?

* update documentation of sparse emline Jacobian to std

further limit ftol to 1e-3 for greater speed.

* attempt to clean up some formatting warnings * add new-style docstrings to emline_fit/utils.py

of functions inner where they have only one caller

* consistently import jit from numba instead of saying '@numba.jit'

moustakas · 2024-09-05T12:07:32Z

py/fastspecfit/continuum.py

@@ -592,7 +589,7 @@ def call_nnls(modelflux, flux, ivar, xparam=None, debug=False,


    @staticmethod
-    @numba.jit(nopython=True, fastmath=True, nogil=True)
+    @jit(nopython=True, nogil=True, fastmath=True)
    def attenuate(M, A, zfactors, wave, dustflux):


Didn't we want to change fastmath=False here and elsewhere?

jdbuhler · 2024-09-05T12:11:06Z

I don't think so. There was one place that it needed to be turned off, which we did a couple of weeks ago. You can do it if you want to see if it makes a big difference. Jeremy On Sep 5, 2024 08:08, Moustakas ***@***.***> wrote: @moustakas commented on this pull request.

________________________________ In py/fastspecfit/continuum.py<#181 (comment)>:

@@ -592,7 +589,7 @@ def call_nnls(modelflux, flux, ivar, xparam=None, debug=False,

@staticmethod - @numba.jit(nopython=True, fastmath=True, nogil=True) + @jit(nopython=True, nogil=True, fastmath=True) def attenuate(M, A, zfactors, wave, dustflux): Didn't we want to change fastmath=False here and elsewhere? — Reply to this email directly, view it on GitHub<#181 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AH4ORJ4TIYP6CL43E7PP2HTZVBCRTAVCNFSM6AAAAABNQXCNSWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDEOBSG4ZDENZVHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

moustakas

The code changes look fine, but either this PR or #180 has led to non-negligible changes in the continuum-fitting results (as you warned me, off-list).

I added just a couple comments. After you make those changes, let's go ahead and merge this PR and then I'm going to test the fitting results on a larger sample.

moustakas · 2024-09-05T12:12:31Z

py/fastspecfit/igm.py

@@ -11,7 +11,7 @@


 class Inoue14(object):
-    r"""
+    """
    IGM absorption from Inoue et al. (2014)


Without the "raw string" formatting, integration tests give a DeprecationWarning. Can we please revert this change?

/global/u2/i/ioannis/code/desihub/fastspecfit/py/fastspecfit/igm.py:14: DeprecationWarning: invalid escape sequence '\m'

moustakas · 2024-09-05T12:14:34Z

py/fastspecfit/continuum.py

        modelflux = np.empty(self.wavelen)
-
+        
        for icam, (s, e) in enumerate(camerapix):


It's a little (a lot?) OCD, but can we remove the added blank 8 spaces on lines 754 and 756? (My configuration of spacemacs shows these blank characters, so I've been cleaning them up.)

moustakas · 2024-09-05T12:15:40Z

py/fastspecfit/templates.py

@@ -206,8 +206,9 @@ def init_ffts(self):
        if find_spec("mkl_fft") is not None:
            import mkl_fft._scipy_fft_backend as be
            sc_fft.set_global_backend(be)
-
+
            self.convolve = sc_sig.convolve


Remove blank spaces.

moustakas · 2024-09-05T12:16:08Z

py/fastspecfit/templates.py

            self.convolve = sc_sig.convolve
+            log.info('Using mkl_fft library for FFTs')
        else:


Maybe this should be a debugging comment?

jdbuhler · 2024-09-05T12:39:49Z

Ok. I'll do this as soon as I am back in front of my computer. The biggest output-altering change was changing ftol from 1e-5 to 1e-3. Jeremy On Sep 5, 2024 08:31, Moustakas ***@***.***> wrote: @moustakas approved this pull request. The code changes look fine, but either this PR or #180<#180> has led to non-negligible changes in the continuum-fitting results (as you warned me, off-list). I added just a couple comments. After you make those changes, let's go ahead and merge this PR and then I'm going to test the fitting results on a larger sample.

________________________________ In py/fastspecfit/igm.py<#181 (comment)>:

@@ -11,7 +11,7 @@

class Inoue14(object): - r""" + """ IGM absorption from Inoue et al. (2014) Without the "raw string" formatting, integration tests give a DeprecationWarning. Can we please revert this change? /global/u2/i/ioannis/code/desihub/fastspecfit/py/fastspecfit/igm.py:14: DeprecationWarning: invalid escape sequence '\m'

________________________________ In py/fastspecfit/continuum.py<#181 (comment)>:

modelflux = np.empty(self.wavelen)

- + for icam, (s, e) in enumerate(camerapix): It's a little (a lot?) OCD, but can we remove the added blank 8 spaces on lines 754 and 756? (My configuration of spacemacs<https://www.spacemacs.org/> shows these blank characters, so I've been cleaning them up.)

________________________________ In py/fastspecfit/templates.py<#181 (comment)>:

@@ -206,8 +206,9 @@ def init_ffts(self):

if find_spec("mkl_fft") is not None: import mkl_fft._scipy_fft_backend as be sc_fft.set_global_backend(be) - + self.convolve = sc_sig.convolve Remove blank spaces.

________________________________ In py/fastspecfit/templates.py<#181 (comment)>:

self.convolve = sc_sig.convolve

+ log.info('Using mkl_fft library for FFTs') else: Maybe this should be a debugging comment? — Reply to this email directly, view it on GitHub<#181 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AH4ORJ4X7OXSGDZ7SJP63HTZVBFIHAVCNFSM6AAAAABNQXCNSWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDEOBSG4ZTGOBXGU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

* make log msg about using mkl_fft debug instead of info * minor whitespace cleanups

* remove shell commands from tops of files that do not contain main or test for __main__

…stmodel

moustakas · 2024-09-16T17:26:02Z

Thanks, @jdbuhler, I'm going to go ahead and merge this and then run on a few 10^4 spectra.

jdbuhler and others added 5 commits August 30, 2024 23:19

Reimplement the DESI resolution matrix using Numba for the multiply,

f266904

which yields a nice speedup. Merge the needed functionality of the emline_fit sparse matrix, so that we don't have to pass two sepearate resolution matrices to different parts of the code.

* tiny cleanup of spectroscopy preprocessing

7f36418

minor cleanup to skip needless array copy in continuum objective

7ced18f

remove unneeded import

ebf4556

jdbuhler added 7 commits September 3, 2024 10:36

* fix missed uses of old res_emline object in QA

14ef0b1

* update documentation of sparse emline Jacobian to std

Update docs for jacobian.py to current std

4a0a6ed

more docstring updates

516aafa

Reduce least_squares overhead by switching from lsmr to exact TRF, and

2df2666

further limit ftol to 1e-3 for greater speed.

* fix markup error in changes.rst

d5d8f64

* attempt to clean up some formatting warnings * add new-style docstrings to emline_fit/utils.py

update docstrings for interface, and make a couple

267571a

of functions inner where they have only one caller

* update util.py for docstring convention

e150b0c

* consistently import jit from numba instead of saying '@numba.jit'

This was referenced Sep 3, 2024

Resolution matrix-vector multiple speedups desihub/desispec#2352

Open

speed up resolution matrix-vector multiplies desihub/redrock#318

Open

jdbuhler added 2 commits September 4, 2024 11:23

remove stray character before comment

c1e5d2e

fix docstrings to conform to rest of code

23aae6f

moustakas reviewed Sep 5, 2024

View reviewed changes

moustakas approved these changes Sep 5, 2024

View reviewed changes

jdbuhler added 6 commits September 5, 2024 08:50

* revert removal of raw tag from string in igm.py

87d4edb

* make log msg about using mkl_fft debug instead of info * minor whitespace cleanups

remove trailing whitespace from all lines, per house style

9db0531

* remove trailing whitespace from lines in all files

ea8db89

* remove shell commands from tops of files that do not contain main or test for __main__

remove header comments with license and charset info

86414a8

revert inclusion of profiling code in fastspecfit.py

0332207

extend trailing whitespace and gratuitous header removal to webapp/fa…

78916bc

…stmodel

moustakas merged commit b71cec6 into main Sep 16, 2024
12 checks passed

moustakas deleted the new-resmul branch September 16, 2024 17:26

moustakas mentioned this pull request Sep 18, 2024

Potentially incorrect resolution matrix in coadds desihub/desispec#2372

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New resolution matrix implementation #181

New resolution matrix implementation #181

jdbuhler commented Sep 2, 2024 •

edited

Loading

moustakas commented Sep 3, 2024

moustakas Sep 5, 2024

jdbuhler commented Sep 5, 2024 via email

moustakas left a comment

moustakas Sep 5, 2024

moustakas Sep 5, 2024

moustakas Sep 5, 2024

moustakas Sep 5, 2024

jdbuhler commented Sep 5, 2024 via email

moustakas commented Sep 16, 2024

		modelflux = np.empty(self.wavelen)


		for icam, (s, e) in enumerate(camerapix):

New resolution matrix implementation #181

New resolution matrix implementation #181

Conversation

jdbuhler commented Sep 2, 2024 • edited Loading

moustakas commented Sep 3, 2024

moustakas Sep 5, 2024

Choose a reason for hiding this comment

jdbuhler commented Sep 5, 2024 via email

moustakas left a comment

Choose a reason for hiding this comment

moustakas Sep 5, 2024

Choose a reason for hiding this comment

moustakas Sep 5, 2024

Choose a reason for hiding this comment

moustakas Sep 5, 2024

Choose a reason for hiding this comment

moustakas Sep 5, 2024

Choose a reason for hiding this comment

jdbuhler commented Sep 5, 2024 via email

moustakas commented Sep 16, 2024

jdbuhler commented Sep 2, 2024 •

edited

Loading