Make tests pass on alpha

[roed314]

Currently, most tests fail when merging code into alpha. To see recent test results, go here.

Since alpha is never intended to be merged into main, fixes should be made on the branches that are being combined into alpha. These branches are:

• Lattices in Eran's fork
• Shimura curves in Eran's fork
• Hilbert modular surfaces in Eran's fork
• Eisenstein series in Eran's fork
• Siegel modular forms in Eran's fork
• Modular curves in Barinder's fork
• Genus 2 curves in Drew's fork
• Isomorphism classes of abelian varieties over finite fields in my fork

Note that mod-l Galois representations are also enabled on alpha, but that code is already in the main branch.

[roed314]

We should look through all of the errors. For example, this one should probably be addressed:

>       assert not collisions, "Template namespace collisions: %s" % collisions
E       AssertionError: Template namespace collisions: [('lat_diagram_page.html', './lmfdb/modular_curves/templates'), ('lat_diagram_page.html', './lmfdb/shimura_curves/templates')]

[roed314]

Failures in Siegel modular forms should get fixed but are probably not urgent:

lmfdb/siegel_modular_forms/test_siegel_modular_forms.py::HomePageTest::test_browse_page FAILED [ 28%]
lmfdb/siegel_modular_forms/test_siegel_modular_forms.py::HomePageTest::test_dimension_tables FAILED [ 42%]
lmfdb/siegel_modular_forms/test_siegel_modular_forms.py::HomePageTest::test_huge_sample FAILED [ 57%]
lmfdb/siegel_modular_forms/test_siegel_modular_forms.py::HomePageTest::test_random_page FAILED [ 71%]
lmfdb/siegel_modular_forms/test_siegel_modular_forms.py::HomePageTest::test_sample_page_Q FAILED [ 85%]
lmfdb/siegel_modular_forms/test_siegel_modular_forms.py::HomePageTest::test_sample_page_nf FAILED [100%]

[roed314]

These are consequences of changing the homepage; the tests should get fixed:

lmfdb/tests/test_homepage.py::HomePageTest::test_all_internal_links FAILED [ 28%]
lmfdb/tests/test_homepage.py::HomePageTest::test_all_links_by_box FAILED [ 42%]
lmfdb/tests/test_homepage.py::HomePageTest::test_box_citations_and_acknowledgments FAILED [ 71%]
lmfdb/tests/test_homepage.py::HomePageTest::test_box_learn_more FAILED   [ 85%]

[roed314]

This looks like a problem with Eisenstein series, but isn't urgent:

_____________________________ CmfTest.test_random ______________________________

self = <lmfdb.classical_modular_forms.test_cmf2.CmfTest testMethod=test_random>

    def test_random(self):
        r"""
        Test that we don't hit any error on a random newform
        """
        def check(page):
            assert 'Newspace' in page.get_data(as_text=True), page.url
            assert 'parameters' in page.get_data(as_text=True), page.url
            assert 'Properties' in page.get_data(as_text=True), page.url
            assert 'Newform' in page.get_data(as_text=True), page.url
            assert 'expansion' in page.get_data(as_text=True), page.url
        for i in range(100):
>           page = self.tc.get('/ModularForm/GL2/Q/holomorphic/random', follow_redirects=True)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

lmfdb/classical_modular_forms/test_cmf2.py:152: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/werkzeug/test.py:1162: in get
    return self.open(*args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/testing.py:235: in open
    response = super().open(
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/werkzeug/test.py:1150: in open
    response = self.resolve_redirect(response, buffered=buffered)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/werkzeug/test.py:1056: in resolve_redirect
    return self.open(builder, buffered=buffered)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/testing.py:235: in open
    response = super().open(
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/werkzeug/test.py:1116: in open
    response_parts = self.run_wsgi_app(request.environ, buffered=buffered)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/werkzeug/test.py:988: in run_wsgi_app
    rv = run_wsgi_app(self.application, environ, buffered=buffered)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/werkzeug/test.py:1264: in run_wsgi_app
    app_rv = app(environ, start_response)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/app.py:1536: in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
lmfdb/app.py:42: in __call__
    return self.app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/app.py:1514: in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/app.py:1511: in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/app.py:919: in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/app.py:917: in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/miniconda/envs/lmfdb/lib/python3.12/site-packages/flask/app.py:902: in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
lmfdb/classical_modular_forms/main.py:594: in by_url_eisenstein_newform_label
    return render_newform_webpage(label)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
lmfdb/classical_modular_forms/main.py:402: in render_newform_webpage
    newform.setup_cc_data(info)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <lmfdb.classical_modular_forms.web_newform.WebNewform object at 0x7f38ecf78110>
info = {'CC_m': [1, 2, 3, 4, 5, 6, ...], 'CC_n': [2, 3, 4, 5, 6, 7, ...], 'default_mrange': '1-8', 'default_nrange': '2-10', ...}

    def setup_cc_data(self, info):
        """
        INPUT:
    
        - ``info`` -- a dictionary with keys
          - ``m`` -- a string describing the embedding indexes desired
          - ``n`` -- a string describing the a_n desired
          - ``CC_m`` -- a list of embedding indexes
          - ``CC_n`` -- a list of desired a_n
          - ``format`` -- one of 'embed', 'analytic_embed', 'satake', or 'satake_angle'
        """
        an_formats = ['embed', 'analytic_embed', None]
        angles_formats = ['satake', 'satake_angle', None]
        analytic_shift_formats = ['embed', None]
        cc_proj = ['conrey_index', 'embedding_index', 'embedding_m', 'embedding_root_real', 'embedding_root_imag']
        format = info.get('format')
        query = {'hecke_orbit_code': self.hecke_orbit_code}
    
        # deal with m
        if self.embedding_label is None:
            m = info.get('m', '1-%s' % (min(self.dim, 20)))
            if '.' in m:
                m = re.sub(r'\d+\.\d+', self.embedding_from_embedding_label, m)
            CC_m = info['CC_m'] if 'CC_m' in info else integer_options(m)
            CC_m = sorted(set(CC_m))
            # if it is a range
            if len(CC_m) - 1 == CC_m[-1] - CC_m[0]:
                query['embedding_m'] = {'$gte': CC_m[0], '$lte': CC_m[-1]}
            else:
                query['embedding_m'] = {'$in': CC_m}
            self.embedding_m = None
        else:
            self.embedding_m = int(info['CC_m'][0])
            cc_proj.extend(['dual_conrey_index', 'dual_embedding_index'])
            query = {'label': self.label + '.' + self.embedding_label}
    
        if format is None and 'CC_n' not in info:
            # for download
            CC_n = (1, self.an_cc_bound)
        else:
            n = info.get('n', '1-10')
            CC_n = info['CC_n'] if 'CC_n' in info else integer_options(n)
            # convert CC_n to an interval in [1,an_bound]
            CC_n = (max(1, min(CC_n)), min(self.an_cc_bound, max(CC_n)))
        an_keys = (CC_n[0] - 1, CC_n[1])
        # extra 5 primes in case we hit too many bad primes
        angles_keys = (
                bisect.bisect_left(primes_for_angles, CC_n[0]),
                min(bisect.bisect_right(primes_for_angles, CC_n[1]) + 5,
                    self.primes_cc_bound)
                )
        an_projection = 'an_normalized[%d:%d]' % an_keys
        angles_projection = 'angles[%d:%d]' % angles_keys
        if format in an_formats:
            cc_proj.append(an_projection)
        if format in angles_formats:
            cc_proj.append(angles_projection)
    
        # !! TODO - Satake parameters for eisenstein newforms are still incorrect !!
        # Write a test for these
        cc_data = list(db.mf_hecke_cc_eis.search(query, projection=cc_proj))
    
        if not cc_data:
            self.has_complex_qexp = False
        else:
            self.has_complex_qexp = True
            self.cc_data = {}
            for embedded_mf in cc_data:
                if format in an_formats:
>                   an_normalized = embedded_mf.pop(an_projection)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E                   KeyError: 'an_normalized[1:10]'

lmfdb/classical_modular_forms/web_newform.py:527: KeyError

[roed314]

Lots of failures in modular curves, but the pages seem to load correctly.

[roed314]

There's an error in cluster pictures coming from an old genus 2 curve label not working correctly: /Genus2Curve/Q/762001/a/762001/1. We should have redirects in place.

[roed314]

Lattice tests are currently broken; we'll work on fixing them after the data has stabilized.

[roed314]

Lots of test failures in genus 2 curves; many seem to result from changes to the labels (of course).

[roed314]

Here are the lint failures:

lmfdb/genus2_curves/main.py:663:27: undefined name 'aut_grp_label'
lmfdb/genus2_curves/main.py:665:32: undefined name 'geom_aut_grp_label'
lmfdb/lattice/genus.py:10:1: 'lmfdb.utils.parse_list' imported but unused
lmfdb/lfunctions/main.py:1527:16: undefined name 'Lfunction_SMF2_scalar_valued'
lmfdb/lfunctions/Lfunction.py:1041:18: undefined name 'Sample'
lmfdb/shimura_curves/web_curve.py:928:20: f-string is missing placeholders
lmfdb/siegel_modular_forms/siegel_modular_form.py:5:1: 'yaml' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:14:1: 'lmfdb.utils.TextBoxNoEg' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:14:1: 'lmfdb.utils.YesNoBox' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:14:1: 'lmfdb.utils.DoubleSelectBox' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:14:1: 'lmfdb.utils.BasicSpacer' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:14:1: 'lmfdb.utils.SelectBoxNoEg' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:27:1: 'lmfdb.utils.search_columns.FloatCol' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:27:1: 'lmfdb.utils.search_columns.CheckCol' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:29:1: 'lmfdb.siegel_modular_forms.web_newform.convert_newformlabel_from_conrey' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:29:1: 'lmfdb.siegel_modular_forms.web_newform.quad_field_knowl' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:39:1: 'lmfdb.sato_tate_groups.main.st_display_knowl' imported but unused
lmfdb/siegel_modular_forms/siegel_modular_form.py:516:5: local variable 'label' is assigned to but never used
lmfdb/siegel_modular_forms/siegel_modular_form.py:546:5: local variable 'label' is assigned to but never used
lmfdb/siegel_modular_forms/siegel_modular_form.py:1472:9: local variable 'stats_url' is assigned to but never used
lmfdb/siegel_modular_forms/siegel_modular_form.py:1745:9: local variable 'num_newforms' is assigned to but never used
lmfdb/siegel_modular_forms/siegel_modular_form.py:1750:9: local variable 'hnum_newforms' is assigned to but never used
lmfdb/siegel_modular_forms/web_newform.py:10:1: 'sage.all.LaurentPolynomialRing' imported but unused
lmfdb/siegel_modular_forms/web_newform.py:10:1: 'sage.all.PowerSeriesRing' imported but unused
lmfdb/siegel_modular_forms/web_newform.py:10:1: 'sage.all.O' imported but unused
lmfdb/siegel_modular_forms/web_newform.py:30:1: 'lmfdb.lfunctions.Lfunctionutilities.seriescoeff' imported but unused
lmfdb/siegel_modular_forms/web_newform.py:195:20: f-string is missing placeholders
lmfdb/siegel_modular_forms/web_newform.py:196:21: f-string is missing placeholders
lmfdb/siegel_modular_forms/web_newform.py:198:20: f-string is missing placeholders
lmfdb/siegel_modular_forms/web_newform.py:199:21: f-string is missing placeholders
lmfdb/siegel_modular_forms/web_newform.py:625:9: local variable 'ns1_label' is assigned to but never used
lmfdb/siegel_modular_forms/web_newform.py:626:9: local variable 'ns1_url' is assigned to but never used
lmfdb/siegel_modular_forms/web_newform.py:673:21: local variable 'lfun_url' is assigned to but never used
lmfdb/siegel_modular_forms/web_newform.py:825:9: local variable 'slabel' is assigned to but never used