issue_comments

Table actions

49 rows where issue = 1551694938 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

author_association 2

issue 1

  • ?_extra= support (draft) · 49 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1480349156 https://github.com/simonw/datasette/pull/1999#issuecomment-1480349156 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YPFXk codecov[bot] 22429695 2023-03-22T22:40:58Z 2023-03-22T22:48:57Z NONE

Codecov Report

Patch coverage: 87.89% and project coverage change: -4.43 :warning:

Comparison is base (56b0758) 92.15% compared to head (921faae) 87.73%.

:exclamation: Current head 921faae differs from pull request most recent head 69a31cd. Consider uploading reports for the commit 69a31cd to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1999 +/- ## ========================================== - Coverage 92.15% 87.73% -4.43% ========================================== Files 38 38 Lines 5560 6066 +506 ========================================== + Hits 5124 5322 +198 - Misses 436 744 +308 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/views/database.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2RhdGFiYXNlLnB5) | `96.61% <ø> (ø)` | | | [datasette/views/row.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3Jvdy5weQ==) | `87.82% <ø> (ø)` | | | [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `69.11% <86.76%> (-23.46%)` | :arrow_down: | | [datasette/renderer.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3JlbmRlcmVyLnB5) | `93.33% <90.90%> (-0.87%)` | :arrow_down: | | [datasette/views/base.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2Jhc2UucHk=) | `92.78% <91.66%> (-2.39%)` | :arrow_down: | | [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.48% <100.00%> (-0.01%)` | :arrow_down: | | [datasette/cli.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `79.93% <100.00%> (ø)` | | | [datasette/hookspecs.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2hvb2tzcGVjcy5weQ==) | `100.00% <100.00%> (ø)` | | | [datasette/publish/cloudrun.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3B1Ymxpc2gvY2xvdWRydW4ucHk=) | `97.29% <100.00%> (ø)` | | | [datasette/utils/\_\_init\_\_.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.59% <100.00%> (-0.27%)` | :arrow_down: | ... and [2 files with indirect coverage changes](https://codecov.io/gh/simonw/datasette/pull/1999/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1480343044 https://github.com/simonw/datasette/pull/1999#issuecomment-1480343044 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YPD4E simonw 9599 2023-03-22T22:33:15Z 2023-03-22T22:33:15Z OWNER

This still needs documentation, but now the tests are passing I'm going to merge it into main!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1480339527 https://github.com/simonw/datasette/pull/1999#issuecomment-1480339527 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YPDBH simonw 9599 2023-03-22T22:28:54Z 2023-03-22T22:28:54Z OWNER

I hacked at the CSV stuff until it worked.

I need to clean it up though, but I can do that in this separate task: - #1101

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1480299765 https://github.com/simonw/datasette/pull/1999#issuecomment-1480299765 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YO5T1 simonw 9599 2023-03-22T21:44:03Z 2023-03-22T21:45:10Z OWNER

Oh this is a bit tricky.

I have a failing test because a plugin that uses the extra_css_urls hook can't see the columns for the page.

Turns out that bit comes from here:

https://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/app.py#L1203-L1217

Which assumes the context has "columns" - but that's only now available if ?_extra=columns was passed.

Actually I think I can cheat here, since it's still getting the HTML context in order to render the template.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1480179217 https://github.com/simonw/datasette/pull/1999#issuecomment-1480179217 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YOb4R simonw 9599 2023-03-22T19:55:31Z 2023-03-22T21:34:02Z OWNER

I rebased from main. Now:

FAILED tests/test_csv.py::test_table_csv - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_cors_headers - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_no_header - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_with_labels - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_with_nullable_labels - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_blob_columns - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_download - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_stream - assert 1 == 101 FAILED tests/test_csv.py::test_csv_trace - AttributeError: 'NoneType' object has no attribute 'text' FAILED tests/test_plugins.py::test_hook_extra_css_urls[/fixtures/sortable-expected_decoded_object2] - AssertionError: assert {'added': 15,...ortable', ...} == {'added': 15,...ortable', ...} FAILED tests/test_plugins.py::test_hook_render_cell_demo - AttributeError: 'NoneType' object has no attribute 'string' FAILED tests/test_plugins.py::test_hook_render_cell_async[/fixtures/simple_primary_key] - assert b'RENDER_CELL_ASYNC_RESULT' in b'<!DOCTYPE html>\n<html>\n<head>\n <title>Error 500</title>\n <link rel="stylesheet" href="/-/static/app.css?d5...ils != detailsClickedWithin\n ).forEach(details => details.open = false);\n});\nexecute_write_fn_connection_exception - Failed: Timeout >\n\n\n\n\nnnection_exception - Failed: Timeout >', ...} == {'1+1': 2, 'c... 0xXXX>', ...} FAILED tests/test_permissions.py::test_permissions_checked[/fixtures/simple_primary_key-permissions3] - assert 500 in (200, 403) FAILED tests/test_table_html.py::test_table_csv_json_export_interface - assert 500 == 200 FAILED tests/test_table_html.py::test_table_metadata - assert 500 == 200 FAILED tests/test_html.py::test_css_classes_on_body[/fixtures/simple_primary_key-expected_classes3] - assert 500 == 200 FAILED tests/test_html.py::test_templates_considered[/fixtures/simple_primary_key-table-fixtures-simple_primary_key.html, *table.html] - assert 500 == 200 FAILED tests/test_plugins.py::test_hook_table_actions[simple_view] - AssertionError: assert [] == [{'href': '/'...simple_view'}] ERROR tests/test_internals_database.py::test_execute_write_fn_connection_exception - Failed: Timeout >1.0s ERROR tests/test_internals_database.py::test_mtime_ns - Failed: Timeout >1.0s ERROR tests/test_internals_database.py::test_mtime_ns_is_none_for_memory - Failed: Timeout >1.0s ERROR tests/test_internals_database.py::test_is_mutable - Failed: Timeout >1.0s ERROR tests/test_internals_database.py::test_database_memory_name - Failed: Timeout >1.0s ERROR tests/test_internals_database.py::test_in_memory_databases_forbid_writes - Failed: Timeout >1.0s =============== 21 failed, 1275 passed, 2 skipped, 1 xfailed, 6 errors in 59.18s ================

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1480163485 https://github.com/simonw/datasette/pull/1999#issuecomment-1480163485 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YOYCd simonw 9599 2023-03-22T19:48:00Z 2023-03-22T19:48:00Z OWNER

Getting close now! Only 13 failures left, mostly relating to CSV. FAILED tests/test_csv.py::test_table_csv - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_cors_headers - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_no_header - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_with_labels - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_with_nullable_labels - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_blob_columns - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_download - assert 500 == 200 FAILED tests/test_csv.py::test_table_csv_stream - assert 1 == 101 FAILED tests/test_plugins.py::test_hook_extra_css_urls[/fixtures/sortable-expected_decoded_object2] - AssertionError: assert {'added': 15,...ortable', ...} == {'added': 15,...ortable', ...} FAILED tests/test_plugins.py::test_hook_register_facet_classes - KeyError: 'suggested_facets' FAILED tests/test_csv.py::test_csv_trace - AttributeError: 'NoneType' object has no attribute 'text' FAILED tests/test_plugins.py::test_hook_extra_body_script[/fixtures/sortable-expected_extra_body_script2] - AssertionError: assert {'added': 15,...ixtures', ...} == {'added': 15,...ixtures', ...} FAILED tests/test_plugins.py::test_hook_register_output_renderer_all_parameters - assert {'1+1': 2, 'c... 0xXXX>', ...} == {'1+1': 2, 'c... 0xXXX>', ...} =============== 13 failed, 1287 passed, 2 skipped, 1 xfailed in 61.57s (0:01:01) ================

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1480154453 https://github.com/simonw/datasette/pull/1999#issuecomment-1480154453 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YOV1V simonw 9599 2023-03-22T19:41:39Z 2023-03-22T19:43:04Z OWNER

To replace this code: https://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/views/base.py#L110-L122

Maybe datasette.render_template() should optionally accept a list of templates.

https://docs.datasette.io/en/stable/internals.html#await-render-template-template-context-none-request-none - turns out it does already:

If this is a list of template file names then the first one that exists will be loaded and rendered.

It doesn't have an easy way to populate that select_templates debug template variable though.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1477082852 https://github.com/simonw/datasette/pull/1999#issuecomment-1477082852 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YCn7k simonw 9599 2023-03-20T23:27:25Z 2023-03-20T23:27:25Z OWNER

Urgh getting CSV to work is going to be so hard, because the logic for that currently lives in a huge chunk of code in BaseView which depends on the old design of the data() method:

https://github.com/simonw/datasette/blob/4bb49848697e40b8b9a1557be42b8e59eac965b3/datasette/views/base.py#L177-L343

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1476860334 https://github.com/simonw/datasette/pull/1999#issuecomment-1476860334 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YBxmu simonw 9599 2023-03-20T20:06:56Z 2023-03-20T22:09:03Z OWNER

pytest -n 8 locally says:

FAILED tests/test_canned_queries.py::test_canned_query_form_csrf_hidden_field[add_name_specify_id-True] - assert '<input type="hidden" name="csrftoken" value="' in '<!DOCTYPE html>\n<html>\n<head>\n <title>Error 500</title>\n <link rel="stylesheet" href="/-/static/app.css?d59...ils != detailsClickedWithin\n ).forEach(details => details.open = false);\n});\n</... FAILED tests/test_canned_queries.py::test_canned_query_form_csrf_hidden_field[add_name-True] - assert '<input type="hidden" name="csrftoken" value="' in '<!DOCTYPE html>\n<html>\n<head>\n <title>Error 500</title>\n <link rel="stylesheet" href="/-/static/app.css?d59...ils != detailsClickedWithin\n ).forEach(details => details.open = false);\n});\n</... FAILED tests/test_canned_queries.py::test_canned_query_permissions - AssertionError: assert 200 == 500 FAILED tests/test_canned_queries.py::test_canned_query_with_named_parameter - KeyError: 'rows' FAILED tests/test_canned_queries.py::test_canned_write_custom_template - assert 500 == 200 FAILED tests/test_canned_queries.py::test_canned_write_query_disabled_for_immutable_database - assert 500 == 200 FAILED tests/test_canned_queries.py::test_custom_params - assert '<input type="text" id="qp3" name="extra" value="foo">' in '<!DOCTYPE html>\n<html>\n<head>\n <title>Error 500</title>\n <link rel="stylesheet" href="/-/static/app.css?d59...ils != detailsClickedWithin\n ).forEach(details => details.open = false);\... FAILED tests/test_canned_queries.py::test_insert - KeyError: 'ds_csrftoken' FAILED tests/test_canned_queries.py::test_insert_error - KeyError: 'ds_csrftoken' FAILED tests/test_canned_queries.py::test_magic_parameters_csrf_json[False-True] - KeyError: 'ds_csrftoken' FAILED tests/test_canned_queries.py::test_magic_parameters_csrf_json[True-True] - KeyError: 'ds_csrftoken' FAILED tests/test_canned_queries.py::test_vary_header - KeyError: 'vary' FAILED tests/test_csv.py::test_csv_trace - AssertionError: assert 'application/...charset=utf-8' == 'text/html; charset=utf-8' FAILED tests/test_csv.py::test_table_csv - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8' FAILED tests/test_csv.py::test_table_csv_blob_columns - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8' FAILED tests/test_csv.py::test_table_csv_download - AssertionError: assert 'application/...charset=utf-8' == 'text/csv; charset=utf-8' FAILED tests/test_csv.py::test_table_csv_no_header - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8' FAILED tests/test_csv.py::test_table_csv_stream - assert 1 == 101 FAILED tests/test_csv.py::test_table_csv_stream_does_not_calculate_counts - AttributeError: 'NoneType' object has no attribute 'text' FAILED tests/test_csv.py::test_table_csv_stream_does_not_calculate_facets - AttributeError: 'NoneType' object has no attribute 'text' FAILED tests/test_csv.py::test_table_csv_with_labels - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8' FAILED tests/test_csv.py::test_table_csv_with_nullable_labels - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8' FAILED tests/test_facets.py::test_array_facet_handle_duplicate_tags - KeyError: 'facet_results' FAILED tests/test_facets.py::test_conflicting_facet_names_json - AssertionError: assert {'results', 'timed_out'} == {'created', '...gs', 'tags_2'} FAILED tests/test_facets.py::test_facet_size - KeyError: 'suggested_facets' FAILED tests/test_facets.py::test_json_array_with_blanks_and_nulls - KeyError: 'suggested_facets' FAILED tests/test_facets.py::test_other_types_of_facet_in_metadata - assert '<strong>created (date)\n' in '<!DOCTYPE html>\n<html>\n<head>\n <title>fixtures: facetable: 15 rows</title>\n <link rel="stylesheet" href="/-...name": "table", "request_path": "/fixtures/facetable", "added": 15, "columns": null};</script>\n\n\n\n</bo... FAILED tests/test_html.py::test_alternate_url_json[/fixtures/facetable-http://localhost/fixtures/facetable.json] - KeyError: 'link' FAILED tests/test_html.py::test_alternate_url_json[/fixtures/table~2Fwith~2Fslashes~2Ecsv-http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json] - KeyError: 'link' FAILED tests/test_html.py::test_templates_considered[/fixtures/simple_primary_key-table-fixtures-simple_primary_key.html, *table.html] - assert '<!-- Templates considered: table-fixtures-simple_primary_key.html, *table.html -->' in '<!DOCTYPE html>\n<html>\n<head>\n <title>fixtures: simple_primary_key: 5 rows</title>\n <link rel="stylesheet" ...able", "request_path": "/fixtures/simple_primary... FAILED tests/test_html.py::test_templates_considered[/fixtures/table~2Fwith~2Fslashes~2Ecsv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html] - assert '<!-- Templates considered: table-fixtures-tablewithslashescsv-fa7563.html, *table.html -->' in '<!DOCTYPE html>\n<html>\n<head>\n <title>fixtures: table/with/slashes.csv: 1 row</title>\n <link rel="styleshee...quest_path": "/fixtures/table~2Fwith~2Fs... FAILED tests/test_plugins.py::test_hook_extra_body_script[/fixtures/sortable-expected_extra_body_script2] - AssertionError: assert {'added': 15,...ixtures', ...} == {'added': 15,...ixtures', ...} FAILED tests/test_plugins.py::test_hook_extra_css_urls[/fixtures/sortable-expected_decoded_object2] - AssertionError: assert {'added': 15,...ortable', ...} == {'added': 15,...ortable', ...} FAILED tests/test_plugins.py::test_hook_register_facet_classes - KeyError: 'suggested_facets' FAILED tests/test_plugins.py::test_hook_register_magic_parameters - KeyError: 'ds_csrftoken' FAILED tests/test_plugins.py::test_hook_register_output_renderer_all_parameters - assert {'next': None...', ...}, ...]} == {'1+1': 2, 'c... 0xXXX>', ...} FAILED tests/test_plugins.py::test_hook_register_output_renderer_no_parameters - assert b'Hello' == b'{"ok": true... "n": null}]}' FAILED tests/test_plugins.py::test_hook_register_output_renderer_returning_broken_value - assert 200 == 500 FAILED tests/test_plugins.py::test_hook_register_output_renderer_returning_response - assert {'next': None...', ...}, ...]} == {'this_is': 'json'} === 39 failed, 1259 passed, 2 skipped, 1 xfailed in 58.07s ===

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1476898261 https://github.com/simonw/datasette/pull/1999#issuecomment-1476898261 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YB63V simonw 9599 2023-03-20T20:37:52Z 2023-03-20T20:37:52Z OWNER

Manual testing spotted a bug.

/content/repos.json?owner=9599&_facet_array=topics - does not return a facet_results key.

/content/repos.json?owner=9599&_facet_array=topics&_facet=owner does.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1476854645 https://github.com/simonw/datasette/pull/1999#issuecomment-1476854645 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YBwN1 simonw 9599 2023-03-20T20:02:24Z 2023-03-20T20:02:24Z OWNER

``` def test_routes(routes, path, expected_class, expected_matches): match, view = resolve_routes(routes, path) if expected_class is None: assert match is None else:

      assert view.view_class.__name__ == expected_class

E AttributeError: 'function' object has no attribute 'view_class' ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1476851525 https://github.com/simonw/datasette/pull/1999#issuecomment-1476851525 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YBvdF simonw 9599 2023-03-20T19:59:51Z 2023-03-20T20:01:40Z OWNER

Three failures in test_html.py: FAILED tests/test_html.py::test_templates_considered[/fixtures/simple_primary_key-table-fixtures-simple_primary_key.html, *table.html] - assert '<!-- Templates considered: table-fixtures-simple_primary_key.html, *table.html -->' in '<!DOCTYPE html>\n<html>\n<head>\n <title>fixtures: simple_primary_key: 5 rows</title>\n <link rel="stylesheet" ...able", "request_path": "/fixtures/simple_primary_key", "ad... FAILED tests/test_html.py::test_templates_considered[/fixtures/table~2Fwith~2Fslashes~2Ecsv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html] - assert '<!-- Templates considered: table-fixtures-tablewithslashescsv-fa7563.html, *table.html -->' in '<!DOCTYPE html>\n<html>\n<head>\n <title>fixtures: table/with/slashes.csv: 1 row</title>\n <link rel="styleshee...quest_path": "/fixtures/table~2Fwith~2Fslashes~2Ec... FAILED tests/test_html.py::test_alternate_url_json[/fixtures/facetable-http://localhost/fixtures/facetable.json] - KeyError: 'link' FAILED tests/test_html.py::test_alternate_url_json[/fixtures/table~2Fwith~2Fslashes~2Ecsv-http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json] - KeyError: 'link'

In CI I'm getting this: https://github.com/simonw/datasette/actions/runs/4472331872/jobs/7858360154

====== 44 failed, 1222 passed, 1 skipped, 1 xfailed in 171.24s (0:02:51) =======

Including failures in tests/test_plugins.py and tests/test_routes.py.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1476781401 https://github.com/simonw/datasette/pull/1999#issuecomment-1476781401 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5YBeVZ simonw 9599 2023-03-20T19:01:35Z 2023-03-20T19:01:35Z OWNER

Down to just these failures: FAILED tests/test_table_html.py::test_sort_links - AssertionError: assert [{'attrs': {'class': ['col-Link'], 'scope': 'col', 'data-column': 'Link', 'data-column-type': '', 'data-column-not-null': '0', 'data-is-pk': '0'}, 'a_href': None}, {'attrs': {'class': ['col-pk1'], 'scope': 'col', 'data-column': 'pk1', 'data-column-typ... FAILED tests/test_table_html.py::test_table_html_simple_primary_key - AssertionError: assert 'id\xa0▼' == 'id' FAILED tests/test_table_html.py::test_table_csv_json_export_interface - AssertionError: assert ['/fixtures/simple_primary_key.json?id__gt=2', '/fixtures/simple_primary_key.testall?id__gt=2', '/fixtures/simple_primary_key.testnone?id__gt=2', '/fixtures/simple_primary_key.testresponse?id__gt=2', '/fixtures/simple_primary_key.csv?id__gt=2&_size=ma... FAILED tests/test_table_html.py::test_csv_json_export_links_include_labels_if_foreign_keys - AssertionError: assert ['/fixtures/facetable.json?_labels=on', '/fixtures/facetable.testall?_labels=on', '/fixtures/facetable.testnone?_labels=on', '/fixtures/facetable.testresponse?_labels=on', '/fixtures/facetable.csv?_labels=on&_size=max', '#export'] == ['/', '', '#export'] FAILED tests/test_table_html.py::test_rowid_sortable_no_primary_key - AssertionError: assert 'rowid\xa0▼' == 'rowid' FAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/infinity-expected_column_options0] - AssertionError: assert ['- column -', 'rowid', 'value'] == ['- column -'] FAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/primary_key_multiple_columns-expected_column_options1] - AssertionError: assert ['- column -', 'id', 'content', 'content2'] == ['- column -'] FAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/compound_primary_key-expected_column_options2] - AssertionError: assert ['- column -', 'pk1', 'pk2', 'content'] == ['- column -'] FAILED tests/test_table_html.py::test_table_html_filter_form_still_shows_nocol_columns - AssertionError: assert ['- column -'] == ['- column -', 'pk1', 'pk2', 'content', 'sortable_with_nulls', 'sortable_with_nulls_2', 'text', 'sortable'] FAILED tests/test_table_html.py::test_metadata_sort - AssertionError: assert ['id', 'name\xa0▼'] == ['id', 'name'] FAILED tests/test_table_html.py::test_metadata_sort_desc - AssertionError: assert ['pk\xa0▲', 'name'] == ['pk', 'name'] === 11 failed, 57 passed in 7.59s ===

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460759358 https://github.com/simonw/datasette/pull/1999#issuecomment-1460759358 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XEWs- simonw 9599 2023-03-08T19:48:13Z 2023-03-20T18:47:12Z OWNER

Breaking this down into smaller steps:

  • [x] Get ?_next= working
  • [x] Implement extensions - so .json is needed again for the JSON version, and anything without an extension is passed through a new code path for HTML
  • [ ] That HTML view should only access JSON data, which can be seen by using .context - this will require a lot of updates to templates (it may be necessary to still provide access to some helper functions though). This will form the basis of the ambition to fully document the template context.
  • [ ] Get a bunch of the existing table HTML and JSON tests to pass
  • [ ] Use those tests to refactor the nasty _next code, see https://github.com/simonw/datasette/pull/1999#issuecomment-1460905469
  • [ ] Figure out how the register_output_renderer(datasette) plugin hook should work
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1475074311 https://github.com/simonw/datasette/pull/1999#issuecomment-1475074311 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5X69kH simonw 9599 2023-03-19T02:16:08Z 2023-03-19T02:17:33Z OWNER

Next big challenge: fix all the tests in test_table_html.py:

FAILED tests/test_table_html.py::test_add_filter_redirects - assert 500 == 302 FAILED tests/test_table_html.py::test_existing_filter_redirects - assert 500 == 302 FAILED tests/test_table_html.py::test_reflected_hidden_form_fields[_facet=_neighborhood-expected_hidden0] - AssertionError: assert {} == {'_facet': '_neighborhood'} FAILED tests/test_table_html.py::test_reflected_hidden_form_fields[_where=1+=+1&_col=_city_id-expected_hidden1] - AssertionError: assert {} == {'_col': '_ci...ere': '1 = 1'} FAILED tests/test_table_html.py::test_reflected_hidden_form_fields[_facet=_neighborhood&_neighborhood__exact=Downtown-expected_hidden2] - AssertionError: assert {} == {'_facet': '_neighborhood'} FAILED tests/test_table_html.py::test_reflected_hidden_form_fields[_facet=_neighborhood&_city_id__gt=1-expected_hidden3] - AssertionError: assert {} == {'_facet': '_neighborhood'} FAILED tests/test_table_html.py::test_empty_search_parameter_gets_removed - assert 500 == 302 FAILED tests/test_table_html.py::test_searchable_view_persists_fts_table - AssertionError: assert [('_fts_table...ts_pk', 'pk')] == [] FAILED tests/test_table_html.py::test_sort_by_desc_redirects - assert 500 == 302 FAILED tests/test_table_html.py::test_sort_links - AssertionError: assert [{'a_href': N...', ...}}, ...] == [{'a_href': N...', ...}}, ...] FAILED tests/test_table_html.py::test_facets_persist_through_filter_form - AssertionError: assert [] == [('_facet', '...ray', 'tags')] FAILED tests/test_table_html.py::test_next_does_not_persist_in_hidden_field - AssertionError: assert [] == [('_size', '1')] FAILED tests/test_table_html.py::test_table_html_simple_primary_key - AttributeError: 'NoneType' object has no attribute 'string' FAILED tests/test_table_html.py::test_table_csv_json_export_interface - AssertionError: assert ['/fixtures/s...x', '#export'] == ['/', '', '#export'] FAILED tests/test_table_html.py::test_csv_json_export_links_include_labels_if_foreign_keys - AssertionError: assert ['/fixtures/f...x', '#export'] == ['/', '', '#export'] FAILED tests/test_table_html.py::test_table_html_no_primary_key - AssertionError: assert ['content', 'a', 'b', 'c'] == ['a', 'b', 'c'] FAILED tests/test_table_html.py::test_rowid_sortable_no_primary_key - AttributeError: 'NoneType' object has no attribute 'string' FAILED tests/test_table_html.py::test_table_html_compound_primary_key - AssertionError: assert 'Link' == 'pk1' FAILED tests/test_table_html.py::test_table_html_foreign_key_links - assert [['<td class=...">\xa0</td>']] == [['<td class=...">\xa0</td>']] FAILED tests/test_table_html.py::test_table_html_disable_foreign_key_links_with_labels - assert [['<td class=...str">b</td>']] == [['<td class=...str">b</td>']] FAILED tests/test_table_html.py::test_table_html_foreign_key_custom_label_column - assert [['<td class=...1</em></td>']] == [['<td class=...1</em></td>']] FAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/infinity-expected_column_options0] - AssertionError: assert ['- column -'...wid', 'value'] == ['- column -'] FAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/primary_key_multiple_columns-expected_column_options1] - AssertionError: assert ['- column -'...', 'content2'] == ['- column -'] FAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/compound_primary_key-expected_column_options2] - AssertionError: assert ['- column -'...2', 'content'] == ['- column -'] FAILED tests/test_table_html.py::test_table_html_filter_form_still_shows_nocol_columns - AssertionError: assert ['- column -'] == ['- column -'...nulls_2', ...] FAILED tests/test_table_html.py::test_compound_primary_key_with_foreign_key_references - assert [['<td class=...ne</a></td>']] == [['<td class=...ne</a></td>']] FAILED tests/test_table_html.py::test_view_html - assert None is not None FAILED tests/test_table_html.py::test_extra_where_clauses - assert [('_where', "...'_city_id=1')] == [] FAILED tests/test_table_html.py::test_other_hidden_form_fields[/fixtures/facetable?_size=10-expected_hidden0] - AssertionError: assert [] == [('_size', '10')] FAILED tests/test_table_html.py::test_other_hidden_form_fields[/fixtures/facetable?_size=10&_ignore=1&_ignore=2-expected_hidden1] - AssertionError: assert [] == [('_size', '1...ignore', '2')] FAILED tests/test_table_html.py::test_search_and_sort_fields_not_duplicated[/fixtures/searchable?_sort=text2&_where=1-expected_hidden3] - AssertionError: assert [] == [('_where', '1')] FAILED tests/test_table_html.py::test_binary_data_display_in_table - assert [['<td class=...">\xa0</td>']] == [['<td class=...">\xa0</td>']] FAILED tests/test_table_html.py::test_metadata_sort - AttributeError: 'NoneType' object has no attribute 'string' FAILED tests/test_table_html.py::test_metadata_sort_desc - AttributeError: 'NoneType' object has no attribute 'string' FAILED tests/test_table_html.py::test_facet_more_links[5-/fixtures/facetable?_facet=_neighborhood-2-True-/fixtures/facetable?_facet=_neighborhood&_facet_size=max] - assert 0 == 2 FAILED tests/test_table_html.py::test_facet_more_links[5-/fixtures/facetable?_facet=_neighborhood&_facet_size=50-5-True-/fixtures/facetable?_facet=_neighborhood&_facet_size=max] - assert 0 == 5 FAILED tests/test_table_html.py::test_facet_total - assert 500 == 200

Deduped that's 30 tests:

FAILED tests/test_table_html.py::test_add_filter_redirects FAILED tests/test_table_html.py::test_binary_data_display_in_table FAILED tests/test_table_html.py::test_compound_primary_key_with_foreign_key_references FAILED tests/test_table_html.py::test_csv_json_export_links_include_labels_if_foreign_keys FAILED tests/test_table_html.py::test_empty_search_parameter_gets_removed FAILED tests/test_table_html.py::test_existing_filter_redirects FAILED tests/test_table_html.py::test_extra_where_clauses FAILED tests/test_table_html.py::test_facet_more_links FAILED tests/test_table_html.py::test_facet_total FAILED tests/test_table_html.py::test_facets_persist_through_filter_form FAILED tests/test_table_html.py::test_metadata_sort FAILED tests/test_table_html.py::test_metadata_sort_desc FAILED tests/test_table_html.py::test_next_does_not_persist_in_hidden_field FAILED tests/test_table_html.py::test_other_hidden_form_fields FAILED tests/test_table_html.py::test_reflected_hidden_form_fields FAILED tests/test_table_html.py::test_rowid_sortable_no_primary_key FAILED tests/test_table_html.py::test_search_and_sort_fields_not_duplicated FAILED tests/test_table_html.py::test_searchable_view_persists_fts_table FAILED tests/test_table_html.py::test_sort_by_desc_redirects FAILED tests/test_table_html.py::test_sort_links FAILED tests/test_table_html.py::test_table_csv_json_export_interface FAILED tests/test_table_html.py::test_table_html_compound_primary_key FAILED tests/test_table_html.py::test_table_html_disable_foreign_key_links_with_labels FAILED tests/test_table_html.py::test_table_html_filter_form_column_options FAILED tests/test_table_html.py::test_table_html_filter_form_still_shows_nocol_columns FAILED tests/test_table_html.py::test_table_html_foreign_key_custom_label_column FAILED tests/test_table_html.py::test_table_html_foreign_key_links FAILED tests/test_table_html.py::test_table_html_no_primary_key FAILED tests/test_table_html.py::test_table_html_simple_primary_key FAILED tests/test_table_html.py::test_view_html

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1475074025 https://github.com/simonw/datasette/pull/1999#issuecomment-1475074025 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5X69fp simonw 9599 2023-03-19T02:14:28Z 2023-03-19T02:14:51Z OWNER

I had to replicate quite a bit of this logic from base.py:

https://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/views/base.py#L526-L544

I should refactor this when I move the canned / arbitrary query views away from that base class too.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1475016834 https://github.com/simonw/datasette/pull/1999#issuecomment-1475016834 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5X6viC simonw 9599 2023-03-18T22:30:31Z 2023-03-18T22:30:31Z OWNER

test_paginate_using_link_header will be tricky.

The reason the tests are failing is that json_renderer() attempts to populate the link header using data["next_url"] - but that's not present unless ?_extra=next_url has been passed:

https://github.com/simonw/datasette/blob/2f38479dcc81f11a4362f4e28511fa88afc34e61/datasette/renderer.py#L101-L102

But I can only rely on next being present, not next_url.

I thought I could maybe assemble the link header using next, by turning that into a next_url link - but there's some custom logic which kicks in for pagination against SQL views (offset/limit pagination) to calculate the next_url which isn't easily replicable at the json_renderer() layer, in the _next_value_and_url() utility function:

https://github.com/simonw/datasette/blob/2f38479dcc81f11a4362f4e28511fa88afc34e61/datasette/views/table.py#L2275-L2282

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1475003292 https://github.com/simonw/datasette/pull/1999#issuecomment-1475003292 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5X6sOc simonw 9599 2023-03-18T21:46:20Z 2023-03-18T21:46:20Z OWNER

Now 25 failures in test_table_api.py: FAILED tests/test_table_api.py::test_expand_labels - assert {'2': {'pk': 2, 'created': '2019-01-14 08:00:00', 'planet_int': 1, 'on_earth': 1, 'state': 'CA', '_city_id': 1, '_neighborhood': 'Dogpatch', 'tags': '["tag1", "tag3"]', 'complex_array'... FAILED tests/test_table_api.py::test_expand_label - AssertionError: assert {'1': {'pk': '1', 'foreign_key_with_label': '1', 'foreign_key_with_blank_label': '3', 'foreign_key_with_no_label': '1', 'foreign_key_compound_pk1': 'a', 'foreign_key_co... FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json-max-age=5] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=invalid-max-age=5] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=10-max-age=10] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=0-no-cache] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_infinity_returned_as_null - AssertionError: assert [{'rowid': 1, 'value': inf}, {'rowid': 2, 'value': -inf}, {'rowid': 3, 'value': 1.5}] == [{'rowid': 1, 'value': None}, {'rowid': 2, 'value': None}, {'rowid': 3, 'value'... FAILED tests/test_table_api.py::test_null_and_compound_foreign_keys_are_not_expanded - AssertionError: assert [{'pk': '1', 'foreign_key_with_label': '1', 'foreign_key_with_blank_label': '3', 'foreign_key_with_no_label': '1', 'foreign_key_compound_pk1': 'a', 'foreign_key_compoun... FAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array-expected_json0-None] - assert [{'rowid': 1, 'data': "b'\\x15\\x1c\\x02\\xc7\\xad\\x05\\xfe'"}, {'rowid': 2, 'data': "b'\\x15\\x1c\\x03\\xc7\\xad\\x05\\xfe'"}, {'rowid': 3, 'data': None}] == [{'rowid': 1, 'data': {'... FAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array&_nl=on-None-{"rowid": 1, "data": {"$base64": true, "encoded": "FRwCx60F/g=="}}\n{"rowid": 2, "data": {"$base64": true, "encoded": "FRwDx60F/g=="}}\n{"rowid": 3, "data": null}] - assert '{"ok": false, "error": "Object of type bytes is not JSON serializable", "status": 500, "title": null}' == '{"rowid": 1, "data": {"$base64": true, "encoded": "FRwCx60F/g=="}}\n{"rowid"... FAILED tests/test_table_api.py::test_paginate_using_link_header[] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrays] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrayfirst] - assert 400 == 200 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=object] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=objects] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array&_nl=on] - assert 1 == 21 FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=created-expected_columns0] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=created-expected_columns1] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created-expected_columns2] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=state-expected_columns3] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created&_nocol=created-expected_columns4] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=state&_facet=state-expected_columns5] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_nocol=content-expected_columns6] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_col=content-expected_columns7] - KeyError: 'columns' ============================================================================= 25 failed, 86 passed, 1 xfailed in 7.18s =============================================================================

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1474704790 https://github.com/simonw/datasette/pull/1999#issuecomment-1474704790 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5X5jWW simonw 9599 2023-03-18T04:52:59Z 2023-03-18T04:52:59Z OWNER

Here are the next set of tests to get passing: % pytest tests/test_table_api.py FAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=state&_facet=_city_id-expected_facet_results0] - KeyError: 'name' FAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=state&_facet=_city_id&state=MI-expected_facet_results1] - KeyError: 'name' FAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=planet_int-expected_facet_results2] - KeyError: 'name' FAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=planet_int&planet_int=1-expected_facet_results3] - KeyError: 'name' FAILED tests/test_table_api.py::test_suggested_facets - KeyError: 'suggested_facets' FAILED tests/test_table_api.py::test_allow_facet_off - KeyError: 'suggested_facets' FAILED tests/test_table_api.py::test_suggest_facets_off - KeyError: 'suggested_facets' FAILED tests/test_table_api.py::test_nofacet[True] - KeyError: 'suggested_facets' FAILED tests/test_table_api.py::test_nofacet[False] - KeyError: 'suggested_facets' FAILED tests/test_table_api.py::test_nosuggest[True] - KeyError: 'suggested_facets' FAILED tests/test_table_api.py::test_nosuggest[False] - KeyError: 'suggested_facets' FAILED tests/test_table_api.py::test_nocount[True-None] - KeyError: 'count' FAILED tests/test_table_api.py::test_nocount[False-15] - KeyError: 'count' FAILED tests/test_table_api.py::test_expand_labels - AssertionError: assert {'13': {'_cit...:00:00', ...}} == {'13': {'_cit...:00:00', ...}} FAILED tests/test_table_api.py::test_expand_label - AssertionError: assert {'1': {'forei...l': '1', ...}} == {'1': {'forei...': '1'}, ...}} FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json-max-age=5] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=invalid-max-age=5] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=10-max-age=10] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=0-no-cache] - KeyError: 'Cache-Control' FAILED tests/test_table_api.py::test_infinity_returned_as_null - AssertionError: assert [{'rowid': 1,...'value': 1.5}] == [{'rowid': 1,...'value': 1.5}] FAILED tests/test_table_api.py::test_null_and_compound_foreign_keys_are_not_expanded - AssertionError: assert [{'foreign_ke...': None, ...}] == [{'foreign_ke...': None, ...}] FAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array-expected_json0-None] - assert [{'data': "b'..., 'rowid': 3}] == [{'data': {'$..., 'rowid': 3}] FAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array&_nl=on-None-{"rowid": 1, "data": {"$base64": true, "encoded": "FRwCx60F/g=="}}\n{"rowid": 2, "data": {"$base64": true, "encoded": "FRwDx60F/g=="}}\n{"rowid": 3, "data": null}] - assert '{"ok": false...title": null}' == '{"rowid": 1,..."data": null}' FAILED tests/test_table_api.py::test_paginate_using_link_header[] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrays] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrayfirst] - assert 400 == 200 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=object] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=objects] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array] - assert 1 == 21 FAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array&_nl=on] - assert 1 == 21 FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=created-expected_columns0] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=created-expected_columns1] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created-expected_columns2] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=state-expected_columns3] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created&_nocol=created-expected_columns4] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=state&_facet=state-expected_columns5] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_nocol=content-expected_columns6] - KeyError: 'columns' FAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_col=content-expected_columns7] - KeyError: 'columns' ============================================================================= 38 failed, 73 passed, 1 xfailed in 7.25s =============================================================================

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1463113856 https://github.com/simonw/datasette/pull/1999#issuecomment-1463113856 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XNViA simonw 9599 2023-03-10T02:13:15Z 2023-03-10T02:13:15Z OWNER

Idea for if this change ends up making a bunch of breaking changes to the templates (which I think it should) - I can generate a GitHub diff link between the old and new templates and include that link in the 1.0 upgrade documentation to help people who wrote custom templates see what they might need to change - with minimal effort from myself to document those changes.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1463024951 https://github.com/simonw/datasette/pull/1999#issuecomment-1463024951 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XM_03 simonw 9599 2023-03-10T00:17:58Z 2023-03-10T00:17:58Z OWNER

Renderers have an impact on three different pages: query results, table page and row page.

The row page feature is incomplete though:

https://congress-legislators.datasettes.com/legislators/social_media/A000055

Why is there no .csv link there?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1463023674 https://github.com/simonw/datasette/pull/1999#issuecomment-1463023674 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XM_g6 simonw 9599 2023-03-10T00:16:03Z 2023-03-10T00:16:03Z OWNER

I also need to figure out the renderers stuff, so I can link to the right URLs for CSV and JSON and other formats:

https://github.com/simonw/datasette/blob/6d07a7da1531cd749844fc6827d9a1e57009b2ea/datasette/views/base.py#L474-L518

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1463022397 https://github.com/simonw/datasette/pull/1999#issuecomment-1463022397 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XM_M9 simonw 9599 2023-03-10T00:14:21Z 2023-03-10T00:14:21Z OWNER

Ironically the thing I most need right now is comprehensive documentation of what variables are being passed to the templates!

One big challenge is that I need to untangle the template context that happens in BaseView - I'm hacking that together at the moment, but I need a real answer for how that should work in a world in which view functions aren't using that base class.

https://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/views/base.py#L110-L145

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1463021383 https://github.com/simonw/datasette/pull/1999#issuecomment-1463021383 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XM-9H simonw 9599 2023-03-10T00:12:50Z 2023-03-10T00:12:50Z OWNER

Now at 34 failed, 34 passed.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1463005744 https://github.com/simonw/datasette/pull/1999#issuecomment-1463005744 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XM7Iw simonw 9599 2023-03-09T23:52:15Z 2023-03-09T23:52:23Z OWNER

I need to figure out what to do about extra_context_from_filters - which was previously passed straight to the HTML context.

https://github.com/simonw/datasette/blob/11f7feb7a3f7166c71389786880863d60ed3d165/datasette/views/table.py#L406-L422

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1462997800 https://github.com/simonw/datasette/pull/1999#issuecomment-1462997800 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XM5Mo simonw 9599 2023-03-09T23:39:47Z 2023-03-09T23:39:47Z OWNER

Found a neat trick: ```diff diff --git a/datasette/app.py b/datasette/app.py index 186f192d..40416713 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -23,7 +23,13 @@ from pathlib import Path

from markupsafe import Markup, escape from itsdangerous import URLSafeSerializer -from jinja2 import ChoiceLoader, Environment, FileSystemLoader, PrefixLoader +from jinja2 import ( + ChoiceLoader, + Environment, + FileSystemLoader, + PrefixLoader, + StrictUndefined, +) from jinja2.environment import Template from jinja2.exceptions import TemplateNotFound

@@ -394,7 +400,10 @@ class Datasette: ] ) self.jinja_env = Environment( - loader=template_loader, autoescape=True, enable_async=True + loader=template_loader, + autoescape=True, + enable_async=True, + undefined=StrictUndefined, ) self.jinja_env.filters["escape_css_string"] = escape_css_string self.jinja_env.filters["quote_plus"] = urllib.parse.quote_plus ``` This causes Jinja to raise a hard error if there are any variables referenced in the template that are not available in the context.

It's helping me spot things that are still missing, rather than just relying on failed unit tests.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461161256 https://github.com/simonw/datasette/pull/1999#issuecomment-1461161256 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XF40o simonw 9599 2023-03-09T02:10:07Z 2023-03-09T02:10:07Z OWNER

Just ran into a no such table: columns error - which I think is because my new view code sometimes bypasses calling this method (currently done in BaseView.dispatch_request()):

https://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/base.py#L101-L103

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461148579 https://github.com/simonw/datasette/pull/1999#issuecomment-1461148579 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XF1uj simonw 9599 2023-03-09T01:54:10Z 2023-03-09T01:55:33Z OWNER

Or... I could temporarily build a quick additional CannedQueryView subclass that just does the necessary bits to get the existing code to work. I'm going to try that.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461148254 https://github.com/simonw/datasette/pull/1999#issuecomment-1461148254 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XF1pe simonw 9599 2023-03-09T01:53:41Z 2023-03-09T01:53:41Z OWNER

Solving this is proving difficult: https://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L1500-L1503

The problem is that calling .data() on QueryView only works here because we expect to ourselves be inside a .data() method, with all of the existing magic that knows how to render things that are returned by that.

So I may need to substantially re-engineer how QueryView works in order to get this to work.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461075648 https://github.com/simonw/datasette/pull/1999#issuecomment-1461075648 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFj7A simonw 9599 2023-03-09T00:24:22Z 2023-03-09T00:24:22Z OWNER

127.0.0.1:8001/fixtures/neighborhood_search fails because the forwarding to a canned query does not yet work.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461074526 https://github.com/simonw/datasette/pull/1999#issuecomment-1461074526 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFjpe simonw 9599 2023-03-09T00:23:06Z 2023-03-09T00:23:06Z OWNER 
pytest tests/test_table_html.py

Currently 44 failed, 24 passed in 7.53s

Failures here: https://gist.github.com/simonw/df0a52cd7d820b776dc3dfc50e7cb778

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461070937 https://github.com/simonw/datasette/pull/1999#issuecomment-1461070937 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFixZ simonw 9599 2023-03-09T00:18:52Z 2023-03-09T00:19:36Z OWNER

I managed to get HTML view working! I did it by continuing to add more things to the extras and the _html bundle until the page loaded for me:

```diff async def extra_extras(): "Available ?_extra= blocks" return { @@ -1981,6 +2053,14 @@ async def extra_extras(): "query", "display_columns", "display_rows", + "database", + "table", + "database_color", + "table_actions", + "filters", + "renderers", + "custom_table_templates", + "sorted_facet_results", ] }

@@ -2006,6 +2086,14 @@ async def extra_extras(): extra_query, extra_metadata, extra_extras, + extra_database, + extra_table, + extra_database_color, + extra_table_actions, + extra_filters, + extra_renderers, + extra_custom_table_templates, + extra_sorted_facet_results, ) `` I'll probably refactor this into something cleaner, and maybe but a bunch of them in a"html"dictionary and update the templates to use{{ html.filters }}` or similar. Will look at that once the tests are passing.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461047607 https://github.com/simonw/datasette/pull/1999#issuecomment-1461047607 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFdE3 simonw 9599 2023-03-08T23:51:46Z 2023-03-08T23:51:46Z OWNER

This feels quite nice:

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461044477 https://github.com/simonw/datasette/pull/1999#issuecomment-1461044477 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFcT9 simonw 9599 2023-03-08T23:47:26Z 2023-03-08T23:47:26Z OWNER

I want to package together all of the extras that are needed for the HTML format. A few options for doing that:

  • Introduce ?_extra=_html where the leading underscore indicates that this is a "bundle" of extras, then define a bundle that's everything needed for the HTML renderer
  • Have some other mechanism whereby different renderers can request a bundle of extras.

I'm leaning towards the first option. I'll try that and see what it looks like.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461023559 https://github.com/simonw/datasette/pull/1999#issuecomment-1461023559 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFXNH simonw 9599 2023-03-08T23:23:02Z 2023-03-08T23:23:02Z OWNER

To get this unblocked, I'm going to allow myself to pass non-JSON-serializable objects to the HTML template version of things. If I can get that working (and get the existing tests to pass) I can consider a later change that makes those JSON serializable - or admit that it's OK for the templates to have non-JSON data passed to them and figure out how best to document those variables independently from the JSON documentation.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1461002039 https://github.com/simonw/datasette/pull/1999#issuecomment-1461002039 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFR83 simonw 9599 2023-03-08T22:58:16Z 2023-03-08T23:02:09Z OWNER

The reason for that Row thing is that it allows custom templates that do things like this:

https://docs.datasette.io/en/stable/changelog.html#easier-custom-templates-for-table-rows

html+jinja {% for row in display_rows %} <div> <h2>{{ row["title"] }}</h2> <p>{{ row["description"] }}<lp> <p>Category: {{ row.display("category_id") }}</p> </div> {% endfor %} Is that a good design? the .display() thing feels weird - I wonder if anyone has ever actually used that.

It's documented here: https://docs.datasette.io/en/0.64.2/custom_templates.html#custom-templates

If you want to output the rendered HTML version of a column, including any links to foreign keys, you can use {{ row.display("column_name") }}.

I can't see any examples of anyone using it in this code search: https://cs.github.com/?scopeName=All+repos&scope=&q=datasette+row.display

It is however useful to have some kind of abstraction layer here that insulates the SQLite Row object, since having an extra layer will help if Datasette ever grows support for alternative database backends such as DuckDB or PostgreSQL.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460988975 https://github.com/simonw/datasette/pull/1999#issuecomment-1460988975 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFOwv simonw 9599 2023-03-08T22:42:57Z 2023-03-08T22:42:57Z OWNER

Aside idea: it might be interesting if there were "lazy" template variables available in the context: things that are not actually executed unless a template author requests them.

Imagine if metadata was a lazy template reference, such that custom templates that don't display any metadata don't trigger it to be resolved (which might involve additional database queries some day).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460986533 https://github.com/simonw/datasette/pull/1999#issuecomment-1460986533 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFOKl simonw 9599 2023-03-08T22:40:28Z 2023-03-08T22:40:28Z OWNER

Figuring out what to do with display_columns_and_rows() is hard. That returns rows as this special kind of object, which is designed to be accessed from the HTML templates:

https://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L45-L71

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460970807 https://github.com/simonw/datasette/pull/1999#issuecomment-1460970807 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFKU3 simonw 9599 2023-03-08T22:31:49Z 2023-03-08T22:33:03Z OWNER

For the HTML version, I need to decide where all of the stuff that happens in async def extra_template() is going to live.

I think it's another one of those extra functions, triggered for ?_extra=context.

https://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L813-L912

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460943097 https://github.com/simonw/datasette/pull/1999#issuecomment-1460943097 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XFDj5 simonw 9599 2023-03-08T22:09:24Z 2023-03-08T22:09:47Z OWNER

The ease with which I added that ?_extra=query feature in https://github.com/simonw/datasette/pull/1999/commits/96e94f9b7b2db53865e61390bcce6761727f26d8 made me feel really confident that this architecture is going in the right direction.

```diff diff --git a/datasette/views/table.py b/datasette/views/table.py index 8d3bb2c930..3e1db9c85f 100644 --- a/datasette/views/table.py +++ b/datasette/views/table.py @@ -1913,6 +1913,13 @@ async def extra_request(): "args": request.args._data, }

  • async def extra_query():
  • "Details of the underlying SQL query"
  • return {
  • "sql": sql,
  • "params": params,
  • } + async def extra_extras(): "Available ?_extra= blocks" return { @@ -1938,6 +1945,7 @@ async def extra_extras(): extra_primary_keys, extra_debug, extra_request,
  • extra_query, extra_extras, ) ```
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460916405 https://github.com/simonw/datasette/pull/1999#issuecomment-1460916405 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XE9C1 simonw 9599 2023-03-08T21:43:27Z 2023-03-08T21:43:27Z OWNER

Just noticed that _json=colname is not working, and that's because it's handled by the renderer here:

https://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/renderer.py#L29-L40

But that's not currently being called by my new code.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460907148 https://github.com/simonw/datasette/pull/1999#issuecomment-1460907148 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XE6yM simonw 9599 2023-03-08T21:34:30Z 2023-03-08T21:34:30Z OWNER

I'm going to hold off on that refactor until later, when I have tests to show me if the refactor works or not.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460906741 https://github.com/simonw/datasette/pull/1999#issuecomment-1460906741 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XE6r1 simonw 9599 2023-03-08T21:34:08Z 2023-03-08T21:34:08Z OWNER

So maybe I can refactor it to look a bit more like this:

https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1602-L1604

One thing that's useful here is that is_view is handled early, like this:

https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L466-L472

So if I omit the is_view bit from the extracted function I can simplify more.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460905469 https://github.com/simonw/datasette/pull/1999#issuecomment-1460905469 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XE6X9 simonw 9599 2023-03-08T21:32:56Z 2023-03-08T21:32:56Z OWNER

I'd really like to extract this ugly logic out into a helper function:

https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1617-L1684

I copied it in and asked ChatGPT to "List all of the variable that are either modified or assigned to by the above code":

Modified variables:

offset
where_clauses
params
order_by

Assigned variables:

sort_value
next_by_pk_clauses
order_by (when sort or sort_desc is not None)

Then I asked which variables were used as inputs, and argued with it a bit about whether it should be counting functions. Eventually got to this:

My apologies for the oversight. Here are the variables needed as input by the above code, excluding any functions:

_next
is_view
sort
sort_desc
use_rowid
pks
order_by

Note that use_rowid is actually defined earlier in terms of two of those other variables: https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1540

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1460760116 https://github.com/simonw/datasette/pull/1999#issuecomment-1460760116 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5XEW40 simonw 9599 2023-03-08T19:48:52Z 2023-03-08T19:48:52Z OWNER

I'm trying to get http://127.0.0.1:8001/fixtures/compound_three_primary_keys?_next=a,d,v to return the correct results.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1421988953 https://github.com/simonw/datasette/pull/1999#issuecomment-1421988953 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5UwdRZ simonw 9599 2023-02-08T04:35:44Z 2023-02-08T05:27:48Z OWNER

Next step: get ?_next=... working (it is ignored at the moment, even though the returned JSON includes the "next" key).

Then... figure out how to render HTML and other requested formats.

Then get the tests to pass!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1399343659 https://github.com/simonw/datasette/pull/1999#issuecomment-1399343659 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5TaEor simonw 9599 2023-01-21T22:19:20Z 2023-02-06T23:02:12Z OWNER

HTML mode needs a list of renderers so it can show links to .geojson etc - can do that as a hidden extra (maybe called renderers), repeating this code:

https://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/views/base.py#L477-L497

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1399341658 https://github.com/simonw/datasette/pull/1999#issuecomment-1399341658 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5TaEJa simonw 9599 2023-01-21T22:06:29Z 2023-01-21T22:07:30Z OWNER

Relevant: - #1101 - #1672 - #1062

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  
1399341151 https://github.com/simonw/datasette/pull/1999#issuecomment-1399341151 https://api.github.com/repos/simonw/datasette/issues/1999 IC_kwDOBm6k_c5TaEBf simonw 9599 2023-01-21T22:03:20Z 2023-01-21T22:03:20Z OWNER

I think I'm going to have to write a new view function from scratch which completely ignores the existing BaseView/DataView/TableView hierarchy.

Here's what I get on the incoming request: (Pdb) request.url, request.full_path, request.host, request.url_vars ('http://127.0.0.1:8001/content/repos.json', '/content/repos.json', '127.0.0.1:8001', {'database': 'content', 'table': 'repos', 'format': 'json'})

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_extra= support (draft) 1551694938  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);