issues

https://github.com/simonw/datasette/compare/0.49.1...c12b7a5def7028845a54a9fdac4052a87a0a8bb8

datasette-graphql works by making internal requests to the TableView class (in order to take advantage of existing pagination logic, plus options like ?_search= and ?_where=) - see #915

I want to support a mod_rewrite style mechanism for putting nicer URLs on top of Datasette pages - I botched that together for a project here using an internal ASGI proxying trick: https://github.com/natbat/tidepools_near_me/commit/ec102c6da5a5d86f17628740d90b6365b671b5e1

If the datasette object provided a documented method for executing internal requests (in a way that makes sense with logging etc - i.e. doesn't get logged as a separate request) both of these use-cases would be much neater.

I'm going to document this in a separate issue.

Originally posted by @simonw in https://github.com/simonw/datasette/pull/1000#issuecomment-706269271

This code: https://github.com/simonw/datasette/blob/7249ac5ca04b5ddc6517750326ee7e522cc49145/tests/utils.py#L1-L8

Predates the introduction of this class method: https://github.com/simonw/datasette/blob/7249ac5ca04b5ddc6517750326ee7e522cc49145/datasette/utils/asgi.py#L108-L121

``` % curl -vv -XOPTIONS https://latest.datasette.io/ * Trying 216.58.195.83:443...

OPTIONS / HTTP/1.1 Host: latest.datasette.io User-Agent: curl/7.70.0 Accept: /

• Mark bundle as not supporting multiuse < HTTP/1.1 500 Internal Server Error ```

A tidied up version of https://til.simonwillison.net/til/til/digitalocean_datasette-on-digitalocean-app-platform.md - but mention that you can deploy to Heroku using the same mechanism.

This issue is about how best to pass additional options to tools used for publishing datasettes. A concrete example is wanting to pass the --tar flag to the heroku CLI tool. I think there are at least two options for doing this: documentation for each publishing tool to explain how to set flags via env variables (if possible) or building a mechanism that lets users pass additional flags through datasette.

When using datasette publish heroku binder-launches.db --extra-options="--config facet_time_limit_ms:35000 --config sql_time_limit_ms:35000" --name=binderlytics --install=datasette-vega to publish https://binderlytics.herokuapp.com/ the following error happens:

``` › Warning: heroku update available from 7.42.1 to 7.43.0. › Warning: heroku update available from 7.42.1 to 7.43.0. › Warning: heroku update available from 7.42.1 to 7.43.0. Setting WEB_CONCURRENCY and restarting ⬢ binderlytics... done, v13 WEB_CONCURRENCY: 1 › Warning: heroku update available from 7.42.1 to 7.43.0. ▸ Couldn't detect GNU tar. Builds could fail due to decompression errors ▸ See https://devcenter.heroku.com/articles/platform-api-deploying-slugs#create-slug-archive ▸ Please install it, or specify the '--tar' option ▸ Falling back to node's built-in compressor buffer.js:358 throw new ERR_INVALID_OPT_VALUE.RangeError('size', size); ^

RangeError [ERR_INVALID_OPT_VALUE]: The value "3303763968" is invalid for option "size" at Function.alloc (buffer.js:367:3) at new Buffer (buffer.js:281:19) at Readable.<anonymous> (/Users/thead/.local/share/heroku/node_modules/archiver-utils/index.js:39:15) at Readable.emit (events.js:322:22) at endReadableNT (/Users/thead/.local/share/heroku/node_modules/readable-stream/lib/_stream_readable.js:1010:12) at processTicksAndRejections (internal/process/task_queues.js:84:21) { code: 'ERR_INVALID_OPT_VALUE' } ```

After installing GNU tar with brew install gnu-tar and modifying datasette/publish/heroku.py to include the --tar=/path/to/gnu-tar publishing works.

I think the problem occurs once your heroku slug reaches a certain size. At least when I add only a few 100 entries to the datasette then the error does not occcur.

datasette version 0.49.1 OSX 10.14.6 (18G103)

I've got some SQLite databases too big to push to Heroku or the other services with built-in support in datasette.

So instead I moved my datasette code and databases to a remote server on Kimsufi. In the folder containing the SQLite databases I run the following code.

nohup datasette serve -h 0.0.0.0 *.db --cors --port 8000 --metadata metadata.json > output.log 2>&1 &.

When I go to http://my-remote-server.com:8000, the site loads. But I know this is not a good long-term solution to running datasette on this server.

What is the "correct" way to have this site run, preferably on server port 80?

Uvicorn uses the WEB_CONCURRENCY variable, if set, to specify the number of workers to use.

Datasette does not work with options for this other than 1:

WEB_CONCURRENCY=2 datasette . WARNING: You must pass the application as an import string to enable 'reload' or 'workers'. This was the cause of the Heroku bug in #627.

I fixed that issue by setting WEB_CONCURRENCY=1 in datasette publish heroku, but a better fix would be to hard-code workers=1 in Datasette itself.

I tried building this:

templates/pages/{topic}.html
templates/pages/{topic}/{slug}.html

And it didn't work - hits to /foo/bar which should have been rendered by the {slug}.html template were instead rendered by the top level {topic.html} template.

On page https://latest.datasette.io/fixtures/sortable?_next=15%2Cg%2Cz&_sort=sortable clicking on sortable_with_nulls > sort_ascending links to https://latest.datasette.io/fixtures/sortable?_next=15%2Cg%2Cz&_sort_desc=sortable_with_nulls - which doesn't make sense.

Changing the sort order needs to reset to the first page.

On https://latest.datasette.io/fixtures/compound_three_primary_keys the column action menu doesn't display for the pk1, pk2 and pk3 columns (because they are primary keys) even though faceting by them is actually useful.

Refs #98

At the very least I'd like a menu on each table column that lets me select sort-asc v.s. sort-desc without having to click twice.

I'd also like to be able to indicate that a column should be used for faceting (possibly only for columns that are not floating point and do not have a unique index on them).

This needs to be built with accessibility in mind - I don't want screenreaders to read out the contents of a menu as the "th" label for any given cell.

Related: #690

Similar to #978.

https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_rrule(%27FREQ%3DHOURLY%3BCOUNT%3D5%27)%2C%0D%0A++dateutil_rrule_date(%0D%0A++++%27FREQ%3DDAILY%3BCOUNT%3D3%27%2C%0D%0A++++%271st+jan+2020%27%0D%0A++)%3B

https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0D%0A++dateutil_easter%28%222020%22%29%2C%0D%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0D%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0D%0A++dateutil_parse%28%222020-03-04%22%29%2C%0D%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%2C%0D%0A++dateutil_easter%282020%29

I'm getting this when latest is deployed to Cloud Run: Traceback (most recent call last): File "/usr/local/bin/datasette", line 8, in <module> sys.exit(cli()) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/datasette/cli.py", line 406, in serve inspect_data = json.load(open(inspect_file)) TypeError: 'bool' object is not callable I think I may have broken things in #970 - a980199e61fe7ccf02c2123849d86172d2ae54ff

This is a request for a "convenience" feature, and only a nice to have. It's based on seeing this feature in several little command line hypertext server apps.

If you run, for example:

datasette.exe serve --open "mydb.s3db"

I would like it if default browser is launched, at the URL that is being served.

The angular cli does this, for example

ng serve <project> --open #see https://angular.io/cli/serve

...as does my usual mini web server of choice when inspecting local static files....

npx http-server -o # see https://www.npmjs.com/package/http-server

Just a tiny thing. Love your work!

From https://docs.python.org/3/library/sqlite3.html#sqlite3.enable_callback_tracebacks

sqlite3.``enable_callback_tracebacks(flag)¶

By default you will not get any tracebacks in user-defined functions, aggregates, converters, authorizer callbacks etc. If you want to debug them, you can call this function with flag set to True. Afterwards, you will get tracebacks from callbacks on sys.stderr. Use False to disable the feature again.

Maybe turn this on for all of Datasette? Are there any disadvantages to doing that?