issue_comments

The Sanic stuff I import at the moment is:

• [x] from sanic.views import HTTPMethodView
• [x] from sanic.request import Request https://github.com/huge-success/sanic/blob/0.7.0/sanic/request.py#L44
• [x] from sanic.exceptions import NotFound
• [x] from sanic import response - used for response.text(""), response.redirect(path), response.HTTPResponse and response.html() https://github.com/huge-success/sanic/blob/0.7.0/sanic/response.py
• [x] from sanic.request import RequestParameters https://github.com/huge-success/sanic/blob/0.7.0/sanic/request.py#L30

It's live! https://a559123.datasette.io/

Last thing is to replace sanic.response:

• response.text("")
• response.html()
• response.redirect(path)
• response.HTTPResponse

Implementations here: https://github.com/huge-success/sanic/blob/0.7.0/sanic/response.py#L175-L285

For the request object.... what are the fields of it I actually use?

• request.url
• request.query_string
• request.path
• request.method
• request.args
• request.raw_args

ALL of those are things that can be derived from the scope - so I think my new Request class (in utils/asgi.py) is just going to be a wrapper around a scope.

The InvalidUsage exception is thrown by Sanic when it gets an unhandled request - usually a HEAD. It was added in efbb4e83374a2c795e436c72fa79f70da72309b8

I'm going to replace it with specific handling for HEAD requests plus a unit test.

The big one: Replace Sanic request and response objects with my own classes, so I can remove Sanic dependency

Replacement for @app.listener("before_server_start") - this is what the ASGI lifespan protocol is for.

I know Uvicorn supports this because it keeps saying ASGI 'lifespan' protocol appears unsupported on the console.

I think the solution here will be to introduce another ASGI wrapper class similar to AsgiTracer. I'll model this on the example in the ASGI lifespan spec.

OK, for Get ?_trace=1 working again. The old code lives in two places:

https://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/app.py#L546-L560

And then:

https://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/app.py#L653-L672

So it's stashing something on the request to tell the rest of the code it should be tracing, then using that collected data from the request to add information to the final body.

One possible shape for the replacement is a new ASGI middleware that wraps everything else. We don't have a mutable request object here though, so we will need to untangle this entirely from the request object.

Also tricky is that in ASGI land we handle streams - we don't usually wait around for the entire response body to be compiled for us. This means the code that modifies the response (adding to the JSON or appending inside the </body>) needs to be reconsidered.

As usual, Starlette seems to have figured this out: https://github.com/encode/starlette/blob/8c8cc2ec0a5cb834a9a15b871ae8b480503abb67/starlette/middleware/gzip.py

Another bug: JSON is being served without a content-type header: ``` ~ $ curl -i 'http://127.0.0.1:8001/fivethirtyeight/ahca-polls%2Fahca_polls.json' HTTP/1.1 200 OK date: Sun, 23 Jun 2019 16:04:01 GMT server: uvicorn referrer-policy: no-referrer transfer-encoding: chunked

{"database": "fivethirtyeight", "table": "ahca-polls/ahca_polls", ... ```

Weird new bug: http://127.0.0.1:8001/fixtures/table%2Fwith%2Fslashes.csv?_format=json is downloading CSV for me now.

https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=json does the right thing.

Mystery solved: that's because I'm constructing my own scope object and testing via ApplicationCommunicator rather than exercising Uvicorn directly.

https://github.com/simonw/datasette/blob/d60fbfcae2658e71cab6d7b3b9f53f8d895064ef/tests/fixtures.py#L42-L57

I don't want to introduce the complexity of launching a real Uvicorn as part of the tests, so I guess I'll have to carefully update my ApplicationCommunicator test harness to more correctly emulate real life.

This is strange: on my local machine http://127.0.0.1:8001/fixtures/table%2Fwith%2Fslashes.csv is returning a 404 BUT there's a test for that which is passing under pytest:

https://github.com/simonw/datasette/blob/d60fbfcae2658e71cab6d7b3b9f53f8d895064ef/tests/test_api.py#L721-L727