issues

I'd like to see spatialite geometry columns rendered more sensibly - at the moment they come through as well-known-binary unless you use custom SQL, and WKB isn't of much use to anyone on the web.

In HTML: they should be shown either as simple lat/long (if it's just a point, for example), or as a sensible placeholder if they're more complex geometries.

In JSON: they should be GeoJSON geometries, (which means they can be automatically fed into a leaflet map with no further messing around).

In CSV: they should be WKT.

I briefly wondered if this should go into a plugin, but I suspect it needs hooking in at a deeper level than the plugin architecture will support any time soon.

This adds two new plugin hooks:

The inspect hook allows plugins to add data to the inspect dictionary.

The prepare_sanic hook allows plugins to hook into the web router. I've attached a warning to this hook in the docs in light of #272 but I want this hook now...

On quick inspection, I don't think it's worthwhile to try and make this hook independent of the web framework (but it looks like Starlette would make the hook implementation a bit nicer).

Ref #14

For some reason I'm hitting a None here with a FTS table. I'm not entirely sure why but this makes the logic work the same as with non-hidden tables.

This adds a straightforward set of classes to aid in the construction of SQL queries.

My plan for this was to allow plugins to manipulate the Datasette-generated SQL in a more structured way. I'm not sure that's going to work, but I feel like this is still a step forward - it reduces the number of intermediate variables in TableView.data which aids readability, and also factors out a lot of the boring string concatenation.

There are a fair number of minor structure changes in here too as I've tried to make the ordering of TableView.data a bit more logical. As far as I can tell, I haven't broken anything...

This stops my automatic editor linting from flagging lines which are too long. It's been lingering in my checkout for ages.

160 is an arbitrary large number - we could alter it if we have any opinions (but I find the line length limit to be my least favourite part of PEP8).

Binary columns (including spatialite geographies) get shown as ugly binary strings in the HTML by default. Nobody wants to see that mess.

Show the size of the column in bytes instead. If you want to decode the binary data, you can use a plugin to do it.

At the moment it's not easy to tell whether the hook is being called in (for example) the row or table view, as in both cases the database and table parameters are provided.

This passes the view_name added in #441 to the extra_body_script hook.

If I have a datasette plugin installed on my system, its hooks are called when running the main datasette tests. This is probably undesirable, especially with the inspect hook in #437, as the plugin may rely on inspected state that the tests don't know about.

It would be nice to have a simple way for plugins to provide additional data export formats. Might require a bit of work on the internals. I can work around this at a lower level with the prepare_sanic hook from #437 in the mean time.

I guess plugins should be able to register a function which takes a row or list of rows and returns the rendered data. They'll also need to provide a file extension and probably a Content-Type.

Datasette could then automatically include this format in the list of export formats on each page.

Looks like this is related to #119.

This changeset refactors out the JSON renderer and then adds a hook and dispatcher system to allow custom output renderers to be registered.

The CSV output renderer is untouched because supporting streaming renderers through this system would be significantly more complex, and probably not worthwhile.

We can't simply allow hooks to be called at request time because we need a list of supported file extensions when the request is being routed in order to resolve ambiguous database/table names. So, renderers need to be registered at startup.

I've tried to make this API independent of Sanic's request/response objects so that this can remain stable during the switch to ASGI. I'm using dictionaries to keep it simple and to make adding additional options in the future easy.

Fixes #440

This allows the row to be identified on row pages. The context here is that I want to access the row's data to plot it on a map.

I considered passing the entire template context through to the hook function. This would expose the actual row data and potentially avoid a further fetch request in JS, but it does make the plugin API a lot more leaky.

(At any rate, using the selected row data is tricky in my case because of Spatialite's infuriating custom binary representation...)

This PR does two things:

• Adds the sqlite column type for each column to the inspected table info.
• Stops binary columns from being rendered to HTML, unless a plugin handles it.

There's a bit more detail in the changeset descriptions.

These changes are intended as a precursor to a plugin which adds first-class support for Spatialite geographic primitives, and perhaps more useful geo-stuff.

These show up as 0 or 1 because sqlite. I think Yes/No would be fine in most cases?

Lets you supply the -n parameter for Heroku deploys, which also lets you update existing Heroku deployments.

It fails with the following JSON traceback if the -n option isn't provided, despite the fact that the command line help says that's not needed for heroku publishes.

This pulls the logic for inspect out into a new file which makes it a bit easier to understand.

This was going to be the first part of an implementation for #276, but it seems like that might take a while so I'm going to PR a few bits of refactoring individually.

When there's a simple (single-column) primary key, it looks weird to duplicate it in the link column.

This change removes the second PK column and treats the link column as if it were the PK column from a header/sorting perspective.

This might make it a bit more difficult to tell what the link for the row is, I'm not sure yet. I feel like the alternative is to change the link column to just have the text "view" or something, instead of repeating the PK. (I doubt it makes much more sense with compound PKs.)

Bonus change in this PR: fix urlencoding of links in the displayed HTML.

Before:

After:

From #203: it would be nice for the JSON API to (optionally) return columns rendered with units in them - if, for example, you're consuming the JSON to render the rows on a map.

I'm not entirely sure how useful this will be though - at the moment my map queries are custom SQL queries (a few have joins in, the rest might be fetching large amounts of data so it makes sense to limit columns fetched). Perhaps the SQL function is a better approach in general.

It would be nice to be able to attach a unit to a column in the metadata, and have it rendered with that unit (and SI prefix) when it's displayed.

It would also be nice to support entering the prefixes in variables when querying.

With my radio licensing app I've put all frequencies in Hz. It's easy enough to special-case the row rendering to add the SI prefixes, but it's pretty unusable when querying by that field.

This also stops it filling up the logs. This happens for HEAD requests at the moment - which perhaps should be handled better, but that's a different issue.

The first commit: * Adds units to exported JSON * Adds units key to metadata skeleton * Adds some docs for units

The second commit adds filtering by units by the first method I mentioned in #203:

Try it here. I think it integrates pretty neatly.

The third commit adds support for registering custom units with Pint from metadata.json. Probably pretty niche, but I need decibels!

This renders unlabeled FKs as simple links. I can't see why this would cause any major problems.

Also includes bonus fixes for two minor issues:

• In foreign key link hrefs the primary key was escaped using HTML escaping rather than URL escaping. This broke some non-integer PKs.
• Print tracebacks to console when handling 500 errors.

This fixes ERROR: conn=<sqlite3.Connection object at 0x10bbb9f10>, sql = 'select ', params = {'id': '1'} caused by an invalid query loading incoming FKs when none exist.

The error was ignored due to async but it still got printed to the console.

Add support for specifying units for a column in metadata.json and rendering them on display using pint.

Example table metadata: json "license_frequency": { "units": { "frequency": "Hz", "channel_width": "Hz", "height": "m", "antenna_height": "m", "azimuth": "degrees" } }

Example result

This works surprisingly well! I'd like to add support for using units when querying but this is PR is pretty usable as-is.

(Pint doesn't seem to support decibels though - it thinks they're decibytes - which is an annoying omission.)

(ref ticket #203)

Currently they just 500. Also cleaned the logic up a bit, I hope I didn't miss anything.

This is issue #184.

They were getting on my nerves.