github

Custom SQL query returning 101 rows (hide)

This data as json, CSV

html_urlissue_urlidnode_idusercreated_atupdated_atauthor_associationbodyreactionsissueperformed_via_github_app
https://github.com/simonw/datasette/issues/860#issuecomment-733288841 https://api.github.com/repos/simonw/datasette/issues/860 733288841 MDEyOklzc3VlQ29tbWVudDczMzI4ODg0MQ== 9599 2020-11-24T23:19:47Z 2020-11-24T23:20:24Z OWNER Here's what I have today - it's an undocumented `datasette.metadata()` method that returns a full JSON dictionary of values OR a single value if the optional `key=` argument is provided: https://github.com/simonw/datasette/blob/f2e2bfcdd9ad4891f3f66c9104c09943d943ffe4/datasette/app.py#L357-L388 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/860#issuecomment-733288522 https://api.github.com/repos/simonw/datasette/issues/860 733288522 MDEyOklzc3VlQ29tbWVudDczMzI4ODUyMg== 9599 2020-11-24T23:18:47Z 2020-11-24T23:18:47Z OWNER In #942 I want to add support for per-column metadata - which means this new lookup mechanism will need to be able to answer the question "what description is available for this column". So what should the `.metadata()` method look like? A couple of options: - `datasette.metadata("description", table=x, database=y)` - can take optional `column=` too. - `datasette.table_metadata("description", table=x, database=y)` and `datasette.database_metadata("description", database=y)` and so on - multiple methods for the different types of metadata. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/860#issuecomment-733287619 https://api.github.com/repos/simonw/datasette/issues/860 733287619 MDEyOklzc3VlQ29tbWVudDczMzI4NzYxOQ== 9599 2020-11-24T23:16:21Z 2020-11-24T23:16:21Z OWNER I'll also allow any key to be looked up - so if users want to invent their own metadata keys other than the default `license_url` etc they can do so. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/860#issuecomment-733287416 https://api.github.com/repos/simonw/datasette/issues/860 733287416 MDEyOklzc3VlQ29tbWVudDczMzI4NzQxNg== 9599 2020-11-24T23:15:44Z 2020-11-24T23:15:44Z OWNER I'm going to go with a plugin hook (and Datasette method) that returns individual values - so you ask it for e.g. the `license_url` for a specific table and it returns a string or `None`. The default plugin hook implementation that ships with Datasette will then implement cascading lookups against `metadata.json` - but other plugins will be able to provide their own implementations, which should make it easy to build a plugin that lets you keep metadata in a database file and edit it interactively. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/1107#issuecomment-733261501 https://api.github.com/repos/simonw/datasette/issues/1107 733261501 MDEyOklzc3VlQ29tbWVudDczMzI2MTUwMQ== 9599 2020-11-24T22:09:11Z 2020-11-24T22:09:11Z OWNER Documentation: https://docs.datasette.io/en/latest/internals.html#setting-key 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 750079085  
https://github.com/simonw/datasette/issues/1107#issuecomment-733257071 https://api.github.com/repos/simonw/datasette/issues/1107 733257071 MDEyOklzc3VlQ29tbWVudDczMzI1NzA3MQ== 9599 2020-11-24T21:59:32Z 2020-11-24T21:59:32Z OWNER I'm going to make this a documented method in https://docs.datasette.io/en/latest/internals.html#datasette-class 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 750079085  
https://github.com/simonw/datasette/issues/1105#issuecomment-733249176 https://api.github.com/repos/simonw/datasette/issues/1105 733249176 MDEyOklzc3VlQ29tbWVudDczMzI0OTE3Ng== 9599 2020-11-24T21:40:28Z 2020-11-24T21:40:28Z OWNER This rebranding is complete - #1107 is a follow-up internal refactor. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749982022  
https://github.com/simonw/datasette/issues/1106#issuecomment-733248437 https://api.github.com/repos/simonw/datasette/issues/1106 733248437 MDEyOklzc3VlQ29tbWVudDczMzI0ODQzNw== 9599 2020-11-24T21:38:50Z 2020-11-24T21:38:50Z OWNER I used an "exact redirect" instead and it worked: <img width="654" alt="Edit_Redirects___Read_the_Docs" src="https://user-images.githubusercontent.com/9599/100154418-60bf0d80-2e5a-11eb-92bf-527a14edb5c5.png"> 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749983857  
https://github.com/simonw/datasette/issues/1106#issuecomment-733247101 https://api.github.com/repos/simonw/datasette/issues/1106 733247101 MDEyOklzc3VlQ29tbWVudDczMzI0NzEwMQ== 9599 2020-11-24T21:35:29Z 2020-11-24T21:36:04Z OWNER <img width="861" alt="Edit_Redirects___Read_the_Docs" src="https://user-images.githubusercontent.com/9599/100153884-a3341a80-2e59-11eb-8d8c-7027c714b9cf.png"> https://docs.datasette.io/en/latest/config.html isn't redirecting though, even after I tried running a rebuild of the `latest` version. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749983857  
https://github.com/simonw/datasette/issues/1106#issuecomment-733245596 https://api.github.com/repos/simonw/datasette/issues/1106 733245596 MDEyOklzc3VlQ29tbWVudDczMzI0NTU5Ng== 9599 2020-11-24T21:32:11Z 2020-11-24T21:32:11Z OWNER https://docs.datasette.io/en/latest/settings.html is now live - need to redirect https://docs.datasette.io/en/latest/config.html to it using the ReadTheDocs redirects interface. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749983857  
https://github.com/simonw/datasette/issues/1107#issuecomment-733245097 https://api.github.com/repos/simonw/datasette/issues/1107 733245097 MDEyOklzc3VlQ29tbWVudDczMzI0NTA5Nw== 9599 2020-11-24T21:31:10Z 2020-11-24T21:31:10Z OWNER Most of these use `plugin_config` which is unaffected. It looks like the only code I need to worry about is this trick in `datasette-graphl`: https://github.com/simonw/datasette-graphql/blob/483c9a9e203bb90365def3df8b8f01dda1e75865/datasette_graphql/utils.py#L456-L460 ```python class DatasetteSpecialConfig(wrapt.ObjectProxy): def config(self, key): if key == "suggest_facets": return False return self.__wrapped__.config(key) ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 750079085  
https://github.com/simonw/datasette/issues/1107#issuecomment-733244471 https://api.github.com/repos/simonw/datasette/issues/1107 733244471 MDEyOklzc3VlQ29tbWVudDczMzI0NDQ3MQ== 9599 2020-11-24T21:29:59Z 2020-11-24T21:29:59Z OWNER I ran `rg '.config\(' datasette-*/` in my top-level directory: ``` datasette-sentry/test_datasette_sentry.py: def plugin_config(self, name): datasette-sentry/datasette_sentry.py: config = datasette.plugin_config("datasette-sentry") or {} datasette-render-markdown/datasette_render_markdown/__init__.py: datasette.plugin_config( datasette-render-images/datasette_render_images.py: plugin_config = datasette.plugin_config("datasette-render-images") or {} datasette-render-html/datasette_render_html.py: config = datasette.plugin_config( datasette-render-timestamps/datasette_render_timestamps/__init__.py: datasette.plugin_config( datasette-permissions-sql/datasette_permissions_sql/__init__.py: for rule in datasette.plugin_config("datasette-permissions-sql") or []: datasette-mask-columns/datasette_mask_columns/__init__.py: datasette.plugin_config("datasette-mask-columns", database=database) or {} datasette-mask-columns/datasette_mask_columns/__init__.py: masks = datasette.plugin_config("datasette-mask-columns", database=database) or {} datasette-media/datasette_media/__init__.py: plugin_config = datasette.plugin_config("datasette-media") or {} datasette-insert/datasette_insert/__init__.py: plugin_config = datasette.plugin_config("datasette-insert") or {} datasette-indieauth/datasette_indieauth/__init__.py: plugin_config = datasette.plugin_config("datasette-indieauth") or {} datasette-graphql/datasette_graphql/__init__.py: config = datasette.plugin_config("datasette-graphql") or {} datasette-graphql/datasette_graphql/__init__.py: config = datasette.plugin_config("datasette-graphql") or {} datasette-graphql/datasette_graphql/utils.py: auto_camelcase=(datasette.plugin_config("datasette-graphql") or {}).get( datasette-graphql/datasette_graphql/utils.py: table_plugin_config = datasette.plugin_config( datasette-graphql/datasette_graphql/utils.py: def config(self, key): datasette-graphql/datasette_graphql/utils.py: return self.__wrapped__.config(key) datasette-init/datasette_init/__init__.py: config = datasette.plugin_config("datasette-init") datasette-edit-templates/datasette_edit_templates/__init__.py: plugin_config = datasette.plugin_config("datasette-edit-templates") or {} datasette-cors/datasette_cors.py: config = datasette.plugin_config("datasette-cors") or {} datasette-cluster-map-old/build/lib/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-cluster-map/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-cluster-map/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-cluster-map/tests/test_cluster_map.py:async def test_plugin_config(db_path, config, table, expected_fragments): datasette-configure-asgi/datasette_configure_asgi.py: configs = datasette.plugin_config("datasette-configure-asgi") or [] datasette-configure-asgi/test_datasette_configure_asgi.py: def plugin_config(self, name): datasette-cluster-map-old/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-auth-passwords/datasette_auth_passwords/__init__.py: config = datasette.plugin_config("datasette-auth-passwords") or {} datasette-auth-github/datasette_auth_github/views.py:def verify_config(config): datasette-auth-github/datasette_auth_github/views.py: config = datasette.plugin_config("datasette-auth-github") datasette-auth-github/datasette_auth_github/views.py: verify_config(config) datasette-auth-github/datasette_auth_github/views.py: config = datasette.plugin_config("datasette-auth-github") datasette-auth-github/datasette_auth_github/views.py: verify_config(config) datasette-atom/datasette_atom/__init__.py: plugin_config = datasette.plugin_config("datasette-atom") datasette-auth-google/datasette_auth_google/__init__.py: config = datasette.plugin_config("datasette-auth-github") or {} datasette-auth-existing-cookies/test_datasette_auth_existing_cookies.py: def plugin_config(self, name): datasette-auth-passwords/build/lib/datasette_auth_passwords/__init__.py: config = datasette.plugin_config("datasette-auth-passwords") or {} datasette-annotate/datasette_annotate/utils.py: plugin_config = datasette.plugin_config("datasette-annotate") or {} datasette-auth-existing-cookies/datasette_auth_existing_cookies/__init__.py: config = datasette.plugin_config("datasette-auth-existing-cookies") or {} datasette-auth-simple/datasette_auth_simple/__init__.py:datasette.plugin_config("datasette-auth-simple") datasette-auth-tokens/datasette_auth_tokens/__init__.py: config = datasette.plugin_config("datasette-auth-tokens") or {} datasette-block-robots/datasette_block_robots/__init__.py: config = datasette.plugin_config("datasette-block-robots") or {} datasette-block-robots/datasette_block_robots/__init__.py: config = datasette.plugin_config("datasette-block-robots") or {} ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 750079085  
https://github.com/simonw/datasette/issues/1107#issuecomment-733241949 https://api.github.com/repos/simonw/datasette/issues/1107 733241949 MDEyOklzc3VlQ29tbWVudDczMzI0MTk0OQ== 9599 2020-11-24T21:24:26Z 2020-11-24T21:24:26Z OWNER Are there any plugins that use this API even though it isn't documented? 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 750079085  
https://github.com/simonw/datasette/issues/1106#issuecomment-733221359 https://api.github.com/repos/simonw/datasette/issues/1106 733221359 MDEyOklzc3VlQ29tbWVudDczMzIyMTM1OQ== 9599 2020-11-24T20:40:21Z 2020-11-24T20:40:21Z OWNER https://readthedocs.org/dashboard/datasette/redirects/ 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749983857  
https://github.com/simonw/datasette/issues/1104#issuecomment-733212084 https://api.github.com/repos/simonw/datasette/issues/1104 733212084 MDEyOklzc3VlQ29tbWVudDczMzIxMjA4NA== 9599 2020-11-24T20:20:33Z 2020-11-24T20:20:33Z OWNER I'll throw an error if a `config.json` file is detected. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749981663  
https://github.com/simonw/datasette/issues/226#issuecomment-733198051 https://api.github.com/repos/simonw/datasette/issues/226 733198051 MDEyOklzc3VlQ29tbWVudDczMzE5ODA1MQ== 9599 2020-11-24T19:52:46Z 2020-11-24T19:52:46Z OWNER This is well handled now: https://github.com/simonw/datasette/tree/0.51.1/tests/plugins 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 315738696  
https://github.com/simonw/datasette/issues/1105#issuecomment-733190827 https://api.github.com/repos/simonw/datasette/issues/1105 733190827 MDEyOklzc3VlQ29tbWVudDczMzE5MDgyNw== 9599 2020-11-24T19:38:02Z 2020-11-24T19:38:02Z OWNER I'd like to redirect https://docs.datasette.io/en/stable/config.html to a new https://docs.datasette.io/en/stable/settings.html page too. I can use https://docs.readthedocs.io/en/stable/user-defined-redirects.html for that. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749982022  
https://github.com/simonw/datasette/issues/1103#issuecomment-733189737 https://api.github.com/repos/simonw/datasette/issues/1103 733189737 MDEyOklzc3VlQ29tbWVudDczMzE4OTczNw== 9599 2020-11-24T19:35:45Z 2020-11-24T19:35:45Z OWNER Part of #1105 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749979454  
https://github.com/simonw/datasette/issues/992#issuecomment-733189693 https://api.github.com/repos/simonw/datasette/issues/992 733189693 MDEyOklzc3VlQ29tbWVudDczMzE4OTY5Mw== 9599 2020-11-24T19:35:38Z 2020-11-24T19:35:38Z OWNER Part of #1105 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 714449879  
https://github.com/simonw/datasette/issues/1104#issuecomment-733189620 https://api.github.com/repos/simonw/datasette/issues/1104 733189620 MDEyOklzc3VlQ29tbWVudDczMzE4OTYyMA== 9599 2020-11-24T19:35:30Z 2020-11-24T19:35:30Z OWNER Part of #1105 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749981663  
https://github.com/simonw/datasette/issues/1103#issuecomment-733187586 https://api.github.com/repos/simonw/datasette/issues/1103 733187586 MDEyOklzc3VlQ29tbWVudDczMzE4NzU4Ng== 9599 2020-11-24T19:31:23Z 2020-11-24T19:31:23Z OWNER I'll set up a redirect from `/-/config` to `/-/settings`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749979454  
https://github.com/simonw/datasette/issues/992#issuecomment-733180289 https://api.github.com/repos/simonw/datasette/issues/992 733180289 MDEyOklzc3VlQ29tbWVudDczMzE4MDI4OQ== 9599 2020-11-24T19:16:30Z 2020-11-24T19:16:30Z OWNER Need to figure out the `--setting foo bar` alternative for this `--config foo:bar` logic: https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/datasette/cli.py#L31-L63 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 714449879  
https://github.com/simonw/datasette/issues/992#issuecomment-733176252 https://api.github.com/repos/simonw/datasette/issues/992 733176252 MDEyOklzc3VlQ29tbWVudDczMzE3NjI1Mg== 9599 2020-11-24T19:07:49Z 2020-11-24T19:07:49Z OWNER I'm going to keep `--config` for the moment but show a deprecation warning that it will be gone in Datasette 1.0. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 714449879  
https://github.com/simonw/datasette/issues/992#issuecomment-733175965 https://api.github.com/repos/simonw/datasette/issues/992 733175965 MDEyOklzc3VlQ29tbWVudDczMzE3NTk2NQ== 9599 2020-11-24T19:07:13Z 2020-11-24T19:07:13Z OWNER This is blocking progress on other metadata tickets like #860 because I want to split the concept of concrete metadata (source, license, etc) from configuration that currently lives in metadata (default sort order, default facets). I'm going to solve this next to unblock that stuff. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 714449879  
https://github.com/simonw/datasette/issues/860#issuecomment-733175454 https://api.github.com/repos/simonw/datasette/issues/860 733175454 MDEyOklzc3VlQ29tbWVudDczMzE3NTQ1NA== 9599 2020-11-24T19:06:07Z 2020-11-24T19:06:07Z OWNER I see two ways this plugin hook could work. It could be asked about a specific instance, database or table and return the full metadata for that object. OR it could ask for a specific metadata field - e.g. `source_url` for table X, and return that. The more finely grained one would allow plugins to implement their own cascading rules pretty easily. Is there a reason it would be better for the hook to return an entire block of JSON for a specific table or database? I also need to decide if this hook is just going to be about source/license/about displayed metadata, or if it will include the functionality that has been sneaking into `metadata.json` over time - stuff like page size, default sort order or default facets. Perhaps I should split those out into a "configuration" concept first, after renaming `--config` to `--setting` in #992. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/1101#issuecomment-732544590 https://api.github.com/repos/simonw/datasette/issues/1101 732544590 MDEyOklzc3VlQ29tbWVudDczMjU0NDU5MA== 9599 2020-11-24T02:22:55Z 2020-11-24T02:22:55Z OWNER The trick I'm using here is to follow the `next_url` in order to paginate through all of the matching results. The loop calls the `data()` method multiple times, once for each page of results: https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/datasette/views/base.py#L304-L307 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749283032  
https://github.com/simonw/datasette/issues/1101#issuecomment-732543700 https://api.github.com/repos/simonw/datasette/issues/1101 732543700 MDEyOklzc3VlQ29tbWVudDczMjU0MzcwMA== 9599 2020-11-24T02:20:30Z 2020-11-24T02:20:30Z OWNER Current design: https://docs.datasette.io/en/stable/plugin_hooks.html#register-output-renderer-datasette ```python @hookimpl def register_output_renderer(datasette): return { "extension": "test", "render": render_demo, "can_render": can_render_demo, # Optional } ``` Where `render_demo` looks something like this: ```python async def render_demo(datasette, columns, rows): db = datasette.get_database() result = await db.execute("select sqlite_version()") first_row = " | ".join(columns) lines = [first_row] lines.append("=" * len(first_row)) for row in rows: lines.append(" | ".join(row)) return Response( "\n".join(lines), content_type="text/plain; charset=utf-8", headers={"x-sqlite-version": result.first()[0]} ) ``` Meanwhile here's where the CSV streaming mode is implemented: https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/datasette/views/base.py#L297-L380 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 749283032  
https://github.com/simonw/datasette/issues/1096#issuecomment-732542285 https://api.github.com/repos/simonw/datasette/issues/1096 732542285 MDEyOklzc3VlQ29tbWVudDczMjU0MjI4NQ== 9599 2020-11-24T02:16:22Z 2020-11-24T02:16:22Z OWNER I'd like to implement this by first extending the `register_output_renderer()` hook to support streaming huge responses, then switching CSV to use the plugin hook in addition to TSV using it. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 743359646  
https://github.com/simonw/datasette/issues/860#issuecomment-731658059 https://api.github.com/repos/simonw/datasette/issues/860 731658059 MDEyOklzc3VlQ29tbWVudDczMTY1ODA1OQ== 9599 2020-11-22T00:31:47Z 2020-11-22T00:33:48Z OWNER Documented behaviour right now, for metadata set at the instance level, is: https://docs.datasette.io/en/stable/metadata.html > The above metadata will be displayed on the index page of your Datasette-powered site. The source and license information will also be included in the footer of every page served by Datasette. > > ... > > Metadata at the top level of the JSON will be shown on the index page and in the footer on every page of the site. The license and source is expected to apply to all of your data. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/860#issuecomment-731657660 https://api.github.com/repos/simonw/datasette/issues/860 731657660 MDEyOklzc3VlQ29tbWVudDczMTY1NzY2MA== 9599 2020-11-22T00:27:32Z 2020-11-22T00:31:54Z OWNER Open question: how should cascading work? If a table is missing a field but the database or instance has it, should that value cascade down to the table? It feels like `license` should definitely cascade: if an instance lists a certain `license` that should absolutely filter through to all databases and tables. But... should the other fields cascade? Cascading `description` doesn't feel right at all, and neither does `title`. What about `about` and `about_url` and `source` and `source_url`? I'm a bit torn on whether they should cascade or not. I'm leaning towards cascading them. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/860#issuecomment-731657486 https://api.github.com/repos/simonw/datasette/issues/860 731657486 MDEyOklzc3VlQ29tbWVudDczMTY1NzQ4Ng== 9599 2020-11-22T00:25:34Z 2020-11-22T00:25:34Z OWNER There are three layers of metadata: table, database and instance. Currently the metadata fields are (ignoring not-quite-metadata like `sort` and `sort_desc`): - `title` - `description` (or `description_html`) - `about` / `about_url` - `source` / `source_url` - `license` / `license_url` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 642651572  
https://github.com/simonw/datasette/issues/1084#issuecomment-731654132 https://api.github.com/repos/simonw/datasette/issues/1084 731654132 MDEyOklzc3VlQ29tbWVudDczMTY1NDEzMg== 9599 2020-11-21T23:45:59Z 2020-11-21T23:45:59Z OWNER https://datasette-graphql-demo.datasette.io/github/users now demonstrates the fix. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737394470  
https://github.com/simonw/datasette/issues/1084#issuecomment-731653083 https://api.github.com/repos/simonw/datasette/issues/1084 731653083 MDEyOklzc3VlQ29tbWVudDczMTY1MzA4Mw== 9599 2020-11-21T23:35:07Z 2020-11-21T23:35:07Z OWNER I got to use CSS `calc()` for this: https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/datasette/static/app.css#L367-L371 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737394470  
https://github.com/simonw/datasette/issues/1084#issuecomment-731652991 https://api.github.com/repos/simonw/datasette/issues/1084 731652991 MDEyOklzc3VlQ29tbWVudDczMTY1Mjk5MQ== 9599 2020-11-21T23:34:22Z 2020-11-21T23:34:22Z OWNER Fixed by positioning the menu relative to the `<div class="page-header">` element rather than the cog icon: <img width="966" alt="data" src="https://user-images.githubusercontent.com/9599/99889950-02eab580-2c0f-11eb-85dc-dd4bc840a92e.png"> 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737394470  
https://github.com/simonw/datasette/issues/1084#issuecomment-731644064 https://api.github.com/repos/simonw/datasette/issues/1084 731644064 MDEyOklzc3VlQ29tbWVudDczMTY0NDA2NA== 9599 2020-11-21T22:10:15Z 2020-11-21T22:10:15Z OWNER Another example of this bug: https://datasette-graphql-demo.datasette.io/github/users <img width="350" alt="github__users__4_823_rows_and_Comparing_1_1___main_·_simonw_datasette-graphql" src="https://user-images.githubusercontent.com/9599/99888639-3889a180-2c03-11eb-83b9-41f2eb12287e.png"> 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737394470  
https://github.com/simonw/datasette/issues/1094#issuecomment-731260091 https://api.github.com/repos/simonw/datasette/issues/1094 731260091 MDEyOklzc3VlQ29tbWVudDczMTI2MDA5MQ== 4808085 2020-11-20T16:11:29Z 2020-11-20T16:11:29Z NONE I can confirm this issue, running version 0.51.1 under Windows. Fixed by commenting out the following line near the top of datasette\utils\asgi.py : `#from os import EX_CANTCREAT` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 743011397  
https://github.com/simonw/datasette/issues/511#issuecomment-730893729 https://api.github.com/repos/simonw/datasette/issues/511 730893729 MDEyOklzc3VlQ29tbWVudDczMDg5MzcyOQ== 4060506 2020-11-20T06:35:13Z 2020-11-20T06:35:13Z NONE Trying to run on Windows today, I get an error from the utils/asgi.py module. It's trying `from os import EX_CANTCREAT` which is Unix-only. I commented this line out, and (so far) it's working. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 456578474  
https://github.com/dogsheep/twitter-to-sqlite/issues/52#issuecomment-729484478 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/52 729484478 MDEyOklzc3VlQ29tbWVudDcyOTQ4NDQ3OA== 4169772 2020-11-18T07:12:45Z 2020-11-18T07:12:45Z NONE I'm so sorry that you already have `--since_id` option and that's enough for the case I've mentioned. Thank you for this excellent tool! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 745393298  
https://github.com/simonw/datasette/issues/1091#issuecomment-729045320 https://api.github.com/repos/simonw/datasette/issues/1091 729045320 MDEyOklzc3VlQ29tbWVudDcyOTA0NTMyMA== 6739646 2020-11-17T16:31:00Z 2020-11-17T16:31:00Z NONE We're using mod_proxy. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/1091#issuecomment-729018386 https://api.github.com/repos/simonw/datasette/issues/1091 729018386 MDEyOklzc3VlQ29tbWVudDcyOTAxODM4Ng== 6739646 2020-11-17T15:48:58Z 2020-11-17T15:48:58Z NONE I don't think we are, but I'll check with Maruan. I think this is the relevant part of our config? ``` Alias "/base/" "/usr/share/corpora/" <Directory "/usr/share/corpora/"> Options +Indexes -Multiviews AllowOverride None </Directory> ProxyPreserveHost On ProxyPass /datasette http://0.0.0.0:8001 ProxyPassReverse /datasette http://0.0.0.0:8001 </VirtualHost> ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/1091#issuecomment-728262974 https://api.github.com/repos/simonw/datasette/issues/1091 728262974 MDEyOklzc3VlQ29tbWVudDcyODI2Mjk3NA== 9599 2020-11-16T19:05:08Z 2020-11-16T19:05:08Z OWNER I have a hunch that there may be some extra configuration in play here - could Apache itself be rewriting some of the links using [mod_proxy_html](https://httpd.apache.org/docs/2.4/mod/mod_proxy_html.html)? 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/dogsheep/swarm-to-sqlite/issues/11#issuecomment-727692413 https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11 727692413 MDEyOklzc3VlQ29tbWVudDcyNzY5MjQxMw== 9599 2020-11-16T02:15:22Z 2020-11-16T02:15:22Z MEMBER Thanks, I'll look into this. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 743400216  
https://github.com/simonw/datasette/issues/1098#issuecomment-727656208 https://api.github.com/repos/simonw/datasette/issues/1098 727656208 MDEyOklzc3VlQ29tbWVudDcyNzY1NjIwOA== 9599 2020-11-15T23:26:14Z 2020-11-15T23:26:14Z OWNER Schema for that broken example: ```sql CREATE TABLE generators_eia860 ( id INTEGER NOT NULL, plant_id_eia INTEGER, generator_id TEXT, report_date DATE, operational_status_code TEXT, operational_status TEXT, ownership_code TEXT, utility_id_eia INTEGER, capacity_mw FLOAT, summer_capacity_mw FLOAT, winter_capacity_mw FLOAT, energy_source_code_1 TEXT, energy_source_code_2 TEXT, energy_source_code_3 TEXT, energy_source_code_4 TEXT, energy_source_code_5 TEXT, energy_source_code_6 TEXT, fuel_type_code_pudl TEXT, multiple_fuels BOOLEAN, deliver_power_transgrid BOOLEAN, syncronized_transmission_grid BOOLEAN, turbines_num INTEGER, planned_modifications BOOLEAN, planned_net_summer_capacity_uprate_mw FLOAT, planned_net_winter_capacity_uprate_mw FLOAT, planned_uprate_date DATE, planned_net_summer_capacity_derate_mw FLOAT, planned_net_winter_capacity_derate_mw FLOAT, planned_derate_date DATE, planned_new_prime_mover_code TEXT, planned_energy_source_code_1 TEXT, planned_repower_date DATE, other_planned_modifications BOOLEAN, other_modifications_date DATE, planned_retirement_date DATE, carbon_capture BOOLEAN, startup_source_code_1 TEXT, startup_source_code_2 TEXT, startup_source_code_3 TEXT, startup_source_code_4 TEXT, technology_description TEXT, turbines_inverters_hydrokinetics TEXT, time_cold_shutdown_full_load_code TEXT, planned_new_capacity_mw FLOAT, cofire_fuels BOOLEAN, switch_oil_gas BOOLEAN, nameplate_power_factor FLOAT, minimum_load_mw FLOAT, uprate_derate_during_year BOOLEAN, uprate_derate_completed_date DATE, current_planned_operating_date DATE, summer_estimated_capability_mw FLOAT, winter_estimated_capability_mw FLOAT, retirement_date DATE, PRIMARY KEY (id), FOREIGN KEY(plant_id_eia, generator_id) REFERENCES generators_entity_eia (plant_id_eia, generator_id), FOREIGN KEY(utility_id_eia) REFERENCES utilities_entity_eia (utility_id_eia), CHECK (multiple_fuels IN (0, 1)), CHECK (deliver_power_transgrid IN (0, 1)), CHECK (syncronized_transmission_grid IN (0, 1)), CHECK (planned_modifications IN (0, 1)), CHECK (other_planned_modifications IN (0, 1)), CHECK (carbon_capture IN (0, 1)), CHECK (cofire_fuels IN (0, 1)), CHECK (switch_oil_gas IN (0, 1)), CHECK (uprate_derate_during_year IN (0, 1)) ); ``` https://pudl-datasette-xl7xwcpe2a-uc.a.run.app/pudl/generators_entity_eia is: ```sql CREATE TABLE generators_entity_eia ( plant_id_eia INTEGER NOT NULL, generator_id TEXT NOT NULL, prime_mover_code TEXT, duct_burners BOOLEAN, operating_date DATE, topping_bottoming_code TEXT, solid_fuel_gasification BOOLEAN, pulverized_coal_tech BOOLEAN, fluidized_bed_tech BOOLEAN, subcritical_tech BOOLEAN, supercritical_tech BOOLEAN, ultrasupercritical_tech BOOLEAN, stoker_tech BOOLEAN, other_combustion_tech BOOLEAN, bypass_heat_recovery BOOLEAN, rto_iso_lmp_node_id TEXT, rto_iso_location_wholesale_reporting_id TEXT, associated_combined_heat_power BOOLEAN, original_planned_operating_date DATE, operating_switch TEXT, previously_canceled BOOLEAN, PRIMARY KEY (plant_id_eia, generator_id), FOREIGN KEY(plant_id_eia) REFERENCES plants_entity_eia (plant_id_eia), CHECK (duct_burners IN (0, 1)), CHECK (solid_fuel_gasification IN (0, 1)), CHECK (pulverized_coal_tech IN (0, 1)), CHECK (fluidized_bed_tech IN (0, 1)), CHECK (subcritical_tech IN (0, 1)), CHECK (supercritical_tech IN (0, 1)), CHECK (ultrasupercritical_tech IN (0, 1)), CHECK (stoker_tech IN (0, 1)), CHECK (other_combustion_tech IN (0, 1)), CHECK (bypass_heat_recovery IN (0, 1)), CHECK (associated_combined_heat_power IN (0, 1)), CHECK (previously_canceled IN (0, 1)) ); ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 743370900  
https://github.com/simonw/datasette/issues/1098#issuecomment-727655636 https://api.github.com/repos/simonw/datasette/issues/1098 727655636 MDEyOklzc3VlQ29tbWVudDcyNzY1NTYzNg== 9599 2020-11-15T23:22:27Z 2020-11-15T23:22:27Z OWNER Need to replicate this in the fixtures, then fix it. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 743370900  
https://github.com/simonw/datasette/pull/1097#issuecomment-727655018 https://api.github.com/repos/simonw/datasette/issues/1097 727655018 MDEyOklzc3VlQ29tbWVudDcyNzY1NTAxOA== 22429695 2020-11-15T23:18:18Z 2020-11-15T23:18:18Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1097?src=pr&el=h1) Report > Merging [#1097](https://codecov.io/gh/simonw/datasette/pull/1097?src=pr&el=desc) (e89211d) into [main](https://codecov.io/gh/simonw/datasette/commit/5eb8e9bf250b26e30b017d39a392c33973997656?el=desc) (5eb8e9b) will **not change** coverage. > The diff coverage is `84.61%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1097/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1097?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1097 +/- ## ======================================= Coverage 91.38% 91.38% ======================================= Files 30 30 Lines 3785 3785 ======================================= Hits 3459 3459 Misses 326 326 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1097?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [datasette/cli.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `73.63% <0.00%> (ø)` | | | [datasette/inspect.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2luc3BlY3QucHk=) | `36.11% <ø> (ø)` | | | [datasette/publish/common.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3B1Ymxpc2gvY29tbW9uLnB5) | `94.73% <ø> (ø)` | | | [datasette/tracer.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3RyYWNlci5weQ==) | `81.60% <0.00%> (ø)` | | | [datasette/utils/testing.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3V0aWxzL3Rlc3RpbmcucHk=) | `95.16% <ø> (ø)` | | | [datasette/publish/heroku.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3B1Ymxpc2gvaGVyb2t1LnB5) | `87.12% <50.00%> (ø)` | | | [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `96.46% <66.66%> (ø)` | | | [datasette/filters.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2ZpbHRlcnMucHk=) | `94.35% <77.77%> (ø)` | | | [datasette/utils/\_\_init\_\_.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.01% <86.20%> (ø)` | | | [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `95.92% <92.30%> (ø)` | | | ... and [9 more](https://codecov.io/gh/simonw/datasette/pull/1097/diff?src=pr&el=tree-more) | | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1097?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1097?src=pr&el=footer). Last update [5eb8e9b...e89211d](https://codecov.io/gh/simonw/datasette/pull/1097?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 743369188  
https://github.com/simonw/datasette/issues/942#issuecomment-727626657 https://api.github.com/repos/simonw/datasette/issues/942 727626657 MDEyOklzc3VlQ29tbWVudDcyNzYyNjY1Nw== 9599 2020-11-15T19:54:44Z 2020-11-15T19:54:44Z OWNER This will also benefit from the metadata plugin hook: #860 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 681334912  
https://github.com/simonw/datasette/issues/1091#issuecomment-727233553 https://api.github.com/repos/simonw/datasette/issues/1091 727233553 MDEyOklzc3VlQ29tbWVudDcyNzIzMzU1Mw== 9599 2020-11-14T16:46:52Z 2020-11-14T16:46:52Z OWNER @tballison could I see the section of your Apache config that configures the proxying to `/datasette/`? 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/1091#issuecomment-726801731 https://api.github.com/repos/simonw/datasette/issues/1091 726801731 MDEyOklzc3VlQ29tbWVudDcyNjgwMTczMQ== 6739646 2020-11-13T14:40:56Z 2020-11-13T14:40:56Z NONE My headers aren't clickable/sortable with custom sql, but I think that's by design. In the default view, https://corpora.tika.apache.org/datasette/file_profiles/file_profiles, ah, y, now I see that the headers should be sortable, but you're right the base_url is not applied. base_url works with "View and Edit SQL" and with "(advanced)" As you point out, does not work with the export csv, json, other or with the "Next page" navigational button at the bottom. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/1091#issuecomment-726798745 https://api.github.com/repos/simonw/datasette/issues/1091 726798745 MDEyOklzc3VlQ29tbWVudDcyNjc5ODc0NQ== 6739646 2020-11-13T14:35:22Z 2020-11-13T14:35:22Z NONE I'm starting this with docker like so: `docker run --name datasette -d -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/file_profiles.db --config sql_time_limit_ms:120000 --config max_returned_rows:100000 --config base_url:/datasette/ --config cache_size_kb:50000` I'm not doing any templating or anything else custom. Apropos of nothing, I swapped out a simpler db, so this query should now work: https://corpora.tika.apache.org/datasette/file_profiles?sql=select%0D%0A++*%0D%0Afrom%0D%0A++file_profiles+fp%0D%0Alimit%0D%0A++10 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/268#issuecomment-726419027 https://api.github.com/repos/simonw/datasette/issues/268 726419027 MDEyOklzc3VlQ29tbWVudDcyNjQxOTAyNw== 9599 2020-11-13T00:09:04Z 2020-11-13T00:09:04Z OWNER Part of the challenge here is that this is the first time the `TableView` will have had a complete rewrite of the SQL it is going to execute. That SQL is currently constructed here: https://github.com/simonw/datasette/blob/5eb8e9bf250b26e30b017d39a392c33973997656/datasette/views/table.py#L628-L636 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 323718842  
https://github.com/simonw/datasette/issues/749#issuecomment-726417847 https://api.github.com/repos/simonw/datasette/issues/749 726417847 MDEyOklzc3VlQ29tbWVudDcyNjQxNzg0Nw== 9599 2020-11-13T00:05:14Z 2020-11-13T00:05:14Z OWNER https://cloud.google.com/blog/products/serverless/cloud-run-now-supports-http-grpc-server-streaming indicates this limit should no longer apply: > With this addition, Cloud Run can now ... Send responses larger than the previous 32 MB limit But I'm still getting errors from Cloud Run attempting to download `.db` files larger than 32 MB. I filed a question in their issue tracker about that here: https://issuetracker.google.com/issues/173038375 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 610829227  
https://github.com/simonw/datasette/issues/1091#issuecomment-726416330 https://api.github.com/repos/simonw/datasette/issues/1091 726416330 MDEyOklzc3VlQ29tbWVudDcyNjQxNjMzMA== 9599 2020-11-13T00:00:43Z 2020-11-13T00:00:43Z OWNER Here's where `url_csv` comes from: https://github.com/simonw/datasette/blob/11eb1e026f3d84cb771f8d6e204939cbaee130cd/datasette/views/base.py#L542-L545 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/1091#issuecomment-726415991 https://api.github.com/repos/simonw/datasette/issues/1091 726415991 MDEyOklzc3VlQ29tbWVudDcyNjQxNTk5MQ== 9599 2020-11-12T23:59:34Z 2020-11-12T23:59:34Z OWNER The sort headers are generated by this template code: https://github.com/simonw/datasette/blob/5eb8e9bf250b26e30b017d39a392c33973997656/datasette/templates/_table.html#L11-L15 The export links use this code: https://github.com/simonw/datasette/blob/5eb8e9bf250b26e30b017d39a392c33973997656/datasette/templates/table.html#L134 https://github.com/simonw/datasette/blob/5eb8e9bf250b26e30b017d39a392c33973997656/datasette/templates/table.html#L180-L201 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/1091#issuecomment-726415019 https://api.github.com/repos/simonw/datasette/issues/1091 726415019 MDEyOklzc3VlQ29tbWVudDcyNjQxNTAxOQ== 9599 2020-11-12T23:56:23Z 2020-11-12T23:56:23Z OWNER @tballison is there any chance you're running any custom templates in that installation? I'm really confused as to why I can't replicate the bug. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/1091#issuecomment-726413829 https://api.github.com/repos/simonw/datasette/issues/1091 726413829 MDEyOklzc3VlQ29tbWVudDcyNjQxMzgyOQ== 9599 2020-11-12T23:52:50Z 2020-11-12T23:54:16Z OWNER Hmm... it's not just the `.csv` and `.json` export links - it's the column headings (which can be clicked to change the sort order) as well. Here's an extract of the HTML from that page: ```html <p class="export-links">This data as <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json">json</a>, <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.csv?_size=max">CSV</a> ( <a href="#export">advanced</a>) </p> <div class="table-wrapper"> <table class="rows-and-columns"> <thead> <tr> <th class="col-Link" scope="col" data-column="Link" data-column-type="" data-column-not-null="0" data-is-pk="0"> Link </th> <th class="col-rowid" scope="col" data-column="rowid" data-column-type="integer" data-column-not-null="0" data-is-pk="1"> <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES?_sort_desc=rowid" rel="nofollow">rowid&nbsp;â–¼</a> </th> <th class="col-PARSE_EXCEPTION_ID" scope="col" data-column="PARSE_EXCEPTION_ID" data-column-type="INTEGER" data-column-not-null="0" data-is-pk="0"> <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES?_sort=PARSE_EXCEPTION_ID" rel="nofollow">PARSE_EXCEPTION_ID</a> </th> <th class="col-PARSE_EXCEPTION_DESCRIPTION" scope="col" data-column="PARSE_EXCEPTION_DESCRIPTION" data-column-type="VARCHAR(128)" data-column-not-null="0" data-is-pk="0"> <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES?_sort=PARSE_EXCEPTION_DESCRIPTION" rel="nofollow">PARSE_EXCEPTION_DESCRIPTION</a> </th> </tr> </thead> <tbody> <tr> <td class="col-Link type-pk"> <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/1">1</a> </td> <td class="col-rowid type-int">1</td> <td class="col-PARSE_EXCEPTION_ID type-int">0</td> <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">RUNTIME</td> </tr> <tr> <td class="col-Link type-pk"> <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/2">2</a> </td> <td class="col-rowid type-int">2</td> <td class="col-PARSE_EXCEPTION_ID type-int">1</td> <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">ENCRYPTION</td> </tr> <tr> <td class="col-Link type-pk"> <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/3">3</a> </td> <td class="col-rowid type-int">3</td> <td class="col-PARSE_EXCEPTION_ID type-int">2</td> <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">ACCESS_PERMISSION</td> </tr> <tr> <td class="col-Link type-pk"> <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/4">4</a> </td> <td class="col-rowid type-int">4</td> <td class="col-PARSE_EXCEPTION_ID type-int">3</td> <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">UNSUPPORTED_VERSION</td> </tr> </tbody> </table> </div> <div id="export" class="advanced-export"> <h3>Advanced export</h3> <p>JSON shape: <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json">default</a>, <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json?_shape=array">array</a>, <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json?_shape=array&amp;_nl=on">newline-delimited</a> </p> <form action="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.csv" method="get"> <p> CSV options: <label> <input type="checkbox" name="_dl"> download file </label> <input type="submit" value="Export CSV"> <input type="hidden" name="_size" value="max"> </p> </form> </div> ``` But here's something _really_ weird - the links to the individual rows DO include the `/datasette/` prefix: ```html <td class="col-Link type-pk"> <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/2">2</a> </td> ``` The navigation bar on that page is correct too: ```html <p class="crumbs"> <a href="/datasette/">home</a> / <a href="/datasette/corpora-metadata">corpora-metadata</a> </p> ``` I've also been unable to replicate this in my own local environment, running `datasette fixtures.db --config base_url:/datasette/`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 742011049  
https://github.com/simonw/datasette/issues/865#issuecomment-726412057 https://api.github.com/repos/simonw/datasette/issues/865 726412057 MDEyOklzc3VlQ29tbWVudDcyNjQxMjA1Nw== 9599 2020-11-12T23:49:23Z 2020-11-12T23:49:23Z OWNER @tballison thanks, I've split that out into a new issue #1091 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 644582921  
https://github.com/simonw/datasette/issues/865#issuecomment-726385782 https://api.github.com/repos/simonw/datasette/issues/865 726385782 MDEyOklzc3VlQ29tbWVudDcyNjM4NTc4Mg== 6739646 2020-11-12T22:41:06Z 2020-11-12T22:41:06Z NONE The same is true if I select advanced export and hit the 'export csv' at the bottom of the page. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 644582921  
https://github.com/simonw/datasette/issues/865#issuecomment-726385422 https://api.github.com/repos/simonw/datasette/issues/865 726385422 MDEyOklzc3VlQ29tbWVudDcyNjM4NTQyMg== 6739646 2020-11-12T22:40:14Z 2020-11-12T22:40:14Z NONE Just tested with the latest Docker image, and it works pretty much everywhere! THANK YOU! I did notice that if I try to export json or csv, the base is not applied. Not sure if I should reopen this issue or open a new one. To see this, go here: https://corpora.tika.apache.org/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES Click/hover over json or CSV and you'll see that the 'datasette' base is not included. Again, many thanks! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 644582921  
https://github.com/simonw/datasette/issues/1089#issuecomment-726127465 https://api.github.com/repos/simonw/datasette/issues/1089 726127465 MDEyOklzc3VlQ29tbWVudDcyNjEyNzQ2NQ== 9599 2020-11-12T14:54:11Z 2020-11-12T14:54:11Z OWNER Suggested list to look out for from that PR: - simply/simple - easy/easier/easiest - obvious/obviously - just - merely - straightforward - ridiculous 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 741665726  
https://github.com/simonw/datasette/issues/1088#issuecomment-725830716 https://api.github.com/repos/simonw/datasette/issues/1088 725830716 MDEyOklzc3VlQ29tbWVudDcyNTgzMDcxNg== 9599 2020-11-12T04:35:38Z 2020-11-12T04:35:38Z OWNER I'm going to fix this without a test, because writing a test for this is a bit fiddly and it's a very minor bug. If it comes back again I'll do the work to test for it. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 741268956  
https://github.com/simonw/datasette/issues/1088#issuecomment-725830533 https://api.github.com/repos/simonw/datasette/issues/1088 725830533 MDEyOklzc3VlQ29tbWVudDcyNTgzMDUzMw== 9599 2020-11-12T04:35:08Z 2020-11-12T04:35:08Z OWNER Yup, swapping `QueryInterrupted` fixed this against my giant database. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 741268956  
https://github.com/simonw/datasette/issues/1088#issuecomment-725829903 https://api.github.com/repos/simonw/datasette/issues/1088 725829903 MDEyOklzc3VlQ29tbWVudDcyNTgyOTkwMw== 9599 2020-11-12T04:33:14Z 2020-11-12T04:33:14Z OWNER I'm suspicious of this code: https://github.com/simonw/datasette/blob/e8e0a6f284ca953b2980186c4356594c07bd1929/datasette/views/table.py#L1032-L1045 This code uses a different exception: https://github.com/simonw/datasette/blob/e8e0a6f284ca953b2980186c4356594c07bd1929/datasette/views/table.py#L658-L663 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 741268956  
https://github.com/simonw/datasette/pull/1085#issuecomment-725731685 https://api.github.com/repos/simonw/datasette/issues/1085 725731685 MDEyOklzc3VlQ29tbWVudDcyNTczMTY4NQ== 22429695 2020-11-12T00:01:18Z 2020-11-12T00:01:18Z NONE # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1085?src=pr&el=h1) Report > Merging [#1085](https://codecov.io/gh/simonw/datasette/pull/1085?src=pr&el=desc) (51e7651) into [main](https://codecov.io/gh/simonw/datasette/commit/2a981e2ac1d13125973904b777d00ea75e8df4e6?el=desc) (2a981e2) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1085/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1085?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1085 +/- ## ======================================= Coverage 91.38% 91.38% ======================================= Files 30 30 Lines 3785 3785 ======================================= Hits 3459 3459 Misses 326 326 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1085?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1085?src=pr&el=footer). Last update [2a981e2...51e7651](https://codecov.io/gh/simonw/datasette/pull/1085?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 740512882  
https://github.com/simonw/datasette/issues/1086#issuecomment-725729857 https://api.github.com/repos/simonw/datasette/issues/1086 725729857 MDEyOklzc3VlQ29tbWVudDcyNTcyOTg1Nw== 9599 2020-11-11T23:55:39Z 2020-11-11T23:55:39Z OWNER Demo: https://latest.datasette.io/fixtures/foreign_key_references?_facet=foreign_key_with_blank_label <img width="772" alt="fixtures__foreign_key_references__2_rows" src="https://user-images.githubusercontent.com/9599/98877868-53575b80-2436-11eb-8669-f24eeb8d0f3c.png"> 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 741021342  
https://github.com/simonw/datasette/issues/1086#issuecomment-725714908 https://api.github.com/repos/simonw/datasette/issues/1086 725714908 MDEyOklzc3VlQ29tbWVudDcyNTcxNDkwOA== 9599 2020-11-11T23:17:26Z 2020-11-11T23:17:26Z OWNER I'm just going to use a regular coloured hyphen. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 741021342  
https://github.com/simonw/datasette/issues/1086#issuecomment-725622784 https://api.github.com/repos/simonw/datasette/issues/1086 725622784 MDEyOklzc3VlQ29tbWVudDcyNTYyMjc4NA== 9599 2020-11-11T19:41:45Z 2020-11-11T19:41:45Z OWNER Maybe use a grey hyphen here? 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 741021342  
https://github.com/simonw/sqlite-utils/issues/168#issuecomment-723829147 https://api.github.com/repos/simonw/sqlite-utils/issues/168 723829147 MDEyOklzc3VlQ29tbWVudDcyMzgyOTE0Nw== 9599 2020-11-09T07:43:30Z 2020-11-09T07:43:30Z OWNER Yeah whatever process the have in place is working great without any extra intervention: they upgraded to 3.0 four hours ago. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 706167456  
https://github.com/simonw/datasette/issues/268#issuecomment-723740546 https://api.github.com/repos/simonw/datasette/issues/268 723740546 MDEyOklzc3VlQ29tbWVudDcyMzc0MDU0Ng== 9599 2020-11-09T04:01:50Z 2020-11-09T04:01:50Z OWNER I should depend on `sqlite-fts4` - I'm doing that in `sqlite-utils` now and it works great: https://github.com/simonw/sqlite-utils/issues/198 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 323718842  
https://github.com/simonw/sqlite-utils/issues/192#issuecomment-723637930 https://api.github.com/repos/simonw/sqlite-utils/issues/192 723637930 MDEyOklzc3VlQ29tbWVudDcyMzYzNzkzMA== 9599 2020-11-08T17:06:56Z 2020-11-08T17:06:56Z OWNER This looks pretty good now! ``` % sqlite-utils search 24ways.db articles simon -c title -c author -t title author ----------------------------------------------------------------------------- ------------------ Don't be eval() Simon Willison DOM Scripting Your Way to Better Blockquotes Jeremy Keith Swooshy Curly Quotes Without Images Simon Collison The Articulate Web Designer of Tomorrow Simon Collison Writing Responsible JavaScript Drew McLellan Going Nuts with CSS Transitions Natalie Downe Managing a Mind Christopher Murphy Design Systems Laura Kalbag Bringing Your Code to the Streets Ruth John Taming Complexity Simon Collison Unobtrusively Mapping Microformats with jQuery Simon Willison Crafting the Front-end Ben Bodien Develop Your Naturalist Superpowers with Observable Notebooks and iNaturalist Natalie Downe Fast Autocomplete Search for Your Website Simon Willison ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735532751  
https://github.com/simonw/sqlite-utils/issues/201#issuecomment-723369033 https://api.github.com/repos/simonw/sqlite-utils/issues/201 723369033 MDEyOklzc3VlQ29tbWVudDcyMzM2OTAzMw== 9599 2020-11-07T01:28:11Z 2020-11-07T01:28:11Z OWNER Need to fix this to close #192 and #197. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 738128913  
https://github.com/simonw/sqlite-utils/issues/194#issuecomment-723368528 https://api.github.com/repos/simonw/sqlite-utils/issues/194 723368528 MDEyOklzc3VlQ29tbWVudDcyMzM2ODUyOA== 9599 2020-11-07T01:24:55Z 2020-11-07T01:24:55Z OWNER Here's the alpha release: https://github.com/simonw/sqlite-utils/releases/tag/3.0a0 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735650864  
https://github.com/simonw/sqlite-utils/issues/197#issuecomment-723365651 https://api.github.com/repos/simonw/sqlite-utils/issues/197 723365651 MDEyOklzc3VlQ29tbWVudDcyMzM2NTY1MQ== 9599 2020-11-07T01:06:32Z 2020-11-07T01:06:32Z OWNER Documentation: https://sqlite-utils.readthedocs.io/en/latest/python-api.html#searching-with-table-search 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737153927  
https://github.com/simonw/sqlite-utils/issues/197#issuecomment-723360842 https://api.github.com/repos/simonw/sqlite-utils/issues/197 723360842 MDEyOklzc3VlQ29tbWVudDcyMzM2MDg0Mg== 9599 2020-11-07T00:40:55Z 2020-11-07T00:40:55Z OWNER The `order=` parameter should be called `order_by` for consistency with this: ```python for row in db["dogs"].rows_where("age > 1", order_by="age"): ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737153927  
https://github.com/simonw/sqlite-utils/issues/200#issuecomment-723357855 https://api.github.com/repos/simonw/sqlite-utils/issues/200 723357855 MDEyOklzc3VlQ29tbWVudDcyMzM1Nzg1NQ== 9599 2020-11-07T00:24:37Z 2020-11-07T00:24:37Z OWNER ``` (sqlite-utils) sqlite-utils % sqlite-utils rows 24ways-fts4.db articles -c title -c author | head -n 3 [{"title": "Why Bother with Accessibility?", "author": "Laura Kalbag"}, {"title": "Levelling Up", "author": "Ashley Baxter"}, {"title": "Project Hubs: A Home Base for Design Projects", "author": "Brad Frost"}, ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 738115165  
https://github.com/simonw/sqlite-utils/issues/192#issuecomment-723357117 https://api.github.com/repos/simonw/sqlite-utils/issues/192 723357117 MDEyOklzc3VlQ29tbWVudDcyMzM1NzExNw== 9599 2020-11-07T00:21:05Z 2020-11-07T00:21:05Z OWNER I'm going to have it only output the exact `-c` columns you requested (if you requested any). Add `--rank` to specify the rank column, since you may not know what its name is. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735532751  
https://github.com/simonw/sqlite-utils/issues/194#issuecomment-723356020 https://api.github.com/repos/simonw/sqlite-utils/issues/194 723356020 MDEyOklzc3VlQ29tbWVudDcyMzM1NjAyMA== 9599 2020-11-07T00:16:06Z 2020-11-07T00:16:06Z OWNER I'm going to release this as an alpha first and sit on it for a few days, since I don't want to ship any mistakes that would result in having to bump straight to 4.0! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735650864  
https://github.com/simonw/sqlite-utils/issues/91#issuecomment-723350956 https://api.github.com/repos/simonw/sqlite-utils/issues/91 723350956 MDEyOklzc3VlQ29tbWVudDcyMzM1MDk1Ng== 9599 2020-11-06T23:53:25Z 2020-11-06T23:53:25Z OWNER This is now possible, for both FTS4 and FTS5 - see #197. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 577302229  
https://github.com/simonw/sqlite-utils/issues/192#issuecomment-723348722 https://api.github.com/repos/simonw/sqlite-utils/issues/192 723348722 MDEyOklzc3VlQ29tbWVudDcyMzM0ODcyMg== 9599 2020-11-06T23:43:09Z 2020-11-06T23:43:09Z OWNER Also that order looks incorrect. It looks like most relevant came back last, not first. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735532751  
https://github.com/simonw/sqlite-utils/issues/192#issuecomment-723348614 https://api.github.com/repos/simonw/sqlite-utils/issues/192 723348614 MDEyOklzc3VlQ29tbWVudDcyMzM0ODYxNA== 9599 2020-11-06T23:42:38Z 2020-11-06T23:42:38Z OWNER This is a bit surprising: ``` (sqlite-utils) sqlite-utils % sqlite-utils search 24ways-fts4.db articles maps -c title [{"rowid": 41, "title": "What Is Vagrant and Why Should I Care?", "rank": -1.9252039178908076}, {"rowid": 298, "title": "First Steps in VR", "rank": -1.9945466378736434}, {"rowid": 43, "title": "Content Production Planning", "rank": -2.1928058363046143}, {"rowid": 100, "title": "Moo'y Christmas", "rank": -2.2698482999851675}, {"rowid": 91, "title": "Infinite Canvas: Moving Beyond the Page", "rank": -2.290928999035195}, {"rowid": 175, "title": "Front-End Code Reusability with CSS and JavaScript", "rank": -2.498731782924352}, {"rowid": 209, "title": "Feeding the Audio Graph", "rank": -2.619968930100356}, {"rowid": 296, "title": "Animation in Design Systems", "rank": -2.62060151817201}, {"rowid": 118, "title": "Ghosts On The Internet", "rank": -2.7224894534521087}, {"rowid": 77, "title": "Colour Accessibility", "rank": -2.7389782859427343}, {"rowid": 245, "title": "Web Content Accessibility Guidelines 2.1\u2014for People Who Haven\u2019t Read the Update", "rank": -2.9750992611162888}, {"rowid": 56, "title": "Helping VIPs Care About Performance", "rank": -3.0819662908932535}, {"rowid": 109, "title": "Geotag Everywhere with Fire Eagle", "rank": -3.1371975973877277}, {"rowid": 203, "title": "Jobs-to-Be-Done in Your UX Toolbox", "rank": -3.2416719461682733}, {"rowid": 276, "title": "Your jQuery: Now With 67% Less Suck", "rank": -3.4947916564653028}, {"rowid": 58, "title": "Beyond the Style Guide", "rank": -3.7508321464447905}, {"rowid": 225, "title": "Good Ideas Grow on Paper", "rank": -4.120077674716844}, {"rowid": 168, "title": "Unobtrusively Mapping Microformats with jQuery", "rank": -4.662224207228984}, {"rowid": 27, "title": "Putting Design on the Map", "rank": -5.667327088267961}, {"rowid": 220, "title": "Finding Your Way with Static Maps", "rank": -9.952534352591737}] ``` I requested just `-c title` but also got back `rowid` and `rank`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735532751  
https://github.com/simonw/sqlite-utils/issues/194#issuecomment-723234493 https://api.github.com/repos/simonw/sqlite-utils/issues/194 723234493 MDEyOklzc3VlQ29tbWVudDcyMzIzNDQ5Mw== 9599 2020-11-06T18:32:34Z 2020-11-06T18:32:34Z OWNER The breaking changes will be: - `table.search()` now returns a generator that produces dictionaries, similar to `table.rows_where()` - The `-c` shortcut no longer works, use `--csv` instead. - The `-f` shortcut no longer works, use `--fmt` instead. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735650864  
https://github.com/simonw/sqlite-utils/issues/197#issuecomment-723230732 https://api.github.com/repos/simonw/sqlite-utils/issues/197 723230732 MDEyOklzc3VlQ29tbWVudDcyMzIzMDczMg== 9599 2020-11-06T18:24:29Z 2020-11-06T18:24:29Z OWNER Still need to update docs. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737153927  
https://github.com/simonw/sqlite-utils/pull/195#issuecomment-723148906 https://api.github.com/repos/simonw/sqlite-utils/issues/195 723148906 MDEyOklzc3VlQ29tbWVudDcyMzE0ODkwNg== 9599 2020-11-06T15:43:51Z 2020-11-06T15:43:51Z OWNER Thanks to #198 (introducing a `rank_bm25()` custom function for FTS4) this feature will be able to offer relevance search for both FTS5 AND FTS4 tables. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735663855  
https://github.com/simonw/sqlite-utils/issues/197#issuecomment-723148310 https://api.github.com/repos/simonw/sqlite-utils/issues/197 723148310 MDEyOklzc3VlQ29tbWVudDcyMzE0ODMxMA== 9599 2020-11-06T15:42:43Z 2020-11-06T15:42:43Z OWNER Having `.search()` return tuples when `.rows_where()` returns dictionaries just feels like bad API design to me - it's inconsistent. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737153927  
https://github.com/simonw/sqlite-utils/issues/199#issuecomment-723147463 https://api.github.com/repos/simonw/sqlite-utils/issues/199 723147463 MDEyOklzc3VlQ29tbWVudDcyMzE0NzQ2Mw== 9599 2020-11-06T15:41:00Z 2020-11-06T15:41:00Z OWNER Something like this: ``` @db.register_function(replace=True) def my_function(a): return a.upper() ``` If `replace=True` then this function will be registered even if a `my_function` of arity 1 has already been registered previously. It defaults to `False` though which means the Database object tracks what functions and arities have been registered in the past and silently ignores any new attempts to register the same name/arity. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737855731  
https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383 https://api.github.com/repos/simonw/sqlite-utils/issues/198 723145383 MDEyOklzc3VlQ29tbWVudDcyMzE0NTM4Mw== 9599 2020-11-06T15:36:47Z 2020-11-06T15:36:47Z OWNER Should I register the custom `rank_bm25` SQLite function for every connection, or should I register it against the connection just the first time the user attempts an FTS4 search? I think I'd rather register it only if it is needed. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737476423  
https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723144893 https://api.github.com/repos/simonw/sqlite-utils/issues/198 723144893 MDEyOklzc3VlQ29tbWVudDcyMzE0NDg5Mw== 9599 2020-11-06T15:35:45Z 2020-11-06T15:35:45Z OWNER Here's a demo of that rank query. I had to sort by `rank` instead of `rank desc` - need to double-check that: https://datasette-sqlite-fts4.datasette.io/24ways-fts4?sql=with+original+as+(%0D%0A++++select%0D%0A++++++++rowid%2C%0D%0A++++++++*%0D%0A++++from+[articles]%0D%0A)%0D%0Aselect%0D%0A++++original.*%2C%0D%0A++++rank_bm25(matchinfo([articles_fts]%2C+%27pcnalx%27))+as+rank%0D%0Afrom%0D%0A++++[original]%0D%0A++++join+[articles_fts]+on+[original].rowid+%3D+[articles_fts].rowid%0D%0Awhere%0D%0A++++[articles_fts]+match+%3Aquery%0D%0Aorder+by%0D%0A++++rank%0D%0Alimit+20&query=jquery+maps 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737476423  
https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723143633 https://api.github.com/repos/simonw/sqlite-utils/issues/198 723143633 MDEyOklzc3VlQ29tbWVudDcyMzE0MzYzMw== 9599 2020-11-06T15:33:12Z 2020-11-06T15:33:12Z OWNER Here's the FTS5 query: ```sql with original as ( select rowid, * from [global-power-plants] ) select original.*, [global-power-plants_fts].rank as rank from [original] join [global-power-plants_fts] on [original].rowid = [global-power-plants_fts].rowid where [global-power-plants_fts] match :query order by rank desc limit 20 ``` The equivalent using `rank_bm25()` for FTS4 would be: ```sql with original as ( select rowid, * from [global-power-plants] ) select original.*, rank_bm25(matchinfo([global-power-plants_fts], 'pcnalx')) as rank from [original] join [global-power-plants_fts] on [original].rowid = [global-power-plants_fts].rowid where [global-power-plants_fts] match :query order by rank desc limit 20 ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737476423  
https://github.com/simonw/sqlite-utils/issues/198#issuecomment-722895825 https://api.github.com/repos/simonw/sqlite-utils/issues/198 722895825 MDEyOklzc3VlQ29tbWVudDcyMjg5NTgyNQ== 9599 2020-11-06T06:29:17Z 2020-11-06T06:29:17Z OWNER I released a 1.0 (and 1.0.1) version of https://github.com/simonw/sqlite-fts4 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737476423  
https://github.com/simonw/sqlite-utils/issues/198#issuecomment-722852262 https://api.github.com/repos/simonw/sqlite-utils/issues/198 722852262 MDEyOklzc3VlQ29tbWVudDcyMjg1MjI2Mg== 9599 2020-11-06T05:41:58Z 2020-11-06T05:41:58Z OWNER Example query (from the tests): ```sql select c0, c1, rank_bm25(matchinfo(search, 'pcnalx')) as bm25 from search where search match ? ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737476423  
https://github.com/simonw/sqlite-utils/issues/198#issuecomment-722849539 https://api.github.com/repos/simonw/sqlite-utils/issues/198 722849539 MDEyOklzc3VlQ29tbWVudDcyMjg0OTUzOQ== 9599 2020-11-06T05:39:17Z 2020-11-06T05:39:17Z OWNER I'd have to copy almost all of the code in https://github.com/simonw/sqlite-fts4/blob/master/sqlite_fts4/__init__.py so I think I will add it as a dependency instead. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737476423  
https://github.com/simonw/sqlite-utils/issues/197#issuecomment-722545442 https://api.github.com/repos/simonw/sqlite-utils/issues/197 722545442 MDEyOklzc3VlQ29tbWVudDcyMjU0NTQ0Mg== 9599 2020-11-05T18:05:33Z 2020-11-05T18:05:33Z OWNER This is likely to result in a 3.0 release due to a backwards-incompatible change to the current `.search()` method - #194 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 737153927  
https://github.com/simonw/sqlite-utils/pull/195#issuecomment-722542895 https://api.github.com/repos/simonw/sqlite-utils/issues/195 722542895 MDEyOklzc3VlQ29tbWVudDcyMjU0Mjg5NQ== 9599 2020-11-05T18:01:33Z 2020-11-05T18:01:33Z OWNER Latest test failure: ``` 114 -> assert [("racoons are biting trash pandas", "USA", "bar")] == table.search( 115 "bite", order="rowid" 116 ) 117 118 119 def test_optimize_fts(fresh_db): (Pdb) table.search("bite") [(2, 'racoons are biting trash pandas', 'USA', 'bar', -9.641434262948206e-07)] ``` The problem here is that the `table.search()` method now behaves differently for FTS4 v.s. FTS5 tables. With FTS4 you get back just the table columns. With FTS5 you also get back the `rowid` as the first column and the `rank` score as the last column. This is weird. It also makes me question whether having `.search()` return a list of tuples is the right API design. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 735663855  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722086105 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722086105 MDEyOklzc3VlQ29tbWVudDcyMjA4NjEwNQ== 9599 2020-11-05T02:29:50Z 2020-11-05T03:39:58Z OWNER The finished monster: ```python _virtual_table_using_re = re.compile(r""" ^ # Start of string \s*CREATE\s+VIRTUAL\s+TABLE\s+ # CREATE VIRTUAL TABLE ( '(?P<squoted_table>[^']*(?:''[^']*)*)' | # single quoted name "(?P<dquoted_table>[^"]*(?:""[^"]*)*)" | # double quoted name `(?P<backtick_table>[^`]+)` | # `backtick` quoted name \[(?P<squarequoted_table>[^\]]+)\] | # [...] quoted name (?P<identifier> # SQLite non-quoted identifier [A-Za-z_\u0080-\uffff] # \u0080-\uffff = "any character larger than u007f" [A-Za-z_\u0080-\uffff0-9\$]* # zero-or-more alphanemuric or $ ) ) \s+(IF\s+NOT\s+EXISTS\s+)? # IF NOT EXISTS (optional) USING\s+(?P<using>\w+) # e.g. USING FTS5 """, re.VERBOSE | re.IGNORECASE) ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722084593 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722084593 MDEyOklzc3VlQ29tbWVudDcyMjA4NDU5Mw== 9599 2020-11-05T02:24:47Z 2020-11-05T02:24:47Z OWNER And an identifier is "ALPHABETIC character and continue with zero or more ALPHANUMERIC characters and/or "$" (u0024) characters" So... [\u0041-\u005a\u0061-\u0071\u007f-\uffff\u005f][\u0041-\u005a\u0061-\u0071\u007f-\uffff\u005f\u0030-\u0039\u0024]+ 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722084213 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722084213 MDEyOklzc3VlQ29tbWVudDcyMjA4NDIxMw== 9599 2020-11-05T02:23:37Z 2020-11-05T02:23:37Z OWNER So... ALPHABETIC: `[\u0041-\u005a\u0061-\u0071\u007f-\uffff\u005f]` NUMERIC: `[\u0030-\u0039]` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722083527 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722083527 MDEyOklzc3VlQ29tbWVudDcyMjA4MzUyNw== 9599 2020-11-05T02:21:26Z 2020-11-05T02:21:26Z OWNER I think that's `\u007F-\uFFFF` in regex range speak. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722082874 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722082874 MDEyOklzc3VlQ29tbWVudDcyMjA4Mjg3NA== 9599 2020-11-05T02:19:18Z 2020-11-05T02:19:18Z OWNER "any other character larger than u007f." Need to figure that out! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722082759 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722082759 MDEyOklzc3VlQ29tbWVudDcyMjA4Mjc1OQ== 9599 2020-11-05T02:18:58Z 2020-11-05T02:18:58Z OWNER More from that document, describing `ALPHANUMERIC`: > **ALPHABETIC** > > Any of the characters in the range u0041 through u005a (letters "A" through "Z") or in the range u0061 through u007a (letters "a" through "z") or the character u005f ("_") or any other character larger than u007f. > > **NUMERIC** > > Any of the characters in the range u0030 through u0039 (digits "0" through "9") > > **ALPHANUMERIC** > > Any character which is either ALPHABETIC or NUMERIC 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722082497 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722082497 MDEyOklzc3VlQ29tbWVudDcyMjA4MjQ5Nw== 9599 2020-11-05T02:18:08Z 2020-11-05T02:18:08Z OWNER I'm missing the case where a table has no quotes around it at all - `create virtual table foo using fts5` So I need to know how to create a regex for a SQLite identifier. https://www.sqlite.org/draft/tokenreq.html seems to be the only available documentation for that. > > ### Identifier tokens > > Identifiers follow the usual rules with the exception that SQLite allows the dollar-sign symbol in the interior of an identifier. The dollar-sign is for compatibility with Microsoft SQL-Server and is not part of the SQL standard. > > > **H41130:** SQLite shall recognize as an ID token any sequence of characters that begins with an ALPHABETIC character and continue with zero or more ALPHANUMERIC characters and/or "$" (u0024) characters and which is not a keyword token. > > Identifiers can be arbitrary character strings within square brackets. This feature is also for compatibility with Microsoft SQL-Server and not a part of the SQL standard. > > > **H41140:** SQLite shall recognize as an ID token any sequence of non-zero characters that begins with "[" (u005b) and continuing through the first "]" (u005d) character. > > The standard way of quoting SQL identifiers is to use double-quotes. > > > **H41150:** SQLite shall recognize as an ID token any sequence of characters that begins with a double-quote (u0022), is followed by zero or more non-zero characters and/or pairs of double-quotes (u0022) and terminates with a double-quote (u0022) that is not part of a pair. > > MySQL allows identifiers to be quoted using the grave accent character. SQLite supports this for interoperability. > > > **H41160:** SQLite shall recognize as an ID token any sequence of characters that begins with a grave accent (u0060), is followed by zero or more non-zero characters and/or pairs ofgrave accents (u0060) and terminates with a grave accent (u0022) that is not part of a pair. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722078361 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722078361 MDEyOklzc3VlQ29tbWVudDcyMjA3ODM2MQ== 9599 2020-11-05T02:04:33Z 2020-11-05T02:04:33Z OWNER Next step: lots of unit tests. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310  
https://github.com/simonw/sqlite-utils/issues/196#issuecomment-722078286 https://api.github.com/repos/simonw/sqlite-utils/issues/196 722078286 MDEyOklzc3VlQ29tbWVudDcyMjA3ODI4Ng== 9599 2020-11-05T02:04:18Z 2020-11-05T02:04:18Z OWNER I think this might be it: ```python create_virtual_table_re = re.compile(r""" \s*CREATE\s+VIRTUAL\s+TABLE\s+ # CREATE VIRTUAL TABLE ( '(?P<squoted_table>[^']*(?:''[^']*)*)' | # single quoted name "(?P<dquoted_table>[^"]*(?:""[^"]*)*)" | # double quoted name `(?P<backtick_table>[^`]+)` | # `backtick` quoted name \[(?P<squarequoted_table>[^\]]+)\] # [...] quoted name ) \s+(IF\s+NOT\s+EXISTS\s+)? # IF NOT EXISTS (optional) USING\s+(?P<using>\w+) """, re.VERBOSE | re.IGNORECASE) ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 736520310