github

Custom SQL query returning 101 rows (hide)

This data as json, CSV

html_urlissue_urlidnode_idusercreated_atupdated_atauthor_associationbodyreactionsissueperformed_via_github_app
https://github.com/simonw/datasette/issues/1922#issuecomment-1332903011 https://api.github.com/repos/simonw/datasette/issues/1922 1332903011 IC_kwDOBm6k_c5Pcnxj 9599 2022-11-30T23:45:29Z 2022-11-30T23:45:29Z OWNER That worked for the preflight request - got this now: <img width="1377" alt="image" src="https://user-images.githubusercontent.com/9599/204931998-d70df6c0-723b-4543-b8c4-601c00fe4b10.png"> So it looks like error responses (in this case for permission denied) are missing CORS headers. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332855687 https://api.github.com/repos/simonw/datasette/issues/1922 1332855687 IC_kwDOBm6k_c5PccOH 9599 2022-11-30T23:09:31Z 2022-11-30T23:09:31Z OWNER Still getting a CORS header. I tried it in Chrome, which outputs helpful messages to the console: <img width="671" alt="image" src="https://user-images.githubusercontent.com/9599/204927523-4b9860f5-4400-4dd5-a036-de9f07f31c5f.png"> 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1923#issuecomment-1332851215 https://api.github.com/repos/simonw/datasette/issues/1923 1332851215 IC_kwDOBm6k_c5PcbIP 9599 2022-11-30T23:04:56Z 2022-11-30T23:04:56Z OWNER That fixed it. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1470320227  
https://github.com/simonw/datasette/issues/1923#issuecomment-1332842435 https://api.github.com/repos/simonw/datasette/issues/1923 1332842435 IC_kwDOBm6k_c5PcY_D 9599 2022-11-30T22:58:33Z 2022-11-30T22:58:33Z OWNER https://stackoverflow.com/questions/74490465/github-actions-failing-for-setup-gcloud/74562740#74562740 suggests trying Python 3.9. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1470320227  
https://github.com/simonw/datasette/issues/1923#issuecomment-1332835664 https://api.github.com/repos/simonw/datasette/issues/1923 1332835664 IC_kwDOBm6k_c5PcXVQ 9599 2022-11-30T22:50:10Z 2022-11-30T22:51:25Z OWNER https://stackoverflow.com/questions/74490465/github-actions-failing-for-setup-gcloud/74562526#74562526 suggests setting `version: '318.0.0'` of `google-github-actions/setup-gcloud`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1470320227  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332698636 https://api.github.com/repos/simonw/datasette/issues/1922 1332698636 IC_kwDOBm6k_c5Pb14M 9599 2022-11-30T20:25:50Z 2022-11-30T20:25:50Z OWNER I just shipped this: Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS I'll try this out on `latest.datasette.io` - but I need to research more to check if this is a safe thing to do or not. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332689547 https://api.github.com/repos/simonw/datasette/issues/1922 1332689547 IC_kwDOBm6k_c5PbzqL 9599 2022-11-30T20:16:21Z 2022-11-30T20:16:46Z OWNER That notebook: ```javascript viewof token = Inputs.text({ label: "Your API token" }) ``` ```javascript viewof createResponse = Inputs.button("Create table", { value: null, reduce: async () => { const response = await fetch( "https://latest.datasette.io/ephemeral/-/create", { method: "POST", mode: "cors", headers: { Authorization: `Bearer {$token}`, "Content-Type": "application/json" }, body: JSON.stringify({ table: "my_new_table", row: { task: "Demonstrate a JSON creation from another domain" } }) } ); return await response.json(); } }) ``` Based on this tip: https://talk.observablehq.com/t/best-pattern-for-click-here-to-submit-your-results-to-an-api-backend/7353/3 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332688245 https://api.github.com/repos/simonw/datasette/issues/1922 1332688245 IC_kwDOBm6k_c5PbzV1 9599 2022-11-30T20:15:08Z 2022-11-30T20:15:08Z OWNER Still getting a CORS error: <img width="1367" alt="image" src="https://user-images.githubusercontent.com/9599/204898990-b1370950-8ae5-4c95-8f7f-42b4a7a717e7.png"> My hunch is this is because I'm not sending `Access-Control-Allow-Methods: GET,HEAD,POST`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332585861 https://api.github.com/repos/simonw/datasette/issues/1922 1332585861 IC_kwDOBm6k_c5PbaWF 9599 2022-11-30T18:43:46Z 2022-11-30T18:43:46Z OWNER Here's what Django Rest Framework does: https://github.com/encode/django-rest-framework/blob/1ae812ea209392ad76cc5d2f35f9f7fb337f63e4/rest_framework/views.py#L514-L521 ```python def options(self, request, *args, **kwargs): """ Handler method for HTTP 'OPTIONS' request. """ if self.metadata_class is None: return self.http_method_not_allowed(request, *args, **kwargs) data = self.metadata_class().determine_metadata(request, self) return Response(data, status=status.HTTP_200_OK) ``` That default `determine_metadata` method looks like this: https://github.com/encode/django-rest-framework/blob/1ae812ea209392ad76cc5d2f35f9f7fb337f63e4/rest_framework/metadata.py#L61-L71 ```python def determine_metadata(self, request, view): metadata = OrderedDict() metadata['name'] = view.get_view_name() metadata['description'] = view.get_view_description() metadata['renders'] = [renderer.media_type for renderer in view.renderer_classes] metadata['parses'] = [parser.media_type for parser in view.parser_classes] if hasattr(view, 'get_serializer'): actions = self.determine_actions(request, view) if actions: metadata['actions'] = actions return metadata ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332580395 https://api.github.com/repos/simonw/datasette/issues/1922 1332580395 IC_kwDOBm6k_c5PbZAr 9599 2022-11-30T18:38:22Z 2022-11-30T18:38:22Z OWNER > [@simon](https://fedi.simonwillison.net/@simon) IMO, it should always be a 2XX series response, typically with no content & an extra `Allow` header with a list of HTTP verbs it responds to. https://mastodon.social/@daniellindsley/109434186252099323 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332572453 https://api.github.com/repos/simonw/datasette/issues/1922 1332572453 IC_kwDOBm6k_c5PbXEl 9599 2022-11-30T18:30:38Z 2022-11-30T18:30:54Z OWNER Started a conversation about how OPTIONS should work on Mastodon: https://fedi.simonwillison.net/@simon/109434148676475291 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332561813 https://api.github.com/repos/simonw/datasette/issues/1922 1332561813 IC_kwDOBm6k_c5PbUeV 9599 2022-11-30T18:20:05Z 2022-11-30T18:20:05Z OWNER Weird, GitHub reply with a 404! ``` ~ % curl -X OPTIONS https://github.com/ -i HTTP/2 404 server: GitHub.com date: Wed, 30 Nov 2022 18:19:39 GMT content-type: text/html; charset=utf-8 content-length: 0 strict-transport-security: max-age=31536000; includeSubdomains; preload x-frame-options: deny x-content-type-options: nosniff x-xss-protection: 0 referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin content-security-policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com objects-origin.githubusercontent.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events; font-src github.githubassets.com; form-action 'self' github.com gist.github.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com objects-origin.githubusercontent.com secured-user-images.githubusercontent.com/ opengraph.githubassets.com github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/ vary: Accept-Encoding, Accept, X-Requested-With x-github-request-id: DD6B:5ACA:102E8A6:1164A99:6387… 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332561059 https://api.github.com/repos/simonw/datasette/issues/1922 1332561059 IC_kwDOBm6k_c5PbUSj 9599 2022-11-30T18:19:20Z 2022-11-30T18:19:20Z OWNER Two test failures: ``` ____________________________ test_homepage_options _____________________________ [gw0] linux -- Python 3.11.0 /opt/hostedtoolcache/Python/3.11.0/x64/bin/python app_client = <datasette.utils.testing.TestClient object at 0x7f4c489269d0> def test_homepage_options(app_client): response = app_client.get("/", method="OPTIONS") > assert response.status == 405 E assert 200 == 405 E + where 200 = <datasette.utils.testing.TestResponse object at 0x7f4c4892f4d0>.status /home/runner/work/datasette/datasette/tests/test_html.py:58: AssertionError ______________________ test_client_methods[options-/-405] ______________________ [gw1] linux -- Python 3.11.0 /opt/hostedtoolcache/Python/3.11.0/x64/bin/python datasette = <datasette.app.Datasette object at 0x7fc33c227550> method = 'options', path = '/', expected_status = 405 @pytest.mark.asyncio @pytest.mark.parametrize( "method,path,expected_status", [ ("get", "/", 200), ("options", "/", 405), ("head", "/", 200), ("put", "/", 405), ("patch", "/", 405), ("delete", "/", 405), ], ) async def test_client_methods(datasette, method, path, expected_status): client_method = getattr(datasette.client, method) response = await client_method(path) assert isinstance(response, httpx.Response) > assert response.status_code == expected_status E assert 200 == 405 E + where 200 = <Response [200 OK]>.status_code /home/runner/work/datasette/datasette/tests/test_internals_datasette_client.py:29: AssertionError =============================== warnings summary =============================== tests/test_cli.py::test_inspect_cli_writes_to_file tests/test_cli.py::test_inspect_cli /home/runner/work/datasette/datasette/datasette/cli.py:163: DeprecationWarning: There is no current event loop loop = asyncio.get_event_loop() tests/test_cli_serve_ge… 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332504654 https://api.github.com/repos/simonw/datasette/issues/1922 1332504654 IC_kwDOBm6k_c5PbGhO 9599 2022-11-30T17:27:39Z 2022-11-30T17:27:39Z OWNER I'll test this once it's deployed to https://latest.datasette.io/ 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332493004 https://api.github.com/repos/simonw/datasette/issues/1922 1332493004 IC_kwDOBm6k_c5PbDrM 9599 2022-11-30T17:18:10Z 2022-11-30T17:18:10Z OWNER Here's why: https://github.com/simonw/datasette/blob/4ddd77e51254bda3bac990ea662bac2e6b29c5e0/datasette/views/base.py#L71-L79 That's code in `BaseView` - but it turns out the code that adds CORS headers is in the `DataView` subclass of that (which the various write API endpoints do not use). https://github.com/simonw/datasette/blob/4ddd77e51254bda3bac990ea662bac2e6b29c5e0/datasette/views/base.py#L158-L162 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1922#issuecomment-1332492092 https://api.github.com/repos/simonw/datasette/issues/1922 1332492092 IC_kwDOBm6k_c5PbDc8 9599 2022-11-30T17:17:21Z 2022-11-30T17:17:21Z OWNER I tried running `fetch()` with a POST from a separate domain and got a browser error because it did a GET against the `/db/-/create` endpoint and the 405 method not supported response did not include the CORS headers. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469973742  
https://github.com/simonw/datasette/issues/1605#issuecomment-1332310772 https://api.github.com/repos/simonw/datasette/issues/1605 1332310772 IC_kwDOBm6k_c5PaXL0 25778 2022-11-30T15:06:37Z 2022-11-30T15:06:37Z CONTRIBUTOR I'll add issues for both and do a documentation PR. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1108671952  
https://github.com/simonw/datasette/issues/1605#issuecomment-1331694246 https://api.github.com/repos/simonw/datasette/issues/1605 1331694246 IC_kwDOBm6k_c5PYAqm 9599 2022-11-30T06:18:41Z 2022-11-30T06:18:41Z OWNER Those sounds to me like they should be promoted to documented, supported internals. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1108671952  
https://github.com/simonw/datasette/issues/1918#issuecomment-1331658629 https://api.github.com/repos/simonw/datasette/issues/1918 1331658629 IC_kwDOBm6k_c5PX3-F 9599 2022-11-30T05:21:51Z 2022-11-30T05:21:51Z OWNER Much better: <img width="1147" alt="image" src="https://user-images.githubusercontent.com/9599/204714148-f21cd802-c268-41cf-bfc1-e9c2f4e48bf9.png"> 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469044738  
https://github.com/simonw/datasette/issues/1919#issuecomment-1331657404 https://api.github.com/repos/simonw/datasette/issues/1919 1331657404 IC_kwDOBm6k_c5PX3q8 9599 2022-11-30T05:19:43Z 2022-11-30T05:19:43Z OWNER This is the test: https://github.com/simonw/datasette/blob/8404b21556d133c89eda4bd1bf5335ed9a0785d6/tests/test_api_write.py#L342-L401 I'm suspicious that there's a timing error of some sort but I can't think what it might be. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469062686  
https://github.com/simonw/datasette/issues/1916#issuecomment-1331651721 https://api.github.com/repos/simonw/datasette/issues/1916 1331651721 IC_kwDOBm6k_c5PX2SJ 9599 2022-11-30T05:10:27Z 2022-11-30T05:10:27Z OWNER They should return 405 method not allowed with an `{"ok":false, "error": "Method not allowed"}` body. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469015001  
https://github.com/simonw/datasette/issues/1917#issuecomment-1331644751 https://api.github.com/repos/simonw/datasette/issues/1917 1331644751 IC_kwDOBm6k_c5PX0lP 9599 2022-11-30T04:59:22Z 2022-11-30T04:59:22Z OWNER Yeah it looks like I introduced this bug here: https://github.com/simonw/datasette/commit/fb7e70d5e72a951efe4b29ad999d8915c032d021 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469043836  
https://github.com/simonw/datasette/issues/1917#issuecomment-1331644078 https://api.github.com/repos/simonw/datasette/issues/1917 1331644078 IC_kwDOBm6k_c5PX0au 9599 2022-11-30T04:58:06Z 2022-11-30T04:58:06Z OWNER The problem might actually be here: https://github.com/simonw/datasette/blob/9f5321ff1eca58c469a45cc406d7eb5ad05accbd/datasette/app.py#L280-L281 `is_mutable` defaults to `True`, so this line should probably be: ```python self.add_database(Database(self, is_mutable=False, is_memory=True), name="_memory") ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1469043836  
https://github.com/simonw/datasette/issues/1915#issuecomment-1331479606 https://api.github.com/repos/simonw/datasette/issues/1915 1331479606 IC_kwDOBm6k_c5PXMQ2 9599 2022-11-30T00:09:06Z 2022-11-30T00:09:06Z OWNER One last feature: I want to show an indication on the table page that the table has X seconds left to live. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468709531  
https://github.com/simonw/datasette/issues/1915#issuecomment-1331479328 https://api.github.com/repos/simonw/datasette/issues/1915 1331479328 IC_kwDOBm6k_c5PXMMg 9599 2022-11-30T00:08:41Z 2022-11-30T00:08:41Z OWNER Five minute has now passed and https://latest.datasette.io/ephemeral/new_table is gone. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468709531  
https://github.com/simonw/datasette/issues/1915#issuecomment-1331476246 https://api.github.com/repos/simonw/datasette/issues/1915 1331476246 IC_kwDOBm6k_c5PXLcW 9599 2022-11-30T00:04:35Z 2022-11-30T00:08:24Z OWNER The new https://github.com/simonw/datasette-ephemeral-tables plugin is live now: https://latest.datasette.io/ephemeral - you have to navigate through https://latest.datasette.io/login-as-root first It work! I created a table using https://latest.datasette.io/-/api#path=%2Fephemeral%2F-%2Fcreate&json=%7B%0A++%22table%22%3A+%22new_table%22%2C%0A++%22columns%22%3A+%5B%0A++++%7B%0A++++++%22name%22%3A+%22id%22%2C%0A++++++%22type%22%3A+%22integer%22%0A++++%7D%2C%0A++++%7B%0A++++++%22name%22%3A+%22name%22%2C%0A++++++%22type%22%3A+%22text%22%0A++++%7D%0A++%5D%2C%0A++%22pk%22%3A+%22id%22%0A%7D&method=POST The table should vanish in a few minutes. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468709531  
https://github.com/simonw/datasette/issues/1915#issuecomment-1331478611 https://api.github.com/repos/simonw/datasette/issues/1915 1331478611 IC_kwDOBm6k_c5PXMBT 9599 2022-11-30T00:07:37Z 2022-11-30T00:07:37Z OWNER Then I created an API token at https://latest.datasette.io/-/create-token and ran this: ``` curl -XPOST 'https://latest.datasette.io/ephemeral/new_table/-/insert' \ -H 'Authorization: Bearer xxx' \ -H 'Content-Type: application/json' \ -d '{"row": {"name": "NAME"}}' ``` And it inserted a row into https://latest.datasette.io/ephemeral/new_table 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468709531  
https://github.com/simonw/datasette/issues/1915#issuecomment-1331432223 https://api.github.com/repos/simonw/datasette/issues/1915 1331432223 IC_kwDOBm6k_c5PXAsf 9599 2022-11-29T23:06:17Z 2022-11-29T23:06:17Z OWNER To (slightly) discourage abuse I'm going to make the demo database only visible to the root user - so people can't create tables with rude names and have them show to the public on https://latest.datasette.io/ 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468709531  
https://github.com/simonw/datasette/issues/1915#issuecomment-1331331082 https://api.github.com/repos/simonw/datasette/issues/1915 1331331082 IC_kwDOBm6k_c5PWoAK 9599 2022-11-29T21:24:59Z 2022-11-29T21:34:53Z OWNER Maybe a plugin called `datasette-temporary-tables` or `datasette-demo-tables` or `datasette-demo-database`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468709531  
https://github.com/simonw/datasette/issues/1850#issuecomment-1331238841 https://api.github.com/repos/simonw/datasette/issues/1850 1331238841 IC_kwDOBm6k_c5PWRe5 9599 2022-11-29T20:11:20Z 2022-11-29T20:11:20Z OWNER Released this in Datasette 1.0a0: - #1913 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1421529723  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331238029 https://api.github.com/repos/simonw/datasette/issues/1913 1331238029 IC_kwDOBm6k_c5PWRSN 9599 2022-11-29T20:10:35Z 2022-11-29T20:10:35Z OWNER Released: - https://pypi.org/project/datasette/1.0a0/ - https://docs.datasette.io/en/latest/changelog.html#a0-2022-11-29 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331226346 https://api.github.com/repos/simonw/datasette/issues/1913 1331226346 IC_kwDOBm6k_c5PWObq 9599 2022-11-29T20:00:16Z 2022-11-29T20:00:36Z OWNER Looks like a fix is coming: https://github.com/pypa/twine/issues/940#issuecomment-1331225509 > > Note that `must_decode` was defined in `pkg_info/_compat.py`, and was thus never an API: before 1.9.0, it was only imported and used in `pkginfo/distribution.py'. > > Nevertheless, I will push out a 1.9.1 release of `pkginfo` which restores a deprecated compatibility alias. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331225277 https://api.github.com/repos/simonw/datasette/issues/1913 1331225277 IC_kwDOBm6k_c5PWOK9 9599 2022-11-29T19:59:14Z 2022-11-29T19:59:34Z OWNER I deleted the tag and tried creating a new release. Now running here: https://github.com/simonw/datasette/actions/runs/3577554546 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331216652 https://api.github.com/repos/simonw/datasette/issues/1913 1331216652 IC_kwDOBm6k_c5PWMEM 9599 2022-11-29T19:54:22Z 2022-11-29T19:54:22Z OWNER Filed a bug report here: https://bugs.launchpad.net/pkginfo/+bug/1998249 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331208206 https://api.github.com/repos/simonw/datasette/issues/1913 1331208206 IC_kwDOBm6k_c5PWKAO 9599 2022-11-29T19:51:31Z 2022-11-29T19:51:31Z OWNER https://pypi.org/project/pkginfo/#history - 1.9.0 came out 39 minutes ago! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331207334 https://api.github.com/repos/simonw/datasette/issues/1913 1331207334 IC_kwDOBm6k_c5PWJym 9599 2022-11-29T19:50:37Z 2022-11-29T19:50:37Z OWNER https://pypi.org/project/setuptools/65.6.3/ came out most recently - 23rd November (wheel and twine are older). No search results at all for that error message. This is very weird, I would have expected it to have been reported by now. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331205613 https://api.github.com/repos/simonw/datasette/issues/1913 1331205613 IC_kwDOBm6k_c5PWJXt 9599 2022-11-29T19:48:52Z 2022-11-29T19:48:52Z OWNER https://github.com/simonw/datasette/blob/07aad511769da9242260c850e8d975cbd8c29552/.github/workflows/publish.yml#L52-L61 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331204360 https://api.github.com/repos/simonw/datasette/issues/1913 1331204360 IC_kwDOBm6k_c5PWJEI 9599 2022-11-29T19:47:40Z 2022-11-29T19:47:40Z OWNER ... but the last step of the deploy failed, when it was meant to push to PyPI! ``` Uploading distributions to https://upload.pypi.org/legacy/ Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.11.0/x64/bin/twine", line 8, in <module> sys.exit(main()) ^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/__main__.py", line 33, in main error = cli.dispatch(sys.argv[1:]) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/cli.py", line 123, in dispatch return main(args.args) ^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 198, in main return upload(upload_settings, parsed_args.dists) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 123, in upload packages_to_upload = [ ^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 124, in <listcomp> _make_package(filename, signatures, upload_settings) for filename in uploads ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 77, in _make_package package = package_file.PackageFile.from_filename(filename, upload_settings.comment) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/package.py", line 96, in from_filename meta = DIST_TYPES[dtype](filename) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/wheel.py", line 42, in __init__ self.extractMetadata() File "/opt/hostedtoolcache/Python/3.11.0/x64/… 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1913#issuecomment-1331203997 https://api.github.com/repos/simonw/datasette/issues/1913 1331203997 IC_kwDOBm6k_c5PWI-d 9599 2022-11-29T19:47:13Z 2022-11-29T19:47:13Z OWNER Weird, retrying the tests DID get them to pass. https://github.com/simonw/datasette/actions/runs/3577355358/jobs/6016518244 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468603401  
https://github.com/simonw/datasette/issues/1877#issuecomment-1331201207 https://api.github.com/repos/simonw/datasette/issues/1877 1331201207 IC_kwDOBm6k_c5PWIS3 9599 2022-11-29T19:44:07Z 2022-11-29T19:44:07Z OWNER I fixed the duplicate logic issue here: https://github.com/simonw/datasette/commit/ee64130fa8a5ff4a24791916c696e10cf2375102 - #1896 Decided not to address `views/table.py`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1432012302  
https://github.com/simonw/datasette/pull/1912#issuecomment-1331196531 https://api.github.com/repos/simonw/datasette/issues/1912 1331196531 IC_kwDOBm6k_c5PWHJz 9599 2022-11-29T19:39:10Z 2022-11-29T19:39:10Z OWNER Annoyingly it looks like I can't rebase this one, and I don't want to squash-merge and lose the commits, so I'm going to do a regular merge instead. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468592292  
https://github.com/simonw/datasette/issues/1605#issuecomment-1331187551 https://api.github.com/repos/simonw/datasette/issues/1605 1331187551 IC_kwDOBm6k_c5PWE9f 25778 2022-11-29T19:29:42Z 2022-11-29T19:29:42Z CONTRIBUTOR Interesting. I started a version using metadata like I outlined up top, but I realized that there's no documented way for a plugin to access either metadata or canned queries. Or at least, I couldn't find a way. There is this method: https://github.com/simonw/datasette/blob/main/datasette/app.py#L472 but I don't want to rely on it if it's not documented. Same with this: https://github.com/simonw/datasette/blob/main/datasette/app.py#L544 If those are safe, I'll build on them. I'm also happy to document them, if that greases the wheels. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1108671952  
https://github.com/simonw/datasette/issues/1891#issuecomment-1331181922 https://api.github.com/repos/simonw/datasette/issues/1891 1331181922 IC_kwDOBm6k_c5PWDli 9599 2022-11-29T19:23:41Z 2022-11-29T19:23:41Z OWNER https://github.com/simonw/datasette/blob/4d49a5a39739476e1ada43f70a0029abcef07977/docs/changelog.rst#10a0-2022-11-29 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1450303205  
https://github.com/simonw/datasette/issues/1891#issuecomment-1331143292 https://api.github.com/repos/simonw/datasette/issues/1891 1331143292 IC_kwDOBm6k_c5PV6J8 9599 2022-11-29T18:57:40Z 2022-11-29T18:57:40Z OWNER I'm going to keep these short - they'll mostly be links to the documentation for the new features. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1450303205  
https://github.com/simonw/datasette/issues/1891#issuecomment-1331140747 https://api.github.com/repos/simonw/datasette/issues/1891 1331140747 IC_kwDOBm6k_c5PV5iL 9599 2022-11-29T18:55:42Z 2022-11-29T18:55:42Z OWNER All features for the alpha are complete now. Release notes should be based on these commits: https://github.com/simonw/datasette/compare/0.63.2...6bda2257868a2cbd70b84b7a86a5bcb47dcc4874 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1450303205  
https://github.com/simonw/datasette/issues/1911#issuecomment-1331135709 https://api.github.com/repos/simonw/datasette/issues/1911 1331135709 IC_kwDOBm6k_c5PV4Td 9599 2022-11-29T18:50:58Z 2022-11-29T18:50:58Z OWNER Updated docs: https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468519699  
https://github.com/simonw/datasette/issues/1911#issuecomment-1331120055 https://api.github.com/repos/simonw/datasette/issues/1911 1331120055 IC_kwDOBm6k_c5PV0e3 9599 2022-11-29T18:36:01Z 2022-11-29T18:36:01Z OWNER Current API design: ``` POST /<database>/-/create ``` ```json { "table": "name_of_new_table", "columns": [ { "name": "id", "type": "integer" }, { "name": "title", "type": "text" } ], "pk": "id" } ``` I'm going to add a new `"pks"` key which is a list, and can be used in place of `"pk"`. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1468519699  
https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156 https://api.github.com/repos/simonw/datasette/issues/1863 1331089156 IC_kwDOBm6k_c5PVs8E 9599 2022-11-29T18:08:53Z 2022-11-29T18:08:53Z OWNER I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1425029242  
https://github.com/simonw/datasette/issues/1863#issuecomment-1330974099 https://api.github.com/repos/simonw/datasette/issues/1863 1330974099 IC_kwDOBm6k_c5PVQ2T 9599 2022-11-29T17:03:00Z 2022-11-29T17:11:05Z OWNER I've decided that I won't do that validation for the first version of this - I'm going to teach `dclient` to send the correct types instead: https://github.com/simonw/dclient/issues/6#issuecomment-1330963953 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1425029242  
https://github.com/simonw/datasette/issues/1909#issuecomment-1329624931 https://api.github.com/repos/simonw/datasette/issues/1909 1329624931 IC_kwDOBm6k_c5PQHdj 9599 2022-11-28T19:19:26Z 2022-11-28T19:19:26Z OWNER The list of states here is a good example of somewhere this might be useful: https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=state&_facet_size=max 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1466952626  
https://github.com/simonw/datasette/issues/1605#issuecomment-1328169472 https://api.github.com/repos/simonw/datasette/issues/1605 1328169472 IC_kwDOBm6k_c5PKkIA 9599 2022-11-27T04:32:14Z 2022-11-27T04:32:14Z OWNER @eyeseast I started work on that plugin: https://github.com/simonw/datasette-export 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1108671952  
https://github.com/simonw/datasette/issues/1706#issuecomment-1325164933 https://api.github.com/repos/simonw/datasette/issues/1706 1325164933 IC_kwDOBm6k_c5O_GmF 1176293 2022-11-23T14:34:54Z 2022-11-23T14:34:54Z NONE This would be helpful. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1198822563  
https://github.com/simonw/datasette/issues/1863#issuecomment-1324539030 https://api.github.com/repos/simonw/datasette/issues/1863 1324539030 IC_kwDOBm6k_c5O8tyW 9599 2022-11-23T04:35:14Z 2022-11-23T04:35:14Z OWNER If I do that I should probably update `insert` to do those validation checks as well. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1425029242  
https://github.com/simonw/datasette/issues/1863#issuecomment-1324531750 https://api.github.com/repos/simonw/datasette/issues/1863 1324531750 IC_kwDOBm6k_c5O8sAm 9599 2022-11-23T04:20:47Z 2022-11-23T04:20:47Z OWNER ... which does imply that I'm going to do an extra layer of validation over what SQLite provides. SQLite will happily allow a text string to be added to a supposedly integer column. I'm not going to allow that - I'll return a validation error instead, unless the string can be safely coerced to the correct type. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1425029242  
https://github.com/simonw/datasette/issues/1863#issuecomment-1324531085 https://api.github.com/repos/simonw/datasette/issues/1863 1324531085 IC_kwDOBm6k_c5O8r2N 9599 2022-11-23T04:19:28Z 2022-11-23T04:19:28Z OWNER Had a design conversation with myself in https://github.com/simonw/dclient/issues/6 where I decided that the API should allow string values to be sent to integer columns which would be automatically converted *if possible to do so* - as an API usability feature. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1425029242  
https://github.com/simonw/datasette/issues/1884#issuecomment-1321460293 https://api.github.com/repos/simonw/datasette/issues/1884 1321460293 IC_kwDOBm6k_c5Ow-JF 15178711 2022-11-21T04:40:55Z 2022-11-21T04:40:55Z CONTRIBUTOR Counting any virtual tables can be pretty tricky. On one hand, counting a [CSV virtual table](https://www.sqlite.org/csv.html) would return the number of rows in the CSV, which is helpful (but can be I/O intensive). Counting a [FTS5 virtual table](https://www.sqlite.org/fts5.html) would return the number of entries in the FTS index, which is kindof helpful, but can be misleading in some cases. On the other hand, arbitrarily running `COUNT(*)` on some virtual tables can be incredibly expensive. SQLite offers new shortcuts/pushdowns on `COUNT(*)` queries for virtual tables, and instead calls the underlying vtab implementation and iterates through all rows in the table without discretion. For example, a virtual table that's backed by a Postgres table would call `select * from pg_table`, which would use up a lot of network and CPU calls. Or a virtual table backed by a [google sheet](https://github.com/0x6b/libgsqlite) would make network/API requests to get all the rows from the sheet just to make a count. The [`pragma_table_list`](https://www.sqlite.org/pragma.html#pragma_table_list) pragma tells you when a table is a regular table or virtual (in the `type` column), but was only added in version 3.37.0 (2021-11-27). Personally, I wouldnt try to `COUNT(*)` virtual tables - it depends on how the virtual table is implemented, it requires that the connection has the proper extensions loaded, and it may accientally cause perf issues for new-age extensions. A few extensions that I'm writing have virtual tables that wouldn't benefit much from `COUNT(*)`, and the fact that SQLite iterates through all rows in a table to count just makes things worse. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1439009231  
https://github.com/simonw/datasette/issues/1903#issuecomment-1321262142 https://api.github.com/repos/simonw/datasette/issues/1903 1321262142 IC_kwDOBm6k_c5OwNw- 9599 2022-11-20T22:35:01Z 2022-11-20T22:35:01Z OWNER A want to call this `datasette/exceptions.py` inspired by Takahē: https://github.com/andrewgodwin/takahe/blob/f491fdb56e2de9200e14b855b5576009ca99dfa5/core/exceptions.py 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1455928469  
https://github.com/simonw/datasette/issues/1886#issuecomment-1321241426 https://api.github.com/repos/simonw/datasette/issues/1886 1321241426 IC_kwDOBm6k_c5OwItS 536941 2022-11-20T20:58:54Z 2022-11-20T20:58:54Z CONTRIBUTOR i wrote up a blog post of how i'm using it! https://bunkum.us/2022/11/20/mgdo-stack.html 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1447050738  
https://github.com/simonw/datasette/issues/1886#issuecomment-1321003094 https://api.github.com/repos/simonw/datasette/issues/1886 1321003094 IC_kwDOBm6k_c5OvOhW 9020979 2022-11-20T00:52:05Z 2022-11-20T00:52:05Z NONE Happy birthday to datasette and thank you Simon for your continued effort on this project! I use datasette (python) as a fast layer on top of search for github projects using https://github.com/dogsheep/github-to-sqlite , and use the JSON API it provides to serve sample data to make Vega-Lite graphing workshop examples that don't require authentication/API keys. It's awesome to have a full SQL API support working without needing to develop any custom API middleware for both filtering and grouping. I've also enjoyed using it as a teaching tool for working with public dataset in [civic data workshops](https://2022.open-data.nyc/event/low-code-visual-data-exploration-with-nyc-public-data/) and as a platform for making visualization [plugins](https://github.com/hydrosquall/datasette-nteract-data-explorer) . I I'm especially excited about datasette-lite, as it will let people participate in future editions of this workshop without having to install anything to make use of their own tables :) 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1447050738  
https://github.com/simonw/datasette/issues/1905#issuecomment-1320721241 https://api.github.com/repos/simonw/datasette/issues/1905 1320721241 IC_kwDOBm6k_c5OuJtZ 9599 2022-11-19T01:12:05Z 2022-11-19T01:12:05Z OWNER Used it to deploy this: https://fivethirtyeight.datasettes.com/-/versions 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1456012874  
https://github.com/simonw/datasette/issues/1905#issuecomment-1320689643 https://api.github.com/repos/simonw/datasette/issues/1905 1320689643 IC_kwDOBm6k_c5OuB_r 9599 2022-11-19T00:17:19Z 2022-11-19T00:41:54Z OWNER The tests don't cover this bit at the moment. Would be easier to write tests if there was a `--generate-dir` option as seen in https://datasette.io/plugins/datasette-publish-vercel 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1456012874  
https://github.com/simonw/datasette/issues/1905#issuecomment-1320706596 https://api.github.com/repos/simonw/datasette/issues/1905 1320706596 IC_kwDOBm6k_c5OuGIk 9599 2022-11-19T00:41:44Z 2022-11-19T00:41:44Z OWNER Tested a deploy: ``` % datasette publish heroku fixtures.db -n datasette-issue-1905 › Warning: heroku update available from 7.63.0 to 7.66.4. › Warning: heroku update available from 7.63.0 to 7.66.4. › Warning: heroku update available from 7.63.0 to 7.66.4. Creating datasette-issue-1905... done › Warning: heroku update available from 7.63.0 to 7.66.4. ▸ Couldn't detect GNU tar. Builds could fail due to decompression errors ▸ See https://devcenter.heroku.com/articles/platform-api-deploying-slugs#create-slug-archive ▸ Please install it, or specify the '--tar' option ▸ Falling back to node's built-in compressor -----> Building on the Heroku-22 stack -----> Determining which buildpack to use for this app -----> Python app detected -----> Using Python version specified in runtime.txt -----> Installing python-3.11.0 -----> Installing pip 22.3.1, setuptools 63.4.3 and wheel 0.37.1 -----> Installing SQLite3 -----> Installing requirements with pip Collecting datasette Downloading datasette-0.63.1-py3-none-any.whl (231 kB) ... -----> Running post-compile hook -----> Discovering process types Procfile declares types -> web -----> Compressing... Done: 28M -----> Launching... Released v3 https://datasette-issue-1905.herokuapp.com/ deployed to Heroku Starting November 28th, 2022, free Heroku Dynos, free Heroku Postgres, and free Heroku Data for Redis® will no longer be available. If you have apps using any of these resources, you must upgrade to paid plans by this date to ensure your apps continue to run and to retain your data. For students, we will announce a new program by the end of September. Learn more at https://blog.heroku.com/next-chapter ``` I had to then pay for the dino because I'd run out of free hours. https://datasette-issue-1905.herokuapp.com/-/versions shows: ```json { "python": { "version": "3.11.0", "full": "3.11.0 (main, Oct 24 2022, 21:34:02) [GCC 11.2.0]" … 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1456012874  
https://github.com/simonw/datasette/issues/1905#issuecomment-1320678715 https://api.github.com/repos/simonw/datasette/issues/1905 1320678715 IC_kwDOBm6k_c5Ot_U7 9599 2022-11-19T00:02:28Z 2022-11-19T00:02:28Z OWNER This is a strong argument for extracting the Heroku support out to a plugin - it would allow this to be fixed with a plugin release without needing to push a full release of Datasette itself. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1456012874  
https://github.com/simonw/datasette/issues/1891#issuecomment-1320625260 https://api.github.com/repos/simonw/datasette/issues/1891 1320625260 IC_kwDOBm6k_c5OtyRs 9599 2022-11-18T23:01:03Z 2022-11-18T23:01:48Z OWNER I think this actually needs to include a whole section of the documentation about the road to 1.0 - what to expect (planned breaking changes) etc. I can add that to the https://docs.datasette.io/en/stable/contributing.html page perhaps - or even create a Roadmap page. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1450303205  
https://github.com/simonw/datasette/issues/1896#issuecomment-1320616559 https://api.github.com/repos/simonw/datasette/issues/1896 1320616559 IC_kwDOBm6k_c5OtwJv 9599 2022-11-18T22:51:14Z 2022-11-18T22:51:14Z OWNER New methods are documented here: https://docs.datasette.io/en/1.0-dev/internals.html#resolve-database-request 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452364777  
https://github.com/simonw/datasette/issues/1903#issuecomment-1320614541 https://api.github.com/repos/simonw/datasette/issues/1903 1320614541 IC_kwDOBm6k_c5OtvqN 9599 2022-11-18T22:47:41Z 2022-11-18T22:47:41Z OWNER When I do this it's important to update the documentation for `resolve_database()` and the like: https://github.com/simonw/datasette/blob/ee64130fa8a5ff4a24791916c696e10cf2375102/docs/internals.rst#L594 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1455928469  
https://github.com/simonw/datasette/issues/1896#issuecomment-1320588299 https://api.github.com/repos/simonw/datasette/issues/1896 1320588299 IC_kwDOBm6k_c5OtpQL 9599 2022-11-18T22:16:59Z 2022-11-18T22:17:06Z OWNER Found myself needing an `await db.view_exists()` method for this, similar to the existing `await db.table_exists()` one. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452364777  
https://github.com/simonw/datasette/issues/1896#issuecomment-1317757949 https://api.github.com/repos/simonw/datasette/issues/1896 1317757949 IC_kwDOBm6k_c5Oi2P9 9599 2022-11-16T22:27:47Z 2022-11-18T21:48:29Z OWNER Open question: should `resolve_table()` know how to identify named canned queries too? I think not, at least for the moment. Feels a bit too specialist to expose in a documented API. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452364777  
https://github.com/simonw/datasette/issues/1863#issuecomment-1320563197 https://api.github.com/repos/simonw/datasette/issues/1863 1320563197 IC_kwDOBm6k_c5OtjH9 9599 2022-11-18T21:47:35Z 2022-11-18T21:48:07Z OWNER Incomplete implementation of this view: ```python class RowUpdateView(BaseView): name = "row-update" def __init__(self, datasette): self.ds = datasette async def post(self, request): database_route = tilde_decode(request.url_vars["database"]) table = tilde_decode(request.url_vars["table"]) try: db = self.ds.get_database(route=database_route) except KeyError: return _error(["Database not found: {}".format(database_route)], 404) database_name = db.name if not await db.table_exists(table): return _error(["Table not found: {}".format(table)], 404) pk_values = urlsafe_components(request.url_vars["pks"]) sql, params, pks = await row_sql_params_pks(db, table, pk_values) results = await db.execute(sql, params, truncate=True) rows = list(results.rows) if not rows: return _error([f"Record not found: {pk_values}"], 404) # Ensure user has permission to update this row if not await self.ds.permission_allowed( request.actor, "update-row", resource=(database_name, table) ): return _error(["Permission denied"], 403) body = await request.post_body() try: data = json.loads(body) except json.JSONDecodeError as e: return _error(["Invalid JSON: {}".format(e)]) if not isinstance(data, dict): return _error(["JSON must be a dictionary"]) def update_row(conn): sqlite_utils.Database(conn)[table].update(pk_values, updates) await db.execute_write_fn(update_row) result = {"ok": True} if data.get("return"): result["row"] = {"row-here": "TODO"} return Response.json(result, status=200) ``` This is before the refactor in: - #1896 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1425029242  
https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1320394127 https://api.github.com/repos/simonw/sqlite-utils/issues/510 1320394127 IC_kwDOCGYnMM5Os52P 1176293 2022-11-18T18:37:51Z 2022-11-18T18:37:51Z NONE I guess it is not incorrect when it says the version is `4`, though it is confusing. Maybe it doesn't even refer to FTS4/FTS5 versions, but something else? In any case, it's not related to sqlite-utils, but SQLite itself. 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1434911255  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319664697 https://api.github.com/repos/simonw/datasette/issues/1900 1319664697 IC_kwDOBm6k_c5OqHw5 419145 2022-11-18T07:59:36Z 2022-11-18T08:00:38Z NONE Okay, my final observations for the night! I've been pushing and pulling the various levers in `utils/__init__.py` to see what makes this work without hard-coding in something for `arm64` and it seems that if I change `/usr/lib/x86_64-linux-gnu/mod_spatialite.so` [here](https://github.com/simonw/datasette/blob/3ecd131e57add427d847b614c920c9624bb2e66b/datasette/utils/__init__.py#L407) to just `mod_spatialite` it's happy. Unfortunately cannot audit that for `x86_64`, but maybe that's a solution that'd be cross-arch compatible? It seems like it's the hard-coding of that path that's tripping it up. (It was actually [this comment from back in 2018 in an entirely unrelated repo](https://github.com/pelias/docker/pull/28#issuecomment-433168462) that nudged me to try this, ha.) 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/pull/1898#issuecomment-1319642535 https://api.github.com/repos/simonw/datasette/issues/1898 1319642535 IC_kwDOBm6k_c5OqCWn 9599 2022-11-18T07:28:45Z 2022-11-18T07:28:45Z OWNER Thanks! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452485922  
https://github.com/simonw/datasette/issues/1899#issuecomment-1319642338 https://api.github.com/repos/simonw/datasette/issues/1899 1319642338 IC_kwDOBm6k_c5OqCTi 9599 2022-11-18T07:28:28Z 2022-11-18T07:28:28Z OWNER Demo: https://latest.datasette.io/fixtures 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452495049  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319641636 https://api.github.com/repos/simonw/datasette/issues/1900 1319641636 IC_kwDOBm6k_c5OqCIk 419145 2022-11-18T07:27:26Z 2022-11-18T07:27:26Z NONE Can confirm that my `uname -a` returns something different at the end: ``` root:xnu-8792.41.9~2/RELEASE_ARM64_T6000 arm64 ``` I'm in `arm64` land, you're in `x86_64`. I am admittedly very fuzzy on how this factors into Docker these days. Honestly thought this was one of the things Docker was suppose to help address. 🤔 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319639462 https://api.github.com/repos/simonw/datasette/issues/1900 1319639462 IC_kwDOBm6k_c5OqBmm 419145 2022-11-18T07:24:19Z 2022-11-18T07:24:19Z NONE Is it, uh, possible we are on different architectures? 😅 I'm using an Apple M1 Pro. I jumped into a bash shell of an unmodified `python:3.11.0-slim-bullseye` container and manually ran `apt-get update` and installed `libsqlite3-mod-spatialite`. I don't end up with with `mod_spatialite.so` in `/usr/lib/x86_64-linux-gnu/` — _mine_ is in `/usr/lib/aarch64-linux-gnu/`. [I swapped that directory in here](https://github.com/simonw/datasette/blob/3db37e9a21f774d6c387fd04bf1e4c870554209e/datasette/utils/__init__.py#L407) in a local copy of `datasette` and poof — it worked! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319525520 https://api.github.com/repos/simonw/datasette/issues/1901 1319525520 IC_kwDOBm6k_c5OplyQ 9599 2022-11-18T04:21:14Z 2022-11-18T07:22:37Z OWNER This search helps too: [https://ripgrep.datasette.io/-/ripgrep?pattern=%7B%25+block+nav&literal=on&ignore=on&glob=%21datasette%2F**](https://ripgrep.datasette.io/-/ripgrep?pattern=%7B%25+block+nav&literal=on&ignore=on&glob=%21datasette%2F**) 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319631421 https://api.github.com/repos/simonw/datasette/issues/1900 1319631421 IC_kwDOBm6k_c5Op_o9 9599 2022-11-18T07:13:00Z 2022-11-18T07:13:00Z OWNER You get: ``` => [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 0.9s => [internal] load build context 2.3s => => transferring context: 72.38MB 2.3s => CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s ``` I get: ``` => [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 1.0s => [internal] load build context 0.0s => => transferring context: 705B 0.0s => CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s ``` Both the image name and the hash are _exactly_ the same. So why are you getting an error while mine works OK? For my machine: ``` ~ % docker --version Docker version 20.10.12, build e91ed57 ~ % uname -a Darwin Simons-MacBook-Pro-2.local 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64 x86_64 ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319629469 https://api.github.com/repos/simonw/datasette/issues/1900 1319629469 IC_kwDOBm6k_c5Op_Kd 9599 2022-11-18T07:10:17Z 2022-11-18T07:10:17Z OWNER This is so weird! What version of Datasette do you get from `datasette --version` there - and what's your Docker version / operating system version? 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319627012 https://api.github.com/repos/simonw/datasette/issues/1901 1319627012 IC_kwDOBm6k_c5Op-kE 9599 2022-11-18T07:07:03Z 2022-11-18T07:07:03Z OWNER Here's the full list of 10 plugin releases for this issue: * [datasette-search-all 1.1.1](https://github.com/simonw/datasette-search-all/releases/tag/1.1.1) * [datasette-ripgrep 0.7.1](https://github.com/simonw/datasette-ripgrep/releases/tag/0.7.1) * [datasette-socrata 0.3.1](https://github.com/simonw/datasette-socrata/releases/tag/0.3.1) * [datasette-configure-fts 1.1.1](https://github.com/simonw/datasette-configure-fts/releases/tag/1.1.1) * [datasette-edit-templates 0.2](https://github.com/simonw/datasette-edit-templates/releases/tag/0.2) * [datasette-copyable 0.3.2](https://github.com/simonw/datasette-copyable/releases/tag/0.3.2) * [datasette-public 0.2.1](https://github.com/simonw/datasette-public/releases/tag/0.2.1) * [datasette-import-table 0.3.1](https://github.com/simonw/datasette-import-table/releases/tag/0.3.1) * [datasette-indieauth 1.2.2](https://github.com/simonw/datasette-indieauth/releases/tag/1.2.2) * [datasette-edit-schema 0.5.2](https://github.com/simonw/datasette-edit-schema/releases/tag/0.5.2) 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319493475 https://api.github.com/repos/simonw/datasette/issues/1901 1319493475 IC_kwDOBm6k_c5Opd9j 9599 2022-11-18T03:19:25Z 2022-11-18T07:03:03Z OWNER Other plugins this looks like it will affect: - [x] `datasette-ripgrep` https://github.com/simonw/datasette-ripgrep/blob/03446464420130368582022eeb5944993f64ec8f/datasette_ripgrep/templates/ripgrep.html#L37-L42 - [x] `datasette-socrata` https://github.com/simonw/datasette-socrata/blob/32fb256a461bf0e790eca10bdc7dd9d96c20f7c4/datasette_socrata/templates/datasette_socrata_error.html#L5-L10 - [x] `datasette-configure-fts` https://github.com/simonw/datasette-configure-fts/blob/eca742e5d4b9190fc22d68bc0a406c575e6d09a0/datasette_configure_fts/templates/configure_fts_database.html#L9-L14 - [x] `datasette-edit-templates` https://github.com/simonw/datasette-edit-templates/blob/f772aff4a2a4080c949746668a8ec6302dbeb0d9/datasette_edit_templates/templates/edit_template.html#L17-L23 - [x] `datasette-copyable` https://github.com/simonw/datasette-copyable/blob/204d5c912a8d48c49155c67fba7339d4bb26ab9a/datasette_copyable/templates/copyable.html#L36-L43 - [x] `datasette-public` https://github.com/simonw/datasette-public/blob/32b6a0ba53bd5714b6b41eddd8705b213c105efc/datasette_public/templates/public_table_change_privacy.html#L5-L11 - [x] `datasette-import-table` https://github.com/simonw/datasettecloud-datasette/blob/37d0fe525c6649c1aec3d1ee8bc35a684570e87f/templates/import_data.html#L5-L10 - [x] `datasette-edit-schema` (three places) - [x] `datasette-indieauth` https://github.com/simonw/datasette-indieauth/blob/a08ce67ddad6098b1240adbeff37d040e4df53b1/datasette_indieauth/templates/indieauth.html#L5-L10 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319623911 https://api.github.com/repos/simonw/datasette/issues/1901 1319623911 IC_kwDOBm6k_c5Op9zn 9599 2022-11-18T07:02:56Z 2022-11-18T07:02:56Z OWNER That's all of them! 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319596087 https://api.github.com/repos/simonw/datasette/issues/1900 1319596087 IC_kwDOBm6k_c5Op3A3 419145 2022-11-18T06:16:33Z 2022-11-18T06:16:33Z NONE Interesting! So I tried this locally using your copy of `nps-spatialite.db` and I got the same error. 🤔 ``` ❯ datasette package nps-spatialite.db --spatialite [+] Building 27.5s (10/10) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 622B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 0.9s => [internal] load build context 2.3s => => transferring context: 72.38MB 2.3s => CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s => [2/6] COPY . /app 0.1s =>… 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319588163 https://api.github.com/repos/simonw/datasette/issues/1901 1319588163 IC_kwDOBm6k_c5Op1FD 9599 2022-11-18T06:05:11Z 2022-11-18T06:05:11Z OWNER For `datasette-copyable` I want to show breadcrumbs that take database/instance permissions into account, so I'm removing `{% block nav %}` entirely and replacing it with this: ```html+jinja {% block crumbs %} {{ crumbs.nav(request=request, database=database, table=table) }} {% endblock %} ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1899#issuecomment-1319584553 https://api.github.com/repos/simonw/datasette/issues/1899 1319584553 IC_kwDOBm6k_c5Op0Mp 9599 2022-11-18T06:00:10Z 2022-11-18T06:01:50Z OWNER I can't actually remember where that `min-height: 70px` came from. I just tried without it and it seems fine - especially since any time you add a newline in the editor it increases its height to fit. I ran this in the DevTools console: ```javascript document.querySelector('.cm-editor').style.minHeight = 'none'; ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452495049  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319583703 https://api.github.com/repos/simonw/datasette/issues/1900 1319583703 IC_kwDOBm6k_c5Opz_X 9599 2022-11-18T05:58:31Z 2022-11-18T05:58:31Z OWNER Could you provide full steps to reproduce plus a SpatiaLite database file that triggered this for you? I'm not able to recreate the problem. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319583281 https://api.github.com/repos/simonw/datasette/issues/1900 1319583281 IC_kwDOBm6k_c5Opz4x 9599 2022-11-18T05:57:44Z 2022-11-18T05:57:44Z OWNER Did you use the `--spatialite` option? I just tried this: datasette package nps-spatialite.db It built the image OK (I didn't see the error you reported), but running the container failed with an error: ``` /tmp % docker run -p 8001:8001 7298e8e6bbfb Usage: datasette serve [OPTIONS] [FILES]... Try 'datasette serve --help' for help. Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module. Read more: https://docs.datasette.io/en/stable/spatialite.html ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319582239 https://api.github.com/repos/simonw/datasette/issues/1900 1319582239 IC_kwDOBm6k_c5Opzof 9599 2022-11-18T05:55:38Z 2022-11-18T05:55:38Z OWNER Trying this out locally with this 69MB SpatiaLite file I happened to have lying around (from testing `shapefile-to-sqlite` a while ago): https://static.simonwillison.net/static/2022/nps-spatialite.db ``` % datasette package nps-spatialite.db --spatialite ... => [2/6] COPY . /app 0.4s => [3/6] WORKDIR /app 0.0s => [4/6] RUN apt-get update && apt-get install -y python3-dev gcc libsqlite3-mod-spatialite && rm -rf /var/lib/apt/lists/* 29.6s => [5/6] RUN pip install -U datasette 12.0s => [6/6] RUN datasette inspect nps-spatialite.db --inspect-file inspect-data.json 2.6s => exporting to image 3.0s => => exporting layers 3.0s => => writing image sha256:4dfef1c373c5c057ef7ac22344f834d522acef24313a1b25d2eba9e500066b8f 0.0s ``` And then: docker run -p 8001:8001 4dfef1c373c5 This worked fine for me. I ran `datasette package` using Datasette 0.63.1. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319574972 https://api.github.com/repos/simonw/datasette/issues/1900 1319574972 IC_kwDOBm6k_c5Opx28 9599 2022-11-18T05:41:28Z 2022-11-18T05:41:28Z OWNER Oh this is with `datasette package`? That should work. Will investigate. 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1900#issuecomment-1319571220 https://api.github.com/repos/simonw/datasette/issues/1900 1319571220 IC_kwDOBm6k_c5Opw8U 9599 2022-11-18T05:34:35Z 2022-11-18T05:34:35Z OWNER Which Docker image are you using here? It looks like it's missing SpatiaLite from the image. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452572348  
https://github.com/simonw/datasette/issues/1897#issuecomment-1319570586 https://api.github.com/repos/simonw/datasette/issues/1897 1319570586 IC_kwDOBm6k_c5Opwya 9599 2022-11-18T05:33:20Z 2022-11-18T05:33:20Z OWNER One of the big changes still left to do for Datasette 1.0 is to unify the JSON representation with the context psssed to the templates (via an `?_extra=` mechanism to add extra context needed by the HTML templates), because a goal for 1.0 is for the template context to be a documented API contract such that custom templates won't break with future releases. As such I expect to do quite a bit of refactoring and cleanup on how the template context works later on. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452457263  
https://github.com/simonw/datasette/issues/1897#issuecomment-1319533445 https://api.github.com/repos/simonw/datasette/issues/1897 1319533445 IC_kwDOBm6k_c5OpnuF 95570 2022-11-18T04:38:03Z 2022-11-18T04:38:03Z CONTRIBUTOR Are you tracking the change to send the JSON over to the frontend separately or was that part of this? Something like this is probably pretty close https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25#diff-0c93232bfd5477eeac96382e52769108b41433d960d5277ffcccf2f464e60abdR9 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452457263  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319528359 https://api.github.com/repos/simonw/datasette/issues/1901 1319528359 IC_kwDOBm6k_c5Opmen 9599 2022-11-18T04:27:00Z 2022-11-18T04:27:00Z OWNER Also `datasette-indieauth` https://github.com/simonw/datasette-indieauth/blob/a08ce67ddad6098b1240adbeff37d040e4df53b1/datasette_indieauth/templates/indieauth.html#L5-L10 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319483555 https://api.github.com/repos/simonw/datasette/issues/1901 1319483555 IC_kwDOBm6k_c5Opbij 9599 2022-11-18T03:02:35Z 2022-11-18T03:02:35Z OWNER Looks like this issue could affect a bunch of other plugins too: https://cs.github.com/?scopeName=All+repos&scope=&q=%3Cp+class%3D%22crumbs%22%3E+user%3Asimonw 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1901#issuecomment-1319482791 https://api.github.com/repos/simonw/datasette/issues/1901 1319482791 IC_kwDOBm6k_c5OpbWn 9599 2022-11-18T03:01:36Z 2022-11-18T03:01:36Z OWNER Good catch. Looks like that bug was introduced by this change: https://github.com/simonw/datasette/commit/1a5e5f2aa951e5bd731067a49819efba68fbe8ef From: - https://github.com/simonw/datasette/issues/1831 The search all plugin includes this code which interacts poorly with that refactor: https://github.com/simonw/datasette-search-all/blob/847b55c368a285e4567627029624d7872ee75cac/datasette_search_all/templates/search_all.html#L31-L36 ```html+jinja {% block nav %} <p class="crumbs"> <a href="{{ urls.instance() }}">home</a> </p> {{ super() }} {% endblock %} ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1453813400  
https://github.com/simonw/datasette/issues/1897#issuecomment-1319478811 https://api.github.com/repos/simonw/datasette/issues/1897 1319478811 IC_kwDOBm6k_c5OpaYb 9599 2022-11-18T02:53:57Z 2022-11-18T02:53:57Z OWNER I decided to just go for the view names, not their columns. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452457263  
https://github.com/simonw/datasette/issues/1897#issuecomment-1319477721 https://api.github.com/repos/simonw/datasette/issues/1897 1319477721 IC_kwDOBm6k_c5OpaHZ 9599 2022-11-18T02:51:40Z 2022-11-18T02:51:40Z OWNER Views aren't currently available in the `_internal` schema. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452457263  
https://github.com/simonw/datasette/issues/1897#issuecomment-1319435374 https://api.github.com/repos/simonw/datasette/issues/1897 1319435374 IC_kwDOBm6k_c5OpPxu 9599 2022-11-18T01:33:30Z 2022-11-18T01:33:30Z OWNER Just noticed that this isn't including views, which it should. 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452457263  
https://github.com/simonw/datasette/issues/1897#issuecomment-1319401843 https://api.github.com/repos/simonw/datasette/issues/1897 1319401843 IC_kwDOBm6k_c5OpHlz 9599 2022-11-18T00:42:03Z 2022-11-18T00:42:23Z OWNER This function works even if the SQLite JSON functions are not available: ```python async def _table_columns(datasette, database_name): internal = datasette.get_database("_internal") result = await internal.execute( "select table_name, name from columns where database_name = ?", [database_name], ) table_columns = {} for row in result.rows: table_columns.setdefault(row["table_name"], []).append(row["name"]) return table_columns ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452457263  
https://github.com/simonw/datasette/issues/1899#issuecomment-1318897922 https://api.github.com/repos/simonw/datasette/issues/1899 1318897922 IC_kwDOBm6k_c5OnMkC 95570 2022-11-17T16:32:42Z 2022-11-17T16:32:42Z CONTRIBUTOR Another idea would be to just not set a min-height and allow the 1 line input to be 1 line heigh 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1452495049  
https://github.com/simonw/sqlite-utils/pull/508#issuecomment-1297788531 https://api.github.com/repos/simonw/sqlite-utils/issues/508 1297788531 IC_kwDOCGYnMM5NWq5z 7908073 2022-10-31T22:54:33Z 2022-11-17T15:11:16Z CONTRIBUTOR Maybe this is actually a problem in the python sqlite bindings. Given [SQLITE's stance on this](https://www.sqlite.org/invalidutf.html) they should probably use `encode('utf-8', 'surrogatepass')`. As far as I understand the error here won't actually be resolved by this PR as-is. We would need to modify the data with `surrogateescape`... :/ or modify the sqlite3 module to use `surrogatepass` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1430563092  
https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1318777114 https://api.github.com/repos/simonw/sqlite-utils/issues/510 1318777114 IC_kwDOCGYnMM5OmvEa 7908073 2022-11-17T15:09:47Z 2022-11-17T15:09:47Z CONTRIBUTOR why close? is the only problem that the _config table that incorrectly says 4 for fts5? if so, that's still something that should be fixed 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1434911255