issue_comments

Table actions

9,106 rows sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

issue >1000

user 329

author_association 4

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1332903011 https://github.com/simonw/datasette/issues/1922#issuecomment-1332903011 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5Pcnxj simonw 9599 2022-11-30T23:45:29Z 2022-11-30T23:45:29Z OWNER

That worked for the preflight request - got this now:

So it looks like error responses (in this case for permission denied) are missing CORS headers.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332855687 https://github.com/simonw/datasette/issues/1922#issuecomment-1332855687 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PccOH simonw 9599 2022-11-30T23:09:31Z 2022-11-30T23:09:31Z OWNER

Still getting a CORS header.

I tried it in Chrome, which outputs helpful messages to the console:

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332851215 https://github.com/simonw/datasette/issues/1923#issuecomment-1332851215 https://api.github.com/repos/simonw/datasette/issues/1923 IC_kwDOBm6k_c5PcbIP simonw 9599 2022-11-30T23:04:56Z 2022-11-30T23:04:56Z OWNER

That fixed it.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 latest.datasette.io Cloud Run deploys failing 1470320227  
1332842435 https://github.com/simonw/datasette/issues/1923#issuecomment-1332842435 https://api.github.com/repos/simonw/datasette/issues/1923 IC_kwDOBm6k_c5PcY_D simonw 9599 2022-11-30T22:58:33Z 2022-11-30T22:58:33Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 latest.datasette.io Cloud Run deploys failing 1470320227  
1332835664 https://github.com/simonw/datasette/issues/1923#issuecomment-1332835664 https://api.github.com/repos/simonw/datasette/issues/1923 IC_kwDOBm6k_c5PcXVQ simonw 9599 2022-11-30T22:50:10Z 2022-11-30T22:51:25Z OWNER

https://stackoverflow.com/questions/74490465/github-actions-failing-for-setup-gcloud/74562526#74562526 suggests setting version: '318.0.0' of google-github-actions/setup-gcloud.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 latest.datasette.io Cloud Run deploys failing 1470320227  
1332698636 https://github.com/simonw/datasette/issues/1922#issuecomment-1332698636 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5Pb14M simonw 9599 2022-11-30T20:25:50Z 2022-11-30T20:25:50Z OWNER

I just shipped this:

Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS

I'll try this out on latest.datasette.io - but I need to research more to check if this is a safe thing to do or not.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332689547 https://github.com/simonw/datasette/issues/1922#issuecomment-1332689547 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbzqL simonw 9599 2022-11-30T20:16:21Z 2022-11-30T20:16:46Z OWNER

That notebook: javascript viewof token = Inputs.text({ label: "Your API token" }) javascript viewof createResponse = Inputs.button("Create table", { value: null, reduce: async () => { const response = await fetch( "https://latest.datasette.io/ephemeral/-/create", { method: "POST", mode: "cors", headers: { Authorization: `Bearer {$token}`, "Content-Type": "application/json" }, body: JSON.stringify({ table: "my_new_table", row: { task: "Demonstrate a JSON creation from another domain" } }) } ); return await response.json(); } }) Based on this tip: https://talk.observablehq.com/t/best-pattern-for-click-here-to-submit-your-results-to-an-api-backend/7353/3

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332688245 https://github.com/simonw/datasette/issues/1922#issuecomment-1332688245 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbzV1 simonw 9599 2022-11-30T20:15:08Z 2022-11-30T20:15:08Z OWNER

Still getting a CORS error:

My hunch is this is because I'm not sending Access-Control-Allow-Methods: GET,HEAD,POST.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332585861 https://github.com/simonw/datasette/issues/1922#issuecomment-1332585861 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbaWF simonw 9599 2022-11-30T18:43:46Z 2022-11-30T18:43:46Z OWNER

Here's what Django Rest Framework does: https://github.com/encode/django-rest-framework/blob/1ae812ea209392ad76cc5d2f35f9f7fb337f63e4/rest_framework/views.py#L514-L521

python def options(self, request, *args, **kwargs): """ Handler method for HTTP 'OPTIONS' request. """ if self.metadata_class is None: return self.http_method_not_allowed(request, *args, **kwargs) data = self.metadata_class().determine_metadata(request, self) return Response(data, status=status.HTTP_200_OK) That default determine_metadata method looks like this: https://github.com/encode/django-rest-framework/blob/1ae812ea209392ad76cc5d2f35f9f7fb337f63e4/rest_framework/metadata.py#L61-L71

python def determine_metadata(self, request, view): metadata = OrderedDict() metadata['name'] = view.get_view_name() metadata['description'] = view.get_view_description() metadata['renders'] = [renderer.media_type for renderer in view.renderer_classes] metadata['parses'] = [parser.media_type for parser in view.parser_classes] if hasattr(view, 'get_serializer'): actions = self.determine_actions(request, view) if actions: metadata['actions'] = actions return metadata

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332580395 https://github.com/simonw/datasette/issues/1922#issuecomment-1332580395 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbZAr simonw 9599 2022-11-30T18:38:22Z 2022-11-30T18:38:22Z OWNER

@simon IMO, it should always be a 2XX series response, typically with no content & an extra Allow header with a list of HTTP verbs it responds to.

https://mastodon.social/@daniellindsley/109434186252099323

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332572453 https://github.com/simonw/datasette/issues/1922#issuecomment-1332572453 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbXEl simonw 9599 2022-11-30T18:30:38Z 2022-11-30T18:30:54Z OWNER

Started a conversation about how OPTIONS should work on Mastodon: https://fedi.simonwillison.net/@simon/109434148676475291

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332561813 https://github.com/simonw/datasette/issues/1922#issuecomment-1332561813 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbUeV simonw 9599 2022-11-30T18:20:05Z 2022-11-30T18:20:05Z OWNER

Weird, GitHub reply with a 404! ~ % curl -X OPTIONS https://github.com/ -i HTTP/2 404 server: GitHub.com date: Wed, 30 Nov 2022 18:19:39 GMT content-type: text/html; charset=utf-8 content-length: 0 strict-transport-security: max-age=31536000; includeSubdomains; preload x-frame-options: deny x-content-type-options: nosniff x-xss-protection: 0 referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin content-security-policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com objects-origin.githubusercontent.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events; font-src github.githubassets.com; form-action 'self' github.com gist.github.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com objects-origin.githubusercontent.com secured-user-images.githubusercontent.com/ opengraph.githubassets.com github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/ vary: Accept-Encoding, Accept, X-Requested-With x-github-request-id: DD6B:5ACA:102E8A6:1164A99:63879EBB

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332561059 https://github.com/simonw/datasette/issues/1922#issuecomment-1332561059 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbUSj simonw 9599 2022-11-30T18:19:20Z 2022-11-30T18:19:20Z OWNER

Two test failures: ``` ____ test_homepage_options _____ [gw0] linux -- Python 3.11.0 /opt/hostedtoolcache/Python/3.11.0/x64/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7f4c489269d0>

def test_homepage_options(app_client):
    response = app_client.get("/", method="OPTIONS")

  assert response.status == 405

E assert 200 == 405 E + where 200 = <datasette.utils.testing.TestResponse object at 0x7f4c4892f4d0>.status

/home/runner/work/datasette/datasette/tests/test_html.py:58: AssertionError ___ test_client_methods[options-/-405] ___ [gw1] linux -- Python 3.11.0 /opt/hostedtoolcache/Python/3.11.0/x64/bin/python

datasette = <datasette.app.Datasette object at 0x7fc33c227550> method = 'options', path = '/', expected_status = 405

@pytest.mark.asyncio
@pytest.mark.parametrize(
    "method,path,expected_status",
    [
        ("get", "/", 200),
        ("options", "/", 405),
        ("head", "/", 200),
        ("put", "/", 405),
        ("patch", "/", 405),
        ("delete", "/", 405),
    ],
)
async def test_client_methods(datasette, method, path, expected_status):
    client_method = getattr(datasette.client, method)
    response = await client_method(path)
    assert isinstance(response, httpx.Response)

  assert response.status_code == expected_status

E assert 200 == 405 E + where 200 = <Response [200 OK]>.status_code

/home/runner/work/datasette/datasette/tests/test_internals_datasette_client.py:29: AssertionError =============================== warnings summary =============================== tests/test_cli.py::test_inspect_cli_writes_to_file tests/test_cli.py::test_inspect_cli /home/runner/work/datasette/datasette/datasette/cli.py:163: DeprecationWarning: There is no current event loop loop = asyncio.get_event_loop()

tests/test_cli_serve_get.py: 2 warnings tests/test_cli.py: 12 warnings tests/test_crossdb.py: 1 warning /home/runner/work/datasette/datasette/datasette/cli.py:591: DeprecationWarning: There is no current event loop asyncio.get_event_loop().run_until_complete(ds.invoke_startup())

tests/test_cli_serve_get.py: 2 warnings tests/test_cli.py: 12 warnings tests/test_crossdb.py: 1 warning /home/runner/work/datasette/datasette/datasette/cli.py:594: DeprecationWarning: There is no current event loop asyncio.get_event_loop().run_until_complete(check_databases(ds))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ FAILED tests/test_html.py::test_homepage_options - assert 200 == 405 + where 200 = <datasette.utils.testing.TestResponse object at 0x7f4c4892f4d0>.status FAILED tests/test_internals_datasette_client.py::test_client_methods[options-/-405] - assert 200 == 405 + where 200 = <Response [200 OK]>.status_code ====== 2 failed, 1195 passed, 1 skipped, 32 warnings in 191.06s (0:03:11) ====== Error: Process completed with exit code 1. `` On reading https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS I feel like I should be a bit more thoughtful about how I treat OPTIONS - maybe it should work for every URL on the site, but return a204 No Content` header?

Comparing a few different sites:

``` ~ % curl -X OPTIONS https://www.google.com/ -i HTTP/2 405 allow: GET, HEAD date: Wed, 30 Nov 2022 18:18:15 GMT content-type: text/html; charset=UTF-8 server: gws content-length: 1592 x-xss-protection: 0 x-frame-options: SAMEORIGIN alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

<html lang=en> <meta charset=utf-8> <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width"> <title>Error 405 (Method Not Allowed)!!1</title>
<style> *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px} </style>


405. <ins>That’s an error.</ins>

The request method OPTIONS is inappropriate for the URL /. <ins>That’s all we know.</ins> ~ % curl -X OPTIONS https://www.mozilla.org/ -i HTTP/2 405 content-type: text/html; charset=utf-8 content-length: 0 server: meinheld/1.0.2 date: Wed, 30 Nov 2022 18:18:38 GMT allow: GET, HEAD x-frame-options: DENY content-security-policy: child-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com www.googletagmanager.com www.google-analytics.com www.youtube-nocookie.com trackertest.org www.surveygizmo.com accounts.firefox.com accounts.firefox.com.cn www.youtube.com; connect-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com www.googletagmanager.com www.google-analytics.com region1.google-analytics.com logs.convertexperiments.com 1003350.metrics.convertexperiments.com 1003343.metrics.convertexperiments.com sentry.prod.mozaws.net o1069899.sentry.io o1069899.ingest.sentry.io https://accounts.firefox.com/ stage.cjms.nonprod.cloudops.mozgcp.net cjms.services.mozilla.com; frame-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com www.googletagmanager.com www.google-analytics.com www.youtube-nocookie.com trackertest.org www.surveygizmo.com accounts.firefox.com accounts.firefox.com.cn www.youtube.com; script-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com 'unsafe-inline' 'unsafe-eval' www.googletagmanager.com www.google-analytics.com tagmanager.google.com www.youtube.com s.ytimg.com cdn-3.convertexperiments.com app.convert.com data.track.convertexperiments.com 1003350.track.convertexperiments.com 1003343.track.convertexperiments.com; img-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com data: mozilla.org www.googletagmanager.com www.google-analytics.com adservice.google.com adservice.google.de adservice.google.dk creativecommons.org cdn-3.convertexperiments.com logs.convertexperiments.com images.ctfassets.net ad.doubleclick.net; style-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com 'unsafe-inline' app.convert.com; default-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com; font-src 'self' cache-control: max-age=600 expires: Wed, 30 Nov 2022 18:28:38 GMT x-backend-server: bedrock-prod-web-b95bc569d-grd25.iowa-a strict-transport-security: max-age=31536000 x-content-type-options: nosniff x-xss-protection: 1; mode=block referrer-policy: strict-origin-when-cross-origin via: 1.1 google, 1.1 6c90b631453c435bd0022caa657b67e8.cloudfront.net (CloudFront) x-cache: Error from cloudfront x-amz-cf-pop: SFO5-P2 x-amz-cf-id: A6-9mLztaE2tz840CbV9bXYiBMZRKEamDj6jGGEl1U7sg8egWfsDqg== ~ % curl -X OPTIONS https://example.com -i HTTP/2 200 allow: OPTIONS, GET, HEAD, POST cache-control: max-age=604800 content-type: text/html; charset=UTF-8 date: Wed, 30 Nov 2022 18:18:59 GMT expires: Wed, 07 Dec 2022 18:18:59 GMT server: EOS (vny/0451) content-length: 0 ``` 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332504654 https://github.com/simonw/datasette/issues/1922#issuecomment-1332504654 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbGhO simonw 9599 2022-11-30T17:27:39Z 2022-11-30T17:27:39Z OWNER

I'll test this once it's deployed to https://latest.datasette.io/

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332493004 https://github.com/simonw/datasette/issues/1922#issuecomment-1332493004 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbDrM simonw 9599 2022-11-30T17:18:10Z 2022-11-30T17:18:10Z OWNER

Here's why:

https://github.com/simonw/datasette/blob/4ddd77e51254bda3bac990ea662bac2e6b29c5e0/datasette/views/base.py#L71-L79

That's code in BaseView - but it turns out the code that adds CORS headers is in the DataView subclass of that (which the various write API endpoints do not use).

https://github.com/simonw/datasette/blob/4ddd77e51254bda3bac990ea662bac2e6b29c5e0/datasette/views/base.py#L158-L162

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332492092 https://github.com/simonw/datasette/issues/1922#issuecomment-1332492092 https://api.github.com/repos/simonw/datasette/issues/1922 IC_kwDOBm6k_c5PbDc8 simonw 9599 2022-11-30T17:17:21Z 2022-11-30T17:17:21Z OWNER

I tried running fetch() with a POST from a separate domain and got a browser error because it did a GET against the /db/-/create endpoint and the 405 method not supported response did not include the CORS headers.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make sure CORS works for write APIs 1469973742  
1332310772 https://github.com/simonw/datasette/issues/1605#issuecomment-1332310772 https://api.github.com/repos/simonw/datasette/issues/1605 IC_kwDOBm6k_c5PaXL0 eyeseast 25778 2022-11-30T15:06:37Z 2022-11-30T15:06:37Z CONTRIBUTOR

I'll add issues for both and do a documentation PR.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Scripted exports 1108671952  
1331694246 https://github.com/simonw/datasette/issues/1605#issuecomment-1331694246 https://api.github.com/repos/simonw/datasette/issues/1605 IC_kwDOBm6k_c5PYAqm simonw 9599 2022-11-30T06:18:41Z 2022-11-30T06:18:41Z OWNER

Those sounds to me like they should be promoted to documented, supported internals.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Scripted exports 1108671952  
1331658629 https://github.com/simonw/datasette/issues/1918#issuecomment-1331658629 https://api.github.com/repos/simonw/datasette/issues/1918 IC_kwDOBm6k_c5PX3-F simonw 9599 2022-11-30T05:21:51Z 2022-11-30T05:21:51Z OWNER

Much better:

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 API explorer should list mutable databases first 1469044738  
1331657404 https://github.com/simonw/datasette/issues/1919#issuecomment-1331657404 https://api.github.com/repos/simonw/datasette/issues/1919 IC_kwDOBm6k_c5PX3q8 simonw 9599 2022-11-30T05:19:43Z 2022-11-30T05:19:43Z OWNER

This is the test: https://github.com/simonw/datasette/blob/8404b21556d133c89eda4bd1bf5335ed9a0785d6/tests/test_api_write.py#L342-L401

I'm suspicious that there's a timing error of some sort but I can't think what it might be.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent `test_delete_row` test failure  1469062686  
1331651721 https://github.com/simonw/datasette/issues/1916#issuecomment-1331651721 https://api.github.com/repos/simonw/datasette/issues/1916 IC_kwDOBm6k_c5PX2SJ simonw 9599 2022-11-30T05:10:27Z 2022-11-30T05:10:27Z OWNER

They should return 405 method not allowed with an {"ok":false, "error": "Method not allowed"} body.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 GET requests against POST endpoints should not 500 error 1469015001  
1331644751 https://github.com/simonw/datasette/issues/1917#issuecomment-1331644751 https://api.github.com/repos/simonw/datasette/issues/1917 IC_kwDOBm6k_c5PX0lP simonw 9599 2022-11-30T04:59:22Z 2022-11-30T04:59:22Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Don't allow writable API to edit the `_memory` database 1469043836  
1331644078 https://github.com/simonw/datasette/issues/1917#issuecomment-1331644078 https://api.github.com/repos/simonw/datasette/issues/1917 IC_kwDOBm6k_c5PX0au simonw 9599 2022-11-30T04:58:06Z 2022-11-30T04:58:06Z OWNER

The problem might actually be here:

https://github.com/simonw/datasette/blob/9f5321ff1eca58c469a45cc406d7eb5ad05accbd/datasette/app.py#L280-L281

is_mutable defaults to True, so this line should probably be:

python self.add_database(Database(self, is_mutable=False, is_memory=True), name="_memory")

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Don't allow writable API to edit the `_memory` database 1469043836  
1331479606 https://github.com/simonw/datasette/issues/1915#issuecomment-1331479606 https://api.github.com/repos/simonw/datasette/issues/1915 IC_kwDOBm6k_c5PXMQ2 simonw 9599 2022-11-30T00:09:06Z 2022-11-30T00:09:06Z OWNER

One last feature: I want to show an indication on the table page that the table has X seconds left to live.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Interactive demo of Datasette 1.0 write APIs 1468709531  
1331479328 https://github.com/simonw/datasette/issues/1915#issuecomment-1331479328 https://api.github.com/repos/simonw/datasette/issues/1915 IC_kwDOBm6k_c5PXMMg simonw 9599 2022-11-30T00:08:41Z 2022-11-30T00:08:41Z OWNER

Five minute has now passed and https://latest.datasette.io/ephemeral/new_table is gone.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Interactive demo of Datasette 1.0 write APIs 1468709531  
1331476246 https://github.com/simonw/datasette/issues/1915#issuecomment-1331476246 https://api.github.com/repos/simonw/datasette/issues/1915 IC_kwDOBm6k_c5PXLcW simonw 9599 2022-11-30T00:04:35Z 2022-11-30T00:08:24Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Interactive demo of Datasette 1.0 write APIs 1468709531  
1331478611 https://github.com/simonw/datasette/issues/1915#issuecomment-1331478611 https://api.github.com/repos/simonw/datasette/issues/1915 IC_kwDOBm6k_c5PXMBT simonw 9599 2022-11-30T00:07:37Z 2022-11-30T00:07:37Z OWNER

Then I created an API token at https://latest.datasette.io/-/create-token and ran this:

curl -XPOST 'https://latest.datasette.io/ephemeral/new_table/-/insert' \ -H 'Authorization: Bearer xxx' \ -H 'Content-Type: application/json' \ -d '{"row": {"name": "NAME"}}' And it inserted a row into https://latest.datasette.io/ephemeral/new_table

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Interactive demo of Datasette 1.0 write APIs 1468709531  
1331432223 https://github.com/simonw/datasette/issues/1915#issuecomment-1331432223 https://api.github.com/repos/simonw/datasette/issues/1915 IC_kwDOBm6k_c5PXAsf simonw 9599 2022-11-29T23:06:17Z 2022-11-29T23:06:17Z OWNER

To (slightly) discourage abuse I'm going to make the demo database only visible to the root user - so people can't create tables with rude names and have them show to the public on https://latest.datasette.io/

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Interactive demo of Datasette 1.0 write APIs 1468709531  
1331331082 https://github.com/simonw/datasette/issues/1915#issuecomment-1331331082 https://api.github.com/repos/simonw/datasette/issues/1915 IC_kwDOBm6k_c5PWoAK simonw 9599 2022-11-29T21:24:59Z 2022-11-29T21:34:53Z OWNER

Maybe a plugin called datasette-temporary-tables or datasette-demo-tables or datasette-demo-database.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Interactive demo of Datasette 1.0 write APIs 1468709531  
1331238841 https://github.com/simonw/datasette/issues/1850#issuecomment-1331238841 https://api.github.com/repos/simonw/datasette/issues/1850 IC_kwDOBm6k_c5PWRe5 simonw 9599 2022-11-29T20:11:20Z 2022-11-29T20:11:20Z OWNER

Released this in Datasette 1.0a0: - #1913

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Write API in Datasette core 1421529723  
1331238029 https://github.com/simonw/datasette/issues/1913#issuecomment-1331238029 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWRSN simonw 9599 2022-11-29T20:10:35Z 2022-11-29T20:10:35Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331226346 https://github.com/simonw/datasette/issues/1913#issuecomment-1331226346 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWObq simonw 9599 2022-11-29T20:00:16Z 2022-11-29T20:00:36Z OWNER

Looks like a fix is coming: https://github.com/pypa/twine/issues/940#issuecomment-1331225509

Note that must_decode was defined in pkg_info/_compat.py, and was thus never an API: before 1.9.0, it was only imported and used in `pkginfo/distribution.py'.

Nevertheless, I will push out a 1.9.1 release of pkginfo which restores a deprecated compatibility alias.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331225277 https://github.com/simonw/datasette/issues/1913#issuecomment-1331225277 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWOK9 simonw 9599 2022-11-29T19:59:14Z 2022-11-29T19:59:34Z OWNER

I deleted the tag and tried creating a new release. Now running here: https://github.com/simonw/datasette/actions/runs/3577554546

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331216652 https://github.com/simonw/datasette/issues/1913#issuecomment-1331216652 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWMEM simonw 9599 2022-11-29T19:54:22Z 2022-11-29T19:54:22Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331208206 https://github.com/simonw/datasette/issues/1913#issuecomment-1331208206 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWKAO simonw 9599 2022-11-29T19:51:31Z 2022-11-29T19:51:31Z OWNER

https://pypi.org/project/pkginfo/#history - 1.9.0 came out 39 minutes ago!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331207334 https://github.com/simonw/datasette/issues/1913#issuecomment-1331207334 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWJym simonw 9599 2022-11-29T19:50:37Z 2022-11-29T19:50:37Z OWNER

https://pypi.org/project/setuptools/65.6.3/ came out most recently - 23rd November (wheel and twine are older).

No search results at all for that error message. This is very weird, I would have expected it to have been reported by now.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331205613 https://github.com/simonw/datasette/issues/1913#issuecomment-1331205613 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWJXt simonw 9599 2022-11-29T19:48:52Z 2022-11-29T19:48:52Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331204360 https://github.com/simonw/datasette/issues/1913#issuecomment-1331204360 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWJEI simonw 9599 2022-11-29T19:47:40Z 2022-11-29T19:47:40Z OWNER

... but the last step of the deploy failed, when it was meant to push to PyPI! Uploading distributions to https://upload.pypi.org/legacy/ Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.11.0/x64/bin/twine", line 8, in <module> sys.exit(main()) ^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/__main__.py", line 33, in main error = cli.dispatch(sys.argv[1:]) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/cli.py", line 123, in dispatch return main(args.args) ^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 198, in main return upload(upload_settings, parsed_args.dists) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 123, in upload packages_to_upload = [ ^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 124, in <listcomp> _make_package(filename, signatures, upload_settings) for filename in uploads ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py", line 77, in _make_package package = package_file.PackageFile.from_filename(filename, upload_settings.comment) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/package.py", line 96, in from_filename meta = DIST_TYPES[dtype](filename) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/wheel.py", line 42, in __init__ self.extractMetadata() File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/pkginfo/distribution.py", line 121, in extractMetadata self.parse(data) File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/wheel.py", line 89, in parse fp = io.StringIO(distribution.must_decode(data)) ^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: module 'pkginfo.distribution' has no attribute 'must_decode'. Did you mean: '_must_decode'? Error: Process completed with exit code 1.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331203997 https://github.com/simonw/datasette/issues/1913#issuecomment-1331203997 https://api.github.com/repos/simonw/datasette/issues/1913 IC_kwDOBm6k_c5PWI-d simonw 9599 2022-11-29T19:47:13Z 2022-11-29T19:47:13Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 1.0a0 1468603401  
1331201207 https://github.com/simonw/datasette/issues/1877#issuecomment-1331201207 https://api.github.com/repos/simonw/datasette/issues/1877 IC_kwDOBm6k_c5PWIS3 simonw 9599 2022-11-29T19:44:07Z 2022-11-29T19:44:07Z OWNER

I fixed the duplicate logic issue here: https://github.com/simonw/datasette/commit/ee64130fa8a5ff4a24791916c696e10cf2375102

  • 1896

Decided not to address views/table.py.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Refactor and tidy up final write API code 1432012302  
1331196531 https://github.com/simonw/datasette/pull/1912#issuecomment-1331196531 https://api.github.com/repos/simonw/datasette/issues/1912 IC_kwDOBm6k_c5PWHJz simonw 9599 2022-11-29T19:39:10Z 2022-11-29T19:39:10Z OWNER

Annoyingly it looks like I can't rebase this one, and I don't want to squash-merge and lose the commits, so I'm going to do a regular merge instead.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Merge 1.0-dev (with initial write API) back into main 1468592292  
1331187551 https://github.com/simonw/datasette/issues/1605#issuecomment-1331187551 https://api.github.com/repos/simonw/datasette/issues/1605 IC_kwDOBm6k_c5PWE9f eyeseast 25778 2022-11-29T19:29:42Z 2022-11-29T19:29:42Z CONTRIBUTOR

Interesting. I started a version using metadata like I outlined up top, but I realized that there's no documented way for a plugin to access either metadata or canned queries. Or at least, I couldn't find a way.

There is this method: https://github.com/simonw/datasette/blob/main/datasette/app.py#L472 but I don't want to rely on it if it's not documented. Same with this: https://github.com/simonw/datasette/blob/main/datasette/app.py#L544

If those are safe, I'll build on them. I'm also happy to document them, if that greases the wheels.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Scripted exports 1108671952  
1331181922 https://github.com/simonw/datasette/issues/1891#issuecomment-1331181922 https://api.github.com/repos/simonw/datasette/issues/1891 IC_kwDOBm6k_c5PWDli simonw 9599 2022-11-29T19:23:41Z 2022-11-29T19:23:41Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1.0a0 release notes 1450303205  
1331143292 https://github.com/simonw/datasette/issues/1891#issuecomment-1331143292 https://api.github.com/repos/simonw/datasette/issues/1891 IC_kwDOBm6k_c5PV6J8 simonw 9599 2022-11-29T18:57:40Z 2022-11-29T18:57:40Z OWNER

I'm going to keep these short - they'll mostly be links to the documentation for the new features.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1.0a0 release notes 1450303205  
1331140747 https://github.com/simonw/datasette/issues/1891#issuecomment-1331140747 https://api.github.com/repos/simonw/datasette/issues/1891 IC_kwDOBm6k_c5PV5iL simonw 9599 2022-11-29T18:55:42Z 2022-11-29T18:55:42Z OWNER

All features for the alpha are complete now. Release notes should be based on these commits:

https://github.com/simonw/datasette/compare/0.63.2...6bda2257868a2cbd70b84b7a86a5bcb47dcc4874

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1.0a0 release notes 1450303205  
1331135709 https://github.com/simonw/datasette/issues/1911#issuecomment-1331135709 https://api.github.com/repos/simonw/datasette/issues/1911 IC_kwDOBm6k_c5PV4Td simonw 9599 2022-11-29T18:50:58Z 2022-11-29T18:50:58Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `/db/-/create` should support creating tables with compound primary keys 1468519699  
1331120055 https://github.com/simonw/datasette/issues/1911#issuecomment-1331120055 https://api.github.com/repos/simonw/datasette/issues/1911 IC_kwDOBm6k_c5PV0e3 simonw 9599 2022-11-29T18:36:01Z 2022-11-29T18:36:01Z OWNER

Current API design: POST /<database>/-/create json { "table": "name_of_new_table", "columns": [ { "name": "id", "type": "integer" }, { "name": "title", "type": "text" } ], "pk": "id" } I'm going to add a new "pks" key which is a list, and can be used in place of "pk".

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `/db/-/create` should support creating tables with compound primary keys 1468519699  
1331089156 https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156 https://api.github.com/repos/simonw/datasette/issues/1863 IC_kwDOBm6k_c5PVs8E simonw 9599 2022-11-29T18:08:53Z 2022-11-29T18:08:53Z OWNER

I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update a single record in an existing table 1425029242  
1330974099 https://github.com/simonw/datasette/issues/1863#issuecomment-1330974099 https://api.github.com/repos/simonw/datasette/issues/1863 IC_kwDOBm6k_c5PVQ2T simonw 9599 2022-11-29T17:03:00Z 2022-11-29T17:11:05Z OWNER

I've decided that I won't do that validation for the first version of this - I'm going to teach dclient to send the correct types instead: https://github.com/simonw/dclient/issues/6#issuecomment-1330963953

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update a single record in an existing table 1425029242  
1329624931 https://github.com/simonw/datasette/issues/1909#issuecomment-1329624931 https://api.github.com/repos/simonw/datasette/issues/1909 IC_kwDOBm6k_c5PQHdj simonw 9599 2022-11-28T19:19:26Z 2022-11-28T19:19:26Z OWNER

The list of states here is a good example of somewhere this might be useful: https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=state&_facet_size=max

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Option to sort facets alphabetically 1466952626  
1328169472 https://github.com/simonw/datasette/issues/1605#issuecomment-1328169472 https://api.github.com/repos/simonw/datasette/issues/1605 IC_kwDOBm6k_c5PKkIA simonw 9599 2022-11-27T04:32:14Z 2022-11-27T04:32:14Z OWNER

@eyeseast I started work on that plugin: https://github.com/simonw/datasette-export

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Scripted exports 1108671952  
1325164933 https://github.com/simonw/datasette/issues/1706#issuecomment-1325164933 https://api.github.com/repos/simonw/datasette/issues/1706 IC_kwDOBm6k_c5O_GmF ar-jan 1176293 2022-11-23T14:34:54Z 2022-11-23T14:34:54Z NONE

This would be helpful.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 [feature] immutable mode for a directory, not just individual sqlite file 1198822563  
1324539030 https://github.com/simonw/datasette/issues/1863#issuecomment-1324539030 https://api.github.com/repos/simonw/datasette/issues/1863 IC_kwDOBm6k_c5O8tyW simonw 9599 2022-11-23T04:35:14Z 2022-11-23T04:35:14Z OWNER

If I do that I should probably update insert to do those validation checks as well.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update a single record in an existing table 1425029242  
1324531750 https://github.com/simonw/datasette/issues/1863#issuecomment-1324531750 https://api.github.com/repos/simonw/datasette/issues/1863 IC_kwDOBm6k_c5O8sAm simonw 9599 2022-11-23T04:20:47Z 2022-11-23T04:20:47Z OWNER

... which does imply that I'm going to do an extra layer of validation over what SQLite provides. SQLite will happily allow a text string to be added to a supposedly integer column. I'm not going to allow that - I'll return a validation error instead, unless the string can be safely coerced to the correct type.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update a single record in an existing table 1425029242  
1324531085 https://github.com/simonw/datasette/issues/1863#issuecomment-1324531085 https://api.github.com/repos/simonw/datasette/issues/1863 IC_kwDOBm6k_c5O8r2N simonw 9599 2022-11-23T04:19:28Z 2022-11-23T04:19:28Z OWNER

Had a design conversation with myself in https://github.com/simonw/dclient/issues/6 where I decided that the API should allow string values to be sent to integer columns which would be automatically converted if possible to do so - as an API usability feature.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update a single record in an existing table 1425029242  
1321460293 https://github.com/simonw/datasette/issues/1884#issuecomment-1321460293 https://api.github.com/repos/simonw/datasette/issues/1884 IC_kwDOBm6k_c5Ow-JF asg017 15178711 2022-11-21T04:40:55Z 2022-11-21T04:40:55Z CONTRIBUTOR

Counting any virtual tables can be pretty tricky. On one hand, counting a CSV virtual table would return the number of rows in the CSV, which is helpful (but can be I/O intensive). Counting a FTS5 virtual table would return the number of entries in the FTS index, which is kindof helpful, but can be misleading in some cases.

On the other hand, arbitrarily running COUNT(*) on some virtual tables can be incredibly expensive. SQLite offers new shortcuts/pushdowns on COUNT(*) queries for virtual tables, and instead calls the underlying vtab implementation and iterates through all rows in the table without discretion. For example, a virtual table that's backed by a Postgres table would call select * from pg_table, which would use up a lot of network and CPU calls. Or a virtual table backed by a google sheet would make network/API requests to get all the rows from the sheet just to make a count.

The pragma_table_list pragma tells you when a table is a regular table or virtual (in the type column), but was only added in version 3.37.0 (2021-11-27).

Personally, I wouldnt try to COUNT(*) virtual tables - it depends on how the virtual table is implemented, it requires that the connection has the proper extensions loaded, and it may accientally cause perf issues for new-age extensions. A few extensions that I'm writing have virtual tables that wouldn't benefit much from COUNT(*), and the fact that SQLite iterates through all rows in a table to count just makes things worse. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Exclude virtual tables from datasette inspect 1439009231  
1321262142 https://github.com/simonw/datasette/issues/1903#issuecomment-1321262142 https://api.github.com/repos/simonw/datasette/issues/1903 IC_kwDOBm6k_c5OwNw- simonw 9599 2022-11-20T22:35:01Z 2022-11-20T22:35:01Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Refactor all error classes into a datasette.exceptions module 1455928469  
1321241426 https://github.com/simonw/datasette/issues/1886#issuecomment-1321241426 https://api.github.com/repos/simonw/datasette/issues/1886 IC_kwDOBm6k_c5OwItS fgregg 536941 2022-11-20T20:58:54Z 2022-11-20T20:58:54Z CONTRIBUTOR

i wrote up a blog post of how i'm using it! https://bunkum.us/2022/11/20/mgdo-stack.html

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Call for birthday presents: if you're using Datasette, let us know how you're using it here 1447050738  
1321003094 https://github.com/simonw/datasette/issues/1886#issuecomment-1321003094 https://api.github.com/repos/simonw/datasette/issues/1886 IC_kwDOBm6k_c5OvOhW hydrosquall 9020979 2022-11-20T00:52:05Z 2022-11-20T00:52:05Z NONE

Happy birthday to datasette and thank you Simon for your continued effort on this project!

I use datasette (python) as a fast layer on top of search for github projects using https://github.com/dogsheep/github-to-sqlite , and use the JSON API it provides to serve sample data to make Vega-Lite graphing workshop examples that don't require authentication/API keys. It's awesome to have a full SQL API support working without needing to develop any custom API middleware for both filtering and grouping.

I've also enjoyed using it as a teaching tool for working with public dataset in civic data workshops and as a platform for making visualization plugins . I

I'm especially excited about datasette-lite, as it will let people participate in future editions of this workshop without having to install anything to make use of their own tables :)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Call for birthday presents: if you're using Datasette, let us know how you're using it here 1447050738  
1320721241 https://github.com/simonw/datasette/issues/1905#issuecomment-1320721241 https://api.github.com/repos/simonw/datasette/issues/1905 IC_kwDOBm6k_c5OuJtZ simonw 9599 2022-11-19T01:12:05Z 2022-11-19T01:12:05Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `publish heroku` failing due to old Python version 1456012874  
1320689643 https://github.com/simonw/datasette/issues/1905#issuecomment-1320689643 https://api.github.com/repos/simonw/datasette/issues/1905 IC_kwDOBm6k_c5OuB_r simonw 9599 2022-11-19T00:17:19Z 2022-11-19T00:41:54Z OWNER

The tests don't cover this bit at the moment.

Would be easier to write tests if there was a --generate-dir option as seen in https://datasette.io/plugins/datasette-publish-vercel

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `publish heroku` failing due to old Python version 1456012874  
1320706596 https://github.com/simonw/datasette/issues/1905#issuecomment-1320706596 https://api.github.com/repos/simonw/datasette/issues/1905 IC_kwDOBm6k_c5OuGIk simonw 9599 2022-11-19T00:41:44Z 2022-11-19T00:41:44Z OWNER

Tested a deploy:

``` % datasette publish heroku fixtures.db -n datasette-issue-1905 › Warning: heroku update available from 7.63.0 to 7.66.4. › Warning: heroku update available from 7.63.0 to 7.66.4. › Warning: heroku update available from 7.63.0 to 7.66.4. Creating datasette-issue-1905... done › Warning: heroku update available from 7.63.0 to 7.66.4. ▸ Couldn't detect GNU tar. Builds could fail due to decompression errors ▸ See https://devcenter.heroku.com/articles/platform-api-deploying-slugs#create-slug-archive ▸ Please install it, or specify the '--tar' option ▸ Falling back to node's built-in compressor

-----> Building on the Heroku-22 stack -----> Determining which buildpack to use for this app -----> Python app detected -----> Using Python version specified in runtime.txt -----> Installing python-3.11.0 -----> Installing pip 22.3.1, setuptools 63.4.3 and wheel 0.37.1 -----> Installing SQLite3 -----> Installing requirements with pip Collecting datasette Downloading datasette-0.63.1-py3-none-any.whl (231 kB) ... -----> Running post-compile hook -----> Discovering process types Procfile declares types -> web

-----> Compressing... Done: 28M -----> Launching... Released v3 https://datasette-issue-1905.herokuapp.com/ deployed to Heroku

Starting November 28th, 2022, free Heroku Dynos, free Heroku Postgres, and free Heroku Data for Redis® will no longer be available.

If you have apps using any of these resources, you must upgrade to paid plans by this date to ensure your apps continue to run and to retain your data. For students, we will announce a new program by the end of September. Learn more at https://blog.heroku.com/next-chapter ``` I had to then pay for the dino because I'd run out of free hours.

https://datasette-issue-1905.herokuapp.com/-/versions shows:

json { "python": { "version": "3.11.0", "full": "3.11.0 (main, Oct 24 2022, 21:34:02) [GCC 11.2.0]" }, "datasette": { "version": "0.63.1" }

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `publish heroku` failing due to old Python version 1456012874  
1320678715 https://github.com/simonw/datasette/issues/1905#issuecomment-1320678715 https://api.github.com/repos/simonw/datasette/issues/1905 IC_kwDOBm6k_c5Ot_U7 simonw 9599 2022-11-19T00:02:28Z 2022-11-19T00:02:28Z OWNER

This is a strong argument for extracting the Heroku support out to a plugin - it would allow this to be fixed with a plugin release without needing to push a full release of Datasette itself.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `publish heroku` failing due to old Python version 1456012874  
1320625260 https://github.com/simonw/datasette/issues/1891#issuecomment-1320625260 https://api.github.com/repos/simonw/datasette/issues/1891 IC_kwDOBm6k_c5OtyRs simonw 9599 2022-11-18T23:01:03Z 2022-11-18T23:01:48Z OWNER

I think this actually needs to include a whole section of the documentation about the road to 1.0 - what to expect (planned breaking changes) etc. I can add that to the https://docs.datasette.io/en/stable/contributing.html page perhaps - or even create a Roadmap page.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1.0a0 release notes 1450303205  
1320616559 https://github.com/simonw/datasette/issues/1896#issuecomment-1320616559 https://api.github.com/repos/simonw/datasette/issues/1896 IC_kwDOBm6k_c5OtwJv simonw 9599 2022-11-18T22:51:14Z 2022-11-18T22:51:14Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Extract logic for resolving a URL to a database / table / row 1452364777  
1320614541 https://github.com/simonw/datasette/issues/1903#issuecomment-1320614541 https://api.github.com/repos/simonw/datasette/issues/1903 IC_kwDOBm6k_c5OtvqN simonw 9599 2022-11-18T22:47:41Z 2022-11-18T22:47:41Z OWNER

When I do this it's important to update the documentation for resolve_database() and the like:

https://github.com/simonw/datasette/blob/ee64130fa8a5ff4a24791916c696e10cf2375102/docs/internals.rst#L594

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Refactor all error classes into a datasette.exceptions module 1455928469  
1320588299 https://github.com/simonw/datasette/issues/1896#issuecomment-1320588299 https://api.github.com/repos/simonw/datasette/issues/1896 IC_kwDOBm6k_c5OtpQL simonw 9599 2022-11-18T22:16:59Z 2022-11-18T22:17:06Z OWNER

Found myself needing an await db.view_exists() method for this, similar to the existing await db.table_exists() one.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Extract logic for resolving a URL to a database / table / row 1452364777  
1317757949 https://github.com/simonw/datasette/issues/1896#issuecomment-1317757949 https://api.github.com/repos/simonw/datasette/issues/1896 IC_kwDOBm6k_c5Oi2P9 simonw 9599 2022-11-16T22:27:47Z 2022-11-18T21:48:29Z OWNER

Open question: should resolve_table() know how to identify named canned queries too?

I think not, at least for the moment. Feels a bit too specialist to expose in a documented API.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Extract logic for resolving a URL to a database / table / row 1452364777  
1320563197 https://github.com/simonw/datasette/issues/1863#issuecomment-1320563197 https://api.github.com/repos/simonw/datasette/issues/1863 IC_kwDOBm6k_c5OtjH9 simonw 9599 2022-11-18T21:47:35Z 2022-11-18T21:48:07Z OWNER

Incomplete implementation of this view: ```python class RowUpdateView(BaseView): name = "row-update"

def __init__(self, datasette):
    self.ds = datasette

async def post(self, request):
    database_route = tilde_decode(request.url_vars["database"])
    table = tilde_decode(request.url_vars["table"])
    try:
        db = self.ds.get_database(route=database_route)
    except KeyError:
        return _error(["Database not found: {}".format(database_route)], 404)

    database_name = db.name
    if not await db.table_exists(table):
        return _error(["Table not found: {}".format(table)], 404)

    pk_values = urlsafe_components(request.url_vars["pks"])

    sql, params, pks = await row_sql_params_pks(db, table, pk_values)
    results = await db.execute(sql, params, truncate=True)
    rows = list(results.rows)
    if not rows:
        return _error([f"Record not found: {pk_values}"], 404)

    # Ensure user has permission to update this row
    if not await self.ds.permission_allowed(
        request.actor, "update-row", resource=(database_name, table)
    ):
        return _error(["Permission denied"], 403)

    body = await request.post_body()
    try:
        data = json.loads(body)
    except json.JSONDecodeError as e:
        return _error(["Invalid JSON: {}".format(e)])
    if not isinstance(data, dict):
        return _error(["JSON must be a dictionary"])

    def update_row(conn):
        sqlite_utils.Database(conn)[table].update(pk_values, updates)

    await db.execute_write_fn(update_row)
    result = {"ok": True}
    if data.get("return"):
        result["row"] = {"row-here": "TODO"}
    return Response.json(result, status=200)

``` This is before the refactor in: - #1896

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update a single record in an existing table 1425029242  
1320394127 https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1320394127 https://api.github.com/repos/simonw/sqlite-utils/issues/510 IC_kwDOCGYnMM5Os52P ar-jan 1176293 2022-11-18T18:37:51Z 2022-11-18T18:37:51Z NONE

I guess it is not incorrect when it says the version is 4, though it is confusing. Maybe it doesn't even refer to FTS4/FTS5 versions, but something else? In any case, it's not related to sqlite-utils, but SQLite itself.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Cannot enable FTS5 despite it being available 1434911255  
1319664697 https://github.com/simonw/datasette/issues/1900#issuecomment-1319664697 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5OqHw5 rdmurphy 419145 2022-11-18T07:59:36Z 2022-11-18T08:00:38Z NONE

Okay, my final observations for the night! I've been pushing and pulling the various levers in utils/__init__.py to see what makes this work without hard-coding in something for arm64 and it seems that if I change /usr/lib/x86_64-linux-gnu/mod_spatialite.so here to just mod_spatialite it's happy.

Unfortunately cannot audit that for x86_64, but maybe that's a solution that'd be cross-arch compatible? It seems like it's the hard-coding of that path that's tripping it up.

(It was actually this comment from back in 2018 in an entirely unrelated repo that nudged me to try this, ha.)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319642535 https://github.com/simonw/datasette/pull/1898#issuecomment-1319642535 https://api.github.com/repos/simonw/datasette/issues/1898 IC_kwDOBm6k_c5OqCWn simonw 9599 2022-11-18T07:28:45Z 2022-11-18T07:28:45Z OWNER

Thanks!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Use DOMContentLoaded instead of load event for CodeMirror initialization 1452485922  
1319642338 https://github.com/simonw/datasette/issues/1899#issuecomment-1319642338 https://api.github.com/repos/simonw/datasette/issues/1899 IC_kwDOBm6k_c5OqCTi simonw 9599 2022-11-18T07:28:28Z 2022-11-18T07:28:28Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused  1452495049  
1319641636 https://github.com/simonw/datasette/issues/1900#issuecomment-1319641636 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5OqCIk rdmurphy 419145 2022-11-18T07:27:26Z 2022-11-18T07:27:26Z NONE

Can confirm that my uname -a returns something different at the end:

root:xnu-8792.41.9~2/RELEASE_ARM64_T6000 arm64

I'm in arm64 land, you're in x86_64. I am admittedly very fuzzy on how this factors into Docker these days. Honestly thought this was one of the things Docker was suppose to help address. 🤔

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319639462 https://github.com/simonw/datasette/issues/1900#issuecomment-1319639462 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5OqBmm rdmurphy 419145 2022-11-18T07:24:19Z 2022-11-18T07:24:19Z NONE

Is it, uh, possible we are on different architectures? 😅 I'm using an Apple M1 Pro.

I jumped into a bash shell of an unmodified python:3.11.0-slim-bullseye container and manually ran apt-get update and installed libsqlite3-mod-spatialite. I don't end up with with mod_spatialite.so in /usr/lib/x86_64-linux-gnu/ — mine is in /usr/lib/aarch64-linux-gnu/.

I swapped that directory in here in a local copy of datasette and poof — it worked!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319525520 https://github.com/simonw/datasette/issues/1901#issuecomment-1319525520 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5OplyQ simonw 9599 2022-11-18T04:21:14Z 2022-11-18T07:22:37Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319631421 https://github.com/simonw/datasette/issues/1900#issuecomment-1319631421 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Op_o9 simonw 9599 2022-11-18T07:13:00Z 2022-11-18T07:13:00Z OWNER

You get:

=> [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 0.9s => [internal] load build context 2.3s => => transferring context: 72.38MB 2.3s => CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s

I get:

=> [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 1.0s => [internal] load build context 0.0s => => transferring context: 705B 0.0s => CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s Both the image name and the hash are exactly the same. So why are you getting an error while mine works OK?

For my machine: ``` ~ % docker --version Docker version 20.10.12, build e91ed57

~ % uname -a Darwin Simons-MacBook-Pro-2.local 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64 x86_64 ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319629469 https://github.com/simonw/datasette/issues/1900#issuecomment-1319629469 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Op_Kd simonw 9599 2022-11-18T07:10:17Z 2022-11-18T07:10:17Z OWNER

This is so weird! What version of Datasette do you get from datasette --version there - and what's your Docker version / operating system version?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319627012 https://github.com/simonw/datasette/issues/1901#issuecomment-1319627012 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5Op-kE simonw 9599 2022-11-18T07:07:03Z 2022-11-18T07:07:03Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319493475 https://github.com/simonw/datasette/issues/1901#issuecomment-1319493475 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5Opd9j simonw 9599 2022-11-18T03:19:25Z 2022-11-18T07:03:03Z OWNER

Other plugins this looks like it will affect:

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319623911 https://github.com/simonw/datasette/issues/1901#issuecomment-1319623911 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5Op9zn simonw 9599 2022-11-18T07:02:56Z 2022-11-18T07:02:56Z OWNER

That's all of them!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319596087 https://github.com/simonw/datasette/issues/1900#issuecomment-1319596087 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Op3A3 rdmurphy 419145 2022-11-18T06:16:33Z 2022-11-18T06:16:33Z NONE

Interesting! So I tried this locally using your copy of nps-spatialite.db and I got the same error. 🤔

``` ❯ datasette package nps-spatialite.db --spatialite [+] Building 27.5s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 622B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 0.9s => [internal] load build context 2.3s => => transferring context: 72.38MB 2.3s => CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s => [2/6] COPY . /app 0.1s => [3/6] WORKDIR /app 0.0s => [4/6] RUN apt-get update && apt-get install -y python3-dev gcc libsqlite3-mod-spatialite && rm -rf /var/lib/apt/lists/* 18.5s => [5/6] RUN pip install -U datasette 4.9s => ERROR [6/6] RUN datasette inspect nps-spatialite.db --inspect-file inspect-data.json 0.7s

[6/6] RUN datasette inspect nps-spatialite.db --inspect-file inspect-data.json:

10 0.681 Traceback (most recent call last):

10 0.681 File "/usr/local/bin/datasette", line 8, in <module>

10 0.681 sys.exit(cli())

10 0.681 ^^^^^

10 0.681 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in call

10 0.682 return self.main(args, *kwargs)

10 0.682 ^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.682 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main

10 0.682 rv = self.invoke(ctx)

10 0.682 ^^^^^^^^^^^^^^^^

10 0.682 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke

10 0.682 return _process_result(sub_ctx.command.invoke(sub_ctx))

10 0.682 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.682 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke

10 0.682 return ctx.invoke(self.callback, **ctx.params)

10 0.682 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.682 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke

10 0.682 return __callback(args, *kwargs)

10 0.682 ^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/cli.py", line 164, in inspect

10 0.683 inspect_data = loop.run_until_complete(inspect_(files, sqlite_extensions))

10 0.683 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete

10 0.683 return future.result()

10 0.683 ^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/cli.py", line 179, in inspect_

10 0.683 counts = await database.table_counts(limit=3600 * 1000)

10 0.683 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 304, in table_counts

10 0.683 for table in await self.table_names():

10 0.683 ^^^^^^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 342, in table_names

10 0.683 results = await self.execute(

10 0.683 ^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 267, in execute

10 0.683 results = await self.execute_fn(sql_operation_in_thread)

10 0.683 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 213, in execute_fn

10 0.683 return await asyncio.get_event_loop().run_in_executor(

10 0.683 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run

10 0.683 result = self.fn(self.args, *self.kwargs)

10 0.683 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/database.py", line 209, in in_thread

10 0.683 self.ds._prepare_connection(conn, self.name)

10 0.683 File "/usr/local/lib/python3.11/site-packages/datasette/app.py", line 593, in _prepare_connection

10 0.683 conn.execute("SELECT load_extension(?)", [extension])

10 0.683 sqlite3.OperationalError: /usr/lib/x86_64-linux-gnu/mod_spatialite.so.so: cannot open shared object file: No such file or directory

executor failed running [/bin/sh -c datasette inspect nps-spatialite.db --inspect-file inspect-data.json]: exit code: 1 ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319588163 https://github.com/simonw/datasette/issues/1901#issuecomment-1319588163 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5Op1FD simonw 9599 2022-11-18T06:05:11Z 2022-11-18T06:05:11Z OWNER

For datasette-copyable I want to show breadcrumbs that take database/instance permissions into account, so I'm removing {% block nav %} entirely and replacing it with this:

html+jinja {% block crumbs %} {{ crumbs.nav(request=request, database=database, table=table) }} {% endblock %}

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319584553 https://github.com/simonw/datasette/issues/1899#issuecomment-1319584553 https://api.github.com/repos/simonw/datasette/issues/1899 IC_kwDOBm6k_c5Op0Mp simonw 9599 2022-11-18T06:00:10Z 2022-11-18T06:01:50Z OWNER

I can't actually remember where that min-height: 70px came from. I just tried without it and it seems fine - especially since any time you add a newline in the editor it increases its height to fit.

I ran this in the DevTools console:

javascript document.querySelector('.cm-editor').style.minHeight = 'none';

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused  1452495049  
1319583703 https://github.com/simonw/datasette/issues/1900#issuecomment-1319583703 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Opz_X simonw 9599 2022-11-18T05:58:31Z 2022-11-18T05:58:31Z OWNER

Could you provide full steps to reproduce plus a SpatiaLite database file that triggered this for you? I'm not able to recreate the problem.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319583281 https://github.com/simonw/datasette/issues/1900#issuecomment-1319583281 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Opz4x simonw 9599 2022-11-18T05:57:44Z 2022-11-18T05:57:44Z OWNER

Did you use the --spatialite option?

I just tried this:

datasette package nps-spatialite.db

It built the image OK (I didn't see the error you reported), but running the container failed with an error:

``` /tmp % docker run -p 8001:8001 7298e8e6bbfb Usage: datasette serve [OPTIONS] [FILES]... Try 'datasette serve --help' for help.

Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module.

Read more: https://docs.datasette.io/en/stable/spatialite.html ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319582239 https://github.com/simonw/datasette/issues/1900#issuecomment-1319582239 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Opzof simonw 9599 2022-11-18T05:55:38Z 2022-11-18T05:55:38Z OWNER

Trying this out locally with this 69MB SpatiaLite file I happened to have lying around (from testing shapefile-to-sqlite a while ago): https://static.simonwillison.net/static/2022/nps-spatialite.db % datasette package nps-spatialite.db --spatialite ... => [2/6] COPY . /app 0.4s => [3/6] WORKDIR /app 0.0s => [4/6] RUN apt-get update && apt-get install -y python3-dev gcc libsqlite3-mod-spatialite && rm -rf /var/lib/apt/lists/* 29.6s => [5/6] RUN pip install -U datasette 12.0s => [6/6] RUN datasette inspect nps-spatialite.db --inspect-file inspect-data.json 2.6s => exporting to image 3.0s => => exporting layers 3.0s => => writing image sha256:4dfef1c373c5c057ef7ac22344f834d522acef24313a1b25d2eba9e500066b8f 0.0s And then:

docker run -p 8001:8001 4dfef1c373c5

This worked fine for me. I ran datasette package using Datasette 0.63.1.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319574972 https://github.com/simonw/datasette/issues/1900#issuecomment-1319574972 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Opx28 simonw 9599 2022-11-18T05:41:28Z 2022-11-18T05:41:28Z OWNER

Oh this is with datasette package? That should work. Will investigate.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319571220 https://github.com/simonw/datasette/issues/1900#issuecomment-1319571220 https://api.github.com/repos/simonw/datasette/issues/1900 IC_kwDOBm6k_c5Opw8U simonw 9599 2022-11-18T05:34:35Z 2022-11-18T05:34:35Z OWNER

Which Docker image are you using here? It looks like it's missing SpatiaLite from the image.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette package --spatialite throws error during build 1452572348  
1319570586 https://github.com/simonw/datasette/issues/1897#issuecomment-1319570586 https://api.github.com/repos/simonw/datasette/issues/1897 IC_kwDOBm6k_c5Opwya simonw 9599 2022-11-18T05:33:20Z 2022-11-18T05:33:20Z OWNER

One of the big changes still left to do for Datasette 1.0 is to unify the JSON representation with the context psssed to the templates (via an ?_extra= mechanism to add extra context needed by the HTML templates), because a goal for 1.0 is for the template context to be a documented API contract such that custom templates won't break with future releases.

As such I expect to do quite a bit of refactoring and cleanup on how the template context works later on.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Serve schema JSON to the SQL editor to enable autocomplete 1452457263  
1319533445 https://github.com/simonw/datasette/issues/1897#issuecomment-1319533445 https://api.github.com/repos/simonw/datasette/issues/1897 IC_kwDOBm6k_c5OpnuF bgrins 95570 2022-11-18T04:38:03Z 2022-11-18T04:38:03Z CONTRIBUTOR

Are you tracking the change to send the JSON over to the frontend separately or was that part of this? Something like this is probably pretty close https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25#diff-0c93232bfd5477eeac96382e52769108b41433d960d5277ffcccf2f464e60abdR9

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Serve schema JSON to the SQL editor to enable autocomplete 1452457263  
1319528359 https://github.com/simonw/datasette/issues/1901#issuecomment-1319528359 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5Opmen simonw 9599 2022-11-18T04:27:00Z 2022-11-18T04:27:00Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319483555 https://github.com/simonw/datasette/issues/1901#issuecomment-1319483555 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5Opbij simonw 9599 2022-11-18T03:02:35Z 2022-11-18T03:02:35Z OWNER

Looks like this issue could affect a bunch of other plugins too: https://cs.github.com/?scopeName=All+repos&scope=&q=%3Cp+class%3D%22crumbs%22%3E+user%3Asimonw

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319482791 https://github.com/simonw/datasette/issues/1901#issuecomment-1319482791 https://api.github.com/repos/simonw/datasette/issues/1901 IC_kwDOBm6k_c5OpbWn simonw 9599 2022-11-18T03:01:36Z 2022-11-18T03:01:36Z OWNER

Good catch. Looks like that bug was introduced by this change: https://github.com/simonw/datasette/commit/1a5e5f2aa951e5bd731067a49819efba68fbe8ef

From:

The search all plugin includes this code which interacts poorly with that refactor:

https://github.com/simonw/datasette-search-all/blob/847b55c368a285e4567627029624d7872ee75cac/datasette_search_all/templates/search_all.html#L31-L36

html+jinja {% block nav %} <p class="crumbs"> <a href="{{ urls.instance() }}">home</a> </p> {{ super() }} {% endblock %}

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Some plugins show "home" breadcrumbs twice in the top left 1453813400  
1319478811 https://github.com/simonw/datasette/issues/1897#issuecomment-1319478811 https://api.github.com/repos/simonw/datasette/issues/1897 IC_kwDOBm6k_c5OpaYb simonw 9599 2022-11-18T02:53:57Z 2022-11-18T02:53:57Z OWNER

I decided to just go for the view names, not their columns.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Serve schema JSON to the SQL editor to enable autocomplete 1452457263  
1319477721 https://github.com/simonw/datasette/issues/1897#issuecomment-1319477721 https://api.github.com/repos/simonw/datasette/issues/1897 IC_kwDOBm6k_c5OpaHZ simonw 9599 2022-11-18T02:51:40Z 2022-11-18T02:51:40Z OWNER

Views aren't currently available in the _internal schema.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Serve schema JSON to the SQL editor to enable autocomplete 1452457263  
1319435374 https://github.com/simonw/datasette/issues/1897#issuecomment-1319435374 https://api.github.com/repos/simonw/datasette/issues/1897 IC_kwDOBm6k_c5OpPxu simonw 9599 2022-11-18T01:33:30Z 2022-11-18T01:33:30Z OWNER

Just noticed that this isn't including views, which it should.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Serve schema JSON to the SQL editor to enable autocomplete 1452457263  
1319401843 https://github.com/simonw/datasette/issues/1897#issuecomment-1319401843 https://api.github.com/repos/simonw/datasette/issues/1897 IC_kwDOBm6k_c5OpHlz simonw 9599 2022-11-18T00:42:03Z 2022-11-18T00:42:23Z OWNER

This function works even if the SQLite JSON functions are not available:

python async def _table_columns(datasette, database_name): internal = datasette.get_database("_internal") result = await internal.execute( "select table_name, name from columns where database_name = ?", [database_name], ) table_columns = {} for row in result.rows: table_columns.setdefault(row["table_name"], []).append(row["name"]) return table_columns

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Serve schema JSON to the SQL editor to enable autocomplete 1452457263  
1318897922 https://github.com/simonw/datasette/issues/1899#issuecomment-1318897922 https://api.github.com/repos/simonw/datasette/issues/1899 IC_kwDOBm6k_c5OnMkC bgrins 95570 2022-11-17T16:32:42Z 2022-11-17T16:32:42Z CONTRIBUTOR

Another idea would be to just not set a min-height and allow the 1 line input to be 1 line heigh

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused  1452495049  
1297788531 https://github.com/simonw/sqlite-utils/pull/508#issuecomment-1297788531 https://api.github.com/repos/simonw/sqlite-utils/issues/508 IC_kwDOCGYnMM5NWq5z chapmanjacobd 7908073 2022-10-31T22:54:33Z 2022-11-17T15:11:16Z CONTRIBUTOR

Maybe this is actually a problem in the python sqlite bindings. Given SQLITE's stance on this they should probably use encode('utf-8', 'surrogatepass'). As far as I understand the error here won't actually be resolved by this PR as-is. We would need to modify the data with surrogateescape... :/ or modify the sqlite3 module to use surrogatepass

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Allow surrogates in parameters 1430563092  

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);