html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/simonw/datasette/issues/1464#issuecomment-917642487,https://api.github.com/repos/simonw/datasette/issues/1464,917642487,IC_kwDOBm6k_c42shz3,51016,2021-09-12T14:03:09Z,2021-09-12T14:03:09Z,CONTRIBUTOR,"haven't had time to get back to this, but idle thought that I'm recording for later investigation: how does the continuous integration handle this installation issue? Is it documented there?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",991191951, https://github.com/simonw/datasette/issues/1464#issuecomment-918621705,https://api.github.com/repos/simonw/datasette/issues/1464,918621705,IC_kwDOBm6k_c42wQ4J,7476523,2021-09-13T22:17:17Z,2021-09-13T22:17:17Z,CONTRIBUTOR,"> haven't had time to get back to this, but idle thought that I'm recording for later investigation: how does the continuous integration handle this installation issue? Is it documented there? Not certain, but I think tests in CI run on Ubuntu and don't appear to install any additional Sqlite-related dependencies, and so my guess is the version of Sqlite installed by default on Ubuntu has the `SQLITE_ENABLE_FTS3_PARENTHESIS` option enabled and so doesn't run into this issue.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",991191951, https://github.com/simonw/datasette/issues/1473#issuecomment-922363640,https://api.github.com/repos/simonw/datasette/issues/1473,922363640,IC_kwDOBm6k_c42-ib4,192568,2021-09-18T19:45:47Z,2021-09-18T19:45:47Z,CONTRIBUTOR,"An update, if I remove the `img` tag and replace it with the text, ""Safer or Toxic?"" it links to the right place. Also, if I keep things exactly as they are, and it improperly, but consistently goes to the `undefined` page, on THAT 404 page, a click on the image properly clicks through to the www.SaferOrToxic.org page. Weird stuff.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",999902754, https://github.com/simonw/datasette/issues/1473#issuecomment-922394999,https://api.github.com/repos/simonw/datasette/issues/1473,922394999,IC_kwDOBm6k_c42-qF3,192568,2021-09-19T00:44:39Z,2021-09-19T00:45:32Z,CONTRIBUTOR,"I replaced: ``` ``` with: ``` ``` I'd still love to know what caused this (and how to troubleshoot to figure it out), so I'll leave it open for a bit, but I do have a functional logo linking to the Hugo home page, at least locally. I'll likely push tomorrow. (Before trying this, I tried to apply a background image to the `a` tag. That didn't work.) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",999902754, https://github.com/simonw/datasette/issues/1480#issuecomment-1268613335,https://api.github.com/repos/simonw/datasette/issues/1480,1268613335,IC_kwDOBm6k_c5LnYDX,536941,2022-10-05T15:45:49Z,2022-10-05T15:45:49Z,CONTRIBUTOR,"running into this as i continue to grow my labor data warehouse. Here a CloudRun PM says the container size should **not** count against memory: https://stackoverflow.com/a/56570717","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369, https://github.com/simonw/datasette/issues/1480#issuecomment-1268629159,https://api.github.com/repos/simonw/datasette/issues/1480,1268629159,IC_kwDOBm6k_c5Lnb6n,536941,2022-10-05T16:00:55Z,2022-10-05T16:00:55Z,CONTRIBUTOR,"as a next step, i'll fetch the docker image from the google registry, and see what memory and disk usage looks like when i run it locally.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369, https://github.com/simonw/datasette/issues/1480#issuecomment-1269847461,https://api.github.com/repos/simonw/datasette/issues/1480,1269847461,IC_kwDOBm6k_c5LsFWl,536941,2022-10-06T11:21:49Z,2022-10-06T11:21:49Z,CONTRIBUTOR,"thanks @simonw, i'll spend a little more time trying to figure out why this isn't working on cloudrun, and then will flip over to fly if i can't. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369, https://github.com/simonw/datasette/issues/1480#issuecomment-1271101072,https://api.github.com/repos/simonw/datasette/issues/1480,1271101072,IC_kwDOBm6k_c5Lw3aQ,536941,2022-10-07T04:39:10Z,2022-10-07T04:39:10Z,CONTRIBUTOR,switching from `immutable=1` to `mode=ro` completely addressed this. see https://github.com/simonw/datasette/issues/1836#issuecomment-1271100651 for details.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369, https://github.com/simonw/datasette/issues/1480#issuecomment-938171377,https://api.github.com/repos/simonw/datasette/issues/1480,938171377,IC_kwDOBm6k_c4361vx,110420,2021-10-07T21:33:12Z,2021-10-07T21:33:12Z,CONTRIBUTOR,"Thanks for the reply @simonw. What services have you had better success with than Cloud Run for larger database? Also, what about my issue description makes you think there may be a workaround? Is there any instrumentation I could add to see at which point in the deploy the memory usage spikes? Should I be able to see this whether it's running under Docker locally, or do you suspect this is Cloud Run-specific?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369, https://github.com/simonw/datasette/issues/1480#issuecomment-947196177,https://api.github.com/repos/simonw/datasette/issues/1480,947196177,IC_kwDOBm6k_c44dRER,110420,2021-10-20T00:05:10Z,2021-10-20T00:05:10Z,CONTRIBUTOR,"I was looking through the Dockerfile-generation code to see if there was anything that would cause memory usage to be a lot during deployment. I noticed that the Dockerfile [runs `datasette --inspect`](https://github.com/simonw/datasette/blob/main/datasette/utils/__init__.py#L354). Is it possible that this is using a lot of memory usage? Or would that come into play when running `gcloud builds submit`, not when it's actually deployed?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369, https://github.com/simonw/datasette/issues/1480#issuecomment-947203725,https://api.github.com/repos/simonw/datasette/issues/1480,947203725,IC_kwDOBm6k_c44dS6N,110420,2021-10-20T00:21:54Z,2021-10-20T00:21:54Z,CONTRIBUTOR,"This StackOverflow post, [sqlite - Cloud Run: Why does my instance need so much RAM?](https://stackoverflow.com/questions/59812405/cloud-run-why-does-my-instance-need-so-much-ram), points to [this section of the Cloud Run docs](https://cloud.google.com/run/docs/troubleshooting) that says: > Note that the Cloud Run container instances run in an environment where the files written to the local filesystem count towards the available memory. This also includes any log files that are not written to /var/log/* or /dev/log. Does datasette write any large files when starting? Or does the [`COPY` command in the Dockerfile](https://github.com/simonw/datasette/blob/main/datasette/utils/__init__.py#L349) count as writing to the local filesystem?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369, https://github.com/simonw/datasette/issues/1522#issuecomment-976117989,https://api.github.com/repos/simonw/datasette/issues/1522,976117989,IC_kwDOBm6k_c46LmDl,813732,2021-11-23T03:00:34Z,2021-11-23T03:00:34Z,CONTRIBUTOR,"I tried deploying the most recent version of the Dockerfile in this thread ([link to comment](https://github.com/simonw/datasette/issues/1522#issuecomment-974605128)), and after trying a few different different combinations, I was only successful when I used `--no-cpu-throttling` (""CPU Is always allocated"" in the UI) Using this method, I got a very similar issue to you: The first time I'd load the site I'd get a 503. But after that first load, I didn't get the issue again. It would re-occur if the service started from cold boot. I suspect this is a race condition in the supervisord configuration. The errors I got were the same `Connection refused: AH00957: http: attempt to connect to 127.0.0.1:8001 (127.0.0.1) failed`, and that seems to indicate that `datasette` hadn't yet started. Looking at the order of logs getting back, the processes reported successfully completing loading after the first 503 was returned, so that makes me think race condition. I can replicate this locally, if I `docker run` and request `localhost:5000/prefix` _before_ I get the `datasette entered RUNNING state` message. Cloud Run wakes up when requests are received, so this test would semi-replicate that, but local docker would be the equivalent of a persistent process, hence it doesn't normally exhibit the same issues. Unfortunately supervisor/supervisor issue 122 (not linking as to prevent cross-project link spam) seems to say that dependency chaining is a feature that's been asked for for a long time, but hasn't been implemented. You could try some suggestions in that thread. ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1058896236, https://github.com/simonw/datasette/issues/1528#issuecomment-1151887842,https://api.github.com/repos/simonw/datasette/issues/1528,1151887842,IC_kwDOBm6k_c5EqGni,25778,2022-06-10T03:23:08Z,2022-06-10T03:23:08Z,CONTRIBUTOR,I just put together a version of this in a plugin: https://github.com/eyeseast/datasette-query-files. Happy to have any feedback.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1060631257, https://github.com/simonw/datasette/issues/1528#issuecomment-975955589,https://api.github.com/repos/simonw/datasette/issues/1528,975955589,IC_kwDOBm6k_c46K-aF,15178711,2021-11-22T22:00:30Z,2021-11-22T22:00:30Z,CONTRIBUTOR,"Oh, another thing to consider: I believe this would be the first `""_file""` key in datasette's metadata, compared to other `""_url""` keys like `""license_url""` or `""about_url""`. Not too sure what considerations to include with this (ex should missing files cause Datasette to stop before starting, should build scripts bundle these sql files somewhere during `datasette package`, etc.)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1060631257, https://github.com/simonw/datasette/issues/1547#issuecomment-997511968,https://api.github.com/repos/simonw/datasette/issues/1547,997511968,IC_kwDOBm6k_c47dNMg,127565,2021-12-20T01:21:59Z,2021-12-20T01:21:59Z,CONTRIBUTOR,"I've installed the alpha version but get an error when starting up Datasette: ``` Traceback (most recent call last): File ""/Users/tim/.pyenv/versions/stock-exchange/bin/datasette"", line 5, in from datasette.cli import cli File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py"", line 15, in from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py"", line 31, in from .views.database import DatabaseDownload, DatabaseView File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py"", line 25, in from datasette.plugins import pm File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py"", line 29, in mod = importlib.import_module(plugin) File ""/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py"", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py"", line 9, in @hookimpl(specname=""filters_from_request"") TypeError: __call__() got an unexpected keyword argument 'specname' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1076388044, https://github.com/simonw/datasette/issues/1547#issuecomment-997519202,https://api.github.com/repos/simonw/datasette/issues/1547,997519202,IC_kwDOBm6k_c47dO9i,127565,2021-12-20T01:36:58Z,2021-12-20T01:36:58Z,CONTRIBUTOR,"Yep, that works -- thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1076388044, https://github.com/simonw/datasette/issues/1549#issuecomment-1087428593,https://api.github.com/repos/simonw/datasette/issues/1549,1087428593,IC_kwDOBm6k_c5A0Nfx,536941,2022-04-04T11:17:13Z,2022-04-04T11:17:13Z,CONTRIBUTOR,"another way to get the behavior of downloading the file is to use the download attribute of the anchor tag https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a#attr-download","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1077620955, https://github.com/simonw/datasette/issues/1549#issuecomment-991754237,https://api.github.com/repos/simonw/datasette/issues/1549,991754237,IC_kwDOBm6k_c47HPf9,536941,2021-12-11T19:14:39Z,2021-12-11T19:14:39Z,CONTRIBUTOR,"that option is not available on [custom queries](https://labordata.bunkum.us/odpr-962a140?sql=with+local_union_filings+as+%28%0D%0A++select+*+from+lm_data+%0D%0A++where%0D%0A++++yr_covered+%3E+cast%28strftime%28%27%25Y%27%2C+%27now%27%2C+%27-5+years%27%29+as+int%29%0D%0A++++and+desig_name+%3D+%27LU%27%0D%0A++order+by+yr_covered+desc%0D%0A%29%2C%0D%0Amost_recent_filing+as+%28%0D%0A++select%0D%0A++++*%0D%0A++from+local_union_filings%0D%0A++group+by%0D%0A++++f_num%0D%0A%29%0D%0Aselect%0D%0A++*%0D%0Afrom%0D%0A++most_recent_filing%0D%0Awhere%0D%0A++next_election+%3E%3D+strftime%28%27%25Y-%25m%27%2C+%27now%27%29%0D%0A++and+next_election+%3C+strftime%28%27%25Y-%25m%27%2C+%27now%27%2C+%27%2B1+year%27%29%0D%0Aorder+by%0D%0A++members+desc%3B). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1077620955, https://github.com/simonw/datasette/issues/1552#issuecomment-995296725,https://api.github.com/repos/simonw/datasette/issues/1552,995296725,IC_kwDOBm6k_c47UwXV,3556,2021-12-15T23:29:32Z,2021-12-15T23:29:32Z,CONTRIBUTOR,"@simonw thank you for your fast answer and your guidance! While digging into the code, I found an undocumented way of doing it: ```yaml facets: [""Facet for a column"", {""array"": ""Facet for an array""}] ``` The only remaining problem with that solution is here: https://github.com/simonw/datasette/blob/250db8192cb8aba5eb8cd301ccc2a49525bc3d24/datasette/facets.py#L33 We have: ```python type, metadata_config = metadata_config.items()[0] ``` But it requires to cast the `dict_items` as a list prior to access the first element: ```python type, metadata_config = list(metadata_config.items())[0] ``` I guess it's an unspotted bug? (I mean, independently of the facets-with-arrays issue.)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1078702875, https://github.com/simonw/datasette/issues/1552#issuecomment-996229007,https://api.github.com/repos/simonw/datasette/issues/1552,996229007,IC_kwDOBm6k_c47YT-P,3556,2021-12-16T22:04:39Z,2021-12-16T22:04:39Z,CONTRIBUTOR,"Wow, that was fast, thank you so much @simonw ! > I'm also not convinced that this configuration syntax is right. It's a bit weird having a `""facets""` list that can either by column-name-strings or `{""type-of-facet"": ""column-name""}` objects. Maybe there's a better design for this? I agree that it's not ideal, my initial naive approach was to detect if it's an array, like what is done here: https://github.com/simonw/datasette/blob/2c07327d23d9c5cf939ada9ba4091c1b8b2ba42d/datasette/facets.py#L312-L313 But it requires an extra query to determine the type, which is a bit problematic, especially for big tables I guess. Taking a look at #510, I wonder if a `facet_delimiter` should be defined for that kind of columns (that would help our team not to have an intermediary conversion step from `foo|bar` to `[""foo"",""bar""]` for instance). To be consistent with the `--extract-column` parameter, maybe an explicit casting/delimiter would be useful: `--set-column 'Foo:Array:|'`. Throwing a lot of ideas without knowing the big picture… but sometimes newcomers have superpowers :).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1078702875, https://github.com/simonw/datasette/issues/1553#issuecomment-992986587,https://api.github.com/repos/simonw/datasette/issues/1553,992986587,IC_kwDOBm6k_c47L8Xb,536941,2021-12-13T22:57:04Z,2021-12-13T22:57:04Z,CONTRIBUTOR,would also be good if the header said the what the max row limit was,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1079111498, https://github.com/simonw/datasette/issues/1553#issuecomment-993014772,https://api.github.com/repos/simonw/datasette/issues/1553,993014772,IC_kwDOBm6k_c47MDP0,536941,2021-12-13T23:46:18Z,2021-12-13T23:46:18Z,CONTRIBUTOR,these headers would also be relevant for json exports of custom queries,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1079111498, https://github.com/simonw/datasette/issues/1561#issuecomment-997128712,https://api.github.com/repos/simonw/datasette/issues/1561,997128712,IC_kwDOBm6k_c47bvoI,536941,2021-12-18T02:35:48Z,2021-12-18T02:35:48Z,CONTRIBUTOR,interesting! i love this feature. this + full caching with cloudflare is really super!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1082765654, https://github.com/simonw/datasette/issues/1581#issuecomment-1077047295,https://api.github.com/repos/simonw/datasette/issues/1581,1077047295,IC_kwDOBm6k_c5AMm__,536941,2022-03-24T04:08:18Z,2022-03-24T04:08:18Z,CONTRIBUTOR,this has been addressed by the datasette-hashed-urls plugin,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1089529555, https://github.com/simonw/datasette/issues/1583#issuecomment-1002825217,https://api.github.com/repos/simonw/datasette/issues/1583,1002825217,IC_kwDOBm6k_c47xeYB,536941,2021-12-30T00:34:16Z,2021-12-30T00:34:16Z,CONTRIBUTOR,"if that is not desirable, it might be good to document that users might want to set up a lifecycle rule to automatically delete these build artifacts. something like https://stackoverflow.com/questions/59937542/can-i-delete-container-images-from-google-cloud-storage-artifacts-bucket","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1090810196, https://github.com/simonw/datasette/issues/1591#issuecomment-1010947634,https://api.github.com/repos/simonw/datasette/issues/1591,1010947634,IC_kwDOBm6k_c48QdYy,82988,2022-01-12T11:32:17Z,2022-01-12T11:32:17Z,CONTRIBUTOR,Is it possible to parse things like `--ext-{plugin}-{arg} VALUE` ?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1100015398, https://github.com/simonw/datasette/issues/160#issuecomment-459915995,https://api.github.com/repos/simonw/datasette/issues/160,459915995,MDEyOklzc3VlQ29tbWVudDQ1OTkxNTk5NQ==,82988,2019-02-02T00:43:16Z,2019-02-02T00:58:20Z,CONTRIBUTOR,"Do you have any simple working examples of how to use `--static`? Inspection of default served files suggests locations such as `http://example.com/-/static/app.css?0e06ee`. If `datasette` is being proxied to `http://example.com/foo/datasette`, what form should arguments to `--static` take so that static files are correctly referenced? Use case is here: https://github.com/psychemedia/jupyterserverproxy-datasette-demo Trying to do a really simple `datasette` demo in MyBinder using jupyter-server-proxy.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",278208011, https://github.com/simonw/datasette/issues/1601#issuecomment-1016651485,https://api.github.com/repos/simonw/datasette/issues/1601,1016651485,IC_kwDOBm6k_c48mN7d,25778,2022-01-19T16:39:03Z,2022-01-19T16:39:03Z,CONTRIBUTOR,"I think both of these are Spatialite specific. They get generated when you first initialize the extension. KNN is actually deprecated in favor of [KNN2](https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN2), as I understand it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1105916061, https://github.com/simonw/datasette/issues/1605#issuecomment-1016994329,https://api.github.com/repos/simonw/datasette/issues/1605,1016994329,IC_kwDOBm6k_c48nhoZ,25778,2022-01-20T00:27:17Z,2022-01-20T00:27:17Z,CONTRIBUTOR,"Right now, I usually have a line in a Makefile like this: ```make combined.geojson: project.db pipenv run datasette project.db --get /project/combined.geojson \ --load-extension spatialite \ --setting sql_time_limit_ms 5000 \ --setting max_returned_rows 20000 \ -m metadata.yml > $@ ``` That all assumes I've loaded whatever I need into `project.db` and created a canned query called `combined` (and then uses `datasette-geojson` for geojson output). It works, but as you can see, it's a lot to manage, a lot of boilerplate, and it wasn't obvious how to get there. If there's an error in the canned query, I get an HTML error page, so that's hard to debug. And it's only one query, so each output needs a line like this. Make isn't ideal, either, for that reason. The thing I really liked with `datafreeze` was doing templated filenames. I have a project now where I need to export a bunch of litttle geojson files, based on queries, and it would be awesome to be able to do something like this: ```yml databases: project: queries: boundaries: sql: ""SELECT * FROM boundaries"" filename: ""boundaries/{id}.geojson"" mode: ""item"" format: geojson ``` And then do: ```sh datasette freeze -m metadata.yml project.db ``` For HTML export, maybe there's a `template` argument, or `format: template` or something. And that gets you a static site generator, kinda for free. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1108671952, https://github.com/simonw/datasette/issues/1605#issuecomment-1018741262,https://api.github.com/repos/simonw/datasette/issues/1605,1018741262,IC_kwDOBm6k_c48uMIO,25778,2022-01-21T18:05:09Z,2022-01-21T18:05:09Z,CONTRIBUTOR,"Thinking about this more, as well as #1356 and various other tickets related to output formats, I think there's a missing plugin hook for formatting results, separate from `register_output_renderer` (or maybe part of it, depending on #1101). Right now, as I understand it, getting output in any format goes through the normal view stack -- a table, a row or a query -- and so by the time `register_output_renderer` gets it, the results have already been truncated or paginated. What I'd want, I think, is to be able to register ways to format results independent of where those results are sent. It's possible this could be done using [`conn.row_factory`](https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.row_factory) (maybe in the `prepare_connection` hook), but I'm not sure that's where it belongs. Another option is some kind of registry of serializers, which `register_output_renderer` and other plugin hooks could use. What I'm trying to avoid here is writing a plugin that also needs plugins for formats I haven't thought of yet.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1108671952, https://github.com/simonw/datasette/issues/1605#issuecomment-1018778667,https://api.github.com/repos/simonw/datasette/issues/1605,1018778667,IC_kwDOBm6k_c48uVQr,25778,2022-01-21T19:00:01Z,2022-01-21T19:00:01Z,CONTRIBUTOR,"Let me know if you want help prototyping any of this, because I'm thinking about it and trying stuff out. Happy to be a sounding board, if it helps.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1108671952, https://github.com/simonw/datasette/issues/1605#issuecomment-1331187551,https://api.github.com/repos/simonw/datasette/issues/1605,1331187551,IC_kwDOBm6k_c5PWE9f,25778,2022-11-29T19:29:42Z,2022-11-29T19:29:42Z,CONTRIBUTOR,"Interesting. I started a version using metadata like I outlined up top, but I realized that there's no documented way for a plugin to access either metadata or canned queries. Or at least, I couldn't find a way. There is this method: https://github.com/simonw/datasette/blob/main/datasette/app.py#L472 but I don't want to rely on it if it's not documented. Same with this: https://github.com/simonw/datasette/blob/main/datasette/app.py#L544 If those are safe, I'll build on them. I'm also happy to document them, if that greases the wheels.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1108671952, https://github.com/simonw/datasette/issues/1605#issuecomment-1332310772,https://api.github.com/repos/simonw/datasette/issues/1605,1332310772,IC_kwDOBm6k_c5PaXL0,25778,2022-11-30T15:06:37Z,2022-11-30T15:06:37Z,CONTRIBUTOR,I'll add issues for both and do a documentation PR.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1108671952, https://github.com/simonw/datasette/issues/1612#issuecomment-1021497165,https://api.github.com/repos/simonw/datasette/issues/1612,1021497165,IC_kwDOBm6k_c484s9N,639012,2022-01-25T18:44:23Z,2022-01-25T18:44:23Z,CONTRIBUTOR,"OMG, this might be the fastest OS ticket I've ever filed, thanks so much @simonw ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1114147905, https://github.com/simonw/datasette/issues/1614#issuecomment-1364345119,https://api.github.com/repos/simonw/datasette/issues/1614,1364345119,IC_kwDOBm6k_c5RUkEf,536941,2022-12-23T21:27:10Z,2022-12-23T21:27:10Z,CONTRIBUTOR,is this issue closed by #1893?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1115435536, https://github.com/simonw/datasette/issues/163#issuecomment-804539729,https://api.github.com/repos/simonw/datasette/issues/163,804539729,MDEyOklzc3VlQ29tbWVudDgwNDUzOTcyOQ==,192568,2021-03-23T02:41:14Z,2021-03-23T02:41:14Z,CONTRIBUTOR,"I'm visiting old issues for context while learning datasette. Let me know if okay to make the occasional comment like this one. querystring argument now located at: https://docs.datasette.io/en/latest/settings.html#sql-time-limit-ms","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",279547886, https://github.com/simonw/datasette/issues/164#issuecomment-804541064,https://api.github.com/repos/simonw/datasette/issues/164,804541064,MDEyOklzc3VlQ29tbWVudDgwNDU0MTA2NA==,192568,2021-03-23T02:45:12Z,2021-03-23T02:45:12Z,CONTRIBUTOR,"""datasette skeleton"" feature removed #476","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",280013907, https://github.com/simonw/datasette/issues/1641#issuecomment-1049879118,https://api.github.com/repos/simonw/datasette/issues/1641,1049879118,IC_kwDOBm6k_c4-k-JO,536941,2022-02-24T13:49:26Z,2022-02-24T13:49:26Z,CONTRIBUTOR,"maybe worth considering adding buttons for paren, asterisk, etc. under the input text box on mobile?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1149310456, https://github.com/simonw/datasette/issues/1655#issuecomment-1062450649,https://api.github.com/repos/simonw/datasette/issues/1655,1062450649,IC_kwDOBm6k_c4_U7XZ,536941,2022-03-09T01:10:46Z,2022-03-09T01:10:46Z,CONTRIBUTOR,"i increased the max_returned_row, because I have some scripts that get CSVs from this site, and this makes doing pagination of CSVs less annoying for many cases. i know that's streaming csvs is something you are hoping to address in 1.0. let me know if there's anything i can do to help with that. as for what if anything can be done about the size of the dom, I don't have any ideas right now, but i'll poke around.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1163369515, https://github.com/simonw/datasette/issues/1655#issuecomment-1258166572,https://api.github.com/repos/simonw/datasette/issues/1655,1258166572,IC_kwDOBm6k_c5K_hks,536941,2022-09-26T14:57:04Z,2022-09-26T14:57:04Z,CONTRIBUTOR,"I think that paginating, even in javascript, could be very helpful. Maybe render json or csv into the page and let javascript loading that into the dom?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1163369515, https://github.com/simonw/datasette/issues/1655#issuecomment-1766994810,https://api.github.com/repos/simonw/datasette/issues/1655,1766994810,IC_kwDOBm6k_c5pUjN6,536941,2023-10-17T19:01:59Z,2023-10-17T19:01:59Z,CONTRIBUTOR,"hi @yejiyang, have your tried using my fork of datasette: https://github.com/fgregg/datasette/tree/no_limit_csv_publish ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1163369515, https://github.com/simonw/datasette/issues/1655#issuecomment-1767219901,https://api.github.com/repos/simonw/datasette/issues/1655,1767219901,IC_kwDOBm6k_c5pVaK9,536941,2023-10-17T21:29:03Z,2023-10-17T21:29:03Z,CONTRIBUTOR,@yejiyang why don’t you move this discussion to my fork to spare simon’s notifications ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1163369515, https://github.com/simonw/datasette/issues/1684#issuecomment-1078126065,https://api.github.com/repos/simonw/datasette/issues/1684,1078126065,IC_kwDOBm6k_c5AQuXx,536941,2022-03-24T20:08:56Z,2022-03-24T20:13:19Z,CONTRIBUTOR,"would be nice if the behavior was 1. try to facet all the columns 2. for bigger tables try to facet the indexed columns 3. for the biggest tables, turn off autofacetting completely This is based on my assumption that what determines autofaceting is the rarity of unique values. Which may not be true!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1179998071, https://github.com/simonw/datasette/issues/1688#issuecomment-1079550754,https://api.github.com/repos/simonw/datasette/issues/1688,1079550754,IC_kwDOBm6k_c5AWKMi,9020979,2022-03-26T01:27:27Z,2022-03-26T03:16:29Z,CONTRIBUTOR,"> Is there a way to serve a static assets when using the plugins/ directory method instead of installing plugins as a new python package? As a workaround, I found I can serve my statics from a non-plugin specific folder using the [--static](https://docs.datasette.io/en/stable/custom_templates.html#serving-static-files) CLI flag. ```bash datasette ~/Library/Safari/History.db \ --plugins-dir=plugins/ \ --static assets:dist/ ``` It's not ideal because it means I'll change the cache pattern path depending on how the plugin is running (via pip install or as a one off script), but it's usable as a workaround. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1181432624, https://github.com/simonw/datasette/issues/1688#issuecomment-1079806857,https://api.github.com/repos/simonw/datasette/issues/1688,1079806857,IC_kwDOBm6k_c5AXIuJ,9020979,2022-03-27T01:01:14Z,2022-03-27T01:01:14Z,CONTRIBUTOR,"Thank you! I went through the cookiecutter template, and published my first package here: https://github.com/hydrosquall/datasette-nteract-data-explorer","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1181432624, https://github.com/simonw/datasette/issues/1696#issuecomment-1407767434,https://api.github.com/repos/simonw/datasette/issues/1696,1407767434,IC_kwDOBm6k_c5T6NOK,193185,2023-01-29T20:56:20Z,2023-01-29T20:56:20Z,CONTRIBUTOR,"I did some horrible things in https://github.com/cldellow/datasette-ui-extras/issues/2 to enable this in my plugin -- example here: https://dux-demo.fly.dev/cooking/posts?_facet=owner_user_id&owner_user_id=67 The implementation relies on two things: - a `filters_from_request` hook that adds a good human description (unfortunately, without the benefit of the CSS styling you mention) - doing something evil to hijack the `exact` and `not` operators in the `Filters` class. We can't leave them as is, or we'll get 2 human descriptions -- the built-in Datasette one and the one from my plugin. We can't remove them, or the filters UI will stop supporting the `=` and `!=` operators This got me thinking: it'd be neat if the list of operators that the filters UI supported wasn't a closed set. A motivating example: adding a geospatial `NEAR` operator. Ideally it'd take two arguments - a target point and a radius, so you could express a filter like `find me all rows whose lat/lng are within 10km of 43.4516° N, 80.4925° W`. (Optionally, the UI could be enhanced if the geonames database was loaded and queried, so a user could say `find me all rows whose lat/lng are within 10km of Kitchener, ON`, and the city gets translated to a lat/lng for them)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1186696202, https://github.com/simonw/datasette/issues/1699#issuecomment-1092357672,https://api.github.com/repos/simonw/datasette/issues/1699,1092357672,IC_kwDOBm6k_c5BHA4o,25778,2022-04-08T01:39:40Z,2022-04-08T01:39:40Z,CONTRIBUTOR,"> My best thought on how to differentiate them so far is plugins: if Datasette plugins that provide alternative outputs - like .geojson and .yml and suchlike - also work for the datasette query command that would make a lot of sense to me. That's my thinking, too. It's really the thing I've been wanting since writing `datasette-geojson`, since I'm always exporting with `datasette --get`. The workflow I'm always looking for is something like this: ```sh cd alltheplaces-datasette datasette query dunkin_in_suffolk -f geojson -o dunkin_in_suffolk.geojson ``` I think this probably needs either a new plugin hook separate from `register_output_renderer` or a way to use that without going through the HTTP stack. Or maybe a render mode that writes to a stream instead of a response. Maybe there's a new key in the dictionary that `register_output_renderer` returns that handles CLI exports.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1193090967, https://github.com/simonw/datasette/issues/1699#issuecomment-1092370880,https://api.github.com/repos/simonw/datasette/issues/1699,1092370880,IC_kwDOBm6k_c5BHEHA,25778,2022-04-08T02:07:40Z,2022-04-08T02:07:40Z,CONTRIBUTOR,"So maybe `render_output_render` returns something like this: ```python @hookimpl def register_output_renderer(datasette): return { ""extension"": ""geojson"", ""render"": render_geojson, ""stream"": stream_geojson, ""can_render"": can_render_geojson, } ``` And stream gets an iterator, instead of a list of rows, so it can efficiently handle large queries. Maybe it also gets passed a destination stream, or it returns an iterator. I'm not sure what makes more sense. Either way, that might cover both CLI exports and streaming responses.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1193090967, https://github.com/simonw/datasette/issues/1699#issuecomment-1092386254,https://api.github.com/repos/simonw/datasette/issues/1699,1092386254,IC_kwDOBm6k_c5BHH3O,25778,2022-04-08T02:39:25Z,2022-04-08T02:39:25Z,CONTRIBUTOR,"And just to think this through a little more, here's what `stream_geojson` might look like: ```python async def stream_geojson(datasette, columns, rows, database, stream): db = datasette.get_database(database) for row in rows: feature = await row_to_geojson(row, db) stream.write(feature + ""\n"") # just assuming newline mode for now ``` Alternately, that could be an async generator, like this: ```python async def stream_geojson(datasette, columns, rows, database): db = datasette.get_database(database) for row in rows: feature = await row_to_geojson(row, db) yield feature ``` Not sure which makes more sense, but I think this pattern would open up a lot of possibility. If you had your [stream_indented_json](https://til.simonwillison.net/python/output-json-array-streaming) function, you could do `yield from stream_indented_json(rows, 2)` and be one your way.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1193090967, https://github.com/simonw/datasette/issues/1699#issuecomment-1094453751,https://api.github.com/repos/simonw/datasette/issues/1699,1094453751,IC_kwDOBm6k_c5BPAn3,25778,2022-04-11T01:32:12Z,2022-04-11T01:32:12Z,CONTRIBUTOR,"Was looking through old issues and realized a bunch of this got discussed in #1101 (including by me!), so sorry to rehash all this. Happy to help with whatever piece of it I can. Would be very excited to be able to use format plugins with exports.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1193090967, https://github.com/simonw/datasette/issues/1713#issuecomment-1099540225,https://api.github.com/repos/simonw/datasette/issues/1713,1099540225,IC_kwDOBm6k_c5BiacB,25778,2022-04-14T19:09:57Z,2022-04-14T19:09:57Z,CONTRIBUTOR,"I wonder if this overlaps with what I outlined in #1605. You could run something like this: ```sh datasette freeze -d exports/ aws s3 cp exports/ s3://my-export-bucket/$(date) ``` And maybe that does what you need. Of course, that plugin isn't built yet. But that's the idea.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1203943272, https://github.com/simonw/datasette/issues/1713#issuecomment-1103312860,https://api.github.com/repos/simonw/datasette/issues/1713,1103312860,IC_kwDOBm6k_c5Bwzfc,536941,2022-04-20T00:52:19Z,2022-04-20T00:52:19Z,CONTRIBUTOR,feels related to #1402 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1203943272, https://github.com/simonw/datasette/issues/1713#issuecomment-1173358747,https://api.github.com/repos/simonw/datasette/issues/1713,1173358747,IC_kwDOBm6k_c5F8Aib,2670795,2022-07-04T05:16:35Z,2022-07-04T05:16:35Z,CONTRIBUTOR,"This feature is pretty important and would be nice if it would be all within Datasette (no separate CLI/deploy required). My workflow now is to basically just copy the result and paste into a Google Sheet, which works, but then it's not discoverable to other journalists browsing the Datasette instance. I started building a plugin similar to [datasette-saved-queries](https://datasette.io/plugins/datasette-saved-queries) but one that maintains its own DB (required if you're working with all immutable DBs), but got bogged down in details.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1203943272, https://github.com/simonw/datasette/issues/1727#issuecomment-1258129113,https://api.github.com/repos/simonw/datasette/issues/1727,1258129113,IC_kwDOBm6k_c5K_YbZ,536941,2022-09-26T14:30:11Z,2022-09-26T14:48:31Z,CONTRIBUTOR,"from your analysis, it seems like the GIL is blocking on loading of the data from sqlite to python, (particularly in the `fetchmany` call) this is probably a simplistic idea, but what if you had the python code in the `execute` method iterate over the cursor and yield out rows or small chunks of rows. something like: ```python with sqlite_timelimit(conn, time_limit_ms): try: cursor = conn.cursor() cursor.execute(sql, params if params is not None else {}) except: ... max_returned_rows = self.ds.max_returned_rows if max_returned_rows == page_size: max_returned_rows += 1 if max_returned_rows and truncate: for i, row in enumerate(cursor): yield row if i == max_returned_rows - 1: break else: for row in cursor: yield row truncated = False ``` this kind of thing works well with a postgres server side cursor, but i'm not sure if it will hold for sqlite. you would still spend about the same amount of time in python and would be contending for the gil, but it would be could be non blocking. depending on the data flow, this could also some benefit for memory. (data stays in more compact sqlite-land until you need it)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1217759117, https://github.com/simonw/datasette/issues/1728#issuecomment-1111705323,https://api.github.com/repos/simonw/datasette/issues/1728,1111705323,IC_kwDOBm6k_c5CQ0br,127565,2022-04-28T03:32:06Z,2022-04-28T03:32:06Z,CONTRIBUTOR,"Ah, that would be it! I have a core set of data which doesn't change to which I want authorised users to be able to submit corrections. I was going to deal with the persistence issue by just grabbing the user corrections at regular intervals and saving to GitHub. I might need to rethink. Thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1218133366, https://github.com/simonw/datasette/issues/1728#issuecomment-1111712953,https://api.github.com/repos/simonw/datasette/issues/1728,1111712953,IC_kwDOBm6k_c5CQ2S5,127565,2022-04-28T03:48:36Z,2022-04-28T03:48:36Z,CONTRIBUTOR,"I don't think that'd work for this project. The db is very big, and my aim was to have an environment where researchers could be making use of the data, but be easily able to add corrections to the HTR/OCR extracted data when they came across problems. It's in its immutable (!) form here: https://sydney-stock-exchange-xqtkxtd5za-ts.a.run.app/stock_exchange/stocks","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1218133366, https://github.com/simonw/datasette/issues/1728#issuecomment-1111751734,https://api.github.com/repos/simonw/datasette/issues/1728,1111751734,IC_kwDOBm6k_c5CQ_w2,127565,2022-04-28T05:09:59Z,2022-04-28T05:09:59Z,CONTRIBUTOR,"Thanks, I'll give it a try!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1218133366, https://github.com/simonw/datasette/issues/1728#issuecomment-1111752676,https://api.github.com/repos/simonw/datasette/issues/1728,1111752676,IC_kwDOBm6k_c5CQ__k,127565,2022-04-28T05:11:54Z,2022-04-28T05:11:54Z,CONTRIBUTOR,"And in terms of the bug, yep I agree that option 2 would be the most useful and least frustrating.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1218133366, https://github.com/simonw/datasette/issues/1742#issuecomment-1128049716,https://api.github.com/repos/simonw/datasette/issues/1742,1128049716,IC_kwDOBm6k_c5DPKw0,25778,2022-05-16T19:24:44Z,2022-05-16T19:24:44Z,CONTRIBUTOR,"Where is `_trace` getting injected? And is it something a plugin should be able to handle? (If it is, I guess I should handle it in this case.)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1237586379, https://github.com/simonw/datasette/issues/1742#issuecomment-1128064864,https://api.github.com/repos/simonw/datasette/issues/1742,1128064864,IC_kwDOBm6k_c5DPOdg,25778,2022-05-16T19:42:13Z,2022-05-16T19:42:13Z,CONTRIBUTOR,"Just to add a wrinkle here, this loads fine: https://alltheplaces-datasette.fly.dev/alltheplaces/places.geojson?_trace=1 But also, this doesn't add any trace data: https://alltheplaces-datasette.fly.dev/alltheplaces/places.json?_trace=1 What am I missing?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1237586379, https://github.com/simonw/datasette/issues/1779#issuecomment-1210675046,https://api.github.com/repos/simonw/datasette/issues/1779,1210675046,IC_kwDOBm6k_c5IKW9m,536941,2022-08-10T13:28:37Z,2022-08-10T13:28:37Z,CONTRIBUTOR,maybe a simpler solution is to set the maxscale to like 2? since datasette is not set up to make use of container scaling anyway?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1334628400, https://github.com/simonw/datasette/issues/1779#issuecomment-1214437408,https://api.github.com/repos/simonw/datasette/issues/1779,1214437408,IC_kwDOBm6k_c5IYtgg,536941,2022-08-14T19:42:58Z,2022-08-14T19:42:58Z,CONTRIBUTOR,thanks @simonw!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1334628400, https://github.com/simonw/datasette/issues/179#issuecomment-360535979,https://api.github.com/repos/simonw/datasette/issues/179,360535979,MDEyOklzc3VlQ29tbWVudDM2MDUzNTk3OQ==,82988,2018-01-25T17:18:24Z,2018-01-25T17:18:24Z,CONTRIBUTOR,"To summarise that thread: - expose full `metadata.json` object to the index page template, eg to allow tables to be referred to by name; - ability to import multiple `metadata.json` files, eg to allow metadata files created for a specific SQLite db to be reused in a datasette referring to several database files; It could also be useful to allow users to import a python file containing custom functions that can that be loaded into scope and made available to custom templates. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",288438570, https://github.com/simonw/datasette/issues/1796#issuecomment-1364345071,https://api.github.com/repos/simonw/datasette/issues/1796,1364345071,IC_kwDOBm6k_c5RUkDv,536941,2022-12-23T21:27:02Z,2022-12-23T21:27:02Z,CONTRIBUTOR,@simonw is this issue closed by #1893?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1355148385, https://github.com/simonw/datasette/issues/1810#issuecomment-1248204219,https://api.github.com/repos/simonw/datasette/issues/1810,1248204219,IC_kwDOBm6k_c5KZhW7,82988,2022-09-15T14:44:47Z,2022-09-15T14:46:26Z,CONTRIBUTOR,"A couple+ of possible use case examples: - someone has a collection of articles indexed with FTS; they want to publish a simple search tool over the results; - someone has an image collection and they want to be able to search over description text to return images; - someone has a set of locations with descriptions, and wants to run a query over places and descriptions and get results as a listing or on a map; - someone has a set of audio or video files with titles, descriptions and/or transcripts, and wants to be able to search over them and return playable versions of returned items. In many cases, I suspect the raw content will be in one table, but the search table will be a second (eg FTS) table. Generally, the search may be over one or more joined tables, and the results constructed from one or more tables (which may or may not be distinct from the search tables).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1374626873, https://github.com/simonw/datasette/issues/1813#issuecomment-1250901367,https://api.github.com/repos/simonw/datasette/issues/1813,1250901367,IC_kwDOBm6k_c5Kjz13,883348,2022-09-19T11:34:45Z,2022-09-19T11:34:45Z,CONTRIBUTOR,"oh and by writing this I just realized the difference: the URL on fly.io is with a custom SQL command whereas the local one is without. It seems that there is no pagination when using custom SQL commands which makes sense Sorry for this useless issue, maybe this can be useful for someone else / me in the future. Thanks again for this wonderful project !","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1377811868, https://github.com/simonw/datasette/issues/1817#issuecomment-1256781274,https://api.github.com/repos/simonw/datasette/issues/1817,1256781274,IC_kwDOBm6k_c5K6PXa,50527,2022-09-23T22:59:46Z,2022-09-23T22:59:46Z,CONTRIBUTOR,"While you are adding features, would you be future-proofing your APIs if you switched over some arguments over to keyword-only arguments or would that be too disruptive? Thinking out loud: ``` async def render_template( self, templates, *, context=None, plugin_context=None, request=None, view_name=None ): ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1384273985, https://github.com/simonw/datasette/issues/1836#issuecomment-1270923537,https://api.github.com/repos/simonw/datasette/issues/1836,1270923537,IC_kwDOBm6k_c5LwMER,536941,2022-10-07T00:46:08Z,2022-10-07T00:46:08Z,CONTRIBUTOR,"i thought it was maybe to do with reading through all the files, but that does not seem to be the case if i make a little test file like: ```python # test_read.py import hashlib import sys import pathlib HASH_BLOCK_SIZE = 1024 * 1024 def inspect_hash(path): """"""Calculate the hash of a database, efficiently."""""" m = hashlib.sha256() with path.open(""rb"") as fp: while True: data = fp.read(HASH_BLOCK_SIZE) if not data: break m.update(data) return m.hexdigest() inspect_hash(pathlib.Path(sys.argv[1])) ``` then a line in the Dockerfile like ```docker RUN python test_read.py nlrb.db && echo ""[]"" > /etc/inspect.json ``` just produes a layer of `3B` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1270936982,https://api.github.com/repos/simonw/datasette/issues/1836,1270936982,IC_kwDOBm6k_c5LwPWW,536941,2022-10-07T00:52:41Z,2022-10-07T00:52:41Z,CONTRIBUTOR,"it's not that the inspect command is somehow changing the db files. if i set them to only read-only, the ""inspect"" layer still has the same very large size.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1270988081,https://api.github.com/repos/simonw/datasette/issues/1836,1270988081,IC_kwDOBm6k_c5Lwb0x,536941,2022-10-07T01:19:01Z,2022-10-07T01:27:35Z,CONTRIBUTOR,"okay, some progress!! running some sql against a database file causes that file to get duplicated even if it doesn't apparently change the file. make a little test script like this: ```python # test_sql.py import sqlite3 import sys db_name = sys.argv[1] conn = sqlite3.connect(f'file:/app/{db_name}', uri=True) cur = conn.cursor() cur.execute('select count(*) from filing') print(cur.fetchone()) ``` then ```docker RUN python test_sql.py nlrb.db ``` produced a layer that's the same size as `nlrb.db`!! ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1270992795,https://api.github.com/repos/simonw/datasette/issues/1836,1270992795,IC_kwDOBm6k_c5Lwc-b,536941,2022-10-07T01:29:15Z,2022-10-07T01:50:14Z,CONTRIBUTOR,"fascinatingly, telling python to open sqlite in read only mode makes this layer have a size of 0 ```python # test_sql_ro.py import sqlite3 import sys db_name = sys.argv[1] conn = sqlite3.connect(f'file:/app/{db_name}?mode=ro', uri=True) cur = conn.cursor() cur.execute('select count(*) from filing') print(cur.fetchone()) ``` that's quite weird because setting the file permissions to read only didn't do anything. (on reflection, that chmod isn't doing anything because the dockerfile commands are run as root)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1271003212,https://api.github.com/repos/simonw/datasette/issues/1836,1271003212,IC_kwDOBm6k_c5LwfhM,536941,2022-10-07T01:52:04Z,2022-10-07T01:52:04Z,CONTRIBUTOR,"and if we try immutable mode, which is how things are opened by `datasette inspect` we duplicate the files!!! ```python # test_sql_immutable.py import sqlite3 import sys db_name = sys.argv[1] conn = sqlite3.connect(f'file:/app/{db_name}?immutable=1', uri=True) cur = conn.cursor() cur.execute('select count(*) from filing') print(cur.fetchone()) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1271008997,https://api.github.com/repos/simonw/datasette/issues/1836,1271008997,IC_kwDOBm6k_c5Lwg7l,536941,2022-10-07T02:00:37Z,2022-10-07T02:00:49Z,CONTRIBUTOR,"yes, and i also think that this is causing the apparent memory problems in #1480. when the container starts up, it will make some operation on the database in `immutable` mode which apparently makes some small change to the db file. if that's so, then the db files will be copied to the read/write layer which counts against cloudrun's memory allocation! running a test of that now. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1271020193,https://api.github.com/repos/simonw/datasette/issues/1836,1271020193,IC_kwDOBm6k_c5Lwjqh,536941,2022-10-07T02:15:05Z,2022-10-07T02:21:08Z,CONTRIBUTOR,"when i hack the connect method to open non mutable files with ""mode=ro"" and not ""immutable=1"" https://github.com/simonw/datasette/blob/eff112498ecc499323c26612d707908831446d25/datasette/database.py#L79 then: ```bash 870 B RUN /bin/sh -c datasette inspect nlrb.db --inspect-file inspect-data.json ``` the `datasette inspect` layer is only the size of the json file!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1271100651,https://api.github.com/repos/simonw/datasette/issues/1836,1271100651,IC_kwDOBm6k_c5Lw3Tr,536941,2022-10-07T04:38:14Z,2022-10-07T04:38:14Z,CONTRIBUTOR,"> yes, and i also think that this is causing the apparent memory problems in #1480. when the container starts up, it will make some operation on the database in `immutable` mode which apparently makes some small change to the db file. if that's so, then the db files will be copied to the read/write layer which counts against cloudrun's memory allocation! > > running a test of that now. this completely addressed #1480 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1271103097,https://api.github.com/repos/simonw/datasette/issues/1836,1271103097,IC_kwDOBm6k_c5Lw355,536941,2022-10-07T04:43:41Z,2022-10-07T04:43:41Z,CONTRIBUTOR,"@simonw, should i open up a new issue for investigating the differences between ""immutable=1"" and ""mode=ro"" and possibly switching to ""mode=ro"". Or would you like to keep that conversation in this issue?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1836#issuecomment-1272357976,https://api.github.com/repos/simonw/datasette/issues/1836,1272357976,IC_kwDOBm6k_c5L1qRY,536941,2022-10-08T16:56:51Z,2022-10-08T16:56:51Z,CONTRIBUTOR,"when you are running from docker, you **always** will want to run as `mode=ro` because the same thing that is causing duplication in the inspect layer will cause duplication in the final container read/write layer when `datasette serve` runs.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908, https://github.com/simonw/datasette/issues/1851#issuecomment-1290615599,https://api.github.com/repos/simonw/datasette/issues/1851,1290615599,IC_kwDOBm6k_c5M7Tsv,25778,2022-10-25T14:05:12Z,2022-10-25T14:05:12Z,CONTRIBUTOR,"This could use a new plugin hook, too. I don't want to complicate your life too much, but for things like GIS, I'd want a way to turn regular JSON into SpatiaLite geometries or combine X/Y coordinates into point geometries and such. Happy to help however I can.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654, https://github.com/simonw/datasette/issues/1851#issuecomment-1291228502,https://api.github.com/repos/simonw/datasette/issues/1851,1291228502,IC_kwDOBm6k_c5M9pVW,25778,2022-10-25T23:02:10Z,2022-10-25T23:02:10Z,CONTRIBUTOR,That's reasonable. Canned queries and custom endpoints are certainly going to give more room for specific needs. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654, https://github.com/simonw/datasette/issues/1851#issuecomment-1292519956,https://api.github.com/repos/simonw/datasette/issues/1851,1292519956,IC_kwDOBm6k_c5NCkoU,15178711,2022-10-26T19:20:33Z,2022-10-26T19:20:33Z,CONTRIBUTOR,"> This could use a new plugin hook, too. I don't want to complicate your life too much, but for things like GIS, I'd want a way to turn regular JSON into SpatiaLite geometries or combine X/Y coordinates into point geometries and such. Happy to help however I can. @eyeseast Maybe you could do this with triggers? Like you can insert JSON-friendly data into a ""raw"" table, and create a trigger that transforms that inserted data into the proper table Here's an example: ```sql -- meant to be updated from a Datasette insert create table points_raw(longitude int, latitude int); -- the target table with proper spatliate geometries create table points(point geometry); CREATE TRIGGER insert_points_raw INSERT ON points_raw BEGIN insert into points(point) values (makepoint(new.longitude, new.latitude)) END; ``` You could then POST a new row to `points_raw` like this: ``` POST /db/points_raw Authorization: Bearer xxx Content-Type: application/json { ""row"": { ""longitude"": 27.64356, ""latitude"": -47.29384 } } ``` Then SQLite with run the trigger and insert a new row in `points` with the correct geometry point. Downside is you'd have duplicated data with `points_raw`, but maybe it could be a `TEMP` table (or have a cron that deletes all rows from that table every so often?)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654, https://github.com/simonw/datasette/issues/1851#issuecomment-1292592210,https://api.github.com/repos/simonw/datasette/issues/1851,1292592210,IC_kwDOBm6k_c5NC2RS,25778,2022-10-26T20:03:46Z,2022-10-26T20:03:46Z,CONTRIBUTOR,"Yeah, every time I see something cool done with triggers, I remember that I need to start using triggers.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654, https://github.com/simonw/datasette/issues/1871#issuecomment-1309650806,https://api.github.com/repos/simonw/datasette/issues/1871,1309650806,IC_kwDOBm6k_c5OD692,3556,2022-11-10T01:38:58Z,2022-11-10T01:38:58Z,CONTRIBUTOR,"> Realized the API explorer doesn't need the API key piece at all - it can work with standard cookie-based auth. > > This also reflects how most plugins are likely to use this API, where they'll be adding JavaScript that uses `fetch()` to call the write API directly. I agree (that's what I did with the previous insert plugin), maybe a complete example using `fetch()` in the documentation would be valuable as a “Getting started with the API” or similar?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1427293909, https://github.com/simonw/datasette/issues/1872#issuecomment-1296076803,https://api.github.com/repos/simonw/datasette/issues/1872,1296076803,IC_kwDOBm6k_c5NQJAD,192568,2022-10-30T02:50:34Z,2022-10-30T02:50:34Z,CONTRIBUTOR,"should this issue be under https://github.com/simonw/datasette-publish-vercel/issues ? Perhaps I just need to update: datasette-publish-vercel==0.11 in requirements.txt? I'll try that and see what happens... ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1428560020, https://github.com/simonw/datasette/issues/1872#issuecomment-1296080804,https://api.github.com/repos/simonw/datasette/issues/1872,1296080804,IC_kwDOBm6k_c5NQJ-k,192568,2022-10-30T03:06:32Z,2022-10-30T03:06:32Z,CONTRIBUTOR,"I updated datasette-publish-vercel to 0.14.2 in requirements.txt And the site is back up! Is there a way that we can get some sort of notice when something like this will have critical impact on website function?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1428560020, https://github.com/simonw/datasette/issues/1884#issuecomment-1309735529,https://api.github.com/repos/simonw/datasette/issues/1884,1309735529,IC_kwDOBm6k_c5OEPpp,25778,2022-11-10T03:57:23Z,2022-11-10T03:57:23Z,CONTRIBUTOR,Here's how to get a list of virtual tables: https://stackoverflow.com/questions/46617118/how-to-fetch-names-of-virtual-tables,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1439009231, https://github.com/simonw/datasette/issues/1884#issuecomment-1313962183,https://api.github.com/repos/simonw/datasette/issues/1884,1313962183,IC_kwDOBm6k_c5OUXjH,25778,2022-11-14T15:46:32Z,2022-11-14T15:46:32Z,CONTRIBUTOR,"It does work, though I think it's probably still worth excluding virtual tables that will always be zero. Here's the same inspection as before, now with `--load-extension spatialite`: ```json { ""alltheplaces"": { ""hash"": ""0843cfe414439ab903c22d1121b7ddbc643418c35c7f0edbcec82ef1452411df"", ""size"": 963375104, ""file"": ""alltheplaces.db"", ""tables"": { ""spatial_ref_sys"": { ""count"": 6215 }, ""spatialite_history"": { ""count"": 18 }, ""sqlite_sequence"": { ""count"": 2 }, ""geometry_columns"": { ""count"": 3 }, ""spatial_ref_sys_aux"": { ""count"": 6164 }, ""views_geometry_columns"": { ""count"": 0 }, ""virts_geometry_columns"": { ""count"": 0 }, ""geometry_columns_statistics"": { ""count"": 3 }, ""views_geometry_columns_statistics"": { ""count"": 0 }, ""virts_geometry_columns_statistics"": { ""count"": 0 }, ""geometry_columns_field_infos"": { ""count"": 0 }, ""views_geometry_columns_field_infos"": { ""count"": 0 }, ""virts_geometry_columns_field_infos"": { ""count"": 0 }, ""geometry_columns_time"": { ""count"": 3 }, ""geometry_columns_auth"": { ""count"": 3 }, ""views_geometry_columns_auth"": { ""count"": 0 }, ""virts_geometry_columns_auth"": { ""count"": 0 }, ""data_licenses"": { ""count"": 10 }, ""sql_statements_log"": { ""count"": 0 }, ""states"": { ""count"": 56 }, ""counties"": { ""count"": 3234 }, ""idx_states_geometry_rowid"": { ""count"": 56 }, ""idx_states_geometry_node"": { ""count"": 3 }, ""idx_states_geometry_parent"": { ""count"": 2 }, ""idx_counties_geometry_rowid"": { ""count"": 3234 }, ""idx_counties_geometry_node"": { ""count"": 98 }, ""idx_counties_geometry_parent"": { ""count"": 97 }, ""idx_places_geometry_rowid"": { ""count"": 1236796 }, ""idx_places_geometry_node"": { ""count"": 38163 }, ""idx_places_geometry_parent"": { ""count"": 38162 }, ""places"": { ""count"": 1332609 }, ""SpatialIndex"": { ""count"": 0 }, ""ElementaryGeometries"": { ""count"": 0 }, ""KNN"": { ""count"": 0 }, ""idx_states_geometry"": { ""count"": 56 }, ""idx_counties_geometry"": { ""count"": 3234 }, ""idx_places_geometry"": { ""count"": 1236796 } } } } ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1439009231, https://github.com/simonw/datasette/issues/1884#issuecomment-1314066229,https://api.github.com/repos/simonw/datasette/issues/1884,1314066229,IC_kwDOBm6k_c5OUw81,25778,2022-11-14T16:48:35Z,2022-11-14T16:48:35Z,CONTRIBUTOR,"I'm realizing I don't know if a virtual table will ever return a count. Maybe it depends on the implementation. For these three, just checking now, it'll always return zero. That said, I'm not sure there's any downside to having them return zero and caching that. (They're hidden, too.) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1439009231, https://github.com/simonw/datasette/issues/1884#issuecomment-1321460293,https://api.github.com/repos/simonw/datasette/issues/1884,1321460293,IC_kwDOBm6k_c5Ow-JF,15178711,2022-11-21T04:40:55Z,2022-11-21T04:40:55Z,CONTRIBUTOR,"Counting any virtual tables can be pretty tricky. On one hand, counting a [CSV virtual table](https://www.sqlite.org/csv.html) would return the number of rows in the CSV, which is helpful (but can be I/O intensive). Counting a [FTS5 virtual table](https://www.sqlite.org/fts5.html) would return the number of entries in the FTS index, which is kindof helpful, but can be misleading in some cases. On the other hand, arbitrarily running `COUNT(*)` on some virtual tables can be incredibly expensive. SQLite offers new shortcuts/pushdowns on `COUNT(*)` queries for virtual tables, and instead calls the underlying vtab implementation and iterates through all rows in the table without discretion. For example, a virtual table that's backed by a Postgres table would call `select * from pg_table`, which would use up a lot of network and CPU calls. Or a virtual table backed by a [google sheet](https://github.com/0x6b/libgsqlite) would make network/API requests to get all the rows from the sheet just to make a count. The [`pragma_table_list`](https://www.sqlite.org/pragma.html#pragma_table_list) pragma tells you when a table is a regular table or virtual (in the `type` column), but was only added in version 3.37.0 (2021-11-27). Personally, I wouldnt try to `COUNT(*)` virtual tables - it depends on how the virtual table is implemented, it requires that the connection has the proper extensions loaded, and it may accientally cause perf issues for new-age extensions. A few extensions that I'm writing have virtual tables that wouldn't benefit much from `COUNT(*)`, and the fact that SQLite iterates through all rows in a table to count just makes things worse. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1439009231, https://github.com/simonw/datasette/issues/1886#issuecomment-1313252879,https://api.github.com/repos/simonw/datasette/issues/1886,1313252879,IC_kwDOBm6k_c5ORqYP,883348,2022-11-14T08:10:23Z,2022-11-14T08:10:23Z,CONTRIBUTOR,"Hi @simonw and thanks for the great tools you're publishing, your dedication is inspiring! I work for the French Ministry of Culture on a surveying tool for objects protected for their historical value. It is part of a program building modern public services called [beta.gouv.fr](https://beta.gouv.fr/). In that context I'm using data published by the Ministry that I have ingested into datasette and published on a free Fly instance : https://collectif-objets-datasette.fly.dev . I have also ingested another data set with infos about french cities on this instance so that I can perform joined queries. The surveying tool synchronizes its data regularly from this datasette instance, and I also use it to perform queries when asked generic questions about the distribution of objects. (The data is not very accessible as it's undocumented and for internal usage mostly)","{""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 3, ""rocket"": 0, ""eyes"": 0}",1447050738, https://github.com/simonw/datasette/issues/1886#issuecomment-1314241058,https://api.github.com/repos/simonw/datasette/issues/1886,1314241058,IC_kwDOBm6k_c5OVboi,25778,2022-11-14T19:06:35Z,2022-11-14T19:06:35Z,CONTRIBUTOR,"This probably counts as a case study: https://github.com/eyeseast/spatial-data-cooking-show. Even has video. Seriously, though, this workflow has become integral to my work with reporters and editors across USA TODAY Network. Very often, I get sent a folder of data in mixed formats, with a vague ask of how we should communicate some part of it to users. Datasette and its constellation of tools makes it easy to get a quick look at that data, run exploratory queries, map it and ask questions to figure out what's important to show. And then I export a version of the data that's exactly what I need for display. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1447050738, https://github.com/simonw/datasette/issues/1886#issuecomment-1321003094,https://api.github.com/repos/simonw/datasette/issues/1886,1321003094,IC_kwDOBm6k_c5OvOhW,9020979,2022-11-20T00:52:05Z,2022-11-20T00:52:05Z,CONTRIBUTOR,"Happy birthday to datasette and thank you Simon for your continued effort on this project! I use datasette (python) as a fast layer on top of search for github projects using https://github.com/dogsheep/github-to-sqlite , and use the JSON API it provides to serve sample data to make Vega-Lite graphing workshop examples that don't require authentication/API keys. It's awesome to have a full SQL API support working without needing to develop any custom API middleware for both filtering and grouping. I've also enjoyed using it as a teaching tool for working with public dataset in [civic data workshops](https://2022.open-data.nyc/event/low-code-visual-data-exploration-with-nyc-public-data/) and as a platform for making visualization [plugins](https://github.com/hydrosquall/datasette-nteract-data-explorer) . I I'm especially excited about datasette-lite, as it will let people participate in future editions of this workshop without having to install anything to make use of their own tables :)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1447050738, https://github.com/simonw/datasette/issues/1886#issuecomment-1321241426,https://api.github.com/repos/simonw/datasette/issues/1886,1321241426,IC_kwDOBm6k_c5OwItS,536941,2022-11-20T20:58:54Z,2022-11-20T20:58:54Z,CONTRIBUTOR,i wrote up a blog post of how i'm using it! https://bunkum.us/2022/11/20/mgdo-stack.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1447050738, https://github.com/simonw/datasette/issues/1890#issuecomment-1317889323,https://api.github.com/repos/simonw/datasette/issues/1890,1317889323,IC_kwDOBm6k_c5OjWUr,536941,2022-11-17T00:47:36Z,2022-11-17T00:47:36Z,CONTRIBUTOR,amazing! thanks @simonw ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1448143294, https://github.com/simonw/datasette/issues/1897#issuecomment-1319533445,https://api.github.com/repos/simonw/datasette/issues/1897,1319533445,IC_kwDOBm6k_c5OpnuF,95570,2022-11-18T04:38:03Z,2022-11-18T04:38:03Z,CONTRIBUTOR,Are you tracking the change to send the JSON over to the frontend separately or was that part of this? Something like this is probably pretty close https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25#diff-0c93232bfd5477eeac96382e52769108b41433d960d5277ffcccf2f464e60abdR9,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263, https://github.com/simonw/datasette/issues/1899#issuecomment-1317873458,https://api.github.com/repos/simonw/datasette/issues/1899,1317873458,IC_kwDOBm6k_c5OjScy,95570,2022-11-17T00:31:07Z,2022-11-17T00:31:07Z,CONTRIBUTOR,"This is one way to fix it ```patch r.html diff --git a/datasette/static/cm-editor-6.0.1.js b/datasette/static/cm-editor-6.0.1.js index c1fd2ab..68cf398 100644 --- a/datasette/static/cm-editor-6.0.1.js +++ b/datasette/static/cm-editor-6.0.1.js @@ -22,7 +22,14 @@ export function editorFromTextArea(textarea, conf = {}) { // https://github.com/codemirror/lang-sql#user-content-sqlconfig.tables let view = new EditorView({ doc: textarea.value, + extensions: [ + EditorView.theme({ + "".cm-content"": { + // Height on cm-content ensures the editor is focusable by clicking beyond the height of the text + minHeight: ""70px"", + }, + }), keymap.of([ { key: ""Shift-Enter"", diff --git a/datasette/templates/_codemirror.html b/datasette/templates/_codemirror.html index dea4710..c4629ae 100644 --- a/datasette/templates/_codemirror.html +++ b/datasette/templates/_codemirror.html @@ -4,7 +4,6 @@ .cm-editor { resize: both; overflow: hidden; - min-height: 70px; width: 80%; border: 1px solid #ddd; } ``` I don't love it but it seems to work for the default case. You can still retrigger the bug by resizing the editor to be > 70px high. The other approach would be to listen for a click on that empty region and move focus to the editor, or something","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452495049, https://github.com/simonw/datasette/issues/1899#issuecomment-1318897922,https://api.github.com/repos/simonw/datasette/issues/1899,1318897922,IC_kwDOBm6k_c5OnMkC,95570,2022-11-17T16:32:42Z,2022-11-17T16:32:42Z,CONTRIBUTOR,Another idea would be to just not set a min-height and allow the 1 line input to be 1 line heigh,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452495049, https://github.com/simonw/datasette/issues/1929#issuecomment-1339906969,https://api.github.com/repos/simonw/datasette/issues/1929,1339906969,IC_kwDOBm6k_c5P3VuZ,3556,2022-12-06T19:34:20Z,2022-12-06T19:34:20Z,CONTRIBUTOR,I confirm that it works 👍 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473659191, https://github.com/simonw/datasette/issues/1973#issuecomment-1369044959,https://api.github.com/repos/simonw/datasette/issues/1973,1369044959,IC_kwDOBm6k_c5Rmfff,193185,2023-01-02T15:41:40Z,2023-01-02T15:41:40Z,CONTRIBUTOR,"Thanks for the response! Yes, it does seem like a pretty nice developer experience--both the automagical labelling of fkeys, and the ability to index the row by column name in addition to column index.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1515815014, https://github.com/simonw/datasette/issues/1973#issuecomment-1407523547,https://api.github.com/repos/simonw/datasette/issues/1973,1407523547,IC_kwDOBm6k_c5T5Rrb,193185,2023-01-29T00:40:31Z,2023-01-29T00:40:31Z,CONTRIBUTOR,"A +1 for switching to `CustomRow`: I think you currently only get a `CustomRow` if the result set had a column that was an fkey ([this code](https://github.com/simonw/datasette/blob/3c352b7132ef09b829abb69a0da0ad00be5edef9/datasette/views/table.py#L667-L682)) Otherwise you get vanilla `sqlite3.Row`s, which will fail if you try to access `.columns` or lookup the cell by name, which surprised me recently","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1515815014, https://github.com/simonw/datasette/issues/1978#issuecomment-1375708725,https://api.github.com/repos/simonw/datasette/issues/1978,1375708725,IC_kwDOBm6k_c5R_6Y1,25778,2023-01-09T14:30:00Z,2023-01-09T14:30:00Z,CONTRIBUTOR,Totally missed that issue. I can close this as a duplicate.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1522778923, https://github.com/simonw/datasette/issues/1983#issuecomment-1375810027,https://api.github.com/repos/simonw/datasette/issues/1983,1375810027,IC_kwDOBm6k_c5SATHr,25778,2023-01-09T15:35:58Z,2023-01-09T15:35:58Z,CONTRIBUTOR,"Yes please, and thank you. I realized I was maybe getting myself in trouble using that, but I think it's a good way to standardize JSON handling.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1525815985,