github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
313494458 | MDExOlB1bGxSZXF1ZXN0MTgxMDMzMDI0 | 200 | Hide Spatialite system tables | 45057 | closed | 0 | 3 | 2018-04-11T21:26:58Z | 2018-04-12T21:34:48Z | 2018-04-12T21:34:48Z | CONTRIBUTOR | simonw/datasette/pulls/200 | They were getting on my nerves. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
314319372 | MDExOlB1bGxSZXF1ZXN0MTgxNjQyMTE0 | 205 | Support filtering with units and more | 45057 | closed | 0 | 3 | 2018-04-14T10:47:51Z | 2018-04-14T15:24:04Z | 2018-04-14T15:24:04Z | CONTRIBUTOR | simonw/datasette/pulls/205 | The first commit: * Adds units to exported JSON * Adds units key to metadata skeleton * Adds some docs for units The second commit adds filtering by units by the first method I mentioned in #203: ![image](https://user-images.githubusercontent.com/45057/38767463-7193be16-3fd9-11e8-8a5f-ac4159415c6d.png) [Try it here](https://wtr-api.herokuapp.com/wtr-663ea99/license_frequency?frequency__gt=50GHz&height__lt=50ft). I think it integrates pretty neatly. The third commit adds support for registering custom units with Pint from metadata.json. Probably pretty niche, but I need decibels! | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/205/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
377156339 | MDU6SXNzdWUzNzcxNTYzMzk= | 371 | datasette publish digitalocean plugin | 82988 | closed | 0 | 3 | 2018-11-04T14:07:41Z | 2021-01-04T20:14:28Z | 2021-01-04T20:14:28Z | CONTRIBUTOR | Provide support for launching `datasette` on Digital Ocean. Example: [Deploy Docker containers into Digital Ocean](https://blog.machinebox.io/deploy-machine-box-in-digital-ocean-385265fbeafd). Digital Ocean also has a preconfigured VM running Docker that can be launched from the command line via the Digital Ocean API: [Docker One-Click Application](https://www.digitalocean.com/docs/one-clicks/docker/). Related: - Launching containers in Digital Ocean servers running docker: [How To Provision and Manage Remote Docker Hosts with Docker Machine on Ubuntu 16.04](https://www.digitalocean.com/community/tutorials/how-to-provision-and-manage-remote-docker-hosts-with-docker-machine-on-ubuntu-16-04) - [How To Use Doctl, the Official DigitalOcean Command-Line Client](https://www.digitalocean.com/community/tutorials/how-to-use-doctl-the-official-digitalocean-command-line-client) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/371/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
432870248 | MDU6SXNzdWU0MzI4NzAyNDg= | 431 | Datasette doesn't reload when database file changes | 82988 | closed | 0 | 3 | 2019-04-13T16:50:43Z | 2019-05-02T05:13:55Z | 2019-05-02T05:13:54Z | CONTRIBUTOR | My understanding of the `--reload` option was that if the database file changed `datasette` would automatically reload. I'm running on a Mac and from the `datasette` UI queries don't seem to be picking up data in a newly changed db (I checked the db timestamp - it certainly updated). I was also expecting to see some sort of log statement in the datasette logging to say that it had detected a file change and restarted, but don't see anything there? Will try to check on an Ubuntu box when I get a chance to see if this is a Mac thing. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/431/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
442327592 | MDU6SXNzdWU0NDIzMjc1OTI= | 456 | Installing installs the tests package | 7725188 | closed | 0 | 3 | 2019-05-09T16:35:16Z | 2020-07-24T20:39:54Z | 2020-07-24T20:39:54Z | CONTRIBUTOR | Because `setup.py` uses `find_packages` and `tests` is on the top-level, `pip install datasette` will install a top-level package called `tests`, which is probably not desired behavior. The offending line is here: https://github.com/simonw/datasette/blob/bfa2ae0d16d39bb82dbe4da4f3fdc3c7f6257418/setup.py#L40 And only `pip uninstall datasette` with a conflicting package would warn you by default; apparently another package had the same problem, which is why I get this message when uninstalling: ``` $ pip uninstall datasette Uninstalling datasette-0.27: Would remove: /usr/local/bin/datasette /usr/local/lib/python3.7/site-packages/datasette-0.27.dist-info/* /usr/local/lib/python3.7/site-packages/datasette/* /usr/local/lib/python3.7/site-packages/tests/* Would not remove (might be manually added): [ .. snip .. ] Proceed (y/n)? ``` This should be a relatively simple fix, and I could drop a PR if desired! Cheers | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/456/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
492153532 | MDU6SXNzdWU0OTIxNTM1MzI= | 573 | Exposing Datasette via Jupyter-server-proxy | 82988 | closed | 0 | 3 | 2019-09-11T10:32:36Z | 2020-03-26T09:41:30Z | 2020-03-26T09:41:30Z | CONTRIBUTOR | It is possible to expose a running `datasette` service in a Jupyter environment such as a MyBinder environment using the [`jupyter-server-proxy`](https://github.com/jupyterhub/jupyter-server-proxy). For example, using [this demo Binder](https://mybinder.org/v2/gh/binder-examples/r/master?filepath=index.ipynb) which has the server proxy installed, we can then upload a simple test database from the notebook homepage, from a Jupyter termianl install datasette and set it running against the test db on eg port 8001 and then view it via the path `proxy/8001`. Clicking links results in 404s though because the `datasette` links aren't relative to the current path? ![image](https://user-images.githubusercontent.com/82988/64689964-44b69280-d487-11e9-8f9f-3681422bcc9f.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/573/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
505818256 | MDExOlB1bGxSZXF1ZXN0MzI3MTcyNTQ1 | 590 | Handle spaces in DB names | 2657547 | closed | 0 | 3 | 2019-10-11T12:18:22Z | 2019-11-04T23:16:31Z | 2019-11-04T23:16:30Z | CONTRIBUTOR | simonw/datasette/pulls/590 | Closes #503 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/590/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
509535510 | MDExOlB1bGxSZXF1ZXN0MzMwMDc2MjYz | 602 | Offer to format readonly SQL | 2657547 | closed | 0 | 3 | 2019-10-20T02:29:32Z | 2019-11-04T07:29:33Z | 2019-11-04T02:39:56Z | CONTRIBUTOR | simonw/datasette/pulls/602 | Following discussion in #601, this PR adds a "Format SQL" button to read-only SQL (if the SQL actually differs from the formatting result). It also removes a console error on readonly SQL queries. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/602/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
562085508 | MDExOlB1bGxSZXF1ZXN0MzcyNzYzOTA2 | 666 | Use inspect-file, if possible, for total row count | 13896256 | closed | 0 | 3 | 2020-02-08T22:10:35Z | 2020-03-09T02:47:15Z | 2020-02-25T20:19:29Z | CONTRIBUTOR | simonw/datasette/pulls/666 | For large tables, counting the number of rows in the table can take a signficant amount of time. Instead, where an inspect-file is provided for an immutable database, look up the row-count for a plain count(*). | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/666/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
598891570 | MDExOlB1bGxSZXF1ZXN0NDAyNjQ1OTg0 | 725 | Update aiofiles requirement from ~=0.4.0 to >=0.4,<0.6 | 27856297 | closed | 0 | 3 | 2020-04-13T13:32:47Z | 2020-05-04T18:16:54Z | 2020-05-04T16:17:49Z | CONTRIBUTOR | simonw/datasette/pulls/725 | Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/Tinche/aiofiles/commit/9a2141e6c51013995ed8b72b121e4e0a27af77b5"><code>9a2141e</code></a> 0.5.0</li> <li><a href="https://github.com/Tinche/aiofiles/commit/479b7eea2c54a9daadb26d1073770e5a50f9ca27"><code>479b7ee</code></a> Update README</li> <li><a href="https://github.com/Tinche/aiofiles/commit/6c247a2cfd1e12f8359aeff04bfe3510f5ef0dc2"><code>6c247a2</code></a> Modernize tests</li> <li><a href="https://github.com/Tinche/aiofiles/commit/eec75d3f425d53306b6aaa8a6005344e56350312"><code>eec75d3</code></a> Switch to async def wherever possible</li> <li><a href="https://github.com/Tinche/aiofiles/commit/786c3e9293a37f63ae1c72b115498d038b5828a0"><code>786c3e9</code></a> Prepare for 3.8</li> <li><a href="https://github.com/Tinche/aiofiles/commit/145107595fe72b877005fb4ebf9d77ffb754689a"><code>1451075</code></a> Update README.rst</li> <li><a href="https://github.com/Tinche/aiofiles/commit/5db1e380e307d0f96177d18db778f82ea0ef87ce"><code>5db1e38</code></a> Add several async os functions</li> <li><a href="https://github.com/Tinche/aiofiles/commit/a60f19bb7b6bd5b063f853d8cae2c41a6878841f"><code>a60f19b</code></a> Add async remove function</li> <li><a href="https://github.com/Tinche/aiofiles/commit/9cf2ac8f9d43446ecdf71c95d05ad6a95bec7377"><code>9cf2ac8</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/Tinche/aiofiles/issues/53">#53</a> from graingert/patch-1</li> <li><a href="https://github.com/Tinche/aiofiles/commit/b88912c6a7a13597b8c5ac93093d7eef86ab1ebb"><code>b88912c</code></a> <strong>all</strong> should be a List[str]</li> <li>Additional commits viewable in <a href="https://github.com/Tinche/aiofiles/compare/v0.4.0...v0.5.0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/725/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
649907676 | MDU6SXNzdWU2NDk5MDc2NzY= | 889 | asgi_wrapper plugin hook is crashing at startup | 49260 | closed | 0 | 3 | 2020-07-02T12:53:13Z | 2020-09-15T20:40:52Z | 2020-09-15T20:40:52Z | CONTRIBUTOR | Steps to reproduce: 1. Install datasette-media plugin `pip install datasette-media` 2. Launch datasette `datasette databasename.db` 3. Error ``` INFO: Started server process [927704] INFO: Waiting for application startup. ERROR: Exception in 'lifespan' protocol Traceback (most recent call last): File "/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/uvicorn/lifespan/on.py", line 48, in main await app(scope, self.receive, self.send) File "/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__ return await self.app(scope, receive, send) File "/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/datasette_media/__init__.py", line 9, in wrapped_app path = scope["path"] KeyError: 'path' ERROR: Application startup failed. Exiting. ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/889/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688386219 | MDExOlB1bGxSZXF1ZXN0NDc1NjY1OTg0 | 142 | insert_all(..., alter=True) should work for new columns introduced after the first 100 records | 96218 | closed | 0 | 3 | 2020-08-28T22:22:57Z | 2020-08-30T07:28:23Z | 2020-08-28T22:30:14Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/142 | Closes #139. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/142/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
729818242 | MDExOlB1bGxSZXF1ZXN0NTEwMjM1OTA5 | 189 | Allow iterables other than Lists in m2m records | 35681 | closed | 0 | 3 | 2020-10-26T18:47:44Z | 2020-10-27T16:28:37Z | 2020-10-27T16:24:21Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/189 | I was playing around with sqlite-utils, creating a Roam Research dogsheep-style importer for Datasette, and ran into a slight snag. I wanted to use a generator to add an order column in an importer. It looked something like: ``` def order_generator(iterable, attr=None): if attr is None: attr = "order" order: int = 0 for i in iterable: i[attr] = order order += 1 yield i ``` When I used this with `insert_all` and other things, it worked fine--but it didn't work as the `records` argument to `m2m`. I dug into it, and sqlite-utils is explicitly checking if the records argument is a list or a tuple. I flipped the check upside down, and now it checks if the argument is a mapping. If it's a mapping, it wraps it in a list, otherwise it leaves it alone. (I get that it might not really make sense to put the order column on the second table. I changed my import schema a bit, and no longer have a real example, but maybe this change still makes sense.) The automated tests still pass, but I did not add any new ones. Let me know what you think! I'm really loving Datasette and its ecosystem; thanks for everything! | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/189/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
752749485 | MDExOlB1bGxSZXF1ZXN0NTI4OTk3NjE0 | 1112 | Fix --metadata doc usage | 50527 | closed | 0 | 6055094 | 3 | 2020-11-28T19:19:51Z | 2020-11-28T23:28:21Z | 2020-11-28T19:53:48Z | CONTRIBUTOR | simonw/datasette/pulls/1112 | I stumbled on this while trying to figure out how to configure datasette-ripgrep via https://github.com/simonw/datasette-ripgrep/issues/15 You may not want to update the changelog (those are annoying) so I added two commits in case that's easier. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1112/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
771872303 | MDExOlB1bGxSZXF1ZXN0NTQzMjQ2NTM1 | 59 | Remove unneeded exists=True for -a/--auth flag. | 631242 | closed | 0 | 3 | 2020-12-21T06:03:55Z | 2021-05-22T14:06:19Z | 2021-05-19T16:08:12Z | CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/59 | The file does not need to exist when using an environment variable. | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
778126516 | MDExOlB1bGxSZXF1ZXN0NTQ4MjcxNDcy | 1170 | Install Prettier via package.json | 3637 | closed | 0 | 6346396 | 3 | 2021-01-04T14:18:03Z | 2021-01-24T21:21:01Z | 2021-01-04T19:52:34Z | CONTRIBUTOR | simonw/datasette/pulls/1170 | This adds a package.json with Prettier and means that developers/CI will use the same version. It also ensures that NPM packages are cached on GitHub Actions which fixes #1169. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1170/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
797649915 | MDExOlB1bGxSZXF1ZXN0NTY0NjA4MjY0 | 1211 | Use context manager instead of plain open | 4488943 | closed | 0 | 3 | 2021-01-31T07:58:10Z | 2021-03-11T16:15:50Z | 2021-03-11T16:15:50Z | CONTRIBUTOR | simonw/datasette/pulls/1211 | Context manager with open closes the files after usage. Fixes: https://github.com/simonw/datasette/issues/1208 When the object is already a pathlib.Path i used read_text write_text functions In some cases pathlib.Path.open were used in context manager, it is basically the same as builtin open. Tests are passing: 850 passed, 5 xfailed, 10 xpassed | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1211/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
810507413 | MDExOlB1bGxSZXF1ZXN0NTc1MTg3NDU3 | 1229 | ensure immutable databses when starting in configuration directory mode with | 295329 | closed | 0 | 3 | 2021-02-17T20:18:26Z | 2022-04-22T13:16:36Z | 2021-03-29T00:17:32Z | CONTRIBUTOR | simonw/datasette/pulls/1229 | fixes #1224 This PR ensures all databases found in a configuration directory that match the files in `inspect-data.json` will be set to `immutable` as outlined in https://docs.datasette.io/en/latest/settings.html#configuration-directory-mode specifically on building the `datasette` instance it checks: - if `immutables` is an empty tuple - as passed by the cli code - if `immutables` is the default function value `None` - when it's not explicitly set And correctly builds the immutable database list from the `inspect-data[file]` keys. Note for this to work the `inspect-data.json` file must contain `file` paths which are relative to the configuration directory otherwise the file paths won't match and the dbs won't be set to immutable. I couldn't find an easy way to test this due to the way `make_app_client` works, happy to take directions on adding a test for this. I've updated the relevant docs as well, i.e. use the `inspect` cli cmd from the config directory path to create the relevant file ``` cd $config_dir datasette inspect *.db --inspect-file=inspect-data.json ``` https://docs.datasette.io/en/latest/performance.html#using-datasette-inspect | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1229/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
827341657 | MDExOlB1bGxSZXF1ZXN0NTg5MjYzMjk3 | 1256 | Minor type in IP adress | 6371750 | closed | 0 | 3 | 2021-03-10T08:28:22Z | 2021-03-10T18:26:46Z | 2021-03-10T18:26:40Z | CONTRIBUTOR | simonw/datasette/pulls/1256 | 127.0.01 replaced by 127.0.0.1 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1256/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
907642546 | MDU6SXNzdWU5MDc2NDI1NDY= | 264 | Supporting additional output formats, like GeoJSON | 25778 | closed | 0 | 3 | 2021-05-31T18:03:32Z | 2021-06-03T05:12:21Z | 2021-06-03T05:12:21Z | CONTRIBUTOR | I have a project going where it would be useful to do some spatial processing in SQLite (instead of large files) and then output GeoJSON. So my workflow would be something like this: 1. Read Shapefiles, GeoJSON, CSVs into a SQLite database 2. Join, filter, prune as needed 3. Export GeoJSON for just the stuff I need at that moment, while still having a database of things that will be useful later I'm wondering if this is worth adding to SQLite-utils itself (GeoJSON, at least), or if it's better to make a counterpart to the ecosystem of `*-to-sqlite` tools, say a suite of `sqlite-to-*` things. Or would it be crazy to have a plugin system? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/264/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
923697888 | MDU6SXNzdWU5MjM2OTc4ODg= | 278 | Support db as first parameter before subcommand, or as environment variable | 601708 | closed | 0 | 3 | 2021-06-17T09:26:29Z | 2021-06-20T22:39:57Z | 2021-06-18T15:43:19Z | CONTRIBUTOR | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/278/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
988325628 | MDExOlB1bGxSZXF1ZXN0NzI3MjY1MDI1 | 1455 | Add scientists to target groups | 198537 | closed | 0 | 3 | 2021-09-04T16:28:58Z | 2021-09-04T16:32:21Z | 2021-09-04T16:31:38Z | CONTRIBUTOR | simonw/datasette/pulls/1455 | Not sure if you want them mentioned explicitly (it's already a long list), but following up on https://twitter.com/simonw/status/1434176989565382656 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1455/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1026379132 | PR_kwDOBm6k_c4tM0JV | 1489 | Update pyyaml requirement from ~=5.3 to >=5.3,<7.0 | 49699333 | closed | 0 | 3 | 2021-10-14T13:09:33Z | 2021-10-14T18:10:43Z | 2021-10-14T18:10:42Z | CONTRIBUTOR | simonw/datasette/pulls/1489 | Updates the requirements on [pyyaml](https://github.com/yaml/pyyaml) to permit the latest version. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/yaml/pyyaml/blob/master/CHANGES">pyyaml's changelog</a>.</em></p> <blockquote> <p>6.0 (2021-10-13)</p> <ul> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/327">yaml/pyyaml#327</a> -- Change README format to Markdown</li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/483">yaml/pyyaml#483</a> -- Add a test for YAML 1.1 types</li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/497">yaml/pyyaml#497</a> -- fix float resolver to ignore <code>.</code> and <code>._</code></li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/550">yaml/pyyaml#550</a> -- drop Python 2.7</li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/553">yaml/pyyaml#553</a> -- Fix spelling of “hexadecimal”</li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/556">yaml/pyyaml#556</a> -- fix representation of Enum subclasses</li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/557">yaml/pyyaml#557</a> -- fix libyaml extension compiler warnings</li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/560">yaml/pyyaml#560</a> -- fix ResourceWarning on leaked file descriptors</li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/561">yaml/pyyaml#561</a> -- always require <code>Loader</code> arg to <code>yaml.load()</code></li> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/564">yaml/pyyaml#564</a> -- remove remaining direct distutils usage</li> </ul> <p>5.4.1 (2021-01-20)</p> <ul> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/480">yaml/pyyaml#480</a> -- Fix stub compat with older pyyaml versions that may unwittingly load it</li> </ul> <p>5.4 (2021-01-19)</p> <ul> <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/407">yaml/pyya… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1489/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1082765654 | I_kwDOBm6k_c5AibFW | 1561 | add hash id to "_memory" url if hashed url mode is turned on and crossdb is also turned on | 536941 | closed | 0 | 3 | 2021-12-17T00:45:12Z | 2022-03-19T04:45:40Z | 2022-03-19T04:45:40Z | CONTRIBUTOR | If hashed_url mode is turned on and crossdb is also turned on, then queries to _memory should have a hash_id. One way that it could work is to have the _memory hash be a hash of all the individual databases. Otherwise, crossdb queries can get quit out of data if using aggressive caching. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1561/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1120990806 | PR_kwDOBm6k_c4x6zZ5 | 1617 | Ensure template_path always uses "/" to match jinja | 3526913 | closed | 0 | 3 | 2022-02-01T17:20:30Z | 2022-02-03T01:58:35Z | 2022-02-03T01:58:35Z | CONTRIBUTOR | simonw/datasette/pulls/1617 | This PR shoudl fix #1545 The existing code substituted / for \, assuming this was the right behaviour for windows. But on Windows, Jinja still uses / for the template list - See https://github.com/pallets/jinja/blob/896a62135bcc151f2997e028c5125bec2cb2431f/src/jinja2/loaders.py#L225 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1617/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1180778860 | PR_kwDOBm6k_c41BFWj | 1685 | Update jinja2 requirement from <3.1.0,>=2.10.3 to >=2.10.3,<3.2.0 | 49699333 | closed | 0 | 3 | 2022-03-25T13:12:13Z | 2022-09-05T18:36:49Z | 2022-09-05T18:36:48Z | CONTRIBUTOR | simonw/datasette/pulls/1685 | Updates the requirements on [jinja2](https://github.com/pallets/jinja) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pallets/jinja/releases">jinja2's releases</a>.</em></p> <blockquote> <h2>3.1.0</h2> <p>This is a feature release, which includes new features and removes previously deprecated features. The 3.1.x branch is now the supported bugfix branch, the 3.0.x branch has become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all dependencies and control upgrades. We also encourage upgrading to MarkupSafe 2.1.1, the latest version at this time.</p> <ul> <li>Changes: <a href="https://jinja.palletsprojects.com/en/3.1.x/changes/#version-3-1-0">https://jinja.palletsprojects.com/en/3.1.x/changes/#version-3-1-0</a></li> <li>Milestone: <a href="https://github.com/pallets/jinja/milestone/8?closed=1">https://github.com/pallets/jinja/milestone/8?closed=1</a></li> <li>MarkupSafe changes: <a href="https://markupsafe.palletsprojects.com/en/2.1.x/changes/#version-2-1-1">https://markupsafe.palletsprojects.com/en/2.1.x/changes/#version-2-1-1</a></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pallets/jinja/blob/main/CHANGES.rst">jinja2's changelog</a>.</em></p> <blockquote> <h2>Version 3.1.0</h2> <p>Released 2022-03-24</p> <ul> <li> <p>Drop support for Python 3.6. :pr:<code>1534</code></p> </li> <li> <p>Remove previously deprecated code. :pr:<code>1544</code></p> <ul> <li><code>WithExtension</code> and <code>AutoEscapeExtension</code> are built-in now.</li> <li><code>contextfilter</code> and <code>contextfunction</code> are replaced by <code>pass_context</code>. <code>evalcontextfilter</code> and <code>evalcontextfunction</code> are replaced by <code>pass_eval_context</code>. <code>environmentfilter</code> and <code>environmentfunction</code> are replac… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1685/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1181432624 | I_kwDOBm6k_c5Gazsw | 1688 | [plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins? | 9020979 | closed | 0 | 3 | 2022-03-26T01:17:44Z | 2022-03-27T01:01:14Z | 2022-03-26T21:34:47Z | CONTRIBUTOR | I'm trying to make a small plugin that depends on static assets, by following the guide [here](https://docs.datasette.io/en/stable/writing_plugins.html#writing-one-off-plugins). I made a `plugins/` directory with `datasette_nteract_data_explorer.py`. I am trying to follow the example of `datasette_vega`, and serving static assets. I created a `statics/` directory within `plugins/` to serve my JS and CSS. https://github.com/simonw/datasette-vega/blob/00de059ab1ef77394ba9f9547abfacf966c479c4/datasette_vega/__init__.py#L13 Unfortunately, datasette doesn't seem to be able to find my assets. Input: ```bash datasette ~/Library/Safari/History.db --plugins-dir=plugins/ ``` ![Image 2022-03-25 at 9 18 17 PM](https://user-images.githubusercontent.com/9020979/160218979-a3ff474b-5255-4a76-85d1-6f90ab2e3b44.jpg) Output: ![Image 2022-03-25 at 9 11 00 PM](https://user-images.githubusercontent.com/9020979/160218733-ca5144cf-f23f-43d8-a8d3-e3a871e57f3a.jpg) I suspect this issue might go away if I move away from "one-off" plugin mode, but it's been a while since I created a new python package so I'm not sure how much work there is to go between "one off" and "packaged for PyPI". I'd like to try to avoid needing to repackage a new `tar.gz` file and or reinstall my library repeatedly when developing new python code. 1. Is there a way to serve a static assets when using the `plugins/` directory method instead of installing plugins as a new python package? 2. If not, is there a way I can work on developing a plugin without creating and repackaging tar.gz files after every change, or is that the recommended path? Thanks for your help! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1688/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1184850337 | PR_kwDOBm6k_c41OrSL | 1693 | Bump black from 22.1.0 to 22.3.0 | 49699333 | closed | 0 | 3 | 2022-03-29T13:11:09Z | 2022-06-28T13:11:38Z | 2022-06-28T13:11:36Z | CONTRIBUTOR | simonw/datasette/pulls/1693 | Bumps [black](https://github.com/psf/black) from 22.1.0 to 22.3.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p> <blockquote> <h2>22.3.0</h2> <h3>Preview style</h3> <ul> <li>Code cell separators <code>#%%</code> are now standardised to <code># %%</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2919">#2919</a>)</li> <li>Remove unnecessary parentheses from <code>except</code> statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2939">#2939</a>)</li> <li>Remove unnecessary parentheses from tuple unpacking in <code>for</code> loops (<a href="https://github-redirect.dependabot.com/psf/black/issues/2945">#2945</a>)</li> <li>Avoid magic-trailing-comma in single-element subscripts (<a href="https://github-redirect.dependabot.com/psf/black/issues/2942">#2942</a>)</li> </ul> <h3>Configuration</h3> <ul> <li>Do not format <code>__pypackages__</code> directories by default (<a href="https://github-redirect.dependabot.com/psf/black/issues/2836">#2836</a>)</li> <li>Add support for specifying stable version with <code>--required-version</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2832">#2832</a>).</li> <li>Avoid crashing when the user has no homedir (<a href="https://github-redirect.dependabot.com/psf/black/issues/2814">#2814</a>)</li> <li>Avoid crashing when md5 is not available (<a href="https://github-redirect.dependabot.com/psf/black/issues/2905">#2905</a>)</li> <li>Fix handling of directory junctions on Windows (<a href="https://github-redirect.dependabot.com/psf/black/issues/2904">#2904</a>)</li> </ul> <h3>Documentation</h3> <ul> <li>Update pylint config documentation (<a href="https://github-redirect.dependabot.com/psf/black/issues/2931">#2931</a>)</li> </ul> <h3>Integrations</h3> <ul> <li>Move test to disable plugin in Vim/Neovim, which speeds up loading (<a href="https://github-redirect.dependabot.com/psf/black/issues/2896">#2896</a>)</li> </ul> <h3… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1693/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1275523220 | PR_kwDOBm6k_c454SlE | 1759 | Extract facet portions of table.html out into included templates | 19872 | closed | 0 | 3 | 2022-06-17T22:04:04Z | 2022-06-20T18:05:45Z | 2022-06-20T18:05:45Z | CONTRIBUTOR | simonw/datasette/pulls/1759 | To allow users customise the facet content as they would prefer such as sorting of facet results. ordering of suggested facets etc. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1759/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1309542173 | PR_kwDOCGYnMM47pwAb | 455 | in extract code, check equality with IS instead of = for nulls | 536941 | closed | 0 | 3 | 2022-07-19T13:40:25Z | 2022-08-27T14:45:03Z | 2022-08-27T14:45:03Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/455 | sqlite "IS" is equivalent to SQL "IS NOT DISTINCT FROM" closes #423 | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/455/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1375930971 | PR_kwDOBm6k_c4_GVBS | 1812 | Bump furo from 2022.6.21 to 2022.9.15 | 49699333 | closed | 0 | 3 | 2022-09-16T13:10:45Z | 2022-09-16T19:50:53Z | 2022-09-16T19:50:52Z | CONTRIBUTOR | simonw/datasette/pulls/1812 | Bumps [furo](https://github.com/pradyunsg/furo) from 2022.6.21 to 2022.9.15. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pradyunsg/furo/blob/main/docs/changelog.md">furo's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <!-- raw HTML omitted --> <h2>2022.09.15 -- Pragmatic Pistachio</h2> <ul> <li>Add a minimum version constraint on pygments.</li> <li>Add an explicit dependency on <code>sass</code>.</li> <li>Change right sidebar title from "Contents" to "On this page".</li> <li>Correctly position sidebars on small screens.</li> <li>Correctly select only Furo's own <code>svg</code> in related pages <code>nav</code>.</li> <li>Make numpy-style documentation headers consistent.</li> <li>Retitle the reference section.</li> <li>Update npm dependencies.</li> </ul> <h2>2022.06.21 -- Opulent Opal</h2> <ul> <li>Fix <code>docutils <= 0.17.x</code> compatibility.</li> <li>Bump to the latest Node.js LTS.</li> </ul> <h2>2022.06.04.1 -- Naughty Nickel bugfix</h2> <ul> <li>Fix the URL used in the "Edit this page" for Read the Docs builds.</li> </ul> <h2>2022.06.04 -- Naughty Nickel</h2> <ul> <li>✨ Advertise Sphinx 5 compatibility.</li> <li>✨ Change to <code>basic-ng</code> as the base theme (from {pypi}<code>sphinx-basic-ng</code>).</li> <li>Document site-wide announcement banners.</li> <li>Drop the pin on pygments.</li> <li>Improve edit button, using <code>basic-ng</code>'s <code>edit-this-page</code> component.</li> <li>Tweak headings to better match what users expect.</li> <li>Tweak how Sphinx's default HTML is rendered, using docutils post-transforms (this replaces parsing+modifying it with BeautifulSoup).</li> <li>When built with docutils 0.18, footnotes are rendered differently and stylised differently in Furo.</li> </ul> <h2>2022.04.07 -- Magical Mauve</h2> <ul> <li>✨ Make sphinx-copybutton look better.</li> <li>Add margin to indentations in line blocks.</li> <li>Add styling for non-arabic list styles</li> <li>Add support for <code>html_bas… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1812/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1385026210 | I_kwDOBm6k_c5SjdKi | 1819 | Preserve query on timeout | 2182 | closed | 0 | 3 | 2022-09-25T13:32:31Z | 2022-09-26T23:16:15Z | 2022-09-26T23:06:06Z | CONTRIBUTOR | If a query hits the timeout it shows a message like: > SQL query took too long. The time limit is controlled by the [sql_time_limit_ms](https://docs.datasette.io/en/stable/settings.html#sql-time-limit-ms) configuration option. But the query is lost. Hitting the browser back button shows the query _before_ the one that errored. It would be nice if the query that errored was preserved for more tweaking. This would make it similar to how "invalid syntax" works since #1346 / #619. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1819/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1400121355 | PR_kwDOBm6k_c5AVujU | 1835 | use inspect data for hash and file size | 536941 | closed | 0 | 3 | 2022-10-06T18:25:24Z | 2022-10-27T20:51:30Z | 2022-10-06T20:06:07Z | CONTRIBUTOR | simonw/datasette/pulls/1835 | `inspect_data` should already include the hash and the db file size, so this PR takes advantage of using those instead of always recalculating. should help a lot on startup with large DBs. closes #1834 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1835/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1428560020 | I_kwDOBm6k_c5VJhiU | 1872 | SITE-BUSTING ERROR: "render_template() called before await ds.invoke_startup()" | 192568 | closed | 0 | 3 | 2022-10-30T02:28:39Z | 2022-10-30T06:26:01Z | 2022-10-30T06:26:01Z | CONTRIBUTOR | 1. My https://list.saferdisinfectants.org/disinfectants/listN page (linked from https://SaferDisinfectants.org ) has been running beautifully for a year and a half, including a GitHub Actions workflow that's been routinely updating the database. 2. I received a recent report that the list page is down. I don't know when it went down, but the content is replaced with: "render_template() called before await ds.invoke_startup()" 3. The local datasette repo runs without incident. 4. The site is hosted on vercel, linked to my github repo. Perhaps some vercel changes were made, but not by anyone on our side. Here is a screenshot of the current project settings: <img width="877" alt="Screen Shot 2022-10-29 at 9 23 30 PM" src="https://user-images.githubusercontent.com/192568/198859474-341a5504-88b8-452d-bd06-e988cbe75e6c.png"> Here a screenshot of the latest deployment status: <img width="1351" alt="Screen Shot 2022-10-29 at 10 19 21 PM" src="https://user-images.githubusercontent.com/192568/198859641-ad3d99ba-a41d-4b98-88f6-e3a01fe9f238.png"> This is my repository: https://github.com/mroswell/list-N (I notice: datasette==0.59 in my requirements.txt file) Because it's been long while since I actively worked on this or any other datasette project, I forget a lot of what I knew at one point. Perhaps some configuration file could be missing? Or perhaps I just need to know the right incantation to add to that vercel settings page. Help is welcome as the nonprofit org is soon hosting its annual conference, and we'd love to have the page working again. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1872/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1665200812 | PR_kwDOCGYnMM5OKveS | 537 | Support self-referencing FKs in `Table.create` | 544011 | closed | 0 | 3 | 2023-04-12T20:26:59Z | 2023-05-08T22:45:33Z | 2023-05-08T21:10:01Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/537 | <!-- readthedocs-preview sqlite-utils start --> ---- :books: Documentation preview :books:: https://sqlite-utils--537.org.readthedocs.build/en/537/ <!-- readthedocs-preview sqlite-utils end --> | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/537/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1719759468 | PR_kwDOBm6k_c5RBXH_ | 2077 | Bump furo from 2023.3.27 to 2023.5.20 | 49699333 | closed | 0 | 3 | 2023-05-22T13:58:16Z | 2023-07-27T13:40:55Z | 2023-07-27T13:40:53Z | CONTRIBUTOR | simonw/datasette/pulls/2077 | Bumps [furo](https://github.com/pradyunsg/furo) from 2023.3.27 to 2023.5.20. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pradyunsg/furo/blob/main/docs/changelog.md">furo's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <!-- raw HTML omitted --> <h2>2023.05.20 -- Unassuming Ultramarine</h2> <ul> <li>✨ Add support for Sphinx 7.</li> <li>Drop support for Sphinx 5.</li> <li>Improve the screen-reader label for sidebar collapse.</li> <li>Make it easier to create derived themes from Furo.</li> <li>Bump all JS dependencies (NodeJS and npm packages).</li> </ul> <h2>2023.03.27 -- Tasty Tangerine</h2> <ul> <li>Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.</li> <li>Add missing class to Font Awesome examples</li> </ul> <h2>2023.03.23 -- Sassy Saffron</h2> <ul> <li>Update Python version classifiers.</li> <li>Increase the icon size in mobile header.</li> <li>Increase admonition title bg opacity.</li> <li>Change the default API background to transparent.</li> <li>Transition the API background change.</li> <li>Remove the "indent" of API entries which have a background.</li> <li>Break long inline code literals.</li> </ul> <h2>2022.12.07 -- Reverent Raspberry</h2> <ul> <li>✨ Add support for Sphinx 6.</li> <li>✨ Improve footnote presentation with docutils 0.18+.</li> <li>Drop support for Sphinx 4.</li> <li>Improve documentation about what the edit button does.</li> <li>Improve handling of empty-flexboxes for better print experience on Chrome.</li> <li>Improve styling for inline signatures.</li> <li>Replace the <code>meta</code> generator tag with a comment.</li> <li>Tweak labels with icons to prevent users selecting icons as text on touch.</li> </ul> <h2>2022.09.29 -- Quaint Quartz</h2> <ul> <li>Add ability to set arbitrary URLs for edit button.</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pradyunsg/furo/commit/d2c9ca852542820632… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/2077/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1865174661 | PR_kwDOBm6k_c5YsfZ7 | 2152 | Bump the python-packages group with 3 updates | 49699333 | closed | 0 | 3 | 2023-08-24T13:34:44Z | 2023-08-28T13:49:39Z | 2023-08-28T13:49:37Z | CONTRIBUTOR | simonw/datasette/pulls/2152 | Bumps the python-packages group with 3 updates: [sphinx](https://github.com/sphinx-doc/sphinx), [furo](https://github.com/pradyunsg/furo) and [blacken-docs](https://github.com/asottile/blacken-docs). Updates `sphinx` from 7.1.2 to 7.2.3 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/sphinx-doc/sphinx/releases">sphinx's releases</a>.</em></p> <blockquote> <h2>Sphinx 7.2.3</h2> <p>Changelog: <a href="https://www.sphinx-doc.org/en/master/changes.html">https://www.sphinx-doc.org/en/master/changes.html</a></p> <h2>Sphinx 7.2.2</h2> <p>Changelog: <a href="https://www.sphinx-doc.org/en/master/changes.html">https://www.sphinx-doc.org/en/master/changes.html</a></p> <h2>Sphinx 7.2.1</h2> <p>Changelog: <a href="https://www.sphinx-doc.org/en/master/changes.html">https://www.sphinx-doc.org/en/master/changes.html</a></p> <h2>Sphinx 7.2.0</h2> <p>Changelog: <a href="https://www.sphinx-doc.org/en/master/changes.html">https://www.sphinx-doc.org/en/master/changes.html</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/sphinx-doc/sphinx/blob/master/CHANGES">sphinx's changelog</a>.</em></p> <blockquote> <h1>Release 7.2.3 (released Aug 23, 2023)</h1> <h2>Dependencies</h2> <ul> <li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11576">#11576</a>: Require sphinxcontrib-serializinghtml 1.1.9.</li> </ul> <h2>Bugs fixed</h2> <ul> <li>Fix regression in <code>autodoc.Documenter.parse_name()</code>.</li> <li>Fix regression in JSON serialisation.</li> <li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11543">#11543</a>: autodoc: Support positional-only parameters in <code>classmethod</code> methods when <code>autodoc_preserve_defaults</code> is <code>True</code>.</li> <li>Restore support string methods on path objects. This is deprecated and will be removed in Sphinx 8. Use :py:func<code>os.fspath</code> to convert :py:class:<del><code>pathlib.Path</code> objects to strings, or :py:class:</del><code>p… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/2152/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
472097220 | MDU6SXNzdWU0NzIwOTcyMjA= | 7 | Script uses a lot of RAM | 9599 | closed | 0 | 3 | 2019-07-24T06:11:11Z | 2019-07-24T06:35:52Z | 2019-07-24T06:35:52Z | MEMBER | I'm using an XML pull parser which should avoid the need to slurp the whole XML file into memory, but it's not working - the script still uses over 1GB of RAM when it runs according to Activity Monitor. I think this is because I'm still causing the full root element to be incrementally loaded into memory just in case I try and access it later. http://effbot.org/elementtree/iterparse.htm says I should use `elem.clear()` as I go. It also says: > The above pattern has one drawback; it does not clear the root element, so you will end up with a single element with lots of empty child elements. If your files are huge, rather than just large, this might be a problem. To work around this, you need to get your hands on the root element. So I will try that recipe and see if it helps. | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
488833698 | MDU6SXNzdWU0ODg4MzM2OTg= | 2 | "twitter-to-sqlite user-timeline" command for pulling tweets by a specific user | 9599 | closed | 0 | 3 | 2019-09-03T21:29:12Z | 2019-09-04T20:02:11Z | 2019-09-04T20:02:11Z | MEMBER | Twitter only allows up to 3,200 tweets to be retrieved from https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html I'm going to do: $ twitter-to-sqlite tweets simonw | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
503053800 | MDU6SXNzdWU1MDMwNTM4MDA= | 12 | Extract "source" into a separate lookup table | 9599 | closed | 0 | 3 | 2019-10-06T05:17:23Z | 2019-10-17T15:49:24Z | 2019-10-17T15:49:24Z | MEMBER | It's pretty bulky and ugly at the moment: <img width="334" alt="trump__tweets__1_820_rows" src="https://user-images.githubusercontent.com/9599/66264630-df23a080-e7bd-11e9-9154-403c2e69f841.png"> | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
503234169 | MDU6SXNzdWU1MDMyMzQxNjk= | 2 | Track and use the 'since' value | 9599 | closed | 0 | 3 | 2019-10-07T05:02:59Z | 2020-03-27T22:22:30Z | 2020-03-27T22:22:30Z | MEMBER | Pocket says: > Whenever possible, you should use the since parameter, or count and and offset parameters when retrieving a user's list. After retrieving the list, you should store the current time (which is provided along with the list response) and pass that in the next request for the list. This way the server only needs to return a small set (changes since that time) instead of the user's entire list every time. At the bottom of https://getpocket.com/developer/docs/v3/retrieve | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
506087267 | MDU6SXNzdWU1MDYwODcyNjc= | 19 | since_id support for home-timeline | 9599 | closed | 0 | 3 | 2019-10-11T22:48:24Z | 2019-10-16T19:13:06Z | 2019-10-16T19:12:46Z | MEMBER | Currently every time you run `home-timeline` we pull all 800 available tweets. We should offer to support `since_id` (which can be provided or can be pulled directly from the database) in order to work more efficiently if this command is executed e.g. on a cron. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
506268945 | MDU6SXNzdWU1MDYyNjg5NDU= | 20 | --since support for various commands for refresh-by-cron | 9599 | closed | 0 | 3 | 2019-10-13T03:40:46Z | 2019-10-21T03:32:04Z | 2019-10-16T19:26:11Z | MEMBER | I want to run a cron that updates my Twitter database every X minutes. It should be able to retrieve the following without needing to paginate through everything: - [x] Tweets I have tweeted - [x] My home timeline (see #19) - [x] Tweets I have favourited It would be nice if this could be standardized across all commands as a `--since` option. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
516967682 | MDU6SXNzdWU1MTY5Njc2ODI= | 10 | Add this repos_starred view | 9599 | closed | 0 | 3 | 2019-11-04T05:44:38Z | 2020-05-02T16:37:36Z | 2020-05-02T16:37:36Z | MEMBER | ```sql create view repos_starred as select stars.starred_at, users.login, repos.* from repos join stars on repos.id = stars.repo join users on repos.owner = users.id order by starred_at desc; ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
585353598 | MDU6SXNzdWU1ODUzNTM1OTg= | 37 | Handle "User not found" error | 9599 | closed | 0 | 3 | 2020-03-20T22:14:32Z | 2020-04-17T23:43:46Z | 2020-04-17T23:43:46Z | MEMBER | While running `user-timeline` I got this bug (because a screen name I asked for didn't exist): ``` File "/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py", line 185, in transform_user user["created_at"] = parser.parse(user["created_at"]) KeyError: 'created_at' >>> import pdb >>> pdb.pm() > /Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py(185)transform_user() -> user["created_at"] = parser.parse(user["created_at"]) (Pdb) user {'errors': [{'code': 50, 'message': 'User not found.'}]} ``` | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
601333634 | MDU6SXNzdWU2MDEzMzM2MzQ= | 28 | Pull repository contributors | 9599 | closed | 0 | 3 | 2020-04-16T18:46:40Z | 2020-04-18T15:05:10Z | 2020-04-18T15:05:10Z | MEMBER | https://developer.github.com/v3/repos/#list-contributors `GET /repos/:owner/:repo/contributors` Not sure if this should be a separate command or should be part of the existing `repos` command. I'm leaning towards a new `contributors` command. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
603624862 | MDU6SXNzdWU2MDM2MjQ4NjI= | 31 | Issue and milestone should have foreign key to repo | 9599 | closed | 0 | 3 | 2020-04-21T00:46:24Z | 2020-04-22T01:20:19Z | 2020-04-22T01:20:19Z | MEMBER | Currently the `repo` column on those tables is a string `simonw/datasette` rather than an ID referencing a row in `repos`. _Originally posted by @simonw in https://github.com/dogsheep/github-to-sqlite/issues/29#issuecomment-616883275_ | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
604222295 | MDU6SXNzdWU2MDQyMjIyOTU= | 32 | Issue comments don't appear to populate issues foreign key | 9599 | closed | 0 | 3 | 2020-04-21T19:17:32Z | 2020-04-22T01:17:44Z | 2020-04-22T01:17:44Z | MEMBER | https://github-to-sqlite.dogsheep.net/github?sql=select+html_url%2C+id%2C+issue+from+issue_comments+order+by+updated_at+desc+limit+101 <img width="774" alt="Screen Shot 2020-04-21 at 12 16 57 PM" src="https://user-images.githubusercontent.com/9599/79904752-0b091b00-83ca-11ea-9b15-b2893f05dfdc.png"> | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610511450 | MDU6SXNzdWU2MTA1MTE0NTA= | 35 | Create index on issue_comments(user) and other foreign keys | 9599 | closed | 0 | 3 | 2020-05-01T02:06:56Z | 2020-05-02T18:26:24Z | 2020-05-02T18:26:24Z | MEMBER | ``` create index issue_comments_user on issue_comments(user) ``` I'm sure there are other user columns that could benefit from an index. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610843136 | MDU6SXNzdWU2MTA4NDMxMzY= | 37 | Mechanism for creating views if they don't yet exist | 9599 | closed | 0 | 3 | 2020-05-01T16:34:10Z | 2020-05-02T16:19:47Z | 2020-05-02T16:19:31Z | MEMBER | Needed for #36 #10 #12 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612860531 | MDU6SXNzdWU2MTI4NjA1MzE= | 17 | Only install osxphotos if running on macOS | 9599 | closed | 0 | 3 | 2020-05-05T20:03:26Z | 2020-05-05T20:20:05Z | 2020-05-05T20:11:23Z | MEMBER | The build is broken right now because you can't `pip install osxphotos` on Ubuntu. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
663976976 | MDU6SXNzdWU2NjM5NzY5NzY= | 48 | Add a table of contents to the README | 9599 | closed | 0 | 3 | 2020-07-22T18:54:33Z | 2020-07-23T17:46:07Z | 2020-07-22T19:03:02Z | MEMBER | Using https://github.com/jonschlinkert/markdown-toc | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
689810340 | MDU6SXNzdWU2ODk4MTAzNDA= | 3 | Datasette plugin to provide custom page for running faceted, ranked searches | 9599 | closed | 0 | 3 | 2020-09-01T05:00:22Z | 2020-09-03T21:01:41Z | 2020-09-03T21:01:41Z | MEMBER | This will be a page at `/-/beta` which renders using a custom template. It will offer a default timeline view plus search and facet by type/date. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
753026003 | MDU6SXNzdWU3NTMwMjYwMDM= | 54 | github-to-sqlite workflows command | 9599 | closed | 0 | 3 | 2020-11-29T21:56:42Z | 2020-11-29T22:08:46Z | 2020-11-29T21:57:17Z | MEMBER | A command that fetches the YAML workflows for different repos, parses them and stores them in relational tables would be really useful for maintaining larger numbers of workflows. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1616347574 | I_kwDOJHON9s5gV4G2 | 1 | Initial proof of concept with ChatGPT | 9599 | closed | 0 | 3 | 2023-03-09T03:44:39Z | 2023-03-09T03:51:55Z | 2023-03-09T03:51:55Z | MEMBER | I'm using ChatGPT to figure out enough AppleScript to get at my notes data. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
276091279 | MDU6SXNzdWUyNzYwOTEyNzk= | 144 | apsw as alternative sqlite3 binding (for full text search) | 649467 | closed | 0 | 3 | 2017-11-22T14:40:39Z | 2018-05-28T21:29:42Z | 2018-05-28T21:29:42Z | NONE | Hey there, Have you considered providing apsw support as an alternative to stock python sqlite3? I use apsw because it keeps up with sqlite3 and is straightforward to bring in extensions like FTS5. FTS really accelerates the kind of searching often done by web clients. I may be able to help (it shouldn't be much code), but there are a couple of stylistic questions that come up when supporting an optional package. Also, apsw is tricky in that it doesn't have a pypi package (author says limitations in providing options to setup.py). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/144/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
306811513 | MDU6SXNzdWUzMDY4MTE1MTM= | 186 | proposal new option to disable user agents cache | 47107 | closed | 0 | 3 | 2018-03-20T10:42:20Z | 2018-03-21T09:07:22Z | 2018-03-21T01:28:31Z | NONE | I think it would be very useful for debugging an option of adding headers to http replies ``` Cache-Control: no-cache ``` especially in the html output | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/186/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
322741659 | MDExOlB1bGxSZXF1ZXN0MTg3NzcwMzQ1 | 258 | Add new metadata key persistent_urls which removes the hash from all database urls | 247131 | closed | 0 | 3 | 2018-05-14T09:39:18Z | 2018-05-21T07:38:15Z | 2018-05-21T07:38:15Z | NONE | simonw/datasette/pulls/258 | Add new metadata key "persistent_urls" which removes the hash from all database urls when set to "true" This PR is just to gauge if this, or something like it, is something you would consider merging? I understand the reason why the substring of the hash is included in the url but there are some use cases where the urls should persist across deployments. For bookmarks for example or for scripts that use the JSON API. This is the initial commit for this feature. Tests and documentation updates to follow. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/258/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
397129564 | MDU6SXNzdWUzOTcxMjk1NjQ= | 397 | Update official datasetteproject/datasette Docker container to SQLite 3.26.0 | 43564 | closed | 0 | 3 | 2019-01-08T22:51:50Z | 2019-01-11T01:25:33Z | 2019-01-11T00:56:18Z | NONE | I try to start datasette on a database that contains the below view It fails in a way that makes me think it does not support the window functions SQL syntax. ``` create view general_ledger as select transactions.account_number, strftime("%Y-%m-%d", verifications.verification_date) as verification_date, verifications.verification_number, verifications.verification_text, case when transactions.centi_amount >= 0 and verifications.verification_number > 0 then printf("%.2f", (transactions.centi_amount/100.0)) end as debit, case when transactions.centi_amount <= 0 and verifications.verification_number > 0 then printf("%.2f", (transactions.centi_amount/100.0)) end as credit, printf("%.2f", sum(transactions.centi_amount) over (partition by transactions.account_number order by verifications.verification_number range between unbounded preceding and current row)/100.0) from verifications inner join transactions on transactions.verification_id = verifications.id order by transactions.account_number, verifications.verification_number; ``` ``` docker run -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/ledger.db Serve! files=('/mnt/ledger.db',) on port 8001 Traceback (most recent call last): File "/usr/local/bin/datasette", line 11, in <module> sys.exit(cli()) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/datase… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/397/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
400229984 | MDU6SXNzdWU0MDAyMjk5ODQ= | 401 | How to pass configuration to plugins? | 1055831 | closed | 0 | 3 | 2019-01-17T11:20:41Z | 2019-01-18T11:48:13Z | 2019-01-18T06:49:07Z | NONE | Hi, Firstly, thanks for your work on datasette, it is a hugely useful tool! I've been working on a fork [https://github.com/dazzag24/datasette-cluster-map] of datasette-cluster-map to allow the tileserver to be easily switched. Primarily because the tiles being served in the current version use localised text for labels and I'd like to have English used for these names instead. It uses http://leaflet-extras.github.io/leaflet-providers/preview/ to allow you to simply set the tile provider using a call like so: ``` let tiles = L.tileLayer.provider('Esri.WorldTopoMap'); ``` instead of the current: ``` let tiles = L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { maxZoom: 19, detectRetina: true, attribution: '© <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors' }), ``` However I've got stuck in trying to work out how to pass the provider string to the plugin. In the documentation: https://datasette.readthedocs.io/en/stable/plugins.html you discuss configuration of plugins and use an example of passing in which latitude and longitude columns should be used. However I cannot seem to see anywhere in the current datasette-cluster-map code where these config params are passed in or used. Can you please point me to an example or how to pass configuration from the metadata.json down into a plugin. Once I've over come this issue I was wondering if you would be interested in taking this change into your version? Many thanks Darren | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/401/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
403922644 | MDU6SXNzdWU0MDM5MjI2NDQ= | 8 | Problems handling column names containing spaces or - | 82988 | closed | 0 | 3 | 2019-01-28T17:23:28Z | 2019-04-14T15:29:33Z | 2019-02-23T21:09:03Z | NONE | Irrrespective of whether using column names containing a space or - character is good practice, SQLite does allow it, but `sqlite-utils` throws an error in the following cases: ```python from sqlite_utils import Database dbname = 'test.db' DB = Database(sqlite3.connect(dbname)) import pandas as pd df = pd.DataFrame({'col1':range(3), 'col2':range(3)}) #Convert pandas dataframe to appropriate list/dict format DB['test1'].insert_all( df.to_dict(orient='records') ) #Works fine ``` However: ```python df = pd.DataFrame({'col 1':range(3), 'col2':range(3)}) DB['test1'].insert_all(df.to_dict(orient='records')) ``` throws: ``` --------------------------------------------------------------------------- OperationalError Traceback (most recent call last) <ipython-input-27-070b758f4f92> in <module>() 1 import pandas as pd 2 df = pd.DataFrame({'col 1':range(3), 'col2':range(3)}) ----> 3 DB['test1'].insert_all(df.to_dict(orient='records')) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) 327 jsonify_if_needed(record.get(key, None)) for key in all_columns 328 ) --> 329 result = self.db.conn.execute(sql, values) 330 self.db.conn.commit() 331 self.last_id = result.lastrowid OperationalError: near "1": syntax error ``` and: ```python df = pd.DataFrame({'col-1':range(3), 'col2':range(3)}) DB['test1'].upsert_all(df.to_dict(orient='records')) ``` results in: ``` --------------------------------------------------------------------------- OperationalError Traceback (most recent call last) <ipython-input-28-654523549d20> in <module>() 1 import pandas as pd 2 df = pd.DataFrame({'col-1':range(3), 'col2':range(3)}) ----> 3 DB['test1'].insert_all(df.to_dict(orient='records')) /usr/local/lib/python3.7/site-packages/sqlite_… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
408376825 | MDU6SXNzdWU0MDgzNzY4MjU= | 409 | Zeit API v1 does not work for new users - need to migrate to v2 | 209967 | closed | 0 | 3 | 2019-02-09T00:50:33Z | 2020-04-06T15:44:46Z | 2020-04-06T15:44:46Z | NONE | Hello there, This looks like a great tool. Thanks. Unfortunately, I hit the following error: ``` michael@hazel ~/src/cc-datasette/data/out datasette publish now cc-datasette.db > WARN! You are using an old version of the Now Platform. More: https://zeit.co/docs/v1-upgrade > Deploying /tmp/tmpjtrxwsyf/datasette under michaelmcandrew > Using project datasette > Error! You tried to create a Now 1.0 deployment. Please use Now 2.0 instead: https://zeit.co/upgrade ``` I'm guessing you might not hit this because you are not a 'new user' of Zeit (https://github.com/zeit/now-cli/issues/1805#issuecomment-452470953). Would it be a lot of work to upgrade to the new Zeit API, do you think? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/409/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
410384988 | MDU6SXNzdWU0MTAzODQ5ODg= | 411 | How to pass named parameter into spatialite MakePoint() function | 1055831 | closed | 0 | 3 | 2019-02-14T16:30:22Z | 2023-10-25T13:23:04Z | 2019-05-05T12:25:04Z | NONE | Hi, datasette version: "0.26.2" extensions: spatialite: "4.4.0-RC0" sqlite version: "3.22.0" I have a table of airports with latitude and longitude columns. I've added spatialite (with KNN support). After creating the db using csvs-to-sqlit, I run these commands to setup the spatialite tables: ``` conn.execute('SELECT InitSpatialMetadata(1)') conn.execute("SELECT AddGeometryColumn('airports', 'point_geom', 4326, 'POINT', 2);") conn.execute('''UPDATE airports SET point_geom = GeomFromText('POINT('||"longitude"||' '||"latitude"||')',4326);''') conn.execute("SELECT CreateSpatialIndex('airports', 'point_geom');") ``` I'm attempting to create a canned query and have this in my metadata.json file: ``` "find_airports_nearest_to_point":{ "sql":"SELECT a.pos AS rank, b.id, b.name, b.country, b.latitude AS latitude, b.longitude AS longitude, a.distance / 1000.0 AS dist_km FROM KNN AS a JOIN airports AS b ON (b.rowid = a.fid) WHERE f_table_name = \"airports\" AND ref_geometry = MakePoint( :Long , :Lat ) AND max_items = 10;"} ``` which doesn't seem to perform the templating of the name parameters correctly and I get no results. Have also tired: ``` MakePoint( || :Long || , || :Lat || ) ``` which returns this error: ``` near "||": syntax error ``` However I cannot seem to find the correct combination of named parameter syntax (:Lat) or sqlite concatenation operator to make it work. Any ideas if using named parameters inside functions is supported? Thanks Darren | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/411/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
418329842 | MDU6SXNzdWU0MTgzMjk4NDI= | 415 | Add query parameter to hide SQL textarea | 36796532 | closed | 0 | 3 | 2019-03-07T14:11:30Z | 2019-03-15T09:30:57Z | 2019-03-15T05:22:43Z | NONE | It would be cool if there was a query parameter to hide / remove the SQL textarea. Then I could simply save a bookmark for a certain query and open it to see the data without having to scroll below the (long) SQL query first. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/415/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
453131917 | MDU6SXNzdWU0NTMxMzE5MTc= | 502 | Exporting sqlite database(s)? | 7936571 | closed | 0 | 3 | 2019-06-06T16:39:53Z | 2021-04-03T05:16:54Z | 2019-06-11T18:50:42Z | NONE | I'm working on datasette from one computer. But if I want to work on it from another computer and want to copy the SQLite database(s) already on the Heroku datasette instance, how to I copy the database(s) to the second computer so that I can then update it and push to online via datasette's command line code that pushes code to Heroku? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/502/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
459936585 | MDU6SXNzdWU0NTk5MzY1ODU= | 527 | Unable to use rank when fts-table generated with csvs-to-sqlite | 2181410 | closed | 0 | 3 | 2019-06-24T14:49:48Z | 2019-06-24T15:21:18Z | 2019-06-24T15:09:10Z | NONE | Hi Simon. If i generate a fts-table with the csvs-to-sqlite f-option, I'm unable to use (in datasette's GUI) the internal ranking of the table for sorting or viewing, but if I generate the fts-table with the enable-fts argument from sqlite-utils, everyrthing works ok. Eg.: datasette, version 0.28 sqlite-utils, version 1.2.1 csvs-to-sqlite, version 0.9 No column named rank with these commands: $ csvs-to-sqlite minutes.csv minutes.db -f text_data $ datasette -i minutes.db select rank, * from minutes_fts where minutes_fts match 'dog' Everything ok with these commands: $ csvs-to-sqlite minutes.csv minutes.db $ sqlite-utils enable-fts minutes.db text_data $ datasette -i minutes.db select rank, * from minutes_fts where minutes_fts match 'dog' Am I doing something wrong? Thank you for a great application! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/527/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
499954048 | MDExOlB1bGxSZXF1ZXN0MzIyNTI5Mzgx | 578 | Added support for multi arch builds | 887095 | closed | 0 | 3 | 2019-09-29T18:43:03Z | 2019-11-13T19:13:15Z | 2019-11-13T19:13:15Z | NONE | simonw/datasette/pulls/578 | Minor changes in Dockerfile and new Makefile to support Docker multi architecture builds. `make`will build one image per architecture and push them as one Docker manifest to Docker Hub. Feel free to change `IMAGE_NAME ` to `datasetteproject/datasette` to update your official Docker Hub image(s). | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/578/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
505512251 | MDU6SXNzdWU1MDU1MTIyNTE= | 588 | Queries per DB table in metadata.json | 12617395 | closed | 0 | 3 | 2019-10-10T21:08:19Z | 2019-10-21T12:58:22Z | 2019-10-21T01:48:42Z | NONE | It doesn't appear possible to have separate queries defined per database table. When I do something like below, my table descriptions show up but not the queries: ` "databases": { "MYDB": { "tables": { "MYFIRSTTABLE": { "source": "Test", "source_url": "https://www.google.com", "queries": { "Query 1": { "sql": "select * from MYFIRSTTABLE", "title": "Query 1", "description": "This is the first query" }, } }, "MYSECONDTABLE": { "source":"Test2", "source_url":"https://www.google.com", "queries": { "Query 2" : { "sql":"select * from MYSECONDTABLE;", "title": "Query 2", "description":"This is the second query" } } } }` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/588/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
506183241 | MDU6SXNzdWU1MDYxODMyNDE= | 593 | make uvicorn optional dependancy (because not ok on windows python yet) | 4312421 | closed | 0 | 3 | 2019-10-12T12:51:07Z | 2019-10-13T06:22:08Z | 2019-10-13T06:22:07Z | NONE | would it be possible to: - remove uvicorn mandatory dependancy ? - eventually make a fallback to hypercorn ? reason: - uvloop not yet supported on Windows/Python-3.8 and below, may happen with Python-3.9 only. - it seems a 6 lines effort (but I'm not expert) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/593/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
506297048 | MDU6SXNzdWU1MDYyOTcwNDg= | 594 | upgrade to uvicorn-0.9 to be Python-3.8 friendly | 4312421 | closed | 0 | 3 | 2019-10-13T09:23:43Z | 2019-11-12T04:47:04Z | 2019-11-12T04:47:04Z | NONE | uvicorn-0.8 relies on websockets-0.7 which lacks python-3.8 compatiblity | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/594/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
534507142 | MDU6SXNzdWU1MzQ1MDcxNDI= | 69 | Feature request: enable extensions loading | 30607 | closed | 0 | 3 | 2019-12-08T08:06:25Z | 2022-02-05T00:04:25Z | 2020-10-16T18:42:49Z | NONE | Hi, it would be great to add a parameter that enables the load of a sqlite extension you need. Something like "-ext modspatialite". In this way your great tool would be even more comfortable and powerful. Thank you very much | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/69/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
539590148 | MDU6SXNzdWU1Mzk1OTAxNDg= | 651 | fts5 syntax error when using punctuation | 2181410 | closed | 0 | 3 | 2019-12-18T10:25:35Z | 2021-07-14T19:26:06Z | 2019-12-30T06:42:55Z | NONE | Hi Simon I get a syntax error when using punctuation or special characters in a fulltext search (using fts5). I created the virtual table using sqlite-utils' "enable-fts"-command. The same error appears on Niche Museums [https://www.niche-museums.com/browse/search?q=park.](https://www.niche-museums.com/browse/search?q=park.), but works fine in most of your other datasette-examples, e.g. register-of-members-interests [https://register-of-members-interests.datasettes.com/regmem-98dc8b7/items?_search=mins.](https://register-of-members-interests.datasettes.com/regmem-98dc8b7/items?_search=mins.) What am I doing wrong? Many thanks! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/651/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
555832585 | MDU6SXNzdWU1NTU4MzI1ODU= | 661 | --port option to expose a port other than 8001 in "datasette package" | 134771 | closed | 0 | 3 | 2020-01-27T21:05:56Z | 2020-01-30T04:17:52Z | 2020-01-29T22:46:45Z | NONE | I see how to alter the port using `datasette serve -p XXX` per the docs. However, I'm packaging up to server the container on AppEngine flexible, which [requires](https://cloud.google.com/appengine/docs/flexible/custom-runtimes/build#listening_to_port_8080) that the container is serving traffic on port 8080. https://github.com/simonw/datasette/blob/7950105c278b140e6cb665c68b59df219870f9bc/Dockerfile#L41 Is there a way to inject a non-default port into the Dockerfile, or should I just do something like `sed` to replace 8001 with 8080 after `dataset package` has done it's thing? Thanks for the advice. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/661/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
559197745 | MDU6SXNzdWU1NTkxOTc3NDU= | 82 | Tutorial command no longer works | 10350886 | closed | 0 | 3 | 2020-02-03T16:36:11Z | 2020-02-27T04:16:43Z | 2020-02-27T04:16:30Z | NONE | Issue with command on [tutorial](https://simonwillison.net/2019/Feb/25/sqlite-utils/) on Simon's site. The following command no longer works, and breaks with the previous too many variables error: #50 ``` cmd > curl "https://data.nasa.gov/resource/y77d-th95.json" | \ sqlite-utils insert meteorites.db meteorites - --pk=id ``` Output: ``` cmd Traceback (most recent call last): File "continuum\miniconda3\envs\main\lib\runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "continuum\miniconda3\envs\main\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "Continuum\miniconda3\envs\main\Scripts\sqlite-utils.exe\__main__.py", line 9, in <module> File "continuum\miniconda3\envs\main\lib\site-packages\click\core.py", line 764, in __call__ return self.main(*args, **kwargs) File "continuum\miniconda3\envs\main\lib\site-packages\click\core.py", line 717, in main rv = self.invoke(ctx) File "continuum\miniconda3\envs\main\lib\site-packages\click\core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "continuum\miniconda3\envs\main\lib\site-packages\click\core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "continuum\miniconda3\envs\main\lib\site-packages\click\core.py", line 555, in invoke return callback(*args, **kwargs) File "continuum\miniconda3\envs\main\lib\site-packages\sqlite_utils\cli.py", line 434, in insert default=default, File "continuum\miniconda3\envs\main\lib\site-packages\sqlite_utils\cli.py", line 384, in insert_upsert_implementation docs, pk=pk, batch_size=batch_size, alter=alter, **extra_kwargs File "continuum\miniconda3\envs\main\lib\site-packages\sqlite_utils\db.py", line 1081, in insert_all result = self.db.conn.execute(query, params) sqlite3.OperationalError: too many SQL variables ``` My thought is that maybe the dataset grew over the last few years and so didn't run into this issue before. No error… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/82/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
606720674 | MDU6SXNzdWU2MDY3MjA2NzQ= | 736 | strange behavior using accented characters | 30607 | closed | 0 | 3 | 2020-04-25T08:34:51Z | 2020-04-28T06:09:28Z | 2020-04-27T18:59:16Z | NONE | Hi, when I search `incompatibilità` [here](https://my-database.now.sh/commissioniComunePalermo/youtube), using full text search, it becomes `incompatibilitÃÂ ` and I have no result. If I encode the `à` char in the URL (`incompatibilit%C3%A0`) I have the right result. ![image](https://user-images.githubusercontent.com/30607/80275201-00a79380-86e0-11ea-865e-f7e1474e8098.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/736/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612378203 | MDU6SXNzdWU2MTIzNzgyMDM= | 757 | Question: Any fixed date for the release with the uft8-encoding fix? | 2181410 | closed | 0 | 3 | 2020-05-05T06:51:20Z | 2020-05-06T18:41:29Z | 2020-05-06T18:41:29Z | NONE | Just a little impatient :) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/757/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612673948 | MDU6SXNzdWU2MTI2NzM5NDg= | 759 | fts search on a column doesn't work anymore due to escape_fts | 133845 | closed | 0 | 3 | 2020-05-05T15:03:44Z | 2021-07-16T02:11:54Z | 2020-05-06T17:50:57Z | NONE | Hi and first, thank you for this awesome work you make with this projet. On a db indexed in full text search, I can't query on indexed column anymore. This request "cauvin language:ita": is running smoothly on a old version of datasette but not on the current version. Compare the current version query `select uuid, title, authors, year, series, language, formats, publisher, tags, identifiers from summary where rowid in (select rowid from summary_fts where summary_fts match escape_fts(:search)) order by uuid limit 101` To an older version: `select title, authors, series, uuid, language, identifiers, tags, publisher, formats, year, links from summary where rowid in (select rowid from summary_fts where summary_fts match :search) order by uuid limit 101` _language_ is a searchable column but now the search string is known as "cauvin language:ita" literally as a search term. columns are not parsed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/759/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
661605489 | MDU6SXNzdWU2NjE2MDU0ODk= | 900 | Some links don't honor base_url | 50220 | closed | 0 | 6026070 | 3 | 2020-07-20T09:40:50Z | 2020-10-23T19:44:04Z | 2020-10-15T22:57:55Z | NONE | Hi, I've been playing with Datasette behind Nginx (awesome tool, thanks !). It seems some URLs are OK but some aren't. For instance in https://github.com/simonw/datasette/blob/master/datasette/templates/query.html#L61 it seems that `url_csv` includes a `/` prefix, resulting in the `base_url` not beeing honored. Actually here, it seems that dropping the prefix `/` to make the link relative is enough (so it may not be strictly related to `base_url`). Additional information: ``` datasette, version 0.45+0.gf1f581b.dirty ``` Relevant Nginx configuration (note that all the trailing slashes have some effect): ``` location /datasette/ { proxy_pass http://127.0.0.1:9001/; proxy_set_header Host $host; } ``` Relelvant Datasette configuration (slashes matter too): ``` --config base_url:/datasette/ ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/900/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
705057955 | MDU6SXNzdWU3MDUwNTc5NTU= | 969 | Add --tar option to "datasette publish heroku" | 1448859 | closed | 0 | 5971510 | 3 | 2020-09-20T06:54:53Z | 2020-10-08T23:55:59Z | 2020-10-08T23:30:59Z | NONE | This issue is about how best to pass additional options to tools used for publishing datasettes. A concrete example is wanting to pass the `--tar` flag to the heroku CLI tool. I think there are at least two options for doing this: documentation for each publishing tool to explain how to set flags via env variables (if possible) or building a mechanism that lets users pass additional flags through datasette. When using `datasette publish heroku binder-launches.db --extra-options="--config facet_time_limit_ms:35000 --config sql_time_limit_ms:35000" --name=binderlytics --install=datasette-vega` to publish https://binderlytics.herokuapp.com/ the following error happens: ``` › Warning: heroku update available from 7.42.1 to 7.43.0. › Warning: heroku update available from 7.42.1 to 7.43.0. › Warning: heroku update available from 7.42.1 to 7.43.0. Setting WEB_CONCURRENCY and restarting ⬢ binderlytics... done, v13 WEB_CONCURRENCY: 1 › Warning: heroku update available from 7.42.1 to 7.43.0. ▸ Couldn't detect GNU tar. Builds could fail due to decompression errors ▸ See https://devcenter.heroku.com/articles/platform-api-deploying-slugs#create-slug-archive ▸ Please install it, or specify the '--tar' option ▸ Falling back to node's built-in compressor buffer.js:358 throw new ERR_INVALID_OPT_VALUE.RangeError('size', size); ^ RangeError [ERR_INVALID_OPT_VALUE]: The value "3303763968" is invalid for option "size" at Function.alloc (buffer.js:367:3) at new Buffer (buffer.js:281:19) at Readable.<anonymous> (/Users/thead/.local/share/heroku/node_modules/archiver-utils/index.js:39:15) at Readable.emit (events.js:322:22) at endReadableNT (/Users/thead/.local/share/heroku/node_modules/readable-stream/lib/_stream_readable.js:1010:12) at processTicksAndRejections (internal/process/task_queues.js:84:21) { code: 'ERR_INVALID_OPT_VALUE' } ``` After installing GNU tar with `brew install gnu-tar` and modifying `datasette/publish/heroku.py` to incl… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/969/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
783778672 | MDU6SXNzdWU3ODM3Nzg2NzI= | 220 | Better error message for *_fts methods against views | 649467 | closed | 0 | 3 | 2021-01-11T23:24:00Z | 2021-02-22T20:44:51Z | 2021-02-14T22:34:26Z | NONE | enable_fts and its related methods only work on tables, not views. Could those methods and possibly others move up to the Queryable superclass? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/220/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
787098146 | MDU6SXNzdWU3ODcwOTgxNDY= | 1190 | `datasette publish upload` mechanism for uploading databases to an existing Datasette instance | 1024355 | closed | 0 | 3 | 2021-01-15T18:18:42Z | 2023-08-30T22:16:39Z | 2023-08-30T22:16:38Z | NONE | If I have a self-hosted instance of Datasette up and running, I'd like to be able to the use the CLI to publish databases to that instance, not only Google or Heroku. Ideally there'd be a `url` parameter or something similar to which one could point the publish command to their instance. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1190/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
796736607 | MDU6SXNzdWU3OTY3MzY2MDc= | 56 | Not all quoted statuses get fetched? | 42315895 | closed | 0 | 3 | 2021-01-29T09:48:44Z | 2021-02-03T10:36:36Z | 2021-02-03T10:36:36Z | NONE | ![image](https://user-images.githubusercontent.com/42315895/106259325-5f75dc80-621f-11eb-8311-db8f2fe2a257.png) In my database I have 13300 quote tweets, but eta 3600 have `quoted_status` empty. I fetched some of them using `https://api.twitter.com/1.1/statuses/show.json?id=xx` and they did have ids of quoted tweets. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
807817197 | MDU6SXNzdWU4MDc4MTcxOTc= | 229 | Hitting `_csv.Error: field larger than field limit (131072)` | 631242 | closed | 0 | 3 | 2021-02-13T19:52:44Z | 2021-02-14T21:33:33Z | 2021-02-14T21:33:33Z | NONE | I have a csv file where one of the fields is so large it is throwing an exception with this error and stops loading: ``` _csv.Error: field larger than field limit (131072) ``` The stack trace occurs here: https://github.com/simonw/sqlite-utils/blob/3.1/sqlite_utils/cli.py#L633 There is a way to handle this that helps: https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072 One issue I had with this problem was sqlite-utils only provides limited context as to where the problem line is. There is the progress bar, but that is by percent rather than by line number. It would have been helpful if it could have provided a line number. Also, it would have been useful if it had allowed the loading to continue with later lines. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/229/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
814591962 | MDU6SXNzdWU4MTQ1OTE5NjI= | 1240 | Allow facetting on custom queries | 7107523 | closed | 0 | 3 | 2021-02-23T15:52:19Z | 2021-02-26T18:19:46Z | 2021-02-26T18:18:18Z | NONE | Facets are a tremendously useful feature, especially for people peeking at the database for the first time and still having little knowledge about the details of the data. It is of great assistance to discover interesting features to explore futher in advanced queries. Yet, it seems it's impossible to use facets when running a custom SQL query, be it from the little gear icons in column names, the facet suggestions at the top (hidden when performing a custom query), or by appending a facet code to the URL. Is there a technical limitation, or is this something that could be unlocked easily? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
815554385 | MDU6SXNzdWU4MTU1NTQzODU= | 237 | db["my_table"].drop(ignore=True) parameter, plus sqlite-utils drop-table --ignore and drop-view --ignore | 649467 | closed | 0 | 3 | 2021-02-24T14:55:06Z | 2021-02-25T17:11:41Z | 2021-02-25T17:11:41Z | NONE | When I'm generating a derived table in python, I often drop the table and create it from scratch. However, the first time I generate the table, it doesn't exist, so the drop raises an exception. That means more boilerplate. I was going to submit a pull request that adds an "if_exists" option to the `drop` method of tables and views. However, for a utility like sqlite_utils, perhaps the "IF EXISTS" SQL semantics is what you want most of the time, and thus should be the default. What do you think? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
832092321 | MDU6SXNzdWU4MzIwOTIzMjE= | 1261 | Some links aren't properly URL encoded. | 812795 | closed | 0 | 3 | 2021-03-15T18:43:59Z | 2021-03-21T02:06:44Z | 2021-03-20T21:36:06Z | NONE | It seems like a percent sign in the query causes some links to end invalid. The json and CSV links on this page don't behave like expected: https://honeypot-brimston3.vercel.app/honeypot?sql=select+time%2C+count%28time%29+as+count+from+%28select+strftime%28%22%25Y-%25m-%25d%22%2C+_etime%29+as+time+from+ssh+%29+group+by+time+order+by+time%3B I can take a swing at trying to fix this, but my python isn't strong and I need a pointer at the right approach and files to change. Thanks! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1261/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
836123030 | MDU6SXNzdWU4MzYxMjMwMzA= | 1265 | Support for HTTP Basic Authentication | 468612 | closed | 0 | 3 | 2021-03-19T15:31:09Z | 2021-03-19T22:05:12Z | 2021-03-19T21:03:09Z | NONE | It would be nice if datasette could support [HTTP Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication). For now I could ofcourse leverage Nginx for basic authentication, but it would be nice to have support for this in datasette by default or via a plugin like datasette-auth-github. My main usecase is to put the whole datasette instance behind a username/password prompt via Basic Auth and not specific urls. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1265/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
919250621 | MDU6SXNzdWU5MTkyNTA2MjE= | 269 | bool type not supported | 4068 | closed | 0 | 3 | 2021-06-11T22:00:36Z | 2021-06-15T01:34:10Z | 2021-06-15T01:34:10Z | NONE | Hi! Thank you for sharing this very nice tool :) It would be nice to have support for more types, like `bool`: it is not possible to convert to boolean at the moment. My suggestion would be to handle it as `bool(int(value))`, like csvkit does. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/269/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925677191 | MDU6SXNzdWU5MjU2NzcxOTE= | 289 | Mypy fixes for rows_from_file() | 857609 | closed | 0 | 3 | 2021-06-20T20:34:59Z | 2021-06-22T18:44:36Z | 2021-06-22T18:13:26Z | NONE | Following https://github.com/simonw/sqlite-utils/issues/279#issuecomment-864328927 You had two mypy errors. The first: > sqlite_utils/utils.py:157: error: Argument 1 to "BufferedReader" has incompatible type "BinaryIO"; expected "RawIOBase" Looking at the `BufferedReader` docs, it seems to expect a `RawIOBase`, and this [has been copied into typeshed](https://github.com/python/typeshed/blob/9ec2f8712480c57353cea097a65d75a2c4ec1846/stdlib/io.pyi#L100). There may be scope to change how `BufferedReader` is documented and typed upstream, but for now it wouldn't be too bad to use a `typing.cast()`: ``` # Detect the format, then call this recursively buffered = io.BufferedReader( cast(io.RawIOBase, fp), # Undocumented BufferedReader support for BinaryIO buffer_size=4096, ) ``` The second error seems to be flagging a legitimate bug in your code: > sqlite_utils/utils.py:163: error: Argument 1 to "decode" of "bytes" has incompatible type "Optional[str]"; expected "str" From your type hints, `encoding` may be `None`. In the CSV format block, you use `encoding or "utf-8-sig"` to set a default, maybe that's desirable in this case too? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/289/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
991575770 | MDExOlB1bGxSZXF1ZXN0NzMwMDIwODY3 | 1467 | Add Authorization header when CORS flag is set | 3058200 | closed | 0 | 3 | 2021-09-08T22:14:41Z | 2021-10-17T02:29:07Z | 2021-10-14T18:54:18Z | NONE | simonw/datasette/pulls/1467 | This PR adds the [`Access-Control-Allow-Headers`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers) flag when CORS mode is enabled. This would fix https://github.com/simonw/datasette-auth-tokens/issues/4. When making cross-origin requests, the server must respond with all allowable HTTP headers. A Datasette instance using auth tokens must accept the `Authorization` HTTP header in order for cross-origin authenticated requests to take place. Please let me know if there's a better way of doing this! I couldn't figure out a way to change the app's response from the plugin itself, so I'm starting here. If you'd rather this logic live in the plugin, I'd love any guidance you're able to give. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1467/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1004613267 | I_kwDOCGYnMM474S6T | 328 | Invalid JSON output when no rows | 12752 | closed | 0 | 3 | 2021-09-22T18:37:26Z | 2021-09-22T20:21:34Z | 2021-09-22T20:20:18Z | NONE | `sqlite-utils query` generates a JSON output with the result from the query: ```json [{...},{...}] ``` If no rows are returned by the query, I'm expecting an empty JSON array: ```json [] ``` But actually I'm getting an empty string. To be consistent, the output should be `[]` when the request succeeds (return code == `0`). | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/328/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1039037439 | PR_kwDOCGYnMM4t0uaI | 333 | Add functionality to read Parquet files. | 2118708 | closed | 0 | 3 | 2021-10-28T23:43:19Z | 2021-11-25T19:47:35Z | 2021-11-25T19:47:35Z | NONE | simonw/sqlite-utils/pulls/333 | I needed this for a project of mine, and I thought it'd be useful to have it in sqlite-utils (It's also mentioned in #248 ). The current implementation works (data is read & data types are inferred correctly. I've added a single straightforward test case, but @simonw please let me know if there are any non-obvious flags/combinations I should test too. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/333/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1075893249 | I_kwDOBm6k_c5AINQB | 1545 | Custom pages don't work on windows | 559711 | closed | 0 | 3 | 2021-12-09T18:53:05Z | 2022-02-03T02:08:31Z | 2022-02-03T01:58:35Z | NONE | It seems that custom pages don't work when put in templates/pages To reproduce on datasette version 0.59.4 using PowerShell on WIndows 10 with Python 3.10.0 mkdir -p templates/pages echo "hello world" >> templates/pages/about.html Start datasette datasette --template-dir templates/ Navigate to [http://127.0.0.1:8001/about](url) and receive: Error 404: Database not found: about | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1545/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1123903919 | I_kwDOCGYnMM5C_Wmv | 397 | Support IF NOT EXISTS for table creation | 738408 | closed | 0 | 3 | 2022-02-04T07:41:15Z | 2022-02-06T01:30:46Z | 2022-02-06T01:29:01Z | NONE | Currently, I have a bunch of code that looks like this: ```python subjects = db["subjects"] if db["subjects"].exists() else db["subjects"].create({ ... }) ``` It would be neat if sqlite-utils could simplify that by supporting `CREATE TABLE IF NOT EXISTS`, so that I'd be able to write, e.g. ```python subjects = db["subjects"].create({...}, if_not_exists=True) ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/397/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1157182254 | I_kwDOBm6k_c5E-TMu | 1646 | Configuration directory mode does not pick up other file extensions than .db | 15640196 | closed | 0 | 3 | 2022-03-02T13:15:23Z | 2022-10-07T23:06:17Z | 2022-10-07T23:03:35Z | NONE | Hello, I've been trying to run Datasette with the [configuration directory mode](https://docs.datasette.io/en/stable/settings.html#configuration-directory-mode) with a structure such as this one: ```plain some-directory/ example.sqlite3 another-example.db one-more.custom [...] ``` (In my scenario I can't just change the filename extension without other problems arising) Now databases with the `.sqlite3` or the custom filename extension are ignored by Datasette in this case. I'm aware that the docs state that a `.db` extension is required, but I was wondering if there is a reason for restricting this or any workaround available? When I run `datasette example.sqlite3` or `datasette one-more.custom` the databases are served by Datasette without a problem. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1646/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1175744654 | I_kwDOCGYnMM5GFHCO | 417 | insert fails on JSONL with whitespace | 9954 | closed | 0 | 3 | 2022-03-21T17:58:14Z | 2022-03-25T21:19:06Z | 2022-03-25T21:17:13Z | NONE | Any JSON that is newline-delimited and has whitespace (newlines) between the start of a JSON object and an attribute fails due to a parse error. e.g. given the valid JSONL: ```{ "attribute": "value" } { "attribute": "value2" } ``` I would expect that `sqlite-utils insert --nl my.db mytable file.jsonl` would properly import the data into `mytable`. However, the following error is thrown instead: `json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)` It makes sense that since the file is intended to be newline separated, the thing being parsed is "{" (which obviously fails), however the default newline-separated output of `jq` isn't compact. Using `jq -c` avoids this problem, but the fix is unintuitive and undocumented. Proposed solutions: 1. Default to a "loose" newline-separated parse; this could be implemented internally as [the equivalent of] a `jq -c` filter ahead of the insert step. 2. Catch the JSONDecodeError (or pre-empt it in the case of a record === "{\n") and give the user a "it looks like your json isn't _actually_ newline-delimited; try running it through `jq -c` instead" error message. It might just have been too early in the morning when I was playing with this, but running pipes of data through sqlite-utils without the 'knack' of it led to some false starts. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/417/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1243151184 | I_kwDOCGYnMM5KGPtQ | 434 | `detect_fts()` identifies the wrong table if tables have names that are subsets of each other | 559711 | closed | 0 | 3 | 2022-05-20T13:28:31Z | 2022-06-14T23:24:09Z | 2022-06-14T23:24:09Z | NONE | Windows 10 Python 3.9.6 When I was running a full text search through the Python library, I noticed that the query was being run on a different full text search table than the one I was trying to search. I took a look at the following function https://github.com/simonw/sqlite-utils/blob/841ad44bacaff05ec79ef78166d12e80c82ba6d7/sqlite_utils/db.py#L2213 and noticed: ```python sql LIKE '%VIRTUAL TABLE%USING FTS%content=%{table}%' ``` My database contains tables with similar names and %{table}% was matching another table that ended differently in its name. I have included a sample test that shows this occurring: I search for Marsupials in db["books"] and The Clue of the Broken Blade is returned. This occurs since the search for Marsupials was "successfully" done against db["booksb"] and rowid 1 is returned. "The Clue of the Broken Blade" has a rowid of 1 in db["books"] and this is what is returned from the search. ```python def test_fts_search_with_similar_table_names(fresh_db): db = Database(memory=True) db["books"].insert_all( [ { "title": "The Clue of the Broken Blade", "author": "Franklin W. Dixon", }, { "title": "Habits of Australian Marsupials", "author": "Marlee Hawkins", }, ] ) db["booksb"].insert( { "title": "Habits of Australian Marsupials", "author": "Marlee Hawkins", } ) db["booksb"].enable_fts(["title", "author"]) db["books"].enable_fts(["title", "author"]) query = "Marsupials" assert [ { "rowid": 1, "title": "Habits of Australian Marsupials", "author": "Marlee Hawkins", }, ] == list(db["books"].search(query)) ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/434/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1246826792 | I_kwDODLZ_YM5KUREo | 10 | When running `auth` command, don't overwrite an existing auth.json file | 11887 | closed | 0 | 3 | 2022-05-24T16:42:20Z | 2022-09-07T15:07:38Z | 2022-08-22T16:17:19Z | NONE | Ran the `auth` command in the same directory I'd previously set up an auth.json file for `twitter-to-sqlite` and it was completely overwritten. Not the biggest issue, but still unexpected. Ideally, for me, the keys would just be added to the existing file, but getting a warning and a chance to back out would be a good solution as well. | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1345452427 | I_kwDODLZ_YM5QMfmL | 11 | -a option is used for "--auth" and for "--all" | 2467 | closed | 0 | 3 | 2022-08-21T10:50:48Z | 2022-08-21T21:11:57Z | 2022-08-21T21:11:57Z | NONE | I'm not sure which option is best, instead of -a -all. | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1388631785 | I_kwDOBm6k_c5SxNbp | 1826 | render_cell documentation example doesn't match the method signature | 66709385 | closed | 0 | 3 | 2022-09-28T02:37:59Z | 2022-09-28T04:30:28Z | 2022-09-28T04:05:16Z | NONE | Open Datasette stable doc at https://docs.datasette.io/en/stable/plugin_hooks.html?highlight=render_cell#render-cell-row-value-column-table-database-datasette render_cell plugin hook method signature is `render_cell(row, value, column, table, database, datasette)`, the example shown inline uses `render_cell(value)`. ![image](https://user-images.githubusercontent.com/66709385/192674691-34265b81-6cdd-41d2-8424-aa12f8bc8c94.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1826/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1392690202 | I_kwDOCGYnMM5TAsQa | 495 | Support JSON values returned from .convert() functions | 649467 | closed | 0 | 3 | 2022-09-30T16:33:49Z | 2022-10-25T21:23:37Z | 2022-10-25T21:23:28Z | NONE | When using the convert function on a JSON column, the result of the conversion function must be a string. If the return value is either a dict (object) or a list (array), the convert call will error out with an unhelpful user defined function exception. It makes sense that since the original column value was a string and required conversion to data structures, the result should be converted back into a JSON string as well. However, other functions auto-convert to JSON string representation, so the fact that convert doesn't could be surprising. At least the documentation should note this requirement, because the sqlite error messages won't readily reveal the issue. Jf only sqlite's JSON column type meant something :) | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/495/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |