github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
562085508 | MDExOlB1bGxSZXF1ZXN0MzcyNzYzOTA2 | 666 | Use inspect-file, if possible, for total row count | 13896256 | closed | 0 | 3 | 2020-02-08T22:10:35Z | 2020-03-09T02:47:15Z | 2020-02-25T20:19:29Z | CONTRIBUTOR | simonw/datasette/pulls/666 | For large tables, counting the number of rows in the table can take a signficant amount of time. Instead, where an inspect-file is provided for an immutable database, look up the row-count for a plain count(*). | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/666/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
569253072 | MDU6SXNzdWU1NjkyNTMwNzI= | 678 | prepare_connection() plugin hook should accept optional datasette argument | 9599 | closed | 0 | 3 | 2020-02-22T00:50:26Z | 2020-02-22T03:53:19Z | 2020-02-22T02:28:51Z | OWNER | I want to build a plugin that allows users to configure certain database columns to be "masked" - so the `password` column on a users table is never revealed, for example. To do this, I need to use the `conn.set_authorizer()` SQLite mechanism. So the plugin needs to build off the `prepare_connection(conn)` hook. But that hook doesn't currently get passed `datasette` so it doesn't have a way of looking up its plugin configuration! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/678/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
573578548 | MDU6SXNzdWU1NzM1Nzg1NDg= | 89 | Ability to customize columns used by extracts= feature | 9599 | open | 0 | 3 | 2020-03-01T16:54:48Z | 2020-10-16T19:17:50Z | OWNER | @simonw any thoughts on allow extracts to specify the lookup column name? If I'm understanding the documentation right, `.lookup()` allows you to define the "value" column (the documentation uses name), but when you use `extracts` keyword as part of `.insert()`, `.upsert()` etc. the lookup must be done against a column named "value". I have an existing lookup table that I've populated with columns "id" and "name" as opposed to "id" and "value", and seems I can't use `extracts=`, unless I'm missing something... Initial thought on how to do this would be to allow the dictionary value to be a tuple of table name column pair... so: ``` table = db.table("trees", extracts={"species_id": ("Species", "name"}) ``` I haven't dug too much into the existing code yet, but does this make sense? Worth doing? _Originally posted by @chrishas35 in https://github.com/simonw/sqlite-utils/issues/46#issuecomment-592999503_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/89/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
574043218 | MDU6SXNzdWU1NzQwNDMyMTg= | 693 | Variables from extra_template_vars() not exposed in _context=1 | 9599 | closed | 0 | 3 | 2020-03-02T15:14:51Z | 2020-04-05T19:12:48Z | 2020-04-05T19:12:48Z | OWNER | The `_context=1` debugging mode does not show variables that should have been added to the context by the `extra_template_vars()` plugin hook. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/693/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
583970196 | MDU6SXNzdWU1ODM5NzAxOTY= | 701 | Search box CSS doesn't look great on OS X Safari | 9599 | closed | 0 | 5234079 | 3 | 2020-03-18T20:00:52Z | 2020-03-24T22:57:18Z | 2020-03-24T22:57:18Z | OWNER | <img width="1447" alt="twitter__tweets__68_773_rows_where_sorted_by_id_descending_and_twitter__tweets__955_rows" src="https://user-images.githubusercontent.com/9599/77002262-73bb2e80-6918-11ea-87a3-e3f537365c9f.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/701/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
585353598 | MDU6SXNzdWU1ODUzNTM1OTg= | 37 | Handle "User not found" error | 9599 | closed | 0 | 3 | 2020-03-20T22:14:32Z | 2020-04-17T23:43:46Z | 2020-04-17T23:43:46Z | MEMBER | While running `user-timeline` I got this bug (because a screen name I asked for didn't exist): ``` File "/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py", line 185, in transform_user user["created_at"] = parser.parse(user["created_at"]) KeyError: 'created_at' >>> import pdb >>> pdb.pm() > /Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py(185)transform_user() -> user["created_at"] = parser.parse(user["created_at"]) (Pdb) user {'errors': [{'code': 50, 'message': 'User not found.'}]} ``` | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
585597133 | MDExOlB1bGxSZXF1ZXN0MzkxOTI0NTA5 | 703 | WIP implementation of writable canned queries | 9599 | closed | 0 | 3 | 2020-03-21T22:23:51Z | 2020-06-03T00:08:14Z | 2020-06-02T23:57:35Z | OWNER | simonw/datasette/pulls/703 | Refs #698. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/703/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | |||||
598013965 | MDU6SXNzdWU1OTgwMTM5NjU= | 724 | --plugin-secret over-rides existing metadata.json plugin config | 9599 | closed | 0 | 3 | 2020-04-10T17:56:30Z | 2020-04-16T04:58:12Z | 2020-04-10T18:34:21Z | OWNER | This means if you use `--plugin-secret` at all (with e.g. `publish cloudrun`) any existing plugin configuration in your `metadata.json` will be ignored. https://github.com/simonw/datasette/blob/af9cd4ca64652fae262e6f7b5d201f6e0adc989b/datasette/publish/cloudrun.py#L98-L109 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/724/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
598891570 | MDExOlB1bGxSZXF1ZXN0NDAyNjQ1OTg0 | 725 | Update aiofiles requirement from ~=0.4.0 to >=0.4,<0.6 | 27856297 | closed | 0 | 3 | 2020-04-13T13:32:47Z | 2020-05-04T18:16:54Z | 2020-05-04T16:17:49Z | CONTRIBUTOR | simonw/datasette/pulls/725 | Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/Tinche/aiofiles/commit/9a2141e6c51013995ed8b72b121e4e0a27af77b5"><code>9a2141e</code></a> 0.5.0</li> <li><a href="https://github.com/Tinche/aiofiles/commit/479b7eea2c54a9daadb26d1073770e5a50f9ca27"><code>479b7ee</code></a> Update README</li> <li><a href="https://github.com/Tinche/aiofiles/commit/6c247a2cfd1e12f8359aeff04bfe3510f5ef0dc2"><code>6c247a2</code></a> Modernize tests</li> <li><a href="https://github.com/Tinche/aiofiles/commit/eec75d3f425d53306b6aaa8a6005344e56350312"><code>eec75d3</code></a> Switch to async def wherever possible</li> <li><a href="https://github.com/Tinche/aiofiles/commit/786c3e9293a37f63ae1c72b115498d038b5828a0"><code>786c3e9</code></a> Prepare for 3.8</li> <li><a href="https://github.com/Tinche/aiofiles/commit/145107595fe72b877005fb4ebf9d77ffb754689a"><code>1451075</code></a> Update README.rst</li> <li><a href="https://github.com/Tinche/aiofiles/commit/5db1e380e307d0f96177d18db778f82ea0ef87ce"><code>5db1e38</code></a> Add several async os functions</li> <li><a href="https://github.com/Tinche/aiofiles/commit/a60f19bb7b6bd5b063f853d8cae2c41a6878841f"><code>a60f19b</code></a> Add async remove function</li> <li><a href="https://github.com/Tinche/aiofiles/commit/9cf2ac8f9d43446ecdf71c95d05ad6a95bec7377"><code>9cf2ac8</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/Tinche/aiofiles/issues/53">#53</a> from graingert/patch-1</li> <li><a href="https://github.com/Tinche/aiofiles/commit/b88912c6a7a13597b8c5ac93093d7eef86ab1ebb"><code>b88912c</code></a> <strong>all</strong> should be a List[str]</li> <li>Additional commits viewable in <a href="https://github.com/Tinche/aiofiles/compare/v0.4.0...v0.5.0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/725/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
601333634 | MDU6SXNzdWU2MDEzMzM2MzQ= | 28 | Pull repository contributors | 9599 | closed | 0 | 3 | 2020-04-16T18:46:40Z | 2020-04-18T15:05:10Z | 2020-04-18T15:05:10Z | MEMBER | https://developer.github.com/v3/repos/#list-contributors `GET /repos/:owner/:repo/contributors` Not sure if this should be a separate command or should be part of the existing `repos` command. I'm leaning towards a new `contributors` command. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
601358649 | MDU6SXNzdWU2MDEzNTg2NDk= | 100 | Mechanism for forcing column-type, over-riding auto-detection | 9599 | closed | 0 | 3 | 2020-04-16T19:12:52Z | 2020-04-17T23:53:32Z | 2020-04-17T23:53:32Z | OWNER | As seen in https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614843406 - there's a problem where you insert a record with a `None` value for a column and that column is created as `TEXT` - but actually you intended it to be an `INT` (as later examples will demonstrate). Some kind of mechanism for over-riding the detected types of columns would be useful here. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/100/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
603624862 | MDU6SXNzdWU2MDM2MjQ4NjI= | 31 | Issue and milestone should have foreign key to repo | 9599 | closed | 0 | 3 | 2020-04-21T00:46:24Z | 2020-04-22T01:20:19Z | 2020-04-22T01:20:19Z | MEMBER | Currently the `repo` column on those tables is a string `simonw/datasette` rather than an ID referencing a row in `repos`. _Originally posted by @simonw in https://github.com/dogsheep/github-to-sqlite/issues/29#issuecomment-616883275_ | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
604222295 | MDU6SXNzdWU2MDQyMjIyOTU= | 32 | Issue comments don't appear to populate issues foreign key | 9599 | closed | 0 | 3 | 2020-04-21T19:17:32Z | 2020-04-22T01:17:44Z | 2020-04-22T01:17:44Z | MEMBER | https://github-to-sqlite.dogsheep.net/github?sql=select+html_url%2C+id%2C+issue+from+issue_comments+order+by+updated_at+desc+limit+101 <img width="774" alt="Screen Shot 2020-04-21 at 12 16 57 PM" src="https://user-images.githubusercontent.com/9599/79904752-0b091b00-83ca-11ea-9b15-b2893f05dfdc.png"> | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
606720674 | MDU6SXNzdWU2MDY3MjA2NzQ= | 736 | strange behavior using accented characters | 30607 | closed | 0 | 3 | 2020-04-25T08:34:51Z | 2020-04-28T06:09:28Z | 2020-04-27T18:59:16Z | NONE | Hi, when I search `incompatibilità` [here](https://my-database.now.sh/commissioniComunePalermo/youtube), using full text search, it becomes `incompatibilitÃÂ ` and I have no result. If I encode the `à` char in the URL (`incompatibilit%C3%A0`) I have the right result. ![image](https://user-images.githubusercontent.com/30607/80275201-00a79380-86e0-11ea-865e-f7e1474e8098.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/736/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
607107849 | MDExOlB1bGxSZXF1ZXN0NDA5MTUzODcw | 739 | Configuration directory mode | 9599 | closed | 0 | 3 | 2020-04-26T20:37:46Z | 2020-04-27T16:30:25Z | 2020-04-27T16:30:25Z | OWNER | simonw/datasette/pulls/739 | Refs #731 TODO: - [x] Decide how to combine explicit command-line options with items detected from the directory structure - [x] Add unit tests - [x] Implement `inspect-data.json` mechanism for populating `immutables` - [x] Add documentation | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/739/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
610511450 | MDU6SXNzdWU2MTA1MTE0NTA= | 35 | Create index on issue_comments(user) and other foreign keys | 9599 | closed | 0 | 3 | 2020-05-01T02:06:56Z | 2020-05-02T18:26:24Z | 2020-05-02T18:26:24Z | MEMBER | ``` create index issue_comments_user on issue_comments(user) ``` I'm sure there are other user columns that could benefit from an index. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610843136 | MDU6SXNzdWU2MTA4NDMxMzY= | 37 | Mechanism for creating views if they don't yet exist | 9599 | closed | 0 | 3 | 2020-05-01T16:34:10Z | 2020-05-02T16:19:47Z | 2020-05-02T16:19:31Z | MEMBER | Needed for #36 #10 #12 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611252244 | MDU6SXNzdWU2MTEyNTIyNDQ= | 750 | Add notlike table filter | 9599 | closed | 0 | 3 | 2020-05-02T18:54:36Z | 2020-05-02T19:10:44Z | 2020-05-02T19:10:44Z | OWNER | I found myself wanting that for applying the opposite of this: https://github-to-sqlite.dogsheep.net/github/dependent_repos?dependent__like=%25simonw%2F%25&_sort_desc=dependent_stars | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/750/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612378203 | MDU6SXNzdWU2MTIzNzgyMDM= | 757 | Question: Any fixed date for the release with the uft8-encoding fix? | 2181410 | closed | 0 | 3 | 2020-05-05T06:51:20Z | 2020-05-06T18:41:29Z | 2020-05-06T18:41:29Z | NONE | Just a little impatient :) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/757/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612673948 | MDU6SXNzdWU2MTI2NzM5NDg= | 759 | fts search on a column doesn't work anymore due to escape_fts | 133845 | closed | 0 | 3 | 2020-05-05T15:03:44Z | 2021-07-16T02:11:54Z | 2020-05-06T17:50:57Z | NONE | Hi and first, thank you for this awesome work you make with this projet. On a db indexed in full text search, I can't query on indexed column anymore. This request "cauvin language:ita": is running smoothly on a old version of datasette but not on the current version. Compare the current version query `select uuid, title, authors, year, series, language, formats, publisher, tags, identifiers from summary where rowid in (select rowid from summary_fts where summary_fts match escape_fts(:search)) order by uuid limit 101` To an older version: `select title, authors, series, uuid, language, identifiers, tags, publisher, formats, year, links from summary where rowid in (select rowid from summary_fts where summary_fts match :search) order by uuid limit 101` _language_ is a searchable column but now the search string is known as "cauvin language:ita" literally as a search term. columns are not parsed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/759/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612860531 | MDU6SXNzdWU2MTI4NjA1MzE= | 17 | Only install osxphotos if running on macOS | 9599 | closed | 0 | 3 | 2020-05-05T20:03:26Z | 2020-05-05T20:20:05Z | 2020-05-05T20:11:23Z | MEMBER | The build is broken right now because you can't `pip install osxphotos` on Ubuntu. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613422636 | MDU6SXNzdWU2MTM0MjI2MzY= | 760 | Way of seeing full schema for a database | 9599 | open | 0 | 3 | 2020-05-06T15:46:08Z | 2020-05-06T23:49:06Z | OWNER | I find myself wanting to quickly figure out all of the BLOB columns in a database. A `/-/schema` page showing the full schema (actually since it's per-database probably `/dbname/-/schema` or `/-/schema/dbname`) would be really handy. It would need to be carefully constructed from various queries against `sqlite_master` - just doing `select * from sqlite_master where type='table'` isn't quite enough because I also want to show indexes, triggers etc. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/760/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
616012427 | MDU6SXNzdWU2MTYwMTI0Mjc= | 764 | Add PyPI project urls to setup.py | 9599 | closed | 0 | 5471110 | 3 | 2020-05-11T16:23:08Z | 2020-05-27T20:21:36Z | 2020-05-11T18:28:55Z | OWNER | Spotted this example here: ```python project_urls={ "Issues": "https://gitlab.com/Cyb3r-Jak3/ExifReader/issues", "Source Code": "https://gitlab.com/Cyb3r-Jak3/ExifReader/-/tree/publish", "CI": "https://gitlab.com/Cyb3r-Jak3/ExifReader/pipelines", "Releases": "https://github.com/Cyb3r-Jak3/ExifReader" }, ``` Results in this on https://pypi.org/project/ExifReader/ <img width="333" alt="ExifReader_·_PyPI" src="https://user-images.githubusercontent.com/9599/81585430-03a1b580-9369-11ea-9b58-e2f732cc13f8.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/764/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
626663119 | MDU6SXNzdWU2MjY2NjMxMTk= | 781 | request.url and request.scheme should obey force_https_urls config setting | 9599 | closed | 0 | 3 | 2020-05-28T16:54:47Z | 2020-05-28T17:39:54Z | 2020-05-28T17:10:13Z | OWNER | I'm trying to get the https://www.niche-museums.com/browse/feed.atom feed to validate and I git this from https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom > This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations. > > [line 6](https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom#l6), column 73: Self reference doesn't match document location [[help](https://validator.w3.org/feed/docs/warning/SelfDoesntMatchLocation.html "more information about this error")] > > <link href="http://www.niche-museums.com/browse/feed.atom" rel="self"/> I tried to fix this using `force_https_urls` ([commit](https://github.com/simonw/museums/commit/5dc8e2c717c59f9e949b65e47a59878e01f929e4)) but it didn't work - because that setting isn't respected by the Request class: https://github.com/simonw/datasette/blob/40885ef24e32d91502b6b8bbad1c7376f50f2830/datasette/utils/asgi.py#L15-L32 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/781/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
629524205 | MDU6SXNzdWU2Mjk1MjQyMDU= | 793 | CSRF protection for /-/messages tool and writable canned queries | 9599 | closed | 0 | 5512395 | 3 | 2020-06-02T21:22:21Z | 2020-06-06T00:43:41Z | 2020-06-05T19:05:59Z | OWNER | > The `/-/messages` debug tool will need CSRF protection or people will be able to add messages using a hidden form on another website. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/790#issuecomment-637790860_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/793/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
630120235 | MDU6SXNzdWU2MzAxMjAyMzU= | 797 | Documentation for new "params" setting for canned queries | 9599 | closed | 0 | 5512395 | 3 | 2020-06-03T15:55:11Z | 2020-06-09T04:00:40Z | 2020-06-03T21:04:51Z | OWNER | Added here: https://github.com/simonw/datasette/commit/aa82d0370463580f2cb10d9617f1bcbe45cc994a#diff-5e0ffd62fced7d46339b9b2cd167c2f9R236 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/797/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
634112607 | MDU6SXNzdWU2MzQxMTI2MDc= | 812 | Ability to customize what happens when a view permission fails | 9599 | closed | 0 | 5533512 | 3 | 2020-06-08T04:26:14Z | 2020-07-01T04:17:46Z | 2020-07-01T04:17:45Z | OWNER | Currently view permission failures raise a `Forbidden` error which is transformed into a 403. It would be good if this page could offer a way forward - maybe just by linking to (or redirecting to) a login screen. This behaviour will vary based on authentication plugins, so a new plugin hook is probably the best way to do this. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/812/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
635107393 | MDU6SXNzdWU2MzUxMDczOTM= | 823 | Documentation is inconsistent about "id" as required field on actor | 9599 | closed | 0 | 5512395 | 3 | 2020-06-09T04:47:58Z | 2020-06-09T14:58:36Z | 2020-06-09T14:58:19Z | OWNER | Docs at https://github.com/simonw/datasette/blob/5a6a73e3190cac103906b479d56129413e5ef190/docs/authentication.rst#actors say: > The only required field in an actor is `"id"`, which must be a string. But the example here returns `{"token": token}`: ```python @hookimpl def actor_from_request(datasette, request): async def inner(): token = request.args.get("_token") if not token: return None # Look up ?_token=xxx in sessions table result = await datasette.get_database().execute( "select count(*) from sessions where token = ?", [token] ) if result.first()[0]: return {"token": token} else: return None return inner ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/823/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
635519358 | MDU6SXNzdWU2MzU1MTkzNTg= | 826 | Document the ds_actor signed cookie | 9599 | closed | 0 | 5512395 | 3 | 2020-06-09T15:06:52Z | 2020-06-09T22:33:12Z | 2020-06-09T22:32:31Z | OWNER | Most authentication plugins (https://github.com/simonw/datasette-auth-github for example) are likely to work by setting the `ds_actor` signed cookie, which is already magically decoded and supported by default Datasette here: https://github.com/simonw/datasette/blob/4fa7cf68536628344356d3ef8c92c25c249067a0/datasette/actor_auth_cookie.py#L1-L13 I should document this. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/826/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
635914822 | MDU6SXNzdWU2MzU5MTQ4MjI= | 828 | Horizontal scrollbar on changelog page on mobile | 9599 | closed | 0 | 5512395 | 3 | 2020-06-10T04:18:54Z | 2020-06-10T04:28:17Z | 2020-06-10T04:28:17Z | OWNER | You can scroll sideways on that page and it looks bad: <img width="419" alt="Changelog_—_Datasette_documentation" src="https://user-images.githubusercontent.com/9599/84226483-b8c5ab80-aa96-11ea-8fa7-db0c37ab9441.png"> The cause is these long links: <img width="452" alt="Changelog_—_Datasette_documentation" src="https://user-images.githubusercontent.com/9599/84226511-d09d2f80-aa96-11ea-8bd9-891c07098516.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/828/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
636511683 | MDU6SXNzdWU2MzY1MTE2ODM= | 830 | Redesign register_facet_classes plugin hook | 9599 | open | 0 | 3268330 | 3 | 2020-06-10T20:03:27Z | 2021-12-16T19:58:22Z | OWNER | Nothing uses this plugin hook yet, so the design is not yet proven. I'm going to build a real plugin against it and use that process to inform any design changes that may need to be made. I'll add a warning about this to the documentation. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/830/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
639542974 | MDU6SXNzdWU2Mzk1NDI5NzQ= | 47 | Fall back to FTS4 if FTS5 is not available | 73579 | open | 0 | 3 | 2020-06-16T10:11:23Z | 2020-06-17T20:13:48Z | NONE | got this with version 0.21.1 from pypi. twitter-to-sqlite auth worked but then "twitter-to-sqlite user-timeline USER.db" produced a tracekback ending in "no such module: FTS5". | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
642296989 | MDU6SXNzdWU2NDIyOTY5ODk= | 856 | Consider pagination of canned queries | 9599 | open | 0 | 3 | 2020-06-20T03:15:59Z | 2021-05-21T14:22:41Z | OWNER | The new `canned_queries()` plugin hook from #852 combined with plugins like https://github.com/simonw/datasette-saved-queries could mean that some installations end up with hundreds or even thousands of canned queries. I should consider pagination or some other way of ensuring that this doesn't cause performance problems for Datasette. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/856/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
642652808 | MDU6SXNzdWU2NDI2NTI4MDg= | 861 | Script to generate larger SQLite test files | 9599 | closed | 0 | 3 | 2020-06-21T22:30:58Z | 2020-06-23T03:44:18Z | 2020-06-23T03:44:18Z | OWNER | > I'll write a little script which generates a 300MB SQLite file with a bunch of tables with lots of randomly generated rows in to help test this. > > Having a tool like that which can generate larger databases with different gnarly performance characteristics will be useful for other performance work too. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/859#issuecomment-647189948_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/861/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
644161221 | MDU6SXNzdWU2NDQxNjEyMjE= | 117 | Support for compound (composite) foreign keys | 9599 | open | 0 | 3 | 2020-06-23T21:33:42Z | 2020-06-23T21:40:31Z | OWNER | It turns out SQLite supports composite foreign keys: https://www.sqlite.org/foreignkeys.html#fk_composite Their example looks like this: ```sql CREATE TABLE album( albumartist TEXT, albumname TEXT, albumcover BINARY, PRIMARY KEY(albumartist, albumname) ); CREATE TABLE song( songid INTEGER, songartist TEXT, songalbum TEXT, songname TEXT, FOREIGN KEY(songartist, songalbum) REFERENCES album(albumartist, albumname) ); ``` Here's what that looks like in sqlite-utils: ``` In [1]: import sqlite_utils In [2]: import sqlite3 In [3]: conn = sqlite3.connect(":memory:") In [4]: conn Out[4]: <sqlite3.Connection at 0x1087186c0> In [5]: conn.executescript(""" ...: CREATE TABLE album( ...: albumartist TEXT, ...: albumname TEXT, ...: albumcover BINARY, ...: PRIMARY KEY(albumartist, albumname) ...: ); ...: ...: CREATE TABLE song( ...: songid INTEGER, ...: songartist TEXT, ...: songalbum TEXT, ...: songname TEXT, ...: FOREIGN KEY(songartist, songalbum) REFERENCES album(albumartist, albumname) ...: ); ...: """) Out[5]: <sqlite3.Cursor at 0x1088def10> In [6]: db = sqlite_utils.Database(conn) In [7]: db.tables … | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/117/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
646448486 | MDExOlB1bGxSZXF1ZXN0NDQwNzM1ODE0 | 868 | initial windows ci setup | 702729 | open | 0 | 3 | 2020-06-26T18:49:13Z | 2021-07-10T23:41:43Z | FIRST_TIME_CONTRIBUTOR | simonw/datasette/pulls/868 | Picking up the work done on #557 with a new PR. Seeing if I can get this working. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/868/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
647103735 | MDU6SXNzdWU2NDcxMDM3MzU= | 875 | "Logged in as: XXX - logout" navigation item | 9599 | closed | 0 | 5533512 | 3 | 2020-06-29T04:31:14Z | 2020-07-02T00:13:24Z | 2020-06-29T18:43:50Z | OWNER | _Originally posted by @simonw in https://github.com/simonw/datasette/issues/840#issuecomment-650895874_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/875/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
649437530 | MDU6SXNzdWU2NDk0Mzc1MzA= | 887 | Canned query page should show the name of the canned query | 9599 | closed | 0 | 5607421 | 3 | 2020-07-02T00:10:39Z | 2020-07-02T00:31:33Z | 2020-07-02T00:23:45Z | OWNER | This page here - the URL is http://127.0.0.1:8001/data/all_tables but "all_tables" is not shown in the UI: <img width="784" alt="data__select_sqlite_master_name_as_table_name__table_info___from_sqlite_master_join_pragma_table_info_sqlite_master_name__as_table_info_order_by_sqlite_master_name__table_info_cid_and_data__insert_into_saved_queries__name__sql__author_id__v" src="https://user-images.githubusercontent.com/9599/86302528-d978b100-bbbd-11ea-9a07-7b5d7d3de321.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/887/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
649907676 | MDU6SXNzdWU2NDk5MDc2NzY= | 889 | asgi_wrapper plugin hook is crashing at startup | 49260 | closed | 0 | 3 | 2020-07-02T12:53:13Z | 2020-09-15T20:40:52Z | 2020-09-15T20:40:52Z | CONTRIBUTOR | Steps to reproduce: 1. Install datasette-media plugin `pip install datasette-media` 2. Launch datasette `datasette databasename.db` 3. Error ``` INFO: Started server process [927704] INFO: Waiting for application startup. ERROR: Exception in 'lifespan' protocol Traceback (most recent call last): File "/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/uvicorn/lifespan/on.py", line 48, in main await app(scope, self.receive, self.send) File "/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__ return await self.app(scope, receive, send) File "/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/datasette_media/__init__.py", line 9, in wrapped_app path = scope["path"] KeyError: 'path' ERROR: Application startup failed. Exiting. ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/889/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
652700770 | MDU6SXNzdWU2NTI3MDA3NzA= | 119 | Ability to remove a foreign key | 9599 | closed | 0 | 3 | 2020-07-07T22:31:37Z | 2020-09-24T20:36:59Z | 2020-09-24T20:36:59Z | OWNER | Useful if you add one but make a mistake and need to undo it without recreating the database from scratch. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/119/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
652961907 | MDU6SXNzdWU2NTI5NjE5MDc= | 121 | Improved (and better documented) support for transactions | 9599 | open | 0 | 3 | 2020-07-08T04:56:51Z | 2020-09-24T20:36:46Z | OWNER | _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655283393_ We should put some thought into how this library supports and encourages smart use of transactions. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/121/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
661605489 | MDU6SXNzdWU2NjE2MDU0ODk= | 900 | Some links don't honor base_url | 50220 | closed | 0 | 6026070 | 3 | 2020-07-20T09:40:50Z | 2020-10-23T19:44:04Z | 2020-10-15T22:57:55Z | NONE | Hi, I've been playing with Datasette behind Nginx (awesome tool, thanks !). It seems some URLs are OK but some aren't. For instance in https://github.com/simonw/datasette/blob/master/datasette/templates/query.html#L61 it seems that `url_csv` includes a `/` prefix, resulting in the `base_url` not beeing honored. Actually here, it seems that dropping the prefix `/` to make the link relative is enough (so it may not be strictly related to `base_url`). Additional information: ``` datasette, version 0.45+0.gf1f581b.dirty ``` Relevant Nginx configuration (note that all the trailing slashes have some effect): ``` location /datasette/ { proxy_pass http://127.0.0.1:9001/; proxy_set_header Host $host; } ``` Relelvant Datasette configuration (slashes matter too): ``` --config base_url:/datasette/ ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/900/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
663976976 | MDU6SXNzdWU2NjM5NzY5NzY= | 48 | Add a table of contents to the README | 9599 | closed | 0 | 3 | 2020-07-22T18:54:33Z | 2020-07-23T17:46:07Z | 2020-07-22T19:03:02Z | MEMBER | Using https://github.com/jonschlinkert/markdown-toc | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
665400224 | MDU6SXNzdWU2NjU0MDAyMjQ= | 906 | "allow": true for anyone, "allow": false for nobody | 9599 | closed | 0 | 5607421 | 3 | 2020-07-24T20:28:10Z | 2020-07-25T00:07:10Z | 2020-07-25T00:05:04Z | OWNER | The "allow" syntax described at https://datasette.readthedocs.io/en/0.45/authentication.html#defining-permissions-with-allow-blocks currently says this: > An allow block can specify "no-one is allowed to do this" using an empty `{}`: > > ``` > { > "allow": {} > } > ``` `"allow": null` allows all access, though this isn't documented (it should be though). These are not very intuitive. How about also supporting `"allow": true` for "allow anyone" and `"allow": false` for "allow nobody"? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/906/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
665407663 | MDU6SXNzdWU2NjU0MDc2NjM= | 908 | Interactive debugging tool for "allow" blocks | 9599 | closed | 0 | 5607421 | 3 | 2020-07-24T20:43:44Z | 2020-07-25T00:06:15Z | 2020-07-24T22:56:52Z | OWNER | > It might be good to have a little interactive tool which helps debug these things, since there are quite a few edge-cases and the damage caused if people use them incorrectly is substantial. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/907#issuecomment-663726146_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/908/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
666040390 | MDU6SXNzdWU2NjYwNDAzOTA= | 127 | Ability to insert files piped to insert-files stdin | 9599 | closed | 0 | 3 | 2020-07-27T07:09:33Z | 2020-07-30T03:08:52Z | 2020-07-30T03:08:18Z | OWNER | > Inserting files by piping them in should work - but since a filename cannot be derived this will need a `--name blah.gif` option. > > cat blah.gif | sqlite-utils insert-files files.db files - --name=blah.gif > _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/122#issuecomment-664128071_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/127/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
671130371 | MDU6SXNzdWU2NzExMzAzNzE= | 130 | Support tokenize option for FTS | 9599 | closed | 0 | 3 | 2020-08-01T19:27:22Z | 2020-08-01T20:51:28Z | 2020-08-01T20:51:14Z | OWNER | FTS5 supports things like porter stemming using a `tokenize=` option: https://www.sqlite.org/fts5.html#tokenizers Something like this in code: ``` CREATE VIRTUAL TABLE [{table}_fts] USING {fts_version} ( {columns}, tokenize='porter', content=[{table}] ); ``` I tried this out just now and it worked exactly as expected. So... `db[table].enable_fts(...) should accept a 'tokenize=` argument, and `sqlite-utils enable-fts ...` should support a `--tokenize` option. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/130/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
671763164 | MDU6SXNzdWU2NzE3NjMxNjQ= | 915 | Refactor TableView class so things like datasette-graphql can reuse the logic | 9599 | closed | 0 | 3 | 2020-08-03T03:13:33Z | 2020-08-18T22:28:37Z | 2020-08-18T22:28:37Z | OWNER | _Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-667780040_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/915/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
675727366 | MDU6SXNzdWU2NzU3MjczNjY= | 919 | Travis should not build the master branch, only the main branch | 9599 | closed | 0 | 3 | 2020-08-09T16:18:25Z | 2020-08-09T16:26:18Z | 2020-08-09T16:19:37Z | OWNER | Caused by #849 - since we are mirroring the two branches (to ensure old links to `master` keep working) Travis is building both. The following in `.travis.yml` should fix that: ``` branches: except: - master ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/919/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677227912 | MDU6SXNzdWU2NzcyMjc5MTI= | 925 | "datasette install" and "datasette uninstall" commands | 9599 | closed | 0 | 3 | 2020-08-11T22:04:32Z | 2020-08-11T22:34:37Z | 2020-08-11T22:32:12Z | OWNER | When installing Datasette plugins it's crucial that they end up in the same virtual environment as Datasette itself. It's not necessarily obvious how to do this, especially if you install Datasette via pipx or homebrew. Solution: `datasette install datasette-vega` and `datasette uninstall datasette-vega` commands that know how to install to the correct place - a very thin wrapper around `pip install`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/925/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
679646710 | MDU6SXNzdWU2Nzk2NDY3MTA= | 935 | db.execute_write_fn(create_tables, block=True) hangs a thread if connection fails | 9599 | closed | 0 | 3 | 2020-08-15T21:49:17Z | 2020-08-15T22:35:33Z | 2020-08-15T22:35:33Z | OWNER | Discovered in https://github.com/simonw/latest-datasette-with-all-plugins/issues/3#issuecomment-674449757 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/935/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
679700269 | MDU6SXNzdWU2Nzk3MDAyNjk= | 938 | Pass columns to extra CSS/JS/etc plugin hooks | 9599 | closed | 0 | 3 | 2020-08-16T06:37:47Z | 2020-09-30T20:36:12Z | 2020-08-16T18:09:59Z | OWNER | I'd like `datasette-cluster-map` to only add links to JavaScript on pages that have tables with latitude and longitude columns. Passing the names of the columns to the plugin hook can support this and will be backwards compatible thanks to pluggy. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/938/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683805434 | MDU6SXNzdWU2ODM4MDU0MzQ= | 135 | Code for finding SpatiaLite in the usual locations | 9599 | closed | 0 | 3 | 2020-08-21T20:15:34Z | 2022-02-05T00:04:26Z | 2020-08-21T20:30:13Z | OWNER | I built this for `shapefile-to-sqlite` but it would be useful in `sqlite-utils` too: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L16-L19 ```python SPATIALITE_PATHS = ( "/usr/lib/x86_64-linux-gnu/mod_spatialite.so", "/usr/local/lib/mod_spatialite.dylib", ) ``` https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L105-L109 ```python def find_spatialite(): for path in SPATIALITE_PATHS: if os.path.exists(path): return path return None ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/135/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683812642 | MDU6SXNzdWU2ODM4MTI2NDI= | 136 | --load-extension=spatialite shortcut option | 9599 | closed | 0 | 3 | 2020-08-21T20:31:25Z | 2022-02-05T00:04:26Z | 2020-10-16T19:14:32Z | OWNER | In conjunction with #135 - this would do the same thing as `--load-extension=path-to-spatialite` (see #134) | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/136/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688386219 | MDExOlB1bGxSZXF1ZXN0NDc1NjY1OTg0 | 142 | insert_all(..., alter=True) should work for new columns introduced after the first 100 records | 96218 | closed | 0 | 3 | 2020-08-28T22:22:57Z | 2020-08-30T07:28:23Z | 2020-08-28T22:30:14Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/142 | Closes #139. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/142/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
689810340 | MDU6SXNzdWU2ODk4MTAzNDA= | 3 | Datasette plugin to provide custom page for running faceted, ranked searches | 9599 | closed | 0 | 3 | 2020-09-01T05:00:22Z | 2020-09-03T21:01:41Z | 2020-09-03T21:01:41Z | MEMBER | This will be a page at `/-/beta` which renders using a custom template. It will offer a default timeline view plus search and facet by type/date. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
694493566 | MDU6SXNzdWU2OTQ0OTM1NjY= | 16 | Timeline view | 9599 | open | 0 | 3 | 2020-09-06T19:13:58Z | 2020-09-21T02:42:29Z | MEMBER | Ability to browse (and facet) by date. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
695377804 | MDU6SXNzdWU2OTUzNzc4MDQ= | 153 | table.optimize() should delete junk rows from *_fts_docsize | 9599 | closed | 0 | 3 | 2020-09-07T20:31:09Z | 2020-09-24T20:35:46Z | 2020-09-07T21:16:33Z | OWNER | > The second challenge here is cleaning up all of those junk rows in existing `*_fts_docsize` tables. Doing that just to the demo database from https://github-to-sqlite.dogsheep.net/github.db dropped its size from 22MB to 16MB! Here's the SQL: > ```sql > DELETE FROM [licenses_fts_docsize] WHERE id NOT IN ( > SELECT rowid FROM [licenses_fts]); > ``` > I can do that as part of the existing `table.optimize()` method, which optimizes FTS tables. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688501064_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/153/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
701294727 | MDU6SXNzdWU3MDEyOTQ3Mjc= | 965 | Documentation for 404.html, 500.html templates | 9599 | closed | 0 | 5818042 | 3 | 2020-09-14T17:36:59Z | 2020-09-14T18:49:49Z | 2020-09-14T18:47:22Z | OWNER | This mechanism is not documented: https://github.com/simonw/datasette/blob/30b98e4d2955073ca2bca92ca7b3d97fcd0191bf/datasette/app.py#L1119-L1129 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/965/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
705057955 | MDU6SXNzdWU3MDUwNTc5NTU= | 969 | Add --tar option to "datasette publish heroku" | 1448859 | closed | 0 | 5971510 | 3 | 2020-09-20T06:54:53Z | 2020-10-08T23:55:59Z | 2020-10-08T23:30:59Z | NONE | This issue is about how best to pass additional options to tools used for publishing datasettes. A concrete example is wanting to pass the `--tar` flag to the heroku CLI tool. I think there are at least two options for doing this: documentation for each publishing tool to explain how to set flags via env variables (if possible) or building a mechanism that lets users pass additional flags through datasette. When using `datasette publish heroku binder-launches.db --extra-options="--config facet_time_limit_ms:35000 --config sql_time_limit_ms:35000" --name=binderlytics --install=datasette-vega` to publish https://binderlytics.herokuapp.com/ the following error happens: ``` › Warning: heroku update available from 7.42.1 to 7.43.0. › Warning: heroku update available from 7.42.1 to 7.43.0. › Warning: heroku update available from 7.42.1 to 7.43.0. Setting WEB_CONCURRENCY and restarting ⬢ binderlytics... done, v13 WEB_CONCURRENCY: 1 › Warning: heroku update available from 7.42.1 to 7.43.0. ▸ Couldn't detect GNU tar. Builds could fail due to decompression errors ▸ See https://devcenter.heroku.com/articles/platform-api-deploying-slugs#create-slug-archive ▸ Please install it, or specify the '--tar' option ▸ Falling back to node's built-in compressor buffer.js:358 throw new ERR_INVALID_OPT_VALUE.RangeError('size', size); ^ RangeError [ERR_INVALID_OPT_VALUE]: The value "3303763968" is invalid for option "size" at Function.alloc (buffer.js:367:3) at new Buffer (buffer.js:281:19) at Readable.<anonymous> (/Users/thead/.local/share/heroku/node_modules/archiver-utils/index.js:39:15) at Readable.emit (events.js:322:22) at endReadableNT (/Users/thead/.local/share/heroku/node_modules/readable-stream/lib/_stream_readable.js:1010:12) at processTicksAndRejections (internal/process/task_queues.js:84:21) { code: 'ERR_INVALID_OPT_VALUE' } ``` After installing GNU tar with `brew install gnu-tar` and modifying `datasette/publish/heroku.py` to incl… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/969/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
708261775 | MDU6SXNzdWU3MDgyNjE3NzU= | 175 | Add docs for .transform(column_order=) | 9599 | closed | 0 | 3 | 2020-09-24T15:19:04Z | 2020-09-24T20:35:48Z | 2020-09-24T16:00:56Z | OWNER | > Need to update docs for `.transform()` now that `column_order=` is available. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/174#discussion_r494403327_ Maybe also add this as an option to `sqlite-utils transform` - since reordering columns is actually a pretty nice capability. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/175/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
710650633 | MDU6SXNzdWU3MTA2NTA2MzM= | 979 | Default table view JSON should include CREATE TABLE | 9599 | closed | 0 | 3 | 2020-09-28T23:54:58Z | 2023-01-09T15:32:39Z | 2023-01-09T15:32:22Z | OWNER | https://latest.datasette.io/fixtures/facetable.json doesn't currently include the CREATE TABLE statement for the page, even though it's available on the HTML version at https://latest.datasette.io/fixtures/facetable | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/979/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
716756082 | MDU6SXNzdWU3MTY3NTYwODI= | 996 | Better handling of multiple matching template wildcard paths | 9599 | closed | 0 | 5971510 | 3 | 2020-10-07T18:25:40Z | 2020-10-08T23:55:41Z | 2020-10-07T22:51:17Z | OWNER | I tried building this: templates/pages/{topic}.html templates/pages/{topic}/{slug}.html And it didn't work - hits to /foo/bar which should have been rendered by the `{slug}.html` template were instead rendered by the top level `{topic.html}` template. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/996/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
716988478 | MDU6SXNzdWU3MTY5ODg0Nzg= | 997 | Documentation covering buildpack deployment | 9599 | closed | 0 | 5971510 | 3 | 2020-10-08T03:21:52Z | 2020-10-08T23:56:03Z | 2020-10-08T23:32:10Z | OWNER | A tidied up version of https://til.simonwillison.net/til/til/digitalocean_datasette-on-digitalocean-app-platform.md - but mention that you can deploy to Heroku using the same mechanism. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/997/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
717729056 | MDU6SXNzdWU3MTc3MjkwNTY= | 999 | Datasette should default to running Uvicorn with workers=1 | 9599 | closed | 0 | 5971510 | 3 | 2020-10-08T23:07:03Z | 2020-10-08T23:55:46Z | 2020-10-08T23:21:36Z | OWNER | Uvicorn uses the `WEB_CONCURRENCY` variable, if set, to specify the number of workers to use. Datasette does not work with options for this other than 1: ``` WEB_CONCURRENCY=2 datasette . WARNING: You must pass the application as an import string to enable 'reload' or 'workers'. ``` This was the cause of the Heroku bug in #627. I fixed that issue by setting `WEB_CONCURRENCY=1` in `datasette publish heroku`, but a better fix would be to hard-code `workers=1` in Datasette itself. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/999/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
718259202 | MDU6SXNzdWU3MTgyNTkyMDI= | 1005 | Remove xfail tests when new httpx is released | 9599 | closed | 0 | 3268330 | 3 | 2020-10-09T16:00:19Z | 2021-02-28T22:41:08Z | 2021-02-28T22:41:08Z | OWNER | > My `httpx` pull request adding `raw_path` support was just merged: https://github.com/encode/httpx/pull/1357 - but it's not in a release yet. > > I'm going to mark these tests as `xfail` so I can land this change - I'll remove that once an `httpx` release comes out that I can use to get the tests passing. > _Originally posted by @simonw in https://github.com/simonw/datasette/pull/1000#issuecomment-706263157_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1005/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
718484082 | MDU6SXNzdWU3MTg0ODQwODI= | 1010 | json / CSV links are broken in Datasette 0.50 | 9599 | closed | 0 | 3 | 2020-10-10T00:07:42Z | 2020-10-10T02:32:03Z | 2020-10-10T02:32:03Z | OWNER | e.g. on https://latest.datasette.io/fixtures/sortable That export link block is broken. The HTML is: ```html <p class="export-links"> This data as <a href="//fixtures/sortable.json">json</a>, <a href="//fixtures/sortable.csv?_size=max">CSV</a> (<a href="#export">advanced</a>) </p> ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1010/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
718953669 | MDU6SXNzdWU3MTg5NTM2Njk= | 1016 | Add a "delete" icon next to filters (in addition to "remove filter") | 9599 | closed | 0 | 6026070 | 3 | 2020-10-11T23:49:53Z | 2020-10-23T19:44:06Z | 2020-10-12T03:01:58Z | OWNER | The "remove filter" option in the select box is not very discoverable. <img width="549" alt="evernote__notes__85_rows_where_where_source-url_is_not_blank_sorted_by_id" src="https://user-images.githubusercontent.com/9599/95693532-b8265880-0be1-11eb-971e-038dede14cdb.png"> It would be good to have an additional remove icon, pointed to by the pink arrow, which removes a specific selected filter. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1016/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
722673818 | MDU6SXNzdWU3MjI2NzM4MTg= | 1023 | Fix issues relating to base_url | 9599 | closed | 0 | 6026070 | 3 | 2020-10-15T21:02:06Z | 2020-11-24T19:51:44Z | 2020-10-31T20:51:01Z | OWNER | Lots of `base_url` bugs that I'd like to solve at once. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1023/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
725743755 | MDU6SXNzdWU3MjU3NDM3NTU= | 1035 | datasette.urls.table(..., format="json") argument | 9599 | closed | 0 | 6026070 | 3 | 2020-10-20T16:09:34Z | 2020-10-31T18:16:43Z | 2020-10-31T18:16:43Z | OWNER | > That `datasette.urls.table("db", "table") + ".json"` example is bad because if the table name contains a `.` it should be `?_format=json` instead. > > Maybe `.table()` should have a `format="json"` option that knows how to do this. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1026#issuecomment-712962517_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1035/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
726094754 | MDU6SXNzdWU3MjYwOTQ3NTQ= | 1037 | Add horizontal scrollbar to tables | 9599 | closed | 0 | 6026070 | 3 | 2020-10-21T03:13:34Z | 2020-10-27T20:52:04Z | 2020-10-21T03:16:36Z | OWNER | Currently you have to scroll the entire page sideways if a table is wide. Make the table `overflow-x: auto` instead. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1037/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
726910999 | MDExOlB1bGxSZXF1ZXN0NTA3OTAzMzky | 1040 | /db/table/-/blob/pk/column.blob download URL | 9599 | closed | 0 | 6026070 | 3 | 2020-10-21T22:39:15Z | 2020-10-24T23:09:20Z | 2020-10-24T23:09:19Z | OWNER | simonw/datasette/pulls/1040 | Refs #1036. Still needs: - [x] Comprehensive tests across all of the code branches, plus permissions - [x] A bit more refactoring to share logic cleanly with `RowView` - ~~A configuration option to disable this feature (probably)~~ | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1040/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
729818242 | MDExOlB1bGxSZXF1ZXN0NTEwMjM1OTA5 | 189 | Allow iterables other than Lists in m2m records | 35681 | closed | 0 | 3 | 2020-10-26T18:47:44Z | 2020-10-27T16:28:37Z | 2020-10-27T16:24:21Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/189 | I was playing around with sqlite-utils, creating a Roam Research dogsheep-style importer for Datasette, and ran into a slight snag. I wanted to use a generator to add an order column in an importer. It looked something like: ``` def order_generator(iterable, attr=None): if attr is None: attr = "order" order: int = 0 for i in iterable: i[attr] = order order += 1 yield i ``` When I used this with `insert_all` and other things, it worked fine--but it didn't work as the `records` argument to `m2m`. I dug into it, and sqlite-utils is explicitly checking if the records argument is a list or a tuple. I flipped the check upside down, and now it checks if the argument is a mapping. If it's a mapping, it wraps it in a list, otherwise it leaves it alone. (I get that it might not really make sense to put the order column on the second table. I changed my import schema a bit, and no longer have a real example, but maybe this change still makes sense.) The automated tests still pass, but I did not add any new ones. Let me know what you think! I'm really loving Datasette and its ecosystem; thanks for everything! | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/189/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
732685643 | MDU6SXNzdWU3MzI2ODU2NDM= | 1063 | .csv should link to .blob downloads | 9599 | closed | 0 | 6026070 | 3 | 2020-10-29T21:45:58Z | 2021-06-17T18:12:30Z | 2020-10-29T22:47:45Z | OWNER | - [x] Update `.csv` output to link to these things (and get that `xfail` test to pass) - ~~Add a `.csv?_blob_base64=1` argument that causes them to be output in base64 in the CSV~~ > Moving the CSV work to a separate ticket. _Originally posted by @simonw in https://github.com/simonw/datasette/pull/1061#issuecomment-719042601_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1063/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
732859030 | MDU6SXNzdWU3MzI4NTkwMzA= | 1066 | Table actions menu plus plugin hook | 9599 | closed | 0 | 6026070 | 3 | 2020-10-30T03:46:54Z | 2020-10-30T05:18:36Z | 2020-10-30T05:16:50Z | OWNER | > For the table actions: attaching it to a cog icon next to the table name could make sense. > > <img width="503" alt="data__faa-wildlife-strikes__26_rows_where_where_AIRPORT____OIL_RIG__sorted_by_rowid_and_Fix_table_name_in_spatialite_example_command_by_jsfenfen_·_Pull_Request__1022_·_simonw_datasette" src="https://user-images.githubusercontent.com/9599/96168896-45192c80-0ed6-11eb-9d90-6172a8346468.png"> > > This is the column action icon at twice the size, color `#666`. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/690#issuecomment-709497595_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1066/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
733768037 | MDU6SXNzdWU3MzM3NjgwMzc= | 1074 | latest.datasette.io should include plugins from fixtures | 9599 | closed | 0 | 6026070 | 3 | 2020-10-31T17:23:23Z | 2020-10-31T19:47:47Z | 2020-10-31T19:47:47Z | OWNER | > It bothers me that these aren't visible in any public demos. Maybe `latest.datasette.io` should include the `my_plugins.py` and `my_plugins2.py` plugins? _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1067#issuecomment-719961701_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1074/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
733829385 | MDU6SXNzdWU3MzM4MjkzODU= | 1077 | database_actions plugin hook | 9599 | closed | 0 | 6055094 | 3 | 2020-10-31T23:48:12Z | 2020-11-02T18:43:25Z | 2020-11-02T18:29:50Z | OWNER | Like `column_actions` but adds a cog menu to the database page. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1077/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735650864 | MDU6SXNzdWU3MzU2NTA4NjQ= | 194 | 3.0 release with some minor breaking changes | 9599 | closed | 0 | 6079500 | 3 | 2020-11-03T21:36:31Z | 2020-11-08T17:19:35Z | 2020-11-08T17:19:34Z | OWNER | While working on search (#192) I've spotted a few small changes I would like to make that would break backwards compatibility in minor ways, hence requiring a 3.x release. `db[table].search()` - I would like this to default to sorting by rank Also I'd like to free up the `-c` and `-f` options for other purposes from the standard output formats here: https://github.com/simonw/sqlite-utils/blob/43eae8b193d362f2b292df73e087ed6f10838144/sqlite_utils/cli.py#L48-L58 I'd like `-f` to be used to indicate a full-text search column during an insert and `-c` to indicate a column (so you can specify which columns you want to output). | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735663855 | MDExOlB1bGxSZXF1ZXN0NTE1MDE0ODgz | 195 | table.search() improvements plus sqlite-utils search command | 9599 | closed | 0 | 3 | 2020-11-03T22:02:08Z | 2020-11-06T18:30:49Z | 2020-11-06T18:30:42Z | OWNER | simonw/sqlite-utils/pulls/195 | Refs #192. Still needs tests. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/195/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
735852274 | MDU6SXNzdWU3MzU4NTIyNzQ= | 1082 | DigitalOcean buildpack memory errors for large sqlite db? | 39538958 | open | 0 | 3 | 2020-11-04T06:35:32Z | 2020-11-04T19:35:44Z | NONE | 1. Have a sqlite db stored in Dropbox 2. Previously tried the Digital Ocean build pack minimal approach (e.g. Procfile, requirements.txt, bin/post_compile) 3. bin/post_compile with wget from Dropbox 4. download of large sqlite db is successful 5. log reveals that when building Docker container, Digital Ocean runs out of memory for 5gb+ sqlite db but works fine for 2gb+ sqlite db | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1082/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
741021342 | MDU6SXNzdWU3NDEwMjEzNDI= | 1086 | Foreign keys with blank titles result in non-clickable links | 9599 | closed | 0 | 6055094 | 3 | 2020-11-11T19:41:09Z | 2020-11-28T23:28:29Z | 2020-11-11T23:46:20Z | OWNER | <img width="851" alt="index__core_snapshot_tags__3_rows_and_swarm__select_venues_name__checkins_id__createdAt__type__timeZoneOffset__editableUntil___like___isMayor__source__venue__createdBy__event__sticker__created__comments_count__entities__shout__checkins_priv" src="https://user-images.githubusercontent.com/9599/98856656-bafbaf80-2412-11eb-8070-4f3a47511973.png"> The HTML looks like this: ```html <td class="col-tag_id type-int"><a href="/index/core_tag/1"></a> <em>1</em></td> ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1086/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
741268956 | MDU6SXNzdWU3NDEyNjg5NTY= | 1088 | OperationalError('interrupted') can 500 on row page | 9599 | closed | 0 | 6055094 | 3 | 2020-11-12T04:29:55Z | 2020-11-28T23:28:35Z | 2020-11-12T04:36:52Z | OWNER | I got this on my (private) https://dogsheep.simonwillison.net/twitter/tweets/1188612004572880896 page: <img width="1059" alt="Error_500" src="https://user-images.githubusercontent.com/9599/98895899-a6440980-245c-11eb-98e8-fb97d10ec2fd.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1088/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
741862364 | MDU6SXNzdWU3NDE4NjIzNjQ= | 1090 | Custom widgets for canned query forms | 9599 | open | 0 | 3 | 2020-11-12T19:21:07Z | 2021-03-27T16:25:25Z | OWNER | This is an idea that was cut from the first version of writable canned queries: > I really want the option to use a `<textarea>` for a specific value. > > Idea: metadata syntax like this: > > ```json > { > "databases": { > "my-database": { > "queries": { > "add_twitter_handle": { > "sql": "insert into twitter_handles (username) values (:username)", > "write": true, > "params": { > "username": { > "widget": "textarea" > } > } > } > } > } > } > } > ``` > > I can ship with some default widgets and provide a plugin hook for registering extra widgets. > > This opens up some really exciting possibilities for things like map widgets that let you draw polygons. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-608125928_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1090/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
751195017 | MDU6SXNzdWU3NTExOTUwMTc= | 1111 | Accessing a database's `.json` is slow for very large SQLite files | 15178711 | open | 0 | 3 | 2020-11-26T00:27:27Z | 2021-01-04T19:57:53Z | CONTRIBUTOR | I have a SQLite DB that's pretty large, 23GB and something like 300 million rows. I expect that most queries I run on it will be slow, which is fine, but there are some things that Datasette does that makes working with the DB very slow. Specifically, when I access the `.json` metadata for a table (which I believe it comes from `datasette/views/database.py`, it takes 43 seconds for the request to come in: ```bash $ time curl localhost:9999/out.json {"database": "out", "size": 24291454976, "tables": [{"name": "PageviewsHour", "columns": ["file", "code", "page", "pageviews"], "primary_keys": [], "count": null, "hidden": false, "fts_table": null, "foreign_keys": {"incoming": [], "outgoing": [{"other_table": "PageviewsHourFiles", "column": "file", "other_column": "file_id"}]}, "private": false}, {"name": "PageviewsHourFiles", "columns": ["file_id", "filename", "sha256", "size", "day", "hour"], "primary_keys": ["file_id"], "count": null, "hidden": false, "fts_table": null, "foreign_keys": {"incoming": [{"other_table": "PageviewsHour", "column": "file_id", "other_column": "file"}], "outgoing": []}, "private": false}, {"name": "sqlite_sequence", "columns": ["name", "seq"], "primary_keys": [], "count": 1, "hidden": false, "fts_table": null, "foreign_keys": {"incoming": [], "outgoing": []}, "private": false}], "hidden_count": 0, "views": [], "queries": [], "private": false, "allow_execute_sql": true, "query_ms": 43340.23213386536} real 0m43.417s user 0m0.006s sys 0m0.016s ``` I suspect this is because a `COUNT(*)` is happening under the hood, which, when I run it through sqlite directly, does take around the same time: ```bash $ time sqlite3 out.db < <(echo "select count(*) from PageviewsHour;") 362794272 real 0m44.523s user 0m2.497s sys 0m6.703s ``` I'm using the `.json` request in the [Observable Datasette Client](https://observablehq.com/@asg017/datasette-client) to 1) verify that a link passed in is a reachable Datasette instance, and 2) a quick way to look at metadata for a db. A few differe… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1111/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
752749485 | MDExOlB1bGxSZXF1ZXN0NTI4OTk3NjE0 | 1112 | Fix --metadata doc usage | 50527 | closed | 0 | 6055094 | 3 | 2020-11-28T19:19:51Z | 2020-11-28T23:28:21Z | 2020-11-28T19:53:48Z | CONTRIBUTOR | simonw/datasette/pulls/1112 | I stumbled on this while trying to figure out how to configure datasette-ripgrep via https://github.com/simonw/datasette-ripgrep/issues/15 You may not want to update the changelog (those are annoying) so I added two commits in case that's easier. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1112/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
753026003 | MDU6SXNzdWU3NTMwMjYwMDM= | 54 | github-to-sqlite workflows command | 9599 | closed | 0 | 3 | 2020-11-29T21:56:42Z | 2020-11-29T22:08:46Z | 2020-11-29T21:57:17Z | MEMBER | A command that fetches the YAML workflows for different repos, parses them and stores them in relational tables would be really useful for maintaining larger numbers of workflows. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
757481949 | MDU6SXNzdWU3NTc0ODE5NDk= | 1131 | "datasette inspect" outputs invalid JSON if an error is logged | 9599 | closed | 0 | 3 | 2020-12-05T00:00:45Z | 2020-12-05T20:48:34Z | 2020-12-05T05:21:19Z | OWNER | See https://github.com/simonw/register-of-members-interests/issues/6: ``` % datasette inspect regmem.db ERROR: conn=<sqlite3.Connection object at 0x10d2a9c60>, sql = 'select count(*) from [items_fts]', params = None: SQL logic error { "regmem": { "hash": "6fde27e3dea80d6b65f2ac7f89cd8448980fee8c91b505ba29c311ba0393317f", "size": 936198144, ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1131/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
769520939 | MDU6SXNzdWU3Njk1MjA5Mzk= | 1149 | Make it easier to theme Datasette with CSS | 9599 | open | 0 | 3268330 | 3 | 2020-12-17T05:01:26Z | 2021-03-22T21:43:16Z | OWNER | I want to theme https://datasette.io/ so that when you visit https://datasette.io/content (the Datasette UI part of it) the navigation from the parent site is used. I tried dropping in a `base.html` template like this: ```html {% extends "page_base.html" %} {% block base_extra_head %} <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> {% for url in extra_css_urls %} <link rel="stylesheet" href="{{ url.url }}"{% if url.sri %} integrity="{{ url.sri }}" crossorigin="anonymous"{% endif %}> {% endfor %} {% for url in extra_js_urls %} <script src="{{ url.url }}"{% if url.sri %} integrity="{{ url.sri }}" crossorigin="anonymous"{% endif %}></script> {% endfor %} {% block extra_head %}{% endblock %} {% endblock %} {% block extra_body_end %} {% include "_close_open_menus.html" %} {% for body_script in body_scripts %} <script>{{ body_script }}</script> {% endfor %} {% endblock %} ``` But this resulted in pages looking like this: <img width="1067" alt="content__categories__3_rows" src="https://user-images.githubusercontent.com/9599/102446045-c168e280-3fe1-11eb-94d6-e7350798eb96.png"> Note that the cog menu is broken and the filter UI is unstyled. To get these working correctly I would need to copy over a whole lot of Datasette's default CSS - and that means that when Datasette changes in the future those pages could break in subtle ways. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1149/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
771872303 | MDExOlB1bGxSZXF1ZXN0NTQzMjQ2NTM1 | 59 | Remove unneeded exists=True for -a/--auth flag. | 631242 | closed | 0 | 3 | 2020-12-21T06:03:55Z | 2021-05-22T14:06:19Z | 2021-05-19T16:08:12Z | CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/59 | The file does not need to exist when using an environment variable. | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
777145954 | MDU6SXNzdWU3NzcxNDU5NTQ= | 1167 | Add Prettier to contributing documentation | 9599 | closed | 0 | 6346396 | 3 | 2020-12-31T22:00:55Z | 2021-01-25T02:01:19Z | 2021-01-25T01:58:28Z | OWNER | Following #1166 - the docs at https://docs.datasette.io/en/stable/contributing.html should include a section about JavaScript, and it should document how to run Prettier. I run it in VS Code but it can be run on the command-line too: npx prettier 'datasette/static/*[!.min].js' --write | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1167/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
778126516 | MDExOlB1bGxSZXF1ZXN0NTQ4MjcxNDcy | 1170 | Install Prettier via package.json | 3637 | closed | 0 | 6346396 | 3 | 2021-01-04T14:18:03Z | 2021-01-24T21:21:01Z | 2021-01-04T19:52:34Z | CONTRIBUTOR | simonw/datasette/pulls/1170 | This adds a package.json with Prettier and means that developers/CI will use the same version. It also ensures that NPM packages are cached on GitHub Actions which fixes #1169. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1170/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
782692159 | MDU6SXNzdWU3ODI2OTIxNTk= | 1182 | Retire "Ecosystem" page in favour of datasette.io/plugins and /tools | 9599 | closed | 0 | 6346396 | 3 | 2021-01-09T21:54:47Z | 2021-01-24T21:21:09Z | 2021-01-09T22:17:28Z | OWNER | https://docs.datasette.io/en/stable/ecosystem.html is no longer needed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1182/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
783778672 | MDU6SXNzdWU3ODM3Nzg2NzI= | 220 | Better error message for *_fts methods against views | 649467 | closed | 0 | 3 | 2021-01-11T23:24:00Z | 2021-02-22T20:44:51Z | 2021-02-14T22:34:26Z | NONE | enable_fts and its related methods only work on tables, not views. Could those methods and possibly others move up to the Queryable superclass? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/220/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
784628163 | MDU6SXNzdWU3ODQ2MjgxNjM= | 1185 | "Statement may not contain PRAGMA" error is not strictly true | 9599 | closed | 0 | 6346396 | 3 | 2021-01-12T22:07:10Z | 2021-01-24T21:21:37Z | 2021-01-12T22:26:26Z | OWNER | Consider https://latest.datasette.io/fixtures?sql=select+%27select%0D%0A%27+%7C%7C+group_concat%28%27++++case+when+%5B%27+%7C%7C+name+%7C%7C+%27%5D+is+not+null+then+%27+%7C%7C+quote%28name+%7C%7C+%27%2C+%27%29+%7C%7C+%27+else+%27%27%27%27+end%27%2C+%27+%7C%7C%0D%0A%27%29+%7C%7C+%27%0D%0A++as+columns%2C%0D%0A++count%28*%29+as+num_rows%0D%0Afrom%0D%0A++%5B%27+%7C%7C+%3Atable+%7C%7C+%27%5D%0D%0Agroup+by%0D%0A++columns%0D%0Aorder+by%0D%0A++num_rows+desc%27+as+query+from+pragma_ytable_info%28%3Atable%29&table=facetable It says "Statement may not contain PRAGMA" - but that's not actually true. Datasette has an allow-list of PRAGMA that are OK - in this case there was a typo in `pragma_ytable_info` which caused the error, but pragma_table_info` would have been OK. So the error message is misleading. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1185/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
787098146 | MDU6SXNzdWU3ODcwOTgxNDY= | 1190 | `datasette publish upload` mechanism for uploading databases to an existing Datasette instance | 1024355 | closed | 0 | 3 | 2021-01-15T18:18:42Z | 2023-08-30T22:16:39Z | 2023-08-30T22:16:38Z | NONE | If I have a self-hosted instance of Datasette up and running, I'd like to be able to the use the CLI to publish databases to that instance, not only Google or Heroku. Ideally there'd be a `url` parameter or something similar to which one could point the publish command to their instance. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1190/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
788447787 | MDU6SXNzdWU3ODg0NDc3ODc= | 1194 | ?_size= argument is not persisted by hidden form fields in the table filters | 9599 | closed | 0 | 6346396 | 3 | 2021-01-18T17:41:52Z | 2021-01-25T03:10:23Z | 2021-01-25T03:10:23Z | OWNER | Click "Apply" on https://covid-19.datasettes.com/covid/ny_times_us_counties?_size=1000&county__exact=San+Francisco&state__exact=California&_sort_desc=date#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=cases&g.y_type=quantitative and the `?_size=1000` parameter from the URL will no longer apply on the reloaded page. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
793002853 | MDExOlB1bGxSZXF1ZXN0NTYwNzYwMTQ1 | 1204 | WIP: Plugin includes | 9599 | open | 0 | 3 | 2021-01-25T03:59:06Z | 2021-12-17T07:10:49Z | OWNER | simonw/datasette/pulls/1204 | Refs #1191 Next steps: - [ ] Get comfortable that this pattern is the right way to go - [ ] Implement it for all of the other pages, not just the table page - [ ] Add a new set of plugin tests that exercise ALL of these new hook locations - [ ] Document, then ship | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1204/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | ||||||
793027837 | MDU6SXNzdWU3OTMwMjc4Mzc= | 1205 | Rename /:memory: to /_memory | 9599 | closed | 0 | 3268330 | 3 | 2021-01-25T05:04:56Z | 2021-01-28T22:55:02Z | 2021-01-28T22:51:42Z | OWNER | For consistency with `/_internal` - and because then we don't need to escape the `:` characters. This change would need to be in before Datasette 1.0. I could land it earlier and set up redirects from the old URLs though. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1205/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
793086333 | MDExOlB1bGxSZXF1ZXN0NTYwODMxNjM4 | 1206 | Release 0.54 | 9599 | closed | 0 | 3 | 2021-01-25T06:45:47Z | 2021-01-25T17:33:30Z | 2021-01-25T17:33:29Z | OWNER | simonw/datasette/pulls/1206 | Refs #1201 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1206/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
796736607 | MDU6SXNzdWU3OTY3MzY2MDc= | 56 | Not all quoted statuses get fetched? | 42315895 | closed | 0 | 3 | 2021-01-29T09:48:44Z | 2021-02-03T10:36:36Z | 2021-02-03T10:36:36Z | NONE | ![image](https://user-images.githubusercontent.com/42315895/106259325-5f75dc80-621f-11eb-8311-db8f2fe2a257.png) In my database I have 13300 quote tweets, but eta 3600 have `quoted_status` empty. I fetched some of them using `https://api.twitter.com/1.1/statuses/show.json?id=xx` and they did have ids of quoted tweets. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
797097140 | MDU6SXNzdWU3OTcwOTcxNDA= | 60 | Use Data from SQLite in other commands | 22578954 | open | 0 | 3 | 2021-01-29T18:35:52Z | 2021-02-12T18:29:43Z | CONTRIBUTOR | As a total beginner here how could you access data from the sqlite table to run other commands. What I am thinking is I want to get all the repos in an organization then using the repo list pull all the commit messages for each repo. I love this project by the way! | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |