github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1618130434 | I_kwDOJHON9s5gcrYC | 11 | Implement a SQL view to make it easier to query files in a nested folder | 9599 | open | 0 | 3 | 2023-03-09T23:19:28Z | 2023-03-09T23:24:01Z | MEMBER | Working with nested data in SQL is tricky, can I make it easier with a view or canned query? | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1616429236 | I_kwDOJHON9s5gWMC0 | 4 | Support incremental updates | 9599 | open | 0 | 2 | 2023-03-09T05:14:00Z | 2023-03-09T18:20:56Z | MEMBER | Running this script can take several hours against a large notes database. Would be neat if it could run against just the notes that have been modified since it last ran. Could pull the max `updated` date and then keep on looping until it finds one modified before then. Problem is I don't actually know what order it iterates over the notes in. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1616440856 | I_kwDOJHON9s5gWO4Y | 5 | Configure full text search | 9599 | open | 0 | 0 | 2023-03-09T05:20:46Z | 2023-03-09T05:20:46Z | MEMBER | FTS would be useful. Maybe even extract the plain text from the notes to make that index easier to create, rather than creating it against the HTML. Can use the `plaintext` property for that. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1617602868 | I_kwDOJHON9s5gaqk0 | 6 | Character encoding problem | 9599 | open | 0 | 2 | 2023-03-09T16:44:34Z | 2023-04-14T15:22:09Z | MEMBER | I ran against a recent note with this in it: > Or just "Actions ⚙️ " And got back: <img width="367" alt="image" src="https://user-images.githubusercontent.com/9599/224092586-e2475d8f-868c-489d-b495-2bfeb1a7db99.png"> > `Actions ‚öôÔ∏è` Pasting that into https://ftfy.vercel.app/?s=Actions+%E2%80%9A%C3%B6%C3%B4%C3%94%E2%88%8F%C3%A8+ gives this: ```python s = 'Actions â\x80\x9aöôÃ\x94â\x88\x8fè' s = s.encode('latin-1') s = s.decode('utf-8') s = s.encode('macroman') s = s.decode('utf-8') print(s) ``` <img width="816" alt="image" src="https://user-images.githubusercontent.com/9599/224092864-2b796c40-7c68-40b1-ae38-d0c61721b742.png"> | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1617938730 | I_kwDOJHON9s5gb8kq | 9 | Default to just storing plaintext, store HTML if `--html` is passed | 9599 | open | 0 | 0 | 2023-03-09T20:19:06Z | 2023-03-09T20:19:06Z | MEMBER | The full `body` version of the notes can get HUGE, due to embedded images. It turns out for my own purposes I'm usually happy with just the `plaintext` version. I'm tempted to say you don't get HTML unless you pass a `--html` option. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
692202408 | MDU6SXNzdWU2OTIyMDI0MDg= | 12 | Idea: maps and GeoJSON support | 9599 | open | 0 | 0 | 2020-09-03T18:47:10Z | 2020-09-04T01:45:03Z | MEMBER | It would be cool if the `display_sql` could return a column populated with GeoJSON which would the automatically be displayed on a map in the results (or maybe default JS would look for a `class="geojson"` element output by the `display` template) - ala https://github.com/simonw/datasette-leaflet-geojson Then I could render workout routes on a map, or Swarm checkin points. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
694136490 | MDU6SXNzdWU2OTQxMzY0OTA= | 15 | Add a bunch of config examples | 9599 | open | 0 | 1 | 2020-09-05T17:58:43Z | 2020-09-18T23:17:39Z | MEMBER | I can bring these over from my personal Dogsheep. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
694493566 | MDU6SXNzdWU2OTQ0OTM1NjY= | 16 | Timeline view | 9599 | open | 0 | 3 | 2020-09-06T19:13:58Z | 2020-09-21T02:42:29Z | MEMBER | Ability to browse (and facet) by date. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
695553522 | MDU6SXNzdWU2OTU1NTM1MjI= | 18 | Deleted records stay in the search index | 9599 | open | 0 | 2 | 2020-09-08T05:14:23Z | 2020-09-08T05:15:51Z | MEMBER | Here's why: https://github.com/dogsheep/dogsheep-beta/blob/24f7898d41a39218058f174c75ba62f7c0fcfff6/dogsheep_beta/utils.py#L44-L53 That should probably do `DELETE FROM index1.search_index WHERE [table] = ?` first. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
695556681 | MDU6SXNzdWU2OTU1NTY2ODE= | 19 | Figure out incremental re-indexing | 9599 | open | 0 | 2 | 2020-09-08T05:23:31Z | 2020-09-08T05:27:07Z | MEMBER | As tables get bigger reindexing everything on a schedule (essentially recreating the entire index from scratch) will start to become a performance bottleneck. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
705215230 | MDU6SXNzdWU3MDUyMTUyMzA= | 26 | Pagination | 9599 | open | 0 | 7 | 2020-09-21T00:14:37Z | 2020-09-21T02:55:54Z | MEMBER | Useful for #16 (timeline view) since you can now filter to just the items on a specific day - but if there are more than 50 items you can't see them all. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
709789634 | MDU6SXNzdWU3MDk3ODk2MzQ= | 27 | Sort order is not persisted by facet filter links | 9599 | open | 0 | 0 | 2020-09-27T18:22:07Z | 2020-09-27T18:22:07Z | MEMBER | A link to `/-/beta?category=1×tamp__date=2018-08-01&q=swedish` should be to `/-/beta?category=1×tamp__date=2018-08-01&q=swedish&sort=newest` | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
724759588 | MDU6SXNzdWU3MjQ3NTk1ODg= | 29 | Add search highlighting snippets | 9599 | open | 0 | 5 | 2020-10-19T16:00:48Z | 2021-08-26T20:23:11Z | MEMBER | Like on https://til.simonwillison.net/til/search?q=Snippet | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
||||||||
836923194 | MDU6SXNzdWU4MzY5MjMxOTQ= | 32 | JSON API for search results | 9599 | open | 0 | 0 | 2021-03-20T22:21:36Z | 2021-03-20T22:21:36Z | MEMBER | Refs https://github.com/simonw/datasette/issues/878 | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1888477283 | I_kwDOC8SPRc5wj-Bj | 38 | Run `rebuild_fts` after building the index | 9599 | open | 0 | 0 | 2023-09-08T23:17:45Z | 2023-09-08T23:17:45Z | MEMBER | In: - https://github.com/simonw/datasette.io/issues/152#issuecomment-1712323347 This turned out to be the fix: ```bash dogsheep-beta index dogsheep-index.db templates/dogsheep-beta.yml sqlite-utils rebuild-fts dogsheep-index.db ``` | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/38/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
689850810 | MDU6SXNzdWU2ODk4NTA4MTA= | 6 | Set up a demo instance | 9599 | open | 0 | 0 | 2020-09-01T06:20:24Z | 2020-09-01T06:20:24Z | MEMBER | Once I've got the Datasette plugin to a state where it's worth building a demo: #3 I can use data from my public https://github-to-sqlite.dogsheep.net/ demo plus the Pocket data subset I use for the demo in https://github.com/dogsheep/pocket-to-sqlite/issues/5 - I could pull in the https://dogsheep-photos.dogsheep.net/ photos data too. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
602533300 | MDU6SXNzdWU2MDI1MzMzMDA= | 1 | Import photo metadata from Apple Photos into SQLite | 9599 | open | 0 | 5324096 | 8 | 2020-04-18T19:23:26Z | 2020-05-04T02:41:40Z | MEMBER | Faces, albums, locations, that kind of thing. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
606033104 | MDU6SXNzdWU2MDYwMzMxMDQ= | 12 | If less than 500MB, show size in MB not GB | 9599 | open | 0 | 1 | 2020-04-24T04:35:01Z | 2020-04-24T04:35:25Z | MEMBER | Just saw this: ``` Uploading 0.05 GB ``` | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
607888367 | MDU6SXNzdWU2MDc4ODgzNjc= | 13 | Also upload movie files | 9599 | open | 0 | 2 | 2020-04-27T22:11:25Z | 2020-04-28T00:39:45Z | MEMBER | The `upload` command currently only handles static images: https://github.com/dogsheep/photos-to-sqlite/blob/d939455af00e07866686457ee2fcb9b2d1b7194e/photos_to_sqlite/utils.py#L26-L33 Need to cover movies taken by my phone and DSLR too. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
608512747 | MDU6SXNzdWU2MDg1MTI3NDc= | 14 | Annotate photos using the Google Cloud Vision API | 9599 | open | 0 | 5 | 2020-04-28T18:09:03Z | 2020-04-28T18:19:06Z | MEMBER | It can detect faces, run OCR, do image labeling (it knows what a lemur is!) and do object localization where it identifies objects and returns bounding polygons for them. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14/reactions", "total_count": 3, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
||||||||
612287234 | MDU6SXNzdWU2MTIyODcyMzQ= | 16 | Import machine-learning detected labels (dog, llama etc) from Apple Photos | 9599 | open | 0 | 13 | 2020-05-05T02:45:43Z | 2020-05-05T05:38:16Z | MEMBER | Follow-on from #1. Apple Photos runs some very sophisticated machine learning on-device to figure out if photos are of dogs, llamas and so on. I really want to extract those labels out into my own database. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 1, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
612860758 | MDU6SXNzdWU2MTI4NjA3NTg= | 18 | Switch CI solution to GitHub Actions with a macOS runner | 9599 | open | 0 | 1 | 2020-05-05T20:03:50Z | 2020-05-05T23:49:18Z | MEMBER | Refs #17. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
615626118 | MDU6SXNzdWU2MTU2MjYxMTg= | 22 | Try out ExifReader | 9599 | open | 0 | 4 | 2020-05-11T06:32:13Z | 2020-05-14T05:59:53Z | MEMBER | https://pypi.org/project/ExifReader/ New fork that should be able to handle EXIF in HEIC files. Forked here: https://github.com/ianare/exif-py/issues/102#issuecomment-626376522 Refs #3 | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
621323348 | MDU6SXNzdWU2MjEzMjMzNDg= | 24 | Configurable URL for images | 9599 | open | 0 | 1 | 2020-05-19T22:25:56Z | 2020-05-20T06:00:29Z | MEMBER | This is hard-coded at the moment, which is bad: https://github.com/dogsheep/photos-to-sqlite/blob/d5d69b9019703c47bc251444838578dd752801e2/photos_to_sqlite/cli.py#L269-L272 | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/24/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
621486115 | MDU6SXNzdWU2MjE0ODYxMTU= | 27 | photos_with_apple_metadata view should include labels | 9599 | open | 0 | 0 | 2020-05-20T06:06:17Z | 2020-05-20T06:06:17Z | MEMBER | https://dogsheep-photos.dogsheep.net/public/photos_with_apple_metadata?place_city=New+Orleans&_facet=place_city&_facet_array=albums&_facet_array=persons Here's one way to add that: ```sql select rowid, photo, ( select json_group_array( json_object( 'label', normalized_string, 'href', '/photos/labelled?_hide_sql=1&label=' || normalized_string ) ) from labels where labels.uuid = photos_with_apple_metadata.uuid ) as labels, date, ``` | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
602533481 | MDU6SXNzdWU2MDI1MzM0ODE= | 3 | Import EXIF data into SQLite - lens used, ISO, aperture etc | 9599 | open | 0 | 5324096 | 2 | 2020-04-18T19:24:31Z | 2021-10-05T12:38:24Z | MEMBER | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
602585497 | MDU6SXNzdWU2MDI1ODU0OTc= | 7 | Integrate image content hashing | 9599 | open | 0 | 2 | 2020-04-19T00:36:58Z | 2021-08-26T02:01:01Z | MEMBER | To spot duplicate images (where the file content differs such that the sha256 is no longer a match) it would be useful to calculate and store perceptual hashes of some sort. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/7/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
||||||||
718934942 | MDU6SXNzdWU3MTg5MzQ5NDI= | 1 | Documentation on how to use this with Datasette | 9599 | open | 0 | 1 | 2020-10-11T21:56:27Z | 2020-10-11T22:14:00Z | MEMBER | In particular how to use `datasette-render-images` to see the images. | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
718938889 | MDU6SXNzdWU3MTg5Mzg4ODk= | 5 | Figure out how to display images from <en-media> tags inline in Datasette | 9599 | open | 0 | 6 | 2020-10-11T22:17:03Z | 2020-10-16T20:16:28Z | MEMBER | Relates to #1. Evernote XML looks like this: ```xml <?xml version="1.0"?> <en-note> <div>This note includes two images.</div> <div> <b>The Python logo</b> </div> <div> <en-media hash="61098c2c541de7f0a907c301dd6542da" type="image/svg+xml" width="125"/> </div> <div> <b>The Evernote logo</b> </div> <div> <en-media hash="91bd26175acac0b2ffdb6efac199f8ca" type="image/svg+xml" width="125"/> </div> </en-note> ``` That hash is the md5 we use to store resources. It should be possible to turn these into embedded image tags, especially if done in conjunction with the https://github.com/simonw/datasette-media plugin. | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
496415321 | MDU6SXNzdWU0OTY0MTUzMjE= | 1 | Figure out some interesting example SQL queries | 9599 | open | 0 | 9 | 2019-09-20T15:28:07Z | 2021-05-03T03:46:23Z | MEMBER | My knowledge of genetics has left me short here. I'd love to be able to provide some interesting example SELECT queries - maybe one that spots if you are [likely to have red hair?](https://www.snpedia.com/index.php/Rs1805007) | 209590345 | issue | { "url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
530491074 | MDU6SXNzdWU1MzA0OTEwNzQ= | 14 | Command for importing events | 9599 | open | 0 | 3 | 2019-11-29T21:28:58Z | 2020-04-14T19:38:34Z | MEMBER | Eg from https://api.github.com/users/simonw/events Docs here: https://developer.github.com/v3/activity/events/#list-events-performed-by-a-user | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
599776345 | MDU6SXNzdWU1OTk3NzYzNDU= | 24 | Feature idea: github-to-sqlite everything ... | 9599 | open | 0 | 0 | 2020-04-14T18:34:00Z | 2020-04-14T18:34:00Z | MEMBER | At the moment if you want to pull all your repos, issues, issues comments etc you have to do it with a sequence of separate commands. Consider adding a `everything` or `all` command which fetches everything that the tool knows how to fetch, and is designed to be run on a cron in a way that fetches just new stuff each time. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/24/reactions", "total_count": 7, "+1": 7, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
703216044 | MDU6SXNzdWU3MDMyMTYwNDQ= | 49 | Feature: gists and starred gists | 9599 | open | 0 | 0 | 2020-09-17T02:30:52Z | 2020-09-17T02:30:52Z | MEMBER | https://developer.github.com/v3/gists/#list-starred-gists | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/49/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
703218756 | MDU6SXNzdWU3MDMyMTg3NTY= | 50 | Commands for making authenticated API calls | 9599 | open | 0 | 7 | 2020-09-17T02:39:07Z | 2020-10-19T05:01:29Z | MEMBER | Similar to `twitter-to-sqlite fetch`, see https://github.com/dogsheep/twitter-to-sqlite/issues/51 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
703246031 | MDU6SXNzdWU3MDMyNDYwMzE= | 51 | github-to-sqlite should handle rate limits better | 9599 | open | 0 | 4 | 2020-09-17T04:01:50Z | 2022-10-14T16:34:07Z | MEMBER | From #50 - right now it will crash with an error of it hits the rate limit. Since the rate limit information (including reset time) is available in the headers it could automatically sleep and try again instead. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
753000405 | MDU6SXNzdWU3NTMwMDA0MDU= | 53 | Command for fetching file contents | 9599 | open | 0 | 1 | 2020-11-29T20:31:04Z | 2020-11-30T00:36:09Z | MEMBER | Something like this: github-to-sqlite files github.db simonw/datasette This would fetch all files from the `main` branch into a `files` table. Additional options could handle things like pulling files from a branch or tag, or just pulling files that match a specific glob or that exist in a specific directory. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/53/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
897212458 | MDU6SXNzdWU4OTcyMTI0NTg= | 63 | Ability to fetch commits from branches other than the default | 9599 | open | 0 | 0 | 2021-05-20T17:58:08Z | 2021-05-20T17:58:08Z | MEMBER | This tool is currently almost entirely ignorant of the concept of branches. One example: you can't retrieve commits from any branch other than the default (usually main). | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/63/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1071071397 | I_kwDODFdgUs4_10Cl | 69 | View that combines issues and issue comments | 9599 | open | 0 | 1 | 2021-12-04T00:34:33Z | 2021-12-04T00:34:52Z | MEMBER | I want to see a reverse chronologically ordered interface onto both issues and comments - essentially a unified log of comments and issues opened across one or multiple projects. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1570375808 | I_kwDODFdgUs5dmgiA | 79 | Deploy demo job is failing due to rate limit | 9599 | open | 0 | 2 | 2023-02-03T20:05:01Z | 2023-12-08T14:50:15Z | MEMBER | https://github.com/dogsheep/github-to-sqlite/actions/runs/4080058087/jobs/7032116511 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/79/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
821841046 | MDU6SXNzdWU4MjE4NDEwNDY= | 6 | Upgrade to latest sqlite-utils | 9599 | open | 0 | 1 | 2021-03-04T07:21:54Z | 2021-03-04T07:22:51Z | MEMBER | This is pinned to v1 at the moment. | 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
952179830 | MDU6SXNzdWU5NTIxNzk4MzA= | 2 | Command for fetching Hacker News threads from the search API | 9599 | open | 0 | 4 | 2021-07-25T02:00:45Z | 2021-07-25T03:12:57Z | MEMBER | I want to be able to fetch every item for a domain, e.g. https://news.ycombinator.com/from?site=simonwillison.net | 248903544 | issue | { "url": "https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
952189173 | MDU6SXNzdWU5NTIxODkxNzM= | 3 | Use HN algolia endpoint to retrieve trees | 9599 | open | 0 | 3 | 2021-07-25T03:35:27Z | 2021-07-25T18:41:17Z | MEMBER | The `trees` command currently has to make a request for every single comment. Algolia have an endpoint that bundles the entire thread together into a single request. `https://hn.algolia.com/api/v1/items/ID` Here's an example that loads quickly, with about 50 comments: https://hn.algolia.com/api/v1/items/27941108 It doesn't appear to use pagination at all - if a thread is big then the response is big. I ran this search to find some stories with more than 1000 comments: https://hn.algolia.com/api/v1/search?tags=story&numericFilters=num_comments%3E=1000 Here's one: https://news.ycombinator.com/item?id=25015967 with 4759 comments. Hitting the API takes 41s and returns 3.7 MB of JSON! ``` wget 'https://hn.algolia.com/api/v1/items/25015967' 0.03s user 0.04s system 0% cpu 41.368 total /tmp % ls -lah 25015967 -rw-r--r-- 1 simon wheel 3.7M Jul 24 20:31 25015967 ``` | 248903544 | issue | { "url": "https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
727848625 | MDU6SXNzdWU3Mjc4NDg2MjU= | 12 | Some workout columns should be float, not text | 9599 | open | 0 | 4 | 2020-10-23T02:47:02Z | 2022-06-23T04:35:02Z | MEMBER | Columns `duration`, `totalDistance` and `totalEnergyBurned` should be converted to float. https://github.com/dogsheep/healthkit-to-sqlite/blob/71e36e1cf034b96de2a8e6652265d782d3fdf63b/healthkit_to_sqlite/utils.py#L50-L57 | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
975166271 | MDU6SXNzdWU5NzUxNjYyNzE= | 20 | Add index on workout_points.date | 9599 | open | 0 | 2 | 2021-08-20T01:08:04Z | 2021-08-20T01:12:48Z | MEMBER | Sorting that by date makes sense for seeing most recent points, and my DB has 2.5m points in so it's an expensive sort! | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
503243784 | MDU6SXNzdWU1MDMyNDM3ODQ= | 3 | Extract images into separate tables | 9599 | open | 0 | 1 | 2019-10-07T05:43:01Z | 2020-09-01T06:17:45Z | MEMBER | As already done with authors. Slightly harder because images do not have a universally unique ID. Also need to figure out what to do about there being columns for both `image` and `images`. <img width="1522" alt="memory__items" src="https://user-images.githubusercontent.com/9599/66287418-9ab20680-e88a-11e9-96bf-6c80d881eff0.png"> | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
689848827 | MDU6SXNzdWU2ODk4NDg4Mjc= | 6 | ISO timestamps | 9599 | open | 0 | 0 | 2020-09-01T06:16:42Z | 2020-09-01T06:16:42Z | MEMBER | The `time_added`, `time_updated` and `time_read` columns currently store data like this: September 19, 2019 - 00:30:30 UTC Should use ISO instead, e.g. `2020-07-26T01:05:24+00:00` | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
673602857 | MDU6SXNzdWU2NzM2MDI4NTc= | 9 | Define a view that displays photos correctly | 9599 | open | 0 | 0 | 2020-08-05T14:53:39Z | 2020-08-05T14:53:39Z | MEMBER | The `photos` table stores data like this: id | createdAt | source | prefix | suffix | width | height | visibility | created ▲ | user -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 5e12c9708506bc000840262a | January 06, 2020 - 05:45:20 UTC | Swarm for iOS 1 | https://fastly.4sqi.net/img/general/ | /15889193_AXxGk4I1nbzUZuyYqObgbXdJNyEHiwj6AUDq0tPZWtw.jpg | 1920 | 1440 | public | 2020-01-06T05:45:20 | 15889193 The photo URL can be derived from those pieces - define a SQL view which does that (using `datasette-json-html` to display the pictures) | 205429375 | issue | { "url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
505673645 | MDU6SXNzdWU1MDU2NzM2NDU= | 16 | Do a better job with archived direct message threads | 9599 | open | 0 | 0 | 2019-10-11T06:55:21Z | 2019-10-11T06:55:27Z | MEMBER | https://github.com/dogsheep/twitter-to-sqlite/blob/fb2698086d766e0333a55bb73435e7283feeb438/twitter_to_sqlite/archive.py#L98-L99 | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
602619330 | MDU6SXNzdWU2MDI2MTkzMzA= | 45 | Use raise_for_status() everywhere | 9599 | open | 0 | 1 | 2020-04-19T04:38:28Z | 2020-04-19T04:39:22Z | MEMBER | I keep seeing errors which I think are caused by authentication or rate limit problems but which appear to be unexpected JSON responses - presumably because they are actually an error message. Recent example: https://github.com/simonw/jsk-fellows-on-twitter/runs/598892575 Using `response.raise_for_status()` everywhere will make these errors less confusing. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/45/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
488874815 | MDU6SXNzdWU0ODg4NzQ4MTU= | 5 | Write tests that simulate the Twitter API | 9599 | open | 0 | 1 | 2019-09-03T23:55:35Z | 2019-09-03T23:56:28Z | MEMBER | I can use betamax for this: https://pypi.org/project/betamax/ | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
703218448 | MDU6SXNzdWU3MDMyMTg0NDg= | 51 | Documentation for twitter-to-sqlite fetch | 9599 | open | 0 | 0 | 2020-09-17T02:38:10Z | 2020-09-17T02:38:10Z | MEMBER | It's mentioned in passing in the README but it deserves its own section: ``` $ twitter-to-sqlite fetch \ "https://api.twitter.com/1.1/account/verify_credentials.json" \ | grep '"id"' | head -n 1 ``` | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/51/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
718272593 | MDU6SXNzdWU3MTgyNzI1OTM= | 1007 | set-env and add-path commands have been deprecated | 9599 | open | 0 | 1 | 2020-10-09T16:21:18Z | 2020-10-09T16:23:51Z | OWNER | https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/ > Starting today runner version 2.273.5 will begin to warn you if you use the `add-path` or `set-env` commands. We are monitoring telemetry for the usage of these commands and plan to fully disable them in the future. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1007/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
718540751 | MDU6SXNzdWU3MTg1NDA3NTE= | 1012 | For 1.0 update trove classifier in setup.py | 9599 | open | 0 | 3268330 | 5 | 2020-10-10T05:52:08Z | 2021-11-16T13:18:36Z | OWNER | Development Status :: 5 - Production/Stable | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1012/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
718910318 | MDU6SXNzdWU3MTg5MTAzMTg= | 1015 | Research: could Datasette install its own plugins? | 9599 | open | 0 | 1 | 2020-10-11T19:33:06Z | 2020-10-11T19:35:04Z | OWNER | It would be cool if Datasette could offer a plugin browsing interface where users could install plugins by clicking "Install" on them - similar to how VS Code extensions work. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1015/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
721068929 | MDU6SXNzdWU3MjEwNjg5Mjk= | 1020 | Method for datasette.client() to forward on authentication | 9599 | open | 0 | 6 | 2020-10-14T01:47:49Z | 2020-10-19T22:45:01Z | OWNER | I stumbled into this while working on Dogsheep Beta: the requests it re-dispatched through `TableView` did not carry authentication cookies, and since this was against a private instance they were thus denied. https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L223-L231 This made me think that `datasette.client.get()` (which Dogsheep Beta will start using shortly) could benefit from some kind of utility mechanism for passing through the cookies and general authenticated state from the current request. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1020/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
724878151 | MDU6SXNzdWU3MjQ4NzgxNTE= | 1032 | Bring date parsing into Datasette core | 9599 | open | 0 | 8 | 2020-10-19T18:30:45Z | 2020-10-19T19:37:55Z | OWNER | Currently this is mainly handled by a plugin - https://github.com/simonw/datasette-dateutil - but I realise now that this really needs to be core functionality. See also Twitter thread: https://twitter.com/simonw/status/1318234808653213696 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1032/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
728905098 | MDU6SXNzdWU3Mjg5MDUwOTg= | 1048 | Documentation and unit tests for urls.row() urls.row_blob() methods | 9599 | open | 0 | 7 | 2020-10-25T00:13:53Z | 2022-07-10T16:23:57Z | OWNER | https://github.com/simonw/datasette/blob/5db7ae3ce165ded57c7fb1cfbdb3258b1cf06c10/datasette/app.py#L1307-L1313 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1048/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
730210880 | MDU6SXNzdWU3MzAyMTA4ODA= | 1055 | query.html and table.html should share the same table implementation | 9599 | open | 0 | 3268330 | 0 | 2020-10-27T07:58:21Z | 2020-10-27T07:58:29Z | OWNER | In #998 I made a change that affected the table page but didn't affect the query page because I incorrectly assumed they shared rendering logic. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1055/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
732674148 | MDU6SXNzdWU3MzI2NzQxNDg= | 1062 | Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows | 9599 | open | 0 | 3268330 | 5 | 2020-10-29T21:25:02Z | 2022-09-28T14:09:54Z | OWNER | This can drive the upgrade of the `register_output_renderer` hook to be able to handle streaming all rows in a large query. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1062/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
733999615 | MDU6SXNzdWU3MzM5OTk2MTU= | 1079 | Handle long breadcrumbs better with new menu | 9599 | open | 0 | 1 | 2020-11-01T15:57:41Z | 2022-01-13T22:21:29Z | OWNER | On this page when signed in as root: https://latest.datasette.io/fixtures/roadside_attraction_characteristics/1 ![EF921CB1-625F-4D04-A850-490B812A72B3](https://user-images.githubusercontent.com/9599/97807807-db0fbf80-1c17-11eb-9c77-ae5169b12c3d.jpeg) ![A49D8B76-5ACF-4F71-A8B4-21A44F5C8D51](https://user-images.githubusercontent.com/9599/97807809-dea34680-1c17-11eb-9511-a49af56a4bd2.jpeg) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1079/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
734777631 | MDU6SXNzdWU3MzQ3Nzc2MzE= | 1080 | "View all" option for facets, to provide a (paginated) list of ALL of the facet counts plus a link to view them | 9599 | open | 0 | 3268330 | 7 | 2020-11-02T19:55:06Z | 2022-02-04T06:25:18Z | OWNER | Can use `/database/-/...` namespace from #296 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1080/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
736365306 | MDU6SXNzdWU3MzYzNjUzMDY= | 1083 | Advanced CSV export for arbitrary queries | 9599 | open | 0 | 2 | 2020-11-04T19:23:05Z | 2021-06-17T18:12:31Z | OWNER | There's no link to download the CSV file - the table page has that as an advanced export option, but this is missing from the query page. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1083/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
741231849 | MDU6SXNzdWU3NDEyMzE4NDk= | 1087 | Idea: ?_extra=urls for getting back URLs to useful things | 9599 | open | 0 | 0 | 2020-11-12T02:55:41Z | 2021-12-15T18:06:16Z | OWNER | Working on https://github.com/simonw/datasette-search-all/issues/10 made me realize that sometimes it can be difficult to calculate the URL for a database, table or row within Datasette. It would be useful to have an optional extra JSON extension (using `?_extra=` from #262) that can help with this. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1087/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
741862364 | MDU6SXNzdWU3NDE4NjIzNjQ= | 1090 | Custom widgets for canned query forms | 9599 | open | 0 | 3 | 2020-11-12T19:21:07Z | 2021-03-27T16:25:25Z | OWNER | This is an idea that was cut from the first version of writable canned queries: > I really want the option to use a `<textarea>` for a specific value. > > Idea: metadata syntax like this: > > ```json > { > "databases": { > "my-database": { > "queries": { > "add_twitter_handle": { > "sql": "insert into twitter_handles (username) values (:username)", > "write": true, > "params": { > "username": { > "widget": "textarea" > } > } > } > } > } > } > } > ``` > > I can ship with some default widgets and provide a plugin hook for registering extra widgets. > > This opens up some really exciting possibilities for things like map widgets that let you draw polygons. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-608125928_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1090/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
743359646 | MDU6SXNzdWU3NDMzNTk2NDY= | 1096 | TSV should be a default export option | 9599 | open | 0 | 1 | 2020-11-15T22:24:02Z | 2021-06-17T18:12:31Z | OWNER | Refs #1095 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1096/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
743371103 | MDU6SXNzdWU3NDMzNzExMDM= | 1099 | Support linking to compound foreign keys | 9599 | open | 0 | 6 | 2020-11-15T23:23:17Z | 2023-01-25T00:58:26Z | OWNER | Reported as a bug in #1098 because they caused 500 errors - but it would be even better if Datasette could hyperlink to related rows via compound foreign keys. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1099/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
749283032 | MDU6SXNzdWU3NDkyODMwMzI= | 1101 | register_output_renderer() should support streaming data | 9599 | open | 0 | 3268330 | 13 | 2020-11-24T02:17:09Z | 2023-01-21T22:07:19Z | OWNER | > I'd like to implement this by first extending the `register_output_renderer()` hook to support streaming huge responses, then switching CSV to use the plugin hook in addition to TSV using it. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1096#issuecomment-732542285_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1101/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
750089847 | MDU6SXNzdWU3NTAwODk4NDc= | 1109 | Deprecate --config in Datasette 1.0 (in favour of --setting) | 9599 | open | 0 | 3268330 | 0 | 2020-11-24T21:43:57Z | 2020-12-17T22:07:49Z | OWNER | I added a deprecation warning to this in #992. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1109/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
274615452 | MDU6SXNzdWUyNzQ2MTU0NTI= | 111 | Add “updated” to metadata | 9599 | open | 0 | 12 | 2017-11-16T18:22:20Z | 2021-09-21T22:48:27Z | OWNER | To give an indication as to when the data was last updated. This should be a field in the metadata that is then shown on the index page and in the footer, if it is set. Also support setting it using an option to “datasette publish” and “datasette package” - which can either be a string or can be the magic string “today” to set it to today’s date: datasette publish file.db --updated=today | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/111/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
763361458 | MDU6SXNzdWU3NjMzNjE0NTg= | 1142 | "Stream all rows" is not at all obvious | 9599 | open | 0 | 9 | 2020-12-12T06:24:57Z | 2021-06-17T18:12:31Z | OWNER | Got a question about how to download all rows - the current option isn't at all clear. <img width="668" alt="loans__ppp_loans__9_511_rows_where_where_search_matches__tech__sorted_by_rowid" src="https://user-images.githubusercontent.com/9599/101977057-ac660b00-3bff-11eb-88f4-c93ffd03d3e0.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1142/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
765637324 | MDU6SXNzdWU3NjU2MzczMjQ= | 1144 | JavaScript to help plugins interact with the fragment part of the URL | 9599 | open | 0 | 1 | 2020-12-13T20:36:06Z | 2020-12-14T14:47:11Z | OWNER | Suggested by Markus Holtermann on Twitter, who is building https://github.com/MarkusH/datasette-chartjs > I've been looking at datasette-vega for how you persist chart settings between form submissions. I've adopted that for datasette-chartjs. Any thoughts on adding a public JS API to #datasette itself, that plugins can rely on? > > I'm talking about functions like onFragmentChange, serialize, unserialize, ... That turn an object into a URL encoded string and put it into the location's hash. And also updating all links/forms automatically. > > Essentially, a plugins could do something like `document.datasette.setConfigValue('prefix', 'foo', 'bar')` and `.getConfigValue('prefix', 'foo')`. And the functions would take care of updating document.location.hash, all (necessary) a.href and form.action https://twitter.com/m_holtermann/status/1338183973311295492 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1144/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
769520939 | MDU6SXNzdWU3Njk1MjA5Mzk= | 1149 | Make it easier to theme Datasette with CSS | 9599 | open | 0 | 3268330 | 3 | 2020-12-17T05:01:26Z | 2021-03-22T21:43:16Z | OWNER | I want to theme https://datasette.io/ so that when you visit https://datasette.io/content (the Datasette UI part of it) the navigation from the parent site is used. I tried dropping in a `base.html` template like this: ```html {% extends "page_base.html" %} {% block base_extra_head %} <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> {% for url in extra_css_urls %} <link rel="stylesheet" href="{{ url.url }}"{% if url.sri %} integrity="{{ url.sri }}" crossorigin="anonymous"{% endif %}> {% endfor %} {% for url in extra_js_urls %} <script src="{{ url.url }}"{% if url.sri %} integrity="{{ url.sri }}" crossorigin="anonymous"{% endif %}></script> {% endfor %} {% block extra_head %}{% endblock %} {% endblock %} {% block extra_body_end %} {% include "_close_open_menus.html" %} {% for body_script in body_scripts %} <script>{{ body_script }}</script> {% endfor %} {% endblock %} ``` But this resulted in pages looking like this: <img width="1067" alt="content__categories__3_rows" src="https://user-images.githubusercontent.com/9599/102446045-c168e280-3fe1-11eb-94d6-e7350798eb96.png"> Note that the cog menu is broken and the filter UI is unstyled. To get these working correctly I would need to copy over a whole lot of Datasette's default CSS - and that means that when Datasette changes in the future those pages could break in subtle ways. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1149/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
770598024 | MDU6SXNzdWU3NzA1OTgwMjQ= | 1152 | Efficiently calculate list of databases/tables a user can view | 9599 | open | 0 | 12 | 2020-12-18T06:13:01Z | 2021-12-27T23:04:31Z | OWNER | > The homepage currently performs a massive flurry of permission checks - one for each, database, table and view: https://github.com/simonw/datasette/blob/0.53/datasette/views/index.py#L21-L75 > > A paginated version of this is a little daunting as the permission checks would have to be carried out in every single table just to calculate the count that will be paginated. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747864831_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1152/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
775666296 | MDU6SXNzdWU3NzU2NjYyOTY= | 1160 | "datasette insert" command and plugin hook | 9599 | open | 0 | 23 | 2020-12-29T02:37:03Z | 2021-06-17T18:12:32Z | OWNER | Tools for loading data into Datasette currently mostly exist as separate utilities - `yaml-to-sqlite` and `csvs-to-sqlite` and suchlike. Bringing these into Datasette could have some interesting properties: - A `datasette insert` command could be extended with plugins to handle more formats - Any format that can be inserted on the command-line could also be inserted using a web UI or web API - which would benefit from new format plugin hooks - If Datasette ever grows beyond SQLite (see #670) a built-in import mechanism could work for those other databases as well - without me needing to write `yaml-to-postgresql` and suchlike | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1160/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776101101 | MDU6SXNzdWU3NzYxMDExMDE= | 1161 | Update a whole bunch of links to datasette.io instead of datasette.readthedocs.io | 9599 | open | 0 | 1 | 2020-12-29T21:47:31Z | 2020-12-29T21:49:57Z | OWNER | https://ripgrep.datasette.io/-/ripgrep?pattern=%28datasette%7Csqlite-utils%29%5C.readthedocs%5C.io | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1161/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776128269 | MDU6SXNzdWU3NzYxMjgyNjk= | 1162 | First working version of "datasette insert data.db file.csv" | 9599 | open | 0 | 0 | 2020-12-29T23:20:11Z | 2021-06-17T18:12:32Z | OWNER | Refs #1160 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1162/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776128565 | MDU6SXNzdWU3NzYxMjg1NjU= | 1163 | "datasette insert data.db url-to-csv" | 9599 | open | 0 | 1 | 2020-12-29T23:21:21Z | 2021-06-17T18:12:32Z | OWNER | Refs #1160 - get filesystem imports working first for #1162, then add import-from-URL. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1163/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776634318 | MDU6SXNzdWU3NzY2MzQzMTg= | 1164 | Mechanism for minifying JavaScript that ships with Datasette | 9599 | open | 0 | 9 | 2020-12-30T20:59:06Z | 2022-01-13T22:21:29Z | OWNER | > If I'm going to minify it I'll need to figure out a build step in Datasette itself so that I can easily work on that minified version. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752748496_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1164/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776635426 | MDU6SXNzdWU3NzY2MzU0MjY= | 1165 | Mechanism for executing JavaScript unit tests | 9599 | open | 0 | 9 | 2020-12-30T21:02:34Z | 2022-01-13T22:21:29Z | OWNER | > I'm going to need to add JavaScript unit tests for this new plugin system. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752757289_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1165/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
777140799 | MDU6SXNzdWU3NzcxNDA3OTk= | 1166 | Adopt Prettier for JavaScript code formatting | 9599 | open | 0 | 10 | 2020-12-31T21:25:27Z | 2022-01-13T22:22:18Z | OWNER | https://prettier.io/ - I'm going to go with 2 spaces. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1166/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
777333388 | MDU6SXNzdWU3NzczMzMzODg= | 1168 | Mechanism for storing metadata in _metadata tables | 9599 | open | 0 | 21 | 2021-01-01T18:47:27Z | 2023-09-28T18:29:05Z | OWNER | _Original title: Perhaps metadata should all live in a `_metadata` in-memory database_ Inspired by #1150 - metadata should be exposed as an API, and for large Datasette instances that API may need to be paginated. So why not expose it through an in-memory database table? One catch to this: plugins. #860 aims to add a plugin hook for metadata. But if the metadata comes from an in-memory table, how do the plugins interact with it? The need to paginate over metadata does make a plugin hook that returns metadata for an individual table seem less wise, since we don't want to have to do 10,000 plugin hook invocations to show a list of all metadata. If those plugins write directly to the in-memory table how can their contributions survive the server restarting? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1168/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778450486 | MDU6SXNzdWU3Nzg0NTA0ODY= | 1171 | GitHub Actions workflow to build and sign macOS binary executables | 9599 | open | 0 | 8 | 2021-01-04T23:36:59Z | 2021-01-07T19:36:00Z | OWNER | Using PyInstaller, as explored in #93 and https://til.simonwillison.net/python/packaging-pyinstaller The bigger challenge will be the code signing bit. I'll need a Apple Developer account ($99/year) and some extensive CI fiddling. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1171/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778530523 | MDU6SXNzdWU3Nzg1MzA1MjM= | 1172 | /-/static should be excluded from auth and permission checks | 9599 | open | 0 | 0 | 2021-01-05T02:53:41Z | 2021-01-05T02:53:41Z | OWNER | I want to set far future / immutable cache headers on everything served from `/-/static` and `/-/static-plugins` This has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1172/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778682317 | MDU6SXNzdWU3Nzg2ODIzMTc= | 1173 | GitHub Actions workflow to build manylinux binary | 9599 | open | 0 | 1 | 2021-01-05T07:41:11Z | 2021-01-05T07:41:43Z | OWNER | Refs #1171 and #93 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1173/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
779156520 | MDU6SXNzdWU3NzkxNTY1MjA= | 1175 | Use structlog for logging | 9599 | open | 0 | 4 | 2021-01-05T15:11:36Z | 2022-07-26T12:52:10Z | OWNER | To solve #241 JSON logging. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1175/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
780153562 | MDU6SXNzdWU3ODAxNTM1NjI= | 1177 | Ability to stream all rows as newline-delimited JSON | 9599 | open | 0 | 3268330 | 1 | 2021-01-06T07:10:48Z | 2022-03-21T15:08:52Z | OWNER | > Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas: > > pandas.read_json("https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on", lines=True) _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1101#issuecomment-755128038_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1177/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
780278550 | MDU6SXNzdWU3ODAyNzg1NTA= | 1179 | Make original path available to render hooks | 9599 | open | 0 | 8 | 2021-01-06T08:31:45Z | 2021-01-25T04:44:33Z | OWNER | https://github.com/simonw/datasette-export-notebook/blob/0.1/datasette_export_notebook/__init__.py ```python async def render_notebook(datasette, request): return Response.html( await datasette.render_template( "export_notebook.html", { "csv_stream_url": datasette.absolute_url( request, path_with_format( request=request, format="csv", extra_qs={"_stream": "on"} ), ), "json_url": datasette.absolute_url( request, path_with_format( request=request, format="json", extra_qs={"_shape": "array"} ), ), "json": json, }, ) ) ``` This results in https://latest-with-plugins.datasette.io/github/issue_comments.Notebook showing `http://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_format=json&_shape=array` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1179/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
780767542 | MDU6SXNzdWU3ODA3Njc1NDI= | 1180 | Lazily evaluated arguments for call_with_supported_arguments | 9599 | open | 0 | 2 | 2021-01-06T18:43:34Z | 2021-01-07T18:56:24Z | OWNER | While building https://github.com/simonw/datasette-export-notebook I thought it would be nice to be able to show a count of exported records on the page "This will stream 10,422 records to your notebook". None of the documented arguments on https://docs.datasette.io/en/0.53/plugin_hooks.html#register-output-renderer-datasette expose the count. The closest is `sql` which could be executed as `select count(*) from ({sql})` but that's a bit inelegant. So, idea: if your defined render function takes a `total` argument, the caller runs a `count(*)` query and passes you that total. To implement this I would need to teach the `call_with_supported_arguments` that some arguments are lazy - they should execute a function (or an async function) but only if they are needed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1180/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
782708469 | MDU6SXNzdWU3ODI3MDg0Njk= | 1183 | Take advantage of sqlite-utils cached table counts, if available | 9599 | open | 0 | 2 | 2021-01-09T23:51:48Z | 2021-01-12T02:42:08Z | OWNER | sqlite-utils 3.2 now has a mechanism for creating a `_counts` table with triggers to maintain counts for individual tables. Datasette could look for this table and use it for counts, dramatically speeding up places that currently suffer from slow counts. Refs #859. https://sqlite-utils.datasette.io/en/stable/python-api.html#cached-table-counts-using-triggers | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1183/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
787098345 | MDU6SXNzdWU3ODcwOTgzNDU= | 1191 | Ability for plugins to collaborate when adding extra HTML to blocks in default templates | 9599 | open | 0 | 3268330 | 12 | 2021-01-15T18:18:51Z | 2023-09-18T06:55:52Z | OWNER | Sometimes a plugin may want to add content to an existing default template - for example `datasette-search-all` adds a new search box at the top of `index.html`. I also want `datasette-upload-csvs` to add a CTA on the `database.html` page: https://github.com/simonw/datasette-upload-csvs/issues/18 Currently plugins can do this by providing a new version of the `index.html` template - but if multiple plugins try to do that only one of them will succeed. It would be better if there were known areas of those templates which plugins could add additional content to, such that multiple plugins can use the same spot. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1191/reactions", "total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
787173276 | MDU6SXNzdWU3ODcxNzMyNzY= | 1193 | Research plugin hook for alternative database backends | 9599 | open | 0 | 1 | 2021-01-15T20:27:50Z | 2021-03-12T01:01:54Z | OWNER | I started exploring what Datasette would like running against PostgreSQL in #670 and @dazzag24 did some work on Parquet described in #657. I had initially thought this was WAY too much additional complexity, but I'm beginning to think that the `Database` class may be small enough that having it abstract away the details of running queries against alternative database backends could be feasible. A bigger issue is SQL generation, but I realized that most of Datasette's SQL generation code exists just in the `TableView` class that runs the table page. If this was abstracted into some kind of SQL builder that could be then customized per-database it might be reasonable to get it working. Very unlikely for this to make it into Datasette 1.0, but maybe this would be the defining feature of Datasette 2.0? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1193/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
789336592 | MDU6SXNzdWU3ODkzMzY1OTI= | 1195 | view_name = "query" for the query page | 9599 | open | 0 | 4 | 2021-01-19T20:21:36Z | 2021-01-25T04:40:08Z | OWNER | It uses `view_name` of `database` at the moment which isn't as useful. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1195/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
792652391 | MDU6SXNzdWU3OTI2NTIzOTE= | 1199 | Experiment with PRAGMA mmap_size=N | 9599 | open | 0 | 2 | 2021-01-23T21:24:09Z | 2021-07-17T17:39:17Z | OWNER | https://sqlite.org/mmap.html - SQLite supports memory-mapped I/O but it's disabled by default. The `PRAGMA mmap_size=N` option can be used to enable it. It would be very interesting to understand the impact this could have on Datasette performance for various different shapes of data. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
792890765 | MDU6SXNzdWU3OTI4OTA3NjU= | 1200 | ?_size=10 option for the arbitrary query page would be useful | 9599 | open | 0 | 2 | 2021-01-24T20:55:35Z | 2021-02-11T03:13:59Z | OWNER | https://latest.datasette.io/fixtures?sql=select+*+from+compound_three_primary_keys&_size=10 - `_size=10` does not do anything at the moment. It would be useful if it did. Would also be good if it persisted in a hidden form field. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
793002853 | MDExOlB1bGxSZXF1ZXN0NTYwNzYwMTQ1 | 1204 | WIP: Plugin includes | 9599 | open | 0 | 3 | 2021-01-25T03:59:06Z | 2021-12-17T07:10:49Z | OWNER | simonw/datasette/pulls/1204 | Refs #1191 Next steps: - [ ] Get comfortable that this pattern is the right way to go - [ ] Implement it for all of the other pages, not just the table page - [ ] Add a new set of plugin tests that exercise ALL of these new hook locations - [ ] Document, then ship | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1204/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | ||||||
799663959 | MDU6SXNzdWU3OTk2NjM5NTk= | 1213 | gzip support for HTML (and JSON) responses | 9599 | open | 0 | 3 | 2021-02-02T20:36:28Z | 2021-02-02T20:41:55Z | OWNER | This page https://datasette-tiles-demo.datasette.io/San_Francisco/tiles is 2MB because of all of the base64 images. Gzipped it's 1.5MB. Since Datasette is usually deployed without a frontend gzipping proxy, Datasette itself needs to solve for this. Gzipping everything won't work because some endpoints - the all-rows CSV endpoint and the download-database endpoint - are streaming and hence can't be buffered-and-gzipped. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1213/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
803929694 | MDU6SXNzdWU4MDM5Mjk2OTQ= | 1219 | Try profiling Datasette using scalene | 9599 | open | 0 | 2 | 2021-02-08T20:37:06Z | 2021-02-08T22:13:00Z | OWNER | https://github.com/emeryberger/scalene looks like an interesting profiling tool. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1219/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
275125561 | MDU6SXNzdWUyNzUxMjU1NjE= | 123 | Datasette serve should accept paths/URLs to CSVs and other file formats | 9599 | open | 0 | 9 | 2017-11-19T02:05:48Z | 2021-07-19T00:04:32Z | OWNER | This would remove the csvs-to-sqlite step which I end up using for almost everything. I'm hesitant to introduce pandas as a required dependency though since it require compiling numpy. Could build it so this option is only available if you have pandas installed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/123/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
||||||||
811458446 | MDU6SXNzdWU4MTE0NTg0NDY= | 1233 | "datasette publish cloudrun" cannot publish files with spaces in their name | 9599 | open | 0 | 1 | 2021-02-18T21:08:31Z | 2021-02-18T21:10:08Z | OWNER | Got this error: ``` Step 6/9 : RUN datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json ---> Running in db9da0068592 Usage: datasette inspect [OPTIONS] [FILES]... Try 'datasette inspect --help' for help. Error: Invalid value for '[FILES]...': Path 'extra' does not exist. The command '/bin/sh -c datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json' returned a non-zero code: 2 ERROR ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 2 ``` While working on the demo for #1232, using this deploy command: ``` GITHUB_SHA=crossdb datasette publish cloudrun fixtures.db 'extra database.db' \ -m fixtures.json \ --plugins-dir=plugins \ --branch=$GITHUB_SHA \ --version-note=$GITHUB_SHA \ --extra-options="--setting template_debug 1 --crossdb" \ --install=pysqlite3-binary \ --service=datasette-latest-crossdb ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1233/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
811505638 | MDU6SXNzdWU4MTE1MDU2Mzg= | 1234 | Runtime support for ATTACHing multiple databases | 9599 | open | 0 | 1 | 2021-02-18T22:06:47Z | 2021-02-22T21:06:28Z | OWNER | > The implementation in #1232 is ready to land. It's the simplest-thing-that-could-possibly-work: you can run `datasette one.db two.db three.db --crossdb` and then use the `/_memory` page to run joins across tables from multiple databases. > > It only works on the first 10 databases that were passed to the command-line. This means that if you have a Datasette instance with hundreds of attached databases (see [Datasette Library](https://github.com/simonw/datasette/issues/417)) this won't be particularly useful for you. > > So... a better, future version of this feature would be one that lets you join across databases on command - maybe by hitting `/_memory?attach=db1&attach=db2` to get a special connection. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/283#issuecomment-781665560_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
812704869 | MDU6SXNzdWU4MTI3MDQ4Njk= | 1237 | ?_pretty=1 option for pretty-printing JSON output | 9599 | open | 0 | 3268330 | 1 | 2021-02-20T20:54:40Z | 2021-11-16T18:28:33Z | OWNER | Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |