issues
252 rows where comments = 0 and state = "open" sorted by reactions
This data as json, CSV (advanced)
Suggested facets: milestone, author_association, draft, created_at (date), updated_at (date)
repo 16
- datasette 165
- sqlite-utils 17
- twitter-to-sqlite 13
- github-to-sqlite 10
- dogsheep-beta 9
- google-takeout-to-sqlite 7
- healthkit-to-sqlite 5
- dogsheep-photos 5
- apple-notes-to-sqlite 5
- dogsheep.github.io 4
- evernote-to-sqlite 3
- swarm-to-sqlite 2
- inaturalist-to-sqlite 2
- pocket-to-sqlite 2
- hacker-news-to-sqlite 2
- genome-to-sqlite 1
state 1
- open · 252 ✖
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions ▼ | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1650981564 | I_kwDOJHON9s5iZ_q8 | 12 | Error running pytest | amlestin 14314871 | open | 0 | 0 | 2023-04-02T15:02:36Z | 2023-04-02T15:07:10Z | NONE |
Solution: This is likely a PYTHONPATH issue due to having pytest installed both globally and in the venv. We can guarantee the tests run by adding the current directory to sys.path automatically using
The alternative is to activate the venv, install pytest, deactivate, then activate the venv again (https://stackoverflow.com/questions/35045038/how-do-i-use-pytest-with-virtualenv) |
apple-notes-to-sqlite 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1650984552 | PR_kwDOJHON9s5NbyYN | 13 | use universal command | amlestin 14314871 | open | 0 | 0 | 2023-04-02T15:10:54Z | 2023-04-02T15:37:34Z | FIRST_TIME_CONTRIBUTOR | dogsheep/apple-notes-to-sqlite/pulls/13 | apple-notes-to-sqlite 611552758 | pull | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
1880968405 | PR_kwDOJHON9s5ZhYny | 14 | fix: fix the problem of Chinese character garbling | barretlee 2698003 | open | 0 | 0 | 2023-09-04T23:48:28Z | 2023-09-04T23:48:28Z | FIRST_TIME_CONTRIBUTOR | dogsheep/apple-notes-to-sqlite/pulls/14 |
|
apple-notes-to-sqlite 611552758 | pull | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1616440856 | I_kwDOJHON9s5gWO4Y | 5 | Configure full text search | simonw 9599 | open | 0 | 0 | 2023-03-09T05:20:46Z | 2023-03-09T05:20:46Z | MEMBER | FTS would be useful. Maybe even extract the plain text from the notes to make that index easier to create, rather than creating it against the HTML. Can use the |
apple-notes-to-sqlite 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1617938730 | I_kwDOJHON9s5gb8kq | 9 | Default to just storing plaintext, store HTML if `--html` is passed | simonw 9599 | open | 0 | 0 | 2023-03-09T20:19:06Z | 2023-03-09T20:19:06Z | MEMBER | The full I'm tempted to say you don't get HTML unless you pass a |
apple-notes-to-sqlite 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
692202408 | MDU6SXNzdWU2OTIyMDI0MDg= | 12 | Idea: maps and GeoJSON support | simonw 9599 | open | 0 | 0 | 2020-09-03T18:47:10Z | 2020-09-04T01:45:03Z | MEMBER | It would be cool if the Then I could render workout routes on a map, or Swarm checkin points. |
dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
697162939 | MDU6SXNzdWU2OTcxNjI5Mzk= | 20 | Add more tags so people can find your project. | ran88dom99 7902810 | open | 0 | 0 | 2020-09-09T21:14:09Z | 2020-09-09T21:14:09Z | NONE | quantified-self habit-tracking google-fit time-tracking wearables quantifiedself for example |
dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/20/reactions", "total_count": 1, "+1": 0, "-1": 1, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
709789634 | MDU6SXNzdWU3MDk3ODk2MzQ= | 27 | Sort order is not persisted by facet filter links | simonw 9599 | open | 0 | 0 | 2020-09-27T18:22:07Z | 2020-09-27T18:22:07Z | MEMBER | A link to |
dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
836923194 | MDU6SXNzdWU4MzY5MjMxOTQ= | 32 | JSON API for search results | simonw 9599 | open | 0 | 0 | 2021-03-20T22:21:36Z | 2021-03-20T22:21:36Z | MEMBER | dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
983221851 | MDU6SXNzdWU5ODMyMjE4NTE= | 34 | Data folder as index command parameter | humrochagf 1223625 | open | 0 | 0 | 2021-08-30T21:29:33Z | 2021-08-30T21:29:33Z | NONE | Hi, First of all, thank you for this wonderful project :smile: I started to use dogsheep to make my personal data searchable, and by using the project I noticed an issue with the index command. It always expects you are running it from the root folder from where the data is located, so I got some errors while trying to make it work on my setup. I separate all databases inside a Before, I configured
And running the index command like this:
It worked to the normal search feature with no problem this way, but when I started adding So my workaround to that was to cd into the data folder and run the indexer. You can check the way I'm doing it at this line of the makefile: https://github.com/humrochagf/my-dogsheep/blob/main/makefile#L3 It works but it would be nice to have an option to pass the path where the data is located to the index function. |
dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/34/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
987985935 | MDExOlB1bGxSZXF1ZXN0NzI2OTkwNjgw | 35 | Support for Datasette's --base-url setting | brandonrobertz 2670795 | open | 0 | 0 | 2021-09-03T17:47:45Z | 2021-09-03T17:47:45Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep-beta/pulls/35 | This makes it so you can use Dogsheep if you're using Datasette with the |
dogsheep-beta 197431109 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/35/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1751214236 | I_kwDOC8SPRc5oYWic | 36 | Getting sqlite_master may not be modified when creating dogsheep index | khushmeeet 8711912 | open | 0 | 0 | 2023-06-11T03:21:53Z | 2023-06-11T03:21:53Z | NONE | When creating a
Command I ran to get this error
Dogsheep version
Python version
|
dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/36/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1888477283 | I_kwDOC8SPRc5wj-Bj | 38 | Run `rebuild_fts` after building the index | simonw 9599 | open | 0 | 0 | 2023-09-08T23:17:45Z | 2023-09-08T23:17:45Z | MEMBER | In: - https://github.com/simonw/datasette.io/issues/152#issuecomment-1712323347 This turned out to be the fix:
|
dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/38/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
689850810 | MDU6SXNzdWU2ODk4NTA4MTA= | 6 | Set up a demo instance | simonw 9599 | open | 0 | 0 | 2020-09-01T06:20:24Z | 2020-09-01T06:20:24Z | MEMBER | Once I've got the Datasette plugin to a state where it's worth building a demo: #3 I can use data from my public https://github-to-sqlite.dogsheep.net/ demo plus the Pocket data subset I use for the demo in https://github.com/dogsheep/pocket-to-sqlite/issues/5 - I could pull in the https://dogsheep-photos.dogsheep.net/ photos data too. |
dogsheep-beta 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
621486115 | MDU6SXNzdWU2MjE0ODYxMTU= | 27 | photos_with_apple_metadata view should include labels | simonw 9599 | open | 0 | 0 | 2020-05-20T06:06:17Z | 2020-05-20T06:06:17Z | MEMBER | Here's one way to add that:
|
dogsheep-photos 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
655974395 | MDExOlB1bGxSZXF1ZXN0NDQ4MzU1Njgw | 30 | Handle empty bucket on first upload. Allow specifying the endpoint_url for services other than S3 (like b2 and digitalocean spaces) | scanner 110038 | open | 0 | 0 | 2020-07-13T16:15:26Z | 2020-07-13T16:15:26Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep-photos/pulls/30 | Finally got around to trying dogsheep-photos but I want to use backblaze's b2 service instead of AWS S3. Had to add a way to optionally specify the endpoint_url to connect to. Then with the bucket being empty the initial key retrieval would fail. Probably a better way to see that the bucket is empty than doing a test inside the paginator loop. Also probably a better way to specify the endpoint_url as we get and test for it twice using the same code in two different places but did not want to spend too much time worrying about it. |
dogsheep-photos 256834907 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/30/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
830283447 | MDU6SXNzdWU4MzAyODM0NDc= | 34 | bucket name | dsisnero 6213 | open | 0 | 0 | 2021-03-12T16:40:57Z | 2021-03-12T16:40:57Z | NONE | I followed the instructions to setup credentials but I am getting a invalid bucket name. Can you put a sample auth.json file in the base that shows the correct format for this? Thanks |
dogsheep-photos 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/34/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1293698966 | PR_kwDOD079W84600uh | 37 | Fix former command name in readme | DanLipsitt 578773 | open | 0 | 0 | 2022-07-05T02:09:13Z | 2022-07-05T02:09:13Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep-photos/pulls/37 | Looks like a previous commit missed a |
dogsheep-photos 256834907 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/37/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1827436260 | PR_kwDOD079W85WtVyk | 39 | Missing option in datasette instructions | coldclimate 319473 | open | 0 | 0 | 2023-07-29T10:34:48Z | 2023-07-29T10:34:48Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep-photos/pulls/39 | Gotta tell it where to look |
dogsheep-photos 256834907 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/39/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
541274681 | MDU6SXNzdWU1NDEyNzQ2ODE= | 2 | Add linkedin-to-sqlite | mnp 881925 | open | 0 | 0 | 2019-12-21T03:13:40Z | 2019-12-21T03:13:40Z | NONE | There is an API available. https://developer.linkedin.com/docs/rest-api# At the minimum, I would think contact list and messages would be of interest. |
dogsheep.github.io 214746582 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
723499985 | MDExOlB1bGxSZXF1ZXN0NTA1MDc2NDE4 | 5 | Add fitbit-to-sqlite | mrphil007 4632208 | open | 0 | 0 | 2020-10-16T20:04:05Z | 2020-10-16T20:04:05Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep.github.io/pulls/5 | dogsheep.github.io 214746582 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
925384329 | MDExOlB1bGxSZXF1ZXN0NjczODcyOTc0 | 7 | Add instagram-to-sqlite | gavindsouza 36654812 | open | 0 | 0 | 2021-06-19T12:26:16Z | 2021-07-28T07:58:59Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep.github.io/pulls/7 | The tool covers only chat imports at the time of opening this PR but I'm planning to import everything else that I feel inquisitive about |
dogsheep.github.io 214746582 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
927385540 | MDU6SXNzdWU5MjczODU1NDA= | 8 | any guidance / experience on imessage-to-sqlite ? | Casyfill 2675621 | open | 0 | 0 | 2021-06-22T15:46:16Z | 2021-06-22T15:46:16Z | NONE | dogsheep.github.io 214746582 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
892383270 | MDExOlB1bGxSZXF1ZXN0NjQ1MTAwODQ4 | 12 | Recovering of malformed ENEX file | engdan77 8431437 | open | 0 | 0 | 2021-05-15T07:49:31Z | 2021-05-15T19:57:50Z | FIRST_TIMER | dogsheep/evernote-to-sqlite/pulls/12 | Hey .. Awesome work developing this project, that I found very useful to me and saved me some work.. Thanks.. :) Some background to this PR... I've been searching around for a tool allowing me to transforming my personal collection of Evernote notes to a format easier to search and potentially easier import to future services. Now I discovered problem processing my large data ~5GB using the existing source using Pythons builtin xml-parser that unfortunately was unable to succeed without exception breaking the process. My first attempt I tried to adapt to more robust lxml package allowing huge data and with "recover", but even if it worked better it also failed processing the whole data. Even using the memory efficient etree.iterparse() it also unfortunately got into trouble. And with no luck finding any other libraries successfully parsing this enormous file I instead chose to build a "hugexmlparser" module that allows parsing this huge file using yield (on a byte-to-byte-level) and allows you to set a maximum size for <note> to cater for potential malformed or undesirable large attachments to export, should succeed covering potential exceptions. Some cases found where the parses discover malformed XML within <content> so also in those cases try to save as much as possible by escaping (to be dealt at a later stage, better than nothing), and if a missing end </note> before new (malformed?) it would add this after encounter a new start-tag. The code for the recovery process is a bit rough and for certain room for refactoring, but at the moment is seem to achieve what I wanted. Now with the above we pass this a minor changed version of save_note_recovery() assure the existing works. Also adding this as a new recover-enex command to click and kept the original options. A couple of new tests was added as well to check against using this command. Now this currently works to me, but thought I might share a PR in such as you find use for this yourself or found useful to others finding this repository. As a second step .. When the time allows it would have been nice to also be able to easily export from SQLite to formatted HTML/MD and attachments saved... but that might perhaps be better a separate project ... or if you or someone else have something that might shared to save some trouble, I would be interested ;-) |
evernote-to-sqlite 303218369 | pull | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1042759769 | PR_kwDOEhK-wc4uAJb9 | 15 | include note tags in the export | d-rep 436138 | open | 0 | 0 | 2021-11-02T20:04:31Z | 2021-11-02T20:04:31Z | FIRST_TIME_CONTRIBUTOR | dogsheep/evernote-to-sqlite/pulls/15 | When parsing the Evernote Here is an example of how to query the data after the script has run:
My .enex source file is 3+ years old so I am assuming the structure hasn't changed. Interestingly, my notebook names show up in the tags list where the tag name is prefixed with |
evernote-to-sqlite 303218369 | pull | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1943259395 | I_kwDOEhK-wc5z08kD | 16 | time data '2014-11-21T11:44:12.000Z' does not match format '%Y%m%dT%H%M%SZ' | linonetwo 3746270 | open | 0 | 0 | 2023-10-14T13:24:39Z | 2023-10-14T13:24:39Z | NONE |
enex is exported by evernote mac client |
evernote-to-sqlite 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
664793260 | MDU6SXNzdWU2NjQ3OTMyNjA= | 2 | Yak shave | ekg 145425 | open | 0 | 0 | 2020-07-23T22:04:18Z | 2020-07-23T22:04:18Z | NONE | Just a quick note... The 23andme data is not exactly your genome, but a SNP chip of your genome. It's "some of your genotypes." Or about 0.1% of your genome. Nice work in any case! It deserves to be liberated!!!!! |
genome-to-sqlite 209590345 | issue | { "url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
599776345 | MDU6SXNzdWU1OTk3NzYzNDU= | 24 | Feature idea: github-to-sqlite everything ... | simonw 9599 | open | 0 | 0 | 2020-04-14T18:34:00Z | 2020-04-14T18:34:00Z | MEMBER | At the moment if you want to pull all your repos, issues, issues comments etc you have to do it with a sequence of separate commands. Consider adding a |
github-to-sqlite 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/24/reactions", "total_count": 7, "+1": 7, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
703216044 | MDU6SXNzdWU3MDMyMTYwNDQ= | 49 | Feature: gists and starred gists | simonw 9599 | open | 0 | 0 | 2020-09-17T02:30:52Z | 2020-09-17T02:30:52Z | MEMBER | github-to-sqlite 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/49/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
797784080 | MDU6SXNzdWU3OTc3ODQwODA= | 62 | Stargazers and workflows commands always require an auth file when using GITHUB_TOKEN | frosencrantz 631242 | open | 0 | 0 | 2021-01-31T18:56:05Z | 2021-01-31T18:56:05Z | CONTRIBUTOR | Requested fix in https://github.com/dogsheep/github-to-sqlite/pull/59 The stargazers and workflows commands always require an auth file, even when using a |
github-to-sqlite 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/62/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
897212458 | MDU6SXNzdWU4OTcyMTI0NTg= | 63 | Ability to fetch commits from branches other than the default | simonw 9599 | open | 0 | 0 | 2021-05-20T17:58:08Z | 2021-05-20T17:58:08Z | MEMBER | This tool is currently almost entirely ignorant of the concept of branches. One example: you can't retrieve commits from any branch other than the default (usually main). |
github-to-sqlite 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/63/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
981690086 | MDExOlB1bGxSZXF1ZXN0NzIxNjg2NzIx | 67 | Replacing step ID key with step_id | jshcmpbll 16374374 | open | 0 | 0 | 2021-08-28T01:26:41Z | 2021-08-28T01:27:00Z | FIRST_TIME_CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/67 | Workflows that have an e.g.
ChangesI'm proposing that the key for Special thanks to @sarcasticadmin @egiffen and @ruebenramirez for helping a bit on this 😄 |
github-to-sqlite 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/67/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1013506559 | PR_kwDODFdgUs4skaNS | 68 | Add support for retrieving teams / members | philwills 68329 | open | 0 | 0 | 2021-10-01T15:55:02Z | 2021-10-01T15:59:53Z | FIRST_TIME_CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/68 | Adds a method for retrieving all the teams within an organisation and all the members in those teams. The latter is stored as a join table |
github-to-sqlite 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/68/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1149402080 | PR_kwDODFdgUs4zaUta | 70 | scrape-dependents: enable paging through package menu option if present | stanbiryukov 36061055 | open | 0 | 0 | 2022-02-24T15:07:25Z | 2022-02-24T15:07:25Z | FIRST_TIME_CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/70 | Some repos organize network dependents by a Package toggle. This PR adds the ability to page through those options and scrape underlying dependents. |
github-to-sqlite 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/70/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1363244199 | I_kwDODFdgUs5RQXSn | 75 | Fetch repos doesn't support organisations | OverkillGuy 2757699 | open | 0 | 0 | 2022-09-06T12:55:06Z | 2022-09-06T12:55:06Z | NONE | Say I want to get all my Github Org's repos info, for data analysis. Not just the public repos, but also the private/internal repos. The endpoints are different for organisation, and this tool doesn't take it into account: https://github.com/dogsheep/github-to-sqlite/blob/ace13ec3d98090d99bd71871c286a4a612c96a50/github_to_sqlite/utils.py#L453 https://github.com/dogsheep/github-to-sqlite/blob/ace13ec3d98090d99bd71871c286a4a612c96a50/github_to_sqlite/utils.py#L455 The endpoints for organisation repos is instead (source):
Let's add support for organisations repo scraping. |
github-to-sqlite 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/75/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1410548368 | I_kwDODFdgUs5UE0KQ | 77 | Feature: Support GitHub discussions | frosencrantz 631242 | open | 0 | 0 | 2022-10-16T16:53:38Z | 2022-10-16T16:53:38Z | CONTRIBUTOR | Hi @simonw I've been a happy user of this tool. Thank you for writing it and sharing it. I wanted to suggest a feature request to support Discussions. For example the VisiData project has discussions https://github.com/saulpw/visidata/discussions , and it would be useful if there was a way to pull that data into the database. However, I'm not offering a pull request. |
github-to-sqlite 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/77/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1505411725 | I_kwDODFdgUs5ZusKN | 78 | self-hosted or corp github enterprise | ebdavison 549431 | open | 0 | 0 | 2022-12-20T22:51:45Z | 2022-12-20T22:51:45Z | NONE | We use github enterprise at work and I would like to use this tool to pull info from that site rather than the public github.com instance. Is there an option for this? If not, can one be added for a custom repo URL? |
github-to-sqlite 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/78/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
504720731 | MDU6SXNzdWU1MDQ3MjA3MzE= | 1 | Add more details on how to request data from google takeout correctly. | dazzag24 1055831 | open | 0 | 0 | 2019-10-09T15:17:34Z | 2019-10-09T15:17:34Z | NONE | The default is to download everything. This can result in an enormous amount of data when you only really need 2 types of data for now:
In addition unless you specify that "My Activity" is downloaded in JSON format the default is HTML. This then causes the
command to fail as it only contains html files not json files. Thanks |
google-takeout-to-sqlite 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1250287607 | PR_kwDODFE5qs44jvRV | 11 | Update README.md | ashanan 11887 | open | 0 | 0 | 2022-05-27T03:13:59Z | 2022-05-27T03:13:59Z | FIRST_TIME_CONTRIBUTOR | dogsheep/google-takeout-to-sqlite/pulls/11 | Fix typo |
google-takeout-to-sqlite 206649770 | pull | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/11/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1557599877 | I_kwDODFE5qs5c1xaF | 12 | location history changes | gerardrbentley 14809320 | open | 0 | 0 | 2023-01-26T03:57:25Z | 2023-01-26T03:57:25Z | NONE | not sure if each download is unique, but I had to change some things to work with the takeout zip I made 2023-01-25 filename changed from "Location History.json" to "Records.json"
```py def get_timestamp_ms(raw_timestamp): try: return datetime.datetime.strptime(raw_timestamp, "%Y-%m-%dT%H:%M:%SZ").timestamp() except ValueError: return datetime.datetime.strptime(raw_timestamp, "%Y-%m-%dT%H:%M:%S.%fZ").timestamp() def save_location_history(db, zf): location_history = json.load( zf.open("Takeout/Location History/Records.json") ) db["location_history"].upsert_all( ( { "id": id_for_location_history(row), "latitude": row["latitudeE7"] / 1e7, "longitude": row["longitudeE7"] / 1e7, "accuracy": row["accuracy"], "timestampMs": get_timestamp_ms(row["timestamp"]), "when": row["timestamp"], } for row in location_history["locations"] ), pk="id", ) def id_for_location_history(row): # We want an ID that is unique but can be sorted by in # date order - so we use the isoformat date + the first # 6 characters of a hash of the JSON first_six = hashlib.sha1( json.dumps(row, separators=(",", ":"), sort_keys=True).encode("utf8") ).hexdigest()[:6] return "{}-{}".format( row['timestamp'], first_six, ) ``` example locations from mine
|
google-takeout-to-sqlite 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/12/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 2 } |
||||||||
1884499674 | PR_kwDODFE5qs5ZtYMc | 13 | use poetry for packages, asdf for versioning, and gh actions for ci | iloveitaly 150855 | open | 0 | 0 | 2023-09-06T17:59:16Z | 2023-09-06T17:59:16Z | FIRST_TIME_CONTRIBUTOR | dogsheep/google-takeout-to-sqlite/pulls/13 |
|
google-takeout-to-sqlite 206649770 | pull | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
769397742 | MDU6SXNzdWU3NjkzOTc3NDI= | 3 | sqlite-utils error on takeout import | khimaros 231498 | open | 0 | 0 | 2020-12-17T01:18:48Z | 2020-12-17T01:19:04Z | NONE |
there is no table create in additionally, this package and hackernews-to-sqlite have conflicting |
google-takeout-to-sqlite 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/3/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
930946817 | MDU6SXNzdWU5MzA5NDY4MTc= | 7 | KeyError: 'accuracy' when processing Location History | davidwilemski 403152 | open | 0 | 0 | 2021-06-27T14:39:43Z | 2021-06-27T14:39:43Z | NONE | I'm new to both the dogsheep tools and datasette but have been experimenting a bit the last few days and these are really cool tools! I encountered a problem running my Google location history through this tool running the latest release in a docker container:
It looks like the tool assumes the My first attempt at a local patch to get myself going was to convert accessing the Now that I've done a successful import run using my initial fix of calling I'm happy to provide a PR for a fix but figured I'd ask about which direction is preferred first. |
google-takeout-to-sqlite 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1046887492 | PR_kwDODFE5qs4uMsMJ | 9 | Removed space from filename My Activity.json | widadmogral 91880982 | open | 0 | 0 | 2021-11-08T00:04:31Z | 2021-11-08T00:04:31Z | FIRST_TIME_CONTRIBUTOR | dogsheep/google-takeout-to-sqlite/pulls/9 | File name from google takeout has no space. The code only runs without error if filename is "MyActivity.json" and not "My Activity.json". Is it a new change by Google? |
google-takeout-to-sqlite 206649770 | pull | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1205867842 | I_kwDODtX3eM5H4BVC | 4 | Retrieve the top-level story for a comment | telotortium 1755789 | open | 0 | 0 | 2022-04-15T20:25:39Z | 2022-04-15T20:25:39Z | NONE | I think that each comment inserted into the database should include a column |
hacker-news-to-sqlite 248903544 | issue | { "url": "https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1353418822 | PR_kwDODtX3eM497MOV | 5 | The program fails when the user has no submissions | fernand0 2467 | open | 0 | 0 | 2022-08-28T17:25:45Z | 2022-08-28T17:25:45Z | FIRST_TIME_CONTRIBUTOR | dogsheep/hacker-news-to-sqlite/pulls/5 | Tested with:
Result:
There is a problem of style with the patch (but not sure what to do) because with the new inicialization ( submitted = []) the part
is not needed. Maybe there is a more adequate way of doing this. |
hacker-news-to-sqlite 248903544 | pull | { "url": "https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
793907673 | MDExOlB1bGxSZXF1ZXN0NTYxNTEyNTAz | 15 | added try / except to write_records | ryancheley 9857779 | open | 0 | 0 | 2021-01-26T03:56:21Z | 2021-01-26T03:56:21Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/15 | to keep the data write from failing if it came across an error during processing. In particular when trying to convert my HealthKit zip file (and that of my wife's) it would consistently error out with the following: ``` db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: too many SQL variables db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: too many SQL variables db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: table rBodyMass has no column named metadata_HKWasUserEntered healthkit-to-sqlite 8 <module> sys.exit(cli()) core.py 829 call return self.main(args, *kwargs) core.py 782 main rv = self.invoke(ctx) core.py 1066 invoke return ctx.invoke(self.callback, **ctx.params) core.py 610 invoke return callback(args, *kwargs) cli.py 57 cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) utils.py 42 convert_xml_to_sqlite write_records(records, db) utils.py 143 write_records db[table].insert_all( db.py 1899 insert_all self.insert_chunk( db.py 1720 insert_chunk self.insert_chunk( db.py 1720 insert_chunk self.insert_chunk( db.py 1714 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: table rBodyMass has no column named metadata_HKWasUserEntered ``` Adding the try / except in the |
healthkit-to-sqlite 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
830901133 | MDExOlB1bGxSZXF1ZXN0NTkyMzY0MjU1 | 16 | Add a fallback ID, print if no ID found | n8henrie 1234956 | open | 0 | 0 | 2021-03-13T13:38:29Z | 2021-03-13T14:44:04Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/16 | healthkit-to-sqlite 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
836063389 | MDU6SXNzdWU4MzYwNjMzODk= | 17 | Datetime columns are not properly formatted to be recognizes as datetime | n8henrie 1234956 | open | 0 | 0 | 2021-03-19T14:33:04Z | 2021-03-19T14:33:04Z | NONE | Currently, the datetimes are formatted in a way that is not recognized by datasette-vega for plotting with a For example, if you have datasette running locally with
The plot is blank unless you choose The If instead the format for this column is changed slightly: I have a PR that addresses this issue, will submit shortly. |
healthkit-to-sqlite 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
836064851 | MDExOlB1bGxSZXF1ZXN0NTk2NjI3Nzgw | 18 | Add datetime parsing | n8henrie 1234956 | open | 0 | 0 | 2021-03-19T14:34:22Z | 2021-03-19T14:34:22Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/18 | Parses the datetime columns so they are subsequently properly recognized as datetime. Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/17 |
healthkit-to-sqlite 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1515717718 | PR_kwDOC8tyDs5Gc-VH | 23 | Include workout statistics | badboy 2129 | open | 0 | 0 | 2023-01-01T17:29:57Z | 2023-01-01T17:29:57Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/23 | Not sure when this changed (iOS 16 maybe?), but the Adding it as another column at leat allows me to pull these out (using SQLite's JSON support). I'm running with this patch on my own data now. |
healthkit-to-sqlite 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/23/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1353411865 | I_kwDODEpn8M5Qq20Z | 1 | Problem with my user | fernand0 2467 | open | 0 | 0 | 2022-08-28T16:59:37Z | 2022-08-28T16:59:37Z | NONE | If I call the program with:
inaturalist-to-sqlite inaturalist.db ftricas
the program exits with an error:
Additional info, the command:
shows that the correct name can be 'taxons'. There is another small problem with a warning: warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported " |
inaturalist-to-sqlite 206202864 | issue | { "url": "https://api.github.com/repos/dogsheep/inaturalist-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1485017981 | I_kwDODEpn8M5Yg5N9 | 2 | table identifications has no column named previous_observation_taxon | heaversm 520541 | open | 0 | 0 | 2022-12-08T16:47:17Z | 2022-12-08T16:47:17Z | NONE | Installed successfully with pip and ran
|
inaturalist-to-sqlite 206202864 | issue | { "url": "https://api.github.com/repos/dogsheep/inaturalist-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
689848827 | MDU6SXNzdWU2ODk4NDg4Mjc= | 6 | ISO timestamps | simonw 9599 | open | 0 | 0 | 2020-09-01T06:16:42Z | 2020-09-01T06:16:42Z | MEMBER | The
Should use ISO instead, e.g. |
pocket-to-sqlite 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
797728929 | MDU6SXNzdWU3OTc3Mjg5Mjk= | 8 | QUESTION: extract full text | darribas 417363 | open | 0 | 0 | 2021-01-31T14:50:10Z | 2021-01-31T14:50:10Z | NONE | This may be solved or a feature already, but I couldn't figure it out, is it possible to extract and store also full text from the saved pages? The same way that Pocket parses the text, it'd be amazing to be able to store (and thus make searchable later) the text. Thank you very much for the project, it's such an amazing idea! |
pocket-to-sqlite 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1393330070 | PR_kwDODD6af84__DNJ | 14 | Photo links | redmanmale 6782721 | open | 0 | 0 | 2022-10-01T09:44:15Z | 2022-11-18T17:10:49Z | FIRST_TIME_CONTRIBUTOR | dogsheep/swarm-to-sqlite/pulls/14 |
Fixes #9. |
swarm-to-sqlite 205429375 | pull | { "url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
673602857 | MDU6SXNzdWU2NzM2MDI4NTc= | 9 | Define a view that displays photos correctly | simonw 9599 | open | 0 | 0 | 2020-08-05T14:53:39Z | 2020-08-05T14:53:39Z | MEMBER | The id | createdAt | source | prefix | suffix | width | height | visibility | created ▲ | user -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 5e12c9708506bc000840262a | January 06, 2020 - 05:45:20 UTC | Swarm for iOS 1 | https://fastly.4sqi.net/img/general/ | /15889193_AXxGk4I1nbzUZuyYqObgbXdJNyEHiwj6AUDq0tPZWtw.jpg | 1920 | 1440 | public | 2020-01-06T05:45:20 | 15889193 The photo URL can be derived from those pieces - define a SQL view which does that (using |
swarm-to-sqlite 205429375 | issue | { "url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
505673645 | MDU6SXNzdWU1MDU2NzM2NDU= | 16 | Do a better job with archived direct message threads | simonw 9599 | open | 0 | 0 | 2019-10-11T06:55:21Z | 2019-10-11T06:55:27Z | MEMBER | twitter-to-sqlite 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
703218448 | MDU6SXNzdWU3MDMyMTg0NDg= | 51 | Documentation for twitter-to-sqlite fetch | simonw 9599 | open | 0 | 0 | 2020-09-17T02:38:10Z | 2020-09-17T02:38:10Z | MEMBER | It's mentioned in passing in the README but it deserves its own section:
|
twitter-to-sqlite 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/51/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1091850530 | I_kwDODEm0Qs5BFFEi | 63 | Import archive error 'withheld_in_countries' | pauloxnet 521097 | open | 0 | 0 | 2022-01-01T16:58:59Z | 2022-01-01T16:58:59Z | NONE | Importing the twitter archive I received this error:
I found only a single tweet with the key I solved the error removing the key from the |
twitter-to-sqlite 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/63/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1097332098 | I_kwDODEm0Qs5BZ_WC | 64 | Include all entities for tweets | max 111631 | open | 0 | 0 | 2022-01-09T23:35:28Z | 2022-01-09T23:35:28Z | NONE | Per our conversation on Twitter: It would be neat if all entities (including URLs) were captured. This way you can ensure, that URLs are parsed out exactly the same way Twitter parses URLs – we all know parsing URLs with a regex ain't fun. Right now, I believe the tool filters out all entities that are not of type |
twitter-to-sqlite 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/64/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1160327106 | PR_kwDODEm0Qs4z_V3w | 65 | Update Twitter dev link, clarify apps vs projects | rixx 2657547 | open | 0 | 0 | 2022-03-05T11:56:08Z | 2022-03-05T11:56:08Z | FIRST_TIME_CONTRIBUTOR | dogsheep/twitter-to-sqlite/pulls/65 | Twitter pushes you heavily towards v2 projects instead of v1 apps – I know the README mentions v1 API compatibility at the top, but I still nearly got turned around here. |
twitter-to-sqlite 206156866 | pull | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/65/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1244082183 | PR_kwDODEm0Qs44PPLy | 66 | Ageinfo workaround | ashanan 11887 | open | 0 | 0 | 2022-05-21T21:08:29Z | 2022-05-21T21:09:16Z | FIRST_TIME_CONTRIBUTOR | dogsheep/twitter-to-sqlite/pulls/66 | I'm not sure if this is due to a new format or just because my ageinfo file is blank, but trying to import an archive would crash when it got to that file. This PR adds a guard clause in the Let me know if you want any changes! |
twitter-to-sqlite 206156866 | pull | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/66/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1513237712 | PR_kwDODEm0Qs5GUoG_ | 67 | Add support for app-only bearer tokens | sometimes-i-send-pull-requests 26161409 | open | 0 | 0 | 2022-12-28T23:31:20Z | 2022-12-28T23:31:20Z | FIRST_TIME_CONTRIBUTOR | dogsheep/twitter-to-sqlite/pulls/67 | Previously, twitter-to-sqlite only supported OAuth1 authentication, and the token must be on behalf of a user. However, Twitter also supports application-only bearer tokens, documented here: https://developer.twitter.com/en/docs/authentication/oauth-2-0/bearer-tokens This PR adds support to twitter-to-sqlite for using application-only bearer tokens. To use, the auth.json file just needs to contain a "bearer_token" key instead of "api_key", "api_secret_key", etc. |
twitter-to-sqlite 206156866 | pull | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/67/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1513237982 | PR_kwDODEm0Qs5GUoKL | 68 | Archive: Import mute table | sometimes-i-send-pull-requests 26161409 | open | 0 | 0 | 2022-12-28T23:32:06Z | 2022-12-28T23:32:06Z | FIRST_TIME_CONTRIBUTOR | dogsheep/twitter-to-sqlite/pulls/68 | twitter-to-sqlite 206156866 | pull | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/68/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
1513238152 | PR_kwDODEm0Qs5GUoMM | 69 | Archive: Import new tweets table name | sometimes-i-send-pull-requests 26161409 | open | 0 | 0 | 2022-12-28T23:32:44Z | 2022-12-28T23:32:44Z | FIRST_TIME_CONTRIBUTOR | dogsheep/twitter-to-sqlite/pulls/69 | Given the code here, it seems like in the past this file was named "tweet.js". In recent exports, it's named "tweets.js". The archive importer needs to be modified to take this into account. Existing logic is reused for importing this table. (However, the resulting table name will be different, matching the different file name -- archive_tweets, rather than archive_tweet). |
twitter-to-sqlite 206156866 | pull | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/69/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1513238314 | PR_kwDODEm0Qs5GUoN6 | 70 | Archive: Import Twitter Circle data | sometimes-i-send-pull-requests 26161409 | open | 0 | 0 | 2022-12-28T23:33:09Z | 2022-12-28T23:33:09Z | FIRST_TIME_CONTRIBUTOR | dogsheep/twitter-to-sqlite/pulls/70 | twitter-to-sqlite 206156866 | pull | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/70/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
1513238455 | PR_kwDODEm0Qs5GUoPm | 71 | Archive: Fix "ni devices" typo in importer | sometimes-i-send-pull-requests 26161409 | open | 0 | 0 | 2022-12-28T23:33:31Z | 2022-12-28T23:33:31Z | FIRST_TIME_CONTRIBUTOR | dogsheep/twitter-to-sqlite/pulls/71 | twitter-to-sqlite 206156866 | pull | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/71/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
1524431805 | I_kwDODEm0Qs5a3Pu9 | 72 | Import thread, including self- and others' replies | mcint 601708 | open | 0 | 0 | 2023-01-08T09:51:06Z | 2023-01-08T09:51:06Z | NONE | statuses-lookup, home-timeline, mentions (only for auth'ed user) don't cover this.
twitter-to-sqlite focuses on archiving users, but does not easily support archiving conversations or community activity. For reference, this is implemented in twarc, using a search, optionally recursively. Other research suggests that this formerly, or currently, requires a search query, use of undocumented |
twitter-to-sqlite 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/72/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1816830546 | I_kwDODEm0Qs5sSqJS | 73 | Twitter v1 API shutdown | david-perez 6341745 | open | 0 | 0 | 2023-07-22T16:57:41Z | 2023-07-22T16:57:41Z | NONE | I've been using this project reliably over the past two years to periodically download my liked tweets, but unfortunately since 19th July I get:
It appears like Twitter has now shut down their v1 endpoints, which is rather gracious of them, considering they announced they'd be deprecated on 29th April. Unfortunately retrieving likes using the v2 API is not part of their free plan. In fact, with the free plan one can only post and delete tweets and retrieve information about oneself. So I'm afraid this is the end of this very nice project. It was very useful, thank you! |
twitter-to-sqlite 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/73/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
||||||||
730210880 | MDU6SXNzdWU3MzAyMTA4ODA= | 1055 | query.html and table.html should share the same table implementation | simonw 9599 | open | 0 | Datasette 1.0 3268330 | 0 | 2020-10-27T07:58:21Z | 2020-10-27T07:58:29Z | OWNER | In #998 I made a change that affected the table page but didn't affect the query page because I incorrectly assumed they shared rendering logic. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1055/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
741231849 | MDU6SXNzdWU3NDEyMzE4NDk= | 1087 | Idea: ?_extra=urls for getting back URLs to useful things | simonw 9599 | open | 0 | 0 | 2020-11-12T02:55:41Z | 2021-12-15T18:06:16Z | OWNER | Working on https://github.com/simonw/datasette-search-all/issues/10 made me realize that sometimes it can be difficult to calculate the URL for a database, table or row within Datasette. It would be useful to have an optional extra JSON extension (using |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1087/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
750089847 | MDU6SXNzdWU3NTAwODk4NDc= | 1109 | Deprecate --config in Datasette 1.0 (in favour of --setting) | simonw 9599 | open | 0 | Datasette 1.0 3268330 | 0 | 2020-11-24T21:43:57Z | 2020-12-17T22:07:49Z | OWNER | I added a deprecation warning to this in #992. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1109/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
756875827 | MDU6SXNzdWU3NTY4NzU4Mjc= | 1129 | Fix footer to the bottom of the page | abdusco 3243482 | open | 0 | 0 | 2020-12-04T07:28:07Z | 2020-12-04T16:04:29Z | CONTRIBUTOR | Footer doesn't stick to the bottom if the body content isn't long enough to reach the end of viewport. This can be fixed using flexbox. ```css body { min-height: 100vh; display: flex; flex-direction: column; } .content { flex-grow: 1; } ``` |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1129/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776128269 | MDU6SXNzdWU3NzYxMjgyNjk= | 1162 | First working version of "datasette insert data.db file.csv" | simonw 9599 | open | 0 | 0 | 2020-12-29T23:20:11Z | 2021-06-17T18:12:32Z | OWNER | Refs #1160 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1162/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778530523 | MDU6SXNzdWU3Nzg1MzA1MjM= | 1172 | /-/static should be excluded from auth and permission checks | simonw 9599 | open | 0 | 0 | 2021-01-05T02:53:41Z | 2021-01-05T02:53:41Z | OWNER | I want to set far future / immutable cache headers on everything served from This has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1172/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
787104850 | MDU6SXNzdWU3ODcxMDQ4NTA= | 1192 | Form Plugin for in-depth Datasette Querying | tomershvueli 1024355 | open | 0 | 0 | 2021-01-15T18:24:50Z | 2021-01-15T18:24:50Z | NONE | I envision a sort of easy-to-build form plugin that would be able to map a user's inputs to different fields/columns in a Datasette database. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1192/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
808771690 | MDU6SXNzdWU4MDg3NzE2OTA= | 1225 | More flexible formatting of records with CSS grid | mhalle 649467 | open | 0 | 0 | 2021-02-15T19:28:17Z | 2021-02-15T19:28:35Z | NONE | In several applications I've been experimenting with alternate formatting of datasette query results. Lately I've found that CSS grids work very well and seem quite general for formatting rows. In CSS I use grid templates to define the layout of each record and the regions for each field, hiding the fields I don't want. It's pretty flexible and looks good. It's also a great basis for highly responsive layout. I initially thought I'd only use this feature for record detail views, but now I use it for index views as well. However, there are some limitations:
* With the existing table templates, it seems that you can change the It would be helpful to at least have an official example or test that used a grid layout for records to make sure nothing in datasette breaks with it. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1225/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
824750134 | MDU6SXNzdWU4MjQ3NTAxMzQ= | 1251 | facet option not appearing when table is big | verajosemanuel 15836677 | open | 0 | 0 | 2021-03-08T16:54:04Z | 2021-03-08T16:54:16Z | NONE | I have a big table with more than 500.000 rows. Trying to facet by one of my columns, the options are not available as for the other smaller tables. I have tried to set it in URL as:
to no avail. is there any limit? how can I force the option "facet" to appear for big tables? |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1251/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
839367451 | MDU6SXNzdWU4MzkzNjc0NTE= | 1275 | Idea: long-running query mode | simonw 9599 | open | 0 | 0 | 2021-03-24T05:23:20Z | 2021-03-24T05:23:20Z | OWNER | It would be cool if you could run Datasette in a long-running query mode, for use with trusted users - something like this:
This would disable the query limit, but would also enable a feature where if a query takes longer than e.g. 1s to return Datasette returns an HTML page to the browser with a progress indicator and polls the server until the query is complete.... but also provides the user with a "cancel" button. This relates to the |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1275/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
275159710 | MDU6SXNzdWUyNzUxNTk3MTA= | 128 | Every visualization should have an "embed" button | simonw 9599 | open | 0 | 0 | 2017-11-19T13:38:13Z | 2019-05-13T18:33:51Z | OWNER | At least for the first round of visualizations, any time you construct one using the UI the result should include an "embed this" button that returns source code to copy and paste These examples should use unpkg.com (or similarl) urls with SRI hashes, eg https://www.srihash.org - and should load data from the datasette JSON API. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/128/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
843884745 | MDU6SXNzdWU4NDM4ODQ3NDU= | 1283 | advanced #export causes unexpected scrolling | mroswell 192568 | open | 0 | 0 | 2021-03-29T22:46:57Z | 2021-03-29T22:46:57Z | CONTRIBUTOR |
The user remedy seems to be to manually remove the "#export" from the URL. This behavior happens in my project, and in: https://covid-19.datasettes.com/covid/economist_excess_deaths (for instance) but not in this table: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1283/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
847700726 | MDU6SXNzdWU4NDc3MDA3MjY= | 1285 | Feature Request or Plugin Request: Numeric Range Facets | mroswell 192568 | open | 0 | 0 | 2021-04-01T01:50:20Z | 2021-04-01T02:28:19Z | CONTRIBUTOR | It would be great to offer facets for numeric data ranges. The ranges could pull from typical GIS methods of creating choropleth maps. https://gisgeography.com/choropleth-maps-data-classification/ Of the following, for mapping, I've always preferred a Jenks Natural Breaks, or a cross between Jenks and Pretty breaks.
Here are some links for Natural Breaks, in case this method is unfamiliar.
Per that last link, there is a Jenks Python module... They also describe it as data-intensive for larger datasets. Maybe this is a good plugin idea. An example of equal Intervals would be 0 – < 10 10 – < 20 20 – < 30 30 – < 40 It's kind of confusing to have that less-than sign in there. it could also be displayed as: 0 – 10 10 – 20 20 – 30 30 – 40 But then it's not completely clear which category 10 is in, for instance. (Best to right-justify.. and use an "en dash" between numbers.) |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1285/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
849512840 | MDU6SXNzdWU4NDk1MTI4NDA= | 1288 | Facets: show counts for null | jungle-boogie 1111743 | open | 0 | 0 | 2021-04-02T22:33:44Z | 2021-04-02T22:33:44Z | NONE | Hi, Thank you for Datasette and being a fan of SQLite! Not all rows in a record will always contain data. So when using a facet on a column where some records have data and others don't, you don't get an accurate count of the results. Please consider also counting and showing null records with facets. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1288/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
853672224 | MDU6SXNzdWU4NTM2NzIyMjQ= | 1294 | "You can check out any time you like. But you can never leave!" | mroswell 192568 | open | 0 | 0 | 2021-04-08T17:02:15Z | 2021-04-08T18:35:50Z | CONTRIBUTOR | (Feel free to rename this one.)
UPDATE: - It would be helpful to have a "Previous page" available for all but the first table page. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
855451460 | MDU6SXNzdWU4NTU0NTE0NjA= | 1297 | Documentation: json1, and introspection endpoints | mroswell 192568 | open | 0 | 0 | 2021-04-12T00:38:00Z | 2021-04-12T01:29:33Z | CONTRIBUTOR | https://docs.datasette.io/en/stable/facets.html notes that:
When I check -/versions I see two sections relevant to json1:
The ENABLE_JSON1 makes me think json1 is likely available. But the |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1297/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
856895291 | MDU6SXNzdWU4NTY4OTUyOTE= | 1299 | Design better empty states | simonw 9599 | open | 0 | 0 | 2021-04-13T12:06:12Z | 2021-04-13T12:06:12Z | OWNER | Inspiration here: https://emptystat.es/ |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1299/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
860734722 | MDU6SXNzdWU4NjA3MzQ3MjI= | 1302 | Fix disappearing facets | mroswell 192568 | open | 0 | 0 | 2021-04-18T18:42:33Z | 2021-04-20T07:40:15Z | CONTRIBUTOR |
Since my site is devoted to whether disinfectants are Safer or Toxic, having the suggested facet disappear from the suggested facet list is very confusing* to end-users. This, along with a few other issues, unfortunately proved beyond my own programming ability to address. So I hired a Senior-level developer to address a number of issues, including this disappearing act.
I'm not sure how to do a pull request for this, because the plugin contains other functionality that goes beyond this bug. I wanted the facets sorted in a certain order (both in the suggested facet list, and the detail lists) (... the detail lists were hopping around all over the place before...) I wanted the duplicate facets removed (leaving only the one where you can facet by individual item in an array.) I wanted the arrays to be presented in a prettier fashion (I did that in the template... That could be moved over to the plugin at some point) I'm thinking it'll be very helpful if applicable parts of my project's plugin (sort_suggested_facets_plugin.py) will be able to be incorporated back into datasette, but I leave that to you to consider. (* The disappearing facet bug was especially confusing because I'm removing the filters and sql from the table page, at the request of the organization. The filters and sql detail created a lot of confusion for end users who try to find disinfectants used by Hospitals, for instance, as an '=' won't find them, since they are part of the Use_site array.) My disappearing-facet confusion was documented in my own issue: https://github.com/mroswell/list-N/issues/57 (addressed by the plugin). Other facet-related issues here: https://github.com/mroswell/list-N/issues/54 (addressed by the plugin); https://github.com/mroswell/list-N/issues/15 (addressed by template); https://github.com/mroswell/list-N/issues/53 (not yet addressed). |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1302/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
870946764 | MDU6SXNzdWU4NzA5NDY3NjQ= | 1312 | how to query many-to-many relationship via json API? | bram2000 5268174 | open | 0 | 0 | 2021-04-29T12:09:49Z | 2021-04-29T12:09:49Z | NONE | Hi, Firstly thanks for Datasette, it's great! I'm trying to use the JSON API to query data from a Datasette instance. I have a simple 3 table many-to-many relationship, like so:
the Now I want to return "all documents within category X" but I cannot see a way to do this without executing two queries; the first to lookup the row_id of category X, and the second to join I could easily write this in SQL, but this makes programmatic handling of pagination much more difficult (we'd have to dynamically modify the SQL to select the row_id and include the correct where and limit clauses). Is there a way to achieve this using the JSON API? |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
871304967 | MDU6SXNzdWU4NzEzMDQ5Njc= | 1315 | settings.json should be picked up by "datasette publish cloudrun" | simonw 9599 | open | 0 | 0 | 2021-04-29T18:16:41Z | 2021-04-29T18:16:41Z | OWNER | datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
915488244 | MDU6SXNzdWU5MTU0ODgyNDQ= | 1372 | Add section to "writing plugins" about security, e.g. avoiding XSS | simonw 9599 | open | 0 | 0 | 2021-06-08T20:49:33Z | 2021-06-08T20:49:46Z | OWNER | https://docs.datasette.io/en/stable/writing_plugins.html should have tips on writing secure plugins. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1372/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
924203783 | MDU6SXNzdWU5MjQyMDM3ODM= | 1379 | Idea: ?_end=1 option for streaming CSV responses | simonw 9599 | open | 0 | 0 | 2021-06-17T18:11:21Z | 2021-06-17T18:11:30Z | OWNER | As discussed in this thread: https://twitter.com/simonw/status/1405554676993433605 - one of the disadvantages of Datasette's streaming CSV feature is that it's hard to tell if you got the whole file or if the connection ended early - or if an error occurred. Idea: offer an optional
For however many columns the CSV file usually has. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1379/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
925491857 | MDU6SXNzdWU5MjU0OTE4NTc= | 1383 | Improve test coverage for `inspect.py` | simonw 9599 | open | 0 | 0 | 2021-06-20T00:22:43Z | 2021-06-20T00:22:49Z | OWNER | https://codecov.io/gh/simonw/datasette/src/main/datasette/inspect.py shows only 36% coverage for that module at the moment. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1383/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
947596222 | MDExOlB1bGxSZXF1ZXN0NjkyNTU3Mzgx | 1399 | Multiple sort | jgryko5 87192257 | open | 0 | 0 | 2021-07-19T12:20:14Z | 2021-07-19T12:20:14Z | FIRST_TIME_CONTRIBUTOR | simonw/datasette/pulls/1399 | Closes #197. I have added support for sorting by multiple parameters as mentioned in the issue above, and together with that, a suggestion on how to implement such sorting in the user interface. |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1399/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
953218043 | MDU6SXNzdWU5NTMyMTgwNDM= | 1403 | Labels explaining what hidden tables are for | simonw 9599 | open | 0 | 0 | 2021-07-26T19:29:22Z | 2022-03-21T22:20:37Z | OWNER | A reasonable question: "What are those hidden tables for?" This could be answered by adding a small piece of explanatory text to each table - based on if it's related to FTS or to SpatiaLite or configured to be hidden for some other reason. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1403/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
957315684 | MDU6SXNzdWU5NTczMTU2ODQ= | 1410 | Rename settings to `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream` | simonw 9599 | open | 0 | Datasette 1.0 3268330 | 0 | 2021-07-31T20:27:12Z | 2021-07-31T20:27:49Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/datasette/issues/1409#issuecomment-890400425 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1410/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
957347860 | MDU6SXNzdWU5NTczNDc4NjA= | 1412 | Mention WAL mode in the documentation (plus backup tips) | simonw 9599 | open | 0 | 0 | 2021-08-01T00:27:11Z | 2021-08-01T00:27:11Z | OWNER | This is useful for people who are deploying Datasette with any write functionality, especially if they might be using something like EFS. I can add a section about this to the bottom of https://docs.datasette.io/en/stable/deploying.html and then link to that section from both https://docs.datasette.io/en/stable/sql_queries.html#writable-canned-queries and https://docs.datasette.io/en/stable/internals.html#await-db-execute-write-sql-params-none-block-false Also useful: mention that just copying a SQLite database file while it is being written to may not get a consistent file, so tell people to use one of the SQLite backup mechanisms instead. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1412/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
970626625 | MDU6SXNzdWU5NzA2MjY2MjU= | 1435 | Turn off suggest facets on tables with large numbers of columns | simonw 9599 | open | 0 | 0 | 2021-08-13T18:30:48Z | 2021-08-13T18:30:48Z | OWNER | If a table has 200 columns it will take multiple seconds to try and suggest facets. I should either quit after the first 20 or not suggest facets at all. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1435/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
982803408 | MDU6SXNzdWU5ODI4MDM0MDg= | 1454 | Feature Request: Publish to IPFS | blitmap 1560788 | open | 0 | 0 | 2021-08-30T13:36:18Z | 2021-08-30T13:36:18Z | NONE | Hello, I am a huge fan of this being used for exploring data. I think it has a lot of flexibility not found in other tools. I'm not sure if what I'm asking for is possible: Can this be extended to publish to IPFS? IPFS is an attractive hosting option for decentralized journalism. Food for thought ~ |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1454/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
988552851 | MDU6SXNzdWU5ODg1NTI4NTE= | 1456 | conda install results in non-functioning `datasette serve` due to out-of-date asgiref | ctb 51016 | open | 0 | 0 | 2021-09-05T16:59:55Z | 2021-09-05T16:59:55Z | CONTRIBUTOR | Over in https://github.com/ctb/2021-sourmash-datasette, I discovered that the following commands fail:
This appears to be because asgiref 3.3.4 doesn't have WebSocketScope, but later versions do - a simple
fixes the problem for me, at least to the point where I can run datasette and poke around as usual. I note that over in the conda-forge recipe, https://github.com/conda-forge/datasette-feedstock/blob/master/recipe/meta.yaml pins asgiref to < 3.4.0, but I'm not sure why - so I'm not sure how to best resolve this issue :). |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1456/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);
comments 1 ✖