github
| html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
|---|---|---|---|---|---|---|---|---|---|---|---|
| https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-927312650 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54 | 927312650 | IC_kwDODEm0Qs43RasK | 2182 | 2021-09-26T14:09:51Z | 2021-09-26T14:09:51Z | NONE | Similar trouble with ageinfo using 0.22. Here's what my ageinfo.js file looks like: ``` window.YTD.ageinfo.part0 = [ { "ageMeta" : { } } ] ``` Commenting out the registration for ageinfo in archive.py gets my archive to import. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
779088071 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926208819 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926208819 | IC_kwDOCGYnMM43NNMz | 9599 | 2021-09-23T22:26:54Z | 2021-09-23T22:26:54Z | OWNER | I could even have those replacement characters be properties of the `Database` class, so uses can sub-class and change them. ```python class Database: left_brace_replace = "_" right_brace_replace = "_" ... ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926207719 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926207719 | IC_kwDOCGYnMM43NM7n | 9599 | 2021-09-23T22:24:09Z | 2021-09-23T22:24:09Z | OWNER | I think I like the underscore option best. I don't like the idea of injecting surprise `( )` parenthesis, and having them vanish entirely could result in things like `item[price]` becoming `itemprice` which feels confusing. `item_price_` is a little ugly but I think I can live with it. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926207246 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926207246 | IC_kwDOCGYnMM43NM0O | 9599 | 2021-09-23T22:23:09Z | 2021-09-23T22:23:09Z | OWNER | What are my options for replacing those characters? - `[` becomes `(` and `]` becomes `)` - `[` and `]` are removed entirely - `[` and `]` both become `_` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926206705 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926206705 | IC_kwDOCGYnMM43NMrx | 9599 | 2021-09-23T22:21:58Z | 2021-09-23T22:21:58Z | OWNER | I'm inclined to just fix them and not have an option for opting-out of fixing them, since it adds quite a bit of cruft to the overall API design for an option that maybe no-one will ever use. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926206220 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926206220 | IC_kwDOCGYnMM43NMkM | 9599 | 2021-09-23T22:20:55Z | 2021-09-23T22:21:19Z | OWNER | If I add a new parameter for opting in and out of fixing these, what should it be called? A few options: - `fix_columns=False` - `rename_bad_columns=False` - `fix_column_names=False` - `error_on_invalid_column_names=True` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926205047 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926205047 | IC_kwDOCGYnMM43NMR3 | 9599 | 2021-09-23T22:18:34Z | 2021-09-23T22:19:38Z | OWNER | Here's the code where this happens: https://github.com/simonw/sqlite-utils/blob/54191d4dc114d7dc21e849b48a4d5ae4f9e601ca/sqlite_utils/db.py#L2943-L2948 It's called from three different methods in `db.py`: `create_table_sql()`, `update()` and `insert_all()`. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926204046 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926204046 | IC_kwDOCGYnMM43NMCO | 9599 | 2021-09-23T22:16:18Z | 2021-09-23T22:16:18Z | OWNER | So either I do the automatic replacement, or I let the user request automatic replacement, or a third option: I do automatic replacement but let the user opt to receive an error instead. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/329#issuecomment-926203501 | https://api.github.com/repos/simonw/sqlite-utils/issues/329 | 926203501 | IC_kwDOCGYnMM43NL5t | 9599 | 2021-09-23T22:15:07Z | 2021-09-23T22:15:30Z | OWNER | Quoting https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586676856 in full: > I'm not sure what to do about this one. > > I can't fix it: this bug in Python's `sqlite3` module means that even if I write a database out with column names that include `[]` I won't be able to read them back again. > > So... I could do one of the following: > > * Throw an error if a column name includes those characters. That's my preferred option I think. > * Automatically replace `[` in column names with `(` and `]` with `)` > * Do the automatic replacement but show a user-visible warning when I do it > * Throw an error, but give the user an option to run with e.g. `--fix-column-names` which applies that automatic fix. > > > Since this is likely to be an incredibly rare edge-case I think I'd rather minimize the amount of code that deals with it, so my preferred option is to just throw that error and stop. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1005891028 | |
| https://github.com/simonw/sqlite-utils/issues/325#issuecomment-925321439 | https://api.github.com/repos/simonw/sqlite-utils/issues/325 | 925321439 | IC_kwDOCGYnMM43J0jf | 9599 | 2021-09-22T20:52:56Z | 2021-09-22T20:52:56Z | OWNER | Updated documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#running-queries-directly-against-csv-or-json > If two files have the same name they will be assigned a numeric suffix: > > $ sqlite-utils memory foo/data.csv bar/data.csv "select * from data_2" | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
990844088 | |
| https://github.com/simonw/sqlite-utils/issues/325#issuecomment-925303497 | https://api.github.com/repos/simonw/sqlite-utils/issues/325 | 925303497 | IC_kwDOCGYnMM43JwLJ | 9599 | 2021-09-22T20:25:44Z | 2021-09-22T20:25:44Z | OWNER | Here's the relevant code: https://github.com/simonw/sqlite-utils/blob/7427a9137f60de961b6331d0922a3f03da0d1890/sqlite_utils/cli.py#L1289-L1292 I can fix this by checking to see if `csv_table` is already in use and adding a suffix. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
990844088 | |
| https://github.com/simonw/sqlite-utils/issues/325#issuecomment-925301981 | https://api.github.com/repos/simonw/sqlite-utils/issues/325 | 925301981 | IC_kwDOCGYnMM43Jvzd | 9599 | 2021-09-22T20:23:25Z | 2021-09-22T20:23:25Z | OWNER | Oddly I can't replicate this on macOS: ``` (sqlite-utils) sqlite-utils % ls foo/*.csv foo/bug.csv (sqlite-utils) sqlite-utils % ls bar/*.csv bar/bug.csv (sqlite-utils) sqlite-utils % sqlite-utils memory foo/bug.csv bar/bug.csv --schema CREATE TABLE "bug" ( [col1] TEXT, [col2] TEXT ); CREATE VIEW t1 AS select * from [bug]; CREATE VIEW t AS select * from [bug]; ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
990844088 | |
| https://github.com/simonw/sqlite-utils/issues/328#issuecomment-925300720 | https://api.github.com/repos/simonw/sqlite-utils/issues/328 | 925300720 | IC_kwDOCGYnMM43Jvfw | 12752 | 2021-09-22T20:21:33Z | 2021-09-22T20:21:33Z | NONE | Wow, that was fast! Thank you! | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1004613267 | |
| https://github.com/simonw/sqlite-utils/issues/325#issuecomment-925300637 | https://api.github.com/repos/simonw/sqlite-utils/issues/325 | 925300637 | IC_kwDOCGYnMM43Jved | 9599 | 2021-09-22T20:21:26Z | 2021-09-22T20:21:26Z | OWNER | The `t1` and `t2` aliases were meant to handle this case, but the are no good if the tool throws an error. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
990844088 | |
| https://github.com/simonw/sqlite-utils/issues/328#issuecomment-925296085 | https://api.github.com/repos/simonw/sqlite-utils/issues/328 | 925296085 | IC_kwDOCGYnMM43JuXV | 9599 | 2021-09-22T20:14:53Z | 2021-09-22T20:14:53Z | OWNER | The bug is in this code: https://github.com/simonw/sqlite-utils/blob/77c240df56068341561e95e4a412cbfa24dc5bc7/sqlite_utils/cli.py#L2205-L2227 | {
"total_count": 1,
"+1": 1,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1004613267 | |
| https://github.com/simonw/sqlite-utils/issues/328#issuecomment-925292384 | https://api.github.com/repos/simonw/sqlite-utils/issues/328 | 925292384 | IC_kwDOCGYnMM43Jtdg | 9599 | 2021-09-22T20:09:53Z | 2021-09-22T20:09:53Z | OWNER | Good catch, thanks. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1004613267 | |
| https://github.com/simonw/datasette/issues/111#issuecomment-924437942 | https://api.github.com/repos/simonw/datasette/issues/111 | 924437942 | IC_kwDOBm6k_c43Gc22 | 9599 | 2021-09-21T22:32:59Z | 2021-09-21T22:47:07Z | OWNER | Side-note: Django 4.0 [will switch](https://docs.djangoproject.com/en/dev/releases/4.0/#zoneinfo-default-timezone-implementation) from using `pytz` to using the standard library `zoneinfo` module introduced in Python 3.9, which has a backport that works as far back as 3.6: https://github.com/pganssle/zoneinfo (https://pypi.org/project/backports.zoneinfo/) If I need to handle timezones I'll use that, but I think I can get away without it? Django does this: https://github.com/django/django/blob/b0ed619303d2fb723330ca9efa3acf23d49f1d19/setup.cfg#L39-L43 ``` install_requires = asgiref >= 3.3.2 backports.zoneinfo; python_version<"3.9" sqlparse >= 0.2.2 tzdata; sys_platform == 'win32' ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
274615452 | |
| https://github.com/simonw/datasette/issues/111#issuecomment-924443089 | https://api.github.com/repos/simonw/datasette/issues/111 | 924443089 | IC_kwDOBm6k_c43GeHR | 9599 | 2021-09-21T22:45:14Z | 2021-09-21T22:45:26Z | OWNER | The audiences I care about here are: - Producers of this timestamp - generally that will be users who are using `datasette publish` to share their data - Human consumers of this timestamp - end users who look at a Datasette site and want to know how recent the data is - Machine consumers of this timestamp - API integrations that might want to check if a Datasette instance has been updated before downloading new data For producers I think there are going to be two categories. The first is users who run "publish" and want the site to reflect when they did so (probably using `--updated=now` when they publish). The second are users who are willing to spend more time thinking about this - for example my various git scraping projects where I want to use a date derived from the git history. For humans... I'd like to be able to calculate a relative time (3 hours ago) in addition to showing the display time, because that helps avoid confusion over timezones. For machine consumers, it might be nice to offer the option of a calculated Unix timestamp-since-1970, since those can be easier to work with in some languages than running a full ISO date parser. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
274615452 | |
| https://github.com/simonw/datasette/issues/111#issuecomment-924438481 | https://api.github.com/repos/simonw/datasette/issues/111 | 924438481 | IC_kwDOBm6k_c43Gc_R | 9599 | 2021-09-21T22:34:03Z | 2021-09-21T22:34:21Z | OWNER | The simplest possible version of this: it's always represented as a UTC ISO datetime, like this: "updated": "2020-10-31T12:00:00+00:00" Later versions of Datasette could extend this to handle other timezones or support just the date (though that's a backwards incompatible change so probably better to decide on the date thing right now). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
274615452 | |
| https://github.com/simonw/datasette/issues/111#issuecomment-924435971 | https://api.github.com/repos/simonw/datasette/issues/111 | 924435971 | IC_kwDOBm6k_c43GcYD | 9599 | 2021-09-21T22:29:15Z | 2021-09-21T22:29:49Z | OWNER | So this is a metadata key called `updated` which can be applied at the table, database or instance level. It is represented as a `.isoformat()` timestamp. Question: should I support just the date - `yyyy-mm-dd` - in addition to the datetime? I think so. I can easily imagine situations where the exact time of day that a change was made hasn't been recorded, but the overall date is known. But in that case, should the `updated` key sometimes be `yyyy-mm-dd` and sometimes be the full isoformat datetime? Or should there be an `updated_date` key that's used for just the date? | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
274615452 | |
| https://github.com/simonw/datasette/issues/111#issuecomment-924432643 | https://api.github.com/repos/simonw/datasette/issues/111 | 924432643 | IC_kwDOBm6k_c43GbkD | 9599 | 2021-09-21T22:23:23Z | 2021-09-21T22:23:23Z | OWNER | I'm going to use https://github.com/dateutil/dateutil for this - it's been maintained constantly (by an evolving team of contributors) [since 2003](https://github.com/dateutil/dateutil/commit/68ae2757ae15c84bf947d47a82a314b3b975bc9b) and is a very trustworthy dependency. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
274615452 | |
| https://github.com/dogsheep/twitter-to-sqlite/pull/59#issuecomment-924209583 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/59 | 924209583 | IC_kwDODEm0Qs43FlGv | 9599 | 2021-09-21T17:37:34Z | 2021-09-21T17:37:34Z | MEMBER | Thanks for this! | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
984942782 | |
| https://github.com/simonw/datasette/pull/1476#issuecomment-923979964 | https://api.github.com/repos/simonw/datasette/issues/1476 | 923979964 | IC_kwDOBm6k_c43EtC8 | 22429695 | 2021-09-21T13:18:22Z | 2021-09-21T13:18:22Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1476?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#1476](https://codecov.io/gh/simonw/datasette/pull/1476?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (2eb01ff) into [main](https://codecov.io/gh/simonw/datasette/commit/b28b6cd2fe97f7e193a235877abeec2c8eb0a821?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b28b6cd) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/simonw/datasette/pull/1476?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #1476 +/- ## ======================================= Coverage 91.83% 91.83% ======================================= Files 34 34 Lines 4421 4421 ======================================= Hits 4060 4060 Misses 361 361 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1476?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1476?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [b28b6cd...2eb01ff](https://codecov.io/gh/simonw/datasette/pull/1476?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
1002459220 | |
| https://github.com/simonw/datasette/issues/111#issuecomment-923106887 | https://api.github.com/repos/simonw/datasette/issues/111 | 923106887 | IC_kwDOBm6k_c43BX5H | 9599 | 2021-09-20T16:58:39Z | 2021-09-20T16:58:39Z | OWNER | Still a good idea today too! Would be great for https://cdc-vaccination-history.datasette.io/ for example. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
274615452 | |
| https://github.com/simonw/datasette/issues/1473#issuecomment-922394999 | https://api.github.com/repos/simonw/datasette/issues/1473 | 922394999 | IC_kwDOBm6k_c42-qF3 | 192568 | 2021-09-19T00:44:39Z | 2021-09-19T00:45:32Z | CONTRIBUTOR | I replaced: ``` <a class="text-3xl font-bold leading-none" href="https://www.saferortoxic.org"> <img src="https://www.saferortoxic.org/images/logo_hu26e4dce8d5931af1ea33526b28fc8383_9734_c52a4f1635ef88bda858373270551ed2.webp" class="custom-logo" alt="Logo: Safer or Toxic?" width="300px"> </a> ``` with: ``` <img src="https://www.saferortoxic.org/images/logo_hu26e4dce8d5931af1ea33526b28fc8383_9734_c52a4f1635ef88bda858373270551ed2.webp" alt="Logo: Safer or Toxic?" style="width:300px; cursor:pointer;" onclick="window.location.href='https://saferortoxic.org';return false;"> ``` I'd still love to know what caused this (and how to troubleshoot to figure it out), so I'll leave it open for a bit, but I do have a functional logo linking to the Hugo home page, at least locally. I'll likely push tomorrow. (Before trying this, I tried to apply a background image to the `a` tag. That didn't work.) | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
999902754 | |
| https://github.com/simonw/datasette/issues/1473#issuecomment-922363640 | https://api.github.com/repos/simonw/datasette/issues/1473 | 922363640 | IC_kwDOBm6k_c42-ib4 | 192568 | 2021-09-18T19:45:47Z | 2021-09-18T19:45:47Z | CONTRIBUTOR | An update, if I remove the `img` tag and replace it with the text, "Safer or Toxic?" it links to the right place. Also, if I keep things exactly as they are, and it improperly, but consistently goes to the `undefined` page, on THAT 404 page, a click on the image properly clicks through to the www.SaferOrToxic.org page. Weird stuff. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
999902754 | |
| https://github.com/simonw/datasette/issues/236#issuecomment-922075480 | https://api.github.com/repos/simonw/datasette/issues/236 | 922075480 | IC_kwDOBm6k_c429cFY | 9599 | 2021-09-17T20:54:13Z | 2021-09-17T20:54:13Z | OWNER | That's so useful @sethvincent! Really interesting reading your code there, especially clever how you're using the `base_url` config. I'd be very interested to see what your demo looks like without using serverless - completely agree that the less additional dependencies there are for this the better. I'm also very interested in figuring out a way to run Datasette in Lambda but with the SQLite database on an EFS volume. Do you have a feel for how hard that would be? | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
317001500 | |
| https://github.com/simonw/datasette/issues/236#issuecomment-920543967 | https://api.github.com/repos/simonw/datasette/issues/236 | 920543967 | IC_kwDOBm6k_c423mLf | 164214 | 2021-09-16T03:19:08Z | 2021-09-16T03:19:08Z | NONE | :wave: I just put together a small example using the lambda container image support: https://github.com/sethvincent/datasette-aws-lambda-example It uses mangum and AWS's [python runtime interface client](https://github.com/aws/aws-lambda-python-runtime-interface-client) to handle the lambda event stuff. I'd be happy to help with a publish plugin for AWS lambda as I plan to use this for upcoming projects. The example uses the [serverless](https://www.serverless.com) cli for deployment but there might be a more suitable deployment approach for the plugin. It would be cool if users didn't have to install anything additional other than the aws cli and its associated config/credentials setup. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
317001500 | |
| https://github.com/simonw/datasette/pull/1471#issuecomment-919141156 | https://api.github.com/repos/simonw/datasette/issues/1471 | 919141156 | IC_kwDOBm6k_c42yPsk | 22429695 | 2021-09-14T13:16:29Z | 2021-09-14T13:16:29Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1471?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#1471](https://codecov.io/gh/simonw/datasette/pull/1471?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (01b05ed) into [main](https://codecov.io/gh/simonw/datasette/commit/b28b6cd2fe97f7e193a235877abeec2c8eb0a821?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b28b6cd) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/simonw/datasette/pull/1471?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #1471 +/- ## ======================================= Coverage 91.83% 91.83% ======================================= Files 34 34 Lines 4421 4421 ======================================= Hits 4060 4060 Misses 361 361 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1471?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1471?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [b28b6cd...01b05ed](https://codecov.io/gh/simonw/datasette/pull/1471?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
996002181 | |
| https://github.com/simonw/datasette/pull/1453#issuecomment-919135732 | https://api.github.com/repos/simonw/datasette/issues/1453 | 919135732 | IC_kwDOBm6k_c42yOX0 | 49699333 | 2021-09-14T13:10:38Z | 2021-09-14T13:10:38Z | CONTRIBUTOR | Superseded by #1471. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
982780906 | |
| https://github.com/simonw/datasette/issues/1464#issuecomment-918621705 | https://api.github.com/repos/simonw/datasette/issues/1464 | 918621705 | IC_kwDOBm6k_c42wQ4J | 7476523 | 2021-09-13T22:17:17Z | 2021-09-13T22:17:17Z | CONTRIBUTOR | > haven't had time to get back to this, but idle thought that I'm recording for later investigation: how does the continuous integration handle this installation issue? Is it documented there? Not certain, but I think tests in CI run on Ubuntu and don't appear to install any additional Sqlite-related dependencies, and so my guess is the version of Sqlite installed by default on Ubuntu has the `SQLITE_ENABLE_FTS3_PARENTHESIS` option enabled and so doesn't run into this issue. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991191951 | |
| https://github.com/simonw/datasette/pull/1385#issuecomment-869105782 | https://api.github.com/repos/simonw/datasette/issues/1385 | 869105782 | MDEyOklzc3VlQ29tbWVudDg2OTEwNTc4Mg== | 22429695 | 2021-06-27T05:48:55Z | 2021-09-13T17:29:30Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1385?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#1385](https://codecov.io/gh/simonw/datasette/pull/1385?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (db78094) into [main](https://codecov.io/gh/simonw/datasette/commit/ea627baccf980d7d8ebc9e1ffff1fe34d556e56f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ea627ba) will **not change** coverage. > The diff coverage is `n/a`. > :exclamation: Current head db78094 differs from pull request most recent head 8d78c8c. Consider uploading reports for the commit 8d78c8c to get more accurate results [](https://codecov.io/gh/simonw/datasette/pull/1385?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #1385 +/- ## ======================================= Coverage 91.70% 91.70% ======================================= Files 34 34 Lines 4364 4364 ======================================= Hits 4002 4002 Misses 362 362 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1385?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1385?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [ea627ba...8d78c8c](https://codecov.io/gh/simonw/datasette/pull/1385?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
930855052 | |
| https://github.com/simonw/datasette/issues/1466#issuecomment-917840012 | https://api.github.com/repos/simonw/datasette/issues/1466 | 917840012 | IC_kwDOBm6k_c42tSCM | 9599 | 2021-09-13T04:54:59Z | 2021-09-13T04:54:59Z | OWNER | Especially relevant now that 0.2.0 is out which is a much higher quality release. https://github.com/simonw/datasette-app/releases/tag/0.2.0 | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991467558 | |
| https://github.com/simonw/datasette/issues/1468#issuecomment-917839801 | https://api.github.com/repos/simonw/datasette/issues/1468 | 917839801 | IC_kwDOBm6k_c42tR-5 | 9599 | 2021-09-13T04:54:17Z | 2021-09-13T04:54:17Z | OWNER | Here's a already open issue for this: #972 | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
994390593 | |
| https://github.com/simonw/datasette/issues/1468#issuecomment-917839507 | https://api.github.com/repos/simonw/datasette/issues/1468 | 917839507 | IC_kwDOBm6k_c42tR6T | 9599 | 2021-09-13T04:53:22Z | 2021-09-13T04:53:22Z | OWNER | At the moment this isn't possible - though there's a workaround which is to define a SQL view for the query, at which point facets will be displayed again. I did a lot of the work required to support this when I refactored how facets worked a while back - but to finally implement this I need to refactor the table view and the arbitrary query view to share much more logic than they do at the moment. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
994390593 | |
| https://github.com/simonw/datasette/issues/1469#issuecomment-917839062 | https://api.github.com/repos/simonw/datasette/issues/1469 | 917839062 | IC_kwDOBm6k_c42tRzW | 9599 | 2021-09-13T04:52:01Z | 2021-09-13T04:52:01Z | OWNER | Here's the code at fault: https://github.com/simonw/datasette/blob/b28b6cd2fe97f7e193a235877abeec2c8eb0a821/datasette/static/table.js#L137-L146 | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
994450961 | |
| https://github.com/simonw/datasette/issues/1464#issuecomment-917642487 | https://api.github.com/repos/simonw/datasette/issues/1464 | 917642487 | IC_kwDOBm6k_c42shz3 | 51016 | 2021-09-12T14:03:09Z | 2021-09-12T14:03:09Z | NONE | haven't had time to get back to this, but idle thought that I'm recording for later investigation: how does the continuous integration handle this installation issue? Is it documented there? | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991191951 | |
| https://github.com/simonw/sqlite-utils/pull/326#issuecomment-916119657 | https://api.github.com/repos/simonw/sqlite-utils/issues/326 | 916119657 | IC_kwDOCGYnMM42muBp | 191622 | 2021-09-09T13:54:10Z | 2021-09-09T13:54:10Z | NONE | dupe of #293? | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991237645 | |
| https://github.com/simonw/datasette/issues/1464#issuecomment-915343886 | https://api.github.com/repos/simonw/datasette/issues/1464 | 915343886 | IC_kwDOBm6k_c42jwoO | 7476523 | 2021-09-08T15:32:06Z | 2021-09-08T15:32:06Z | CONTRIBUTOR | Thanks, that does look similar! > Unfortunately, pysqlite3-binary isn't available for Mac OS X, so I can't quickly check that that fixes it; will do so later. Ah that makes sense, I guess that's why this isn't just always installed already. I wonder if a possible solution to this issue could be doing feature detection on whether this feature is supported by the current Sqlite version, and if not these tests could be disabled locally? But possibly there's a better way to handle this, will see what @simonw thinks | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991191951 | |
| https://github.com/simonw/sqlite-utils/pull/326#issuecomment-915321467 | https://api.github.com/repos/simonw/sqlite-utils/issues/326 | 915321467 | IC_kwDOCGYnMM42jrJ7 | 22429695 | 2021-09-08T15:05:53Z | 2021-09-08T15:05:53Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#326](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (078a087) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/77c240df56068341561e95e4a412cbfa24dc5bc7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (77c240d) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #326 +/- ## ======================================= Coverage 96.58% 96.58% ======================================= Files 5 5 Lines 2223 2223 ======================================= Hits 2147 2147 Misses 76 76 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [77c240d...078a087](https://codecov.io/gh/simonw/sqlite-utils/pull/326?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991237645 | |
| https://github.com/simonw/datasette/issues/1464#issuecomment-915302885 | https://api.github.com/repos/simonw/datasette/issues/1464 | 915302885 | IC_kwDOBm6k_c42jmnl | 51016 | 2021-09-08T14:44:50Z | 2021-09-08T14:44:50Z | NONE | thanks for the response! full errors attached; excerpt: ``` ... def test_searchmode(table_metadata, querystring, expected_rows): with make_app_client( metadata={"databases": {"fixtures": {"tables": {"searchable": table_metadata}}}} ) as client: response = client.get("/fixtures/searchable.json?" + querystring) > assert expected_rows == response.json["rows"] E AssertionError: assert [[1, 'barry c...sel', 'puma']] == [] E Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther'] E Use -v to get the full diff /Users/t/dev/datasette/tests/test_api.py:1115: AssertionError ``` [errors.txt](https://github.com/simonw/datasette/files/7129719/errors.txt) A quick scan of #1223 suggests you're right. Unfortunately, pysqlite3-binary isn't available for Mac OS X, so I can't quickly check that that fixes it; will do so later. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991191951 | |
| https://github.com/simonw/datasette/issues/1464#issuecomment-915299013 | https://api.github.com/repos/simonw/datasette/issues/1464 | 915299013 | IC_kwDOBm6k_c42jlrF | 7476523 | 2021-09-08T14:40:28Z | 2021-09-08T14:40:28Z | CONTRIBUTOR | What are the full errors you're getting? This *may* be the same issue as described in https://github.com/simonw/datasette/pull/1223 - essentially the test suite (and corresponding Datasette features I assume) are by default implicitly dependent on your Sqlite installation having been compiled with the `SQLITE_ENABLE_FTS3_PARENTHESIS` option. If this is the same issue then I think this can be fixed either by recompiling with that option or (probably more easily) by running `pip install pysqlite3-binary`, which will be used in preference to your system Sqlite installation and has this option enabled. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991191951 | |
| https://github.com/simonw/datasette/issues/1464#issuecomment-915279711 | https://api.github.com/repos/simonw/datasette/issues/1464 | 915279711 | IC_kwDOBm6k_c42jg9f | 51016 | 2021-09-08T14:16:49Z | 2021-09-08T14:16:49Z | NONE | on commit d57ab156b35ec642 | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991191951 | |
| https://github.com/simonw/datasette/pull/1463#issuecomment-915229323 | https://api.github.com/repos/simonw/datasette/issues/1463 | 915229323 | IC_kwDOBm6k_c42jUqL | 22429695 | 2021-09-08T13:15:26Z | 2021-09-08T13:15:26Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1463?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#1463](https://codecov.io/gh/simonw/datasette/pull/1463?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (2dd94c5) into [main](https://codecov.io/gh/simonw/datasette/commit/d57ab156b35ec642549fb69d08279850065027d2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (d57ab15) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/simonw/datasette/pull/1463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #1463 +/- ## ======================================= Coverage 91.83% 91.83% ======================================= Files 34 34 Lines 4421 4421 ======================================= Hits 4060 4060 Misses 361 361 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1463?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1463?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [d57ab15...2dd94c5](https://codecov.io/gh/simonw/datasette/pull/1463?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
991121619 | |
| https://github.com/simonw/datasette/issues/1462#issuecomment-914644260 | https://api.github.com/repos/simonw/datasette/issues/1462 | 914644260 | IC_kwDOBm6k_c42hF0k | 9599 | 2021-09-07T21:34:32Z | 2021-09-07T21:34:32Z | OWNER | I think this is a setting. There are two relevant settings at the moment: ``` "template_debug": false, "trace_debug": false, ``` For consistence then this should be called `something_debug` - but do I want a single setting that exposes the `_internal` database and adds those debug options to the menu, or do I want those as two separate settings? - `internal_debug` to enable access to that `_internal` database - `menu_debug` for those menu options? | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
990367646 | |
| https://github.com/simonw/datasette/issues/1461#issuecomment-914441037 | https://api.github.com/repos/simonw/datasette/issues/1461 | 914441037 | IC_kwDOBm6k_c42gUNN | 9599 | 2021-09-07T16:13:59Z | 2021-09-07T16:13:59Z | OWNER | I don't think I'll adopt it for this project. For example, here: ```diff response = Response.redirect("/") - response.set_cookie("ds_actor", datasette.sign({ - "a": { - "id": "cleopaws" - } - }, "actor")) + response.set_cookie("ds_actor", datasette.sign({"a": {"id": "cleopaws"}}, "actor")) ``` I chose to use the multi-line version to help emphasize the structure - the single-line replacement loses that. I think I'll continue to make my own editorial choices about how the code examples are laid out. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
989986586 | |
| https://github.com/simonw/datasette/issues/1461#issuecomment-914440282 | https://api.github.com/repos/simonw/datasette/issues/1461 | 914440282 | IC_kwDOBm6k_c42gUBa | 9599 | 2021-09-07T16:12:57Z | 2021-09-07T16:12:57Z | OWNER | Here's the diff it produced from that first run: ```diff diff --git a/docs/authentication.rst b/docs/authentication.rst index 0d98cf8..8008023 100644 --- a/docs/authentication.rst +++ b/docs/authentication.rst @@ -381,11 +381,7 @@ Authentication plugins can set signed ``ds_actor`` cookies themselves like so: .. code-block:: python response = Response.redirect("/") - response.set_cookie("ds_actor", datasette.sign({ - "a": { - "id": "cleopaws" - } - }, "actor")) + response.set_cookie("ds_actor", datasette.sign({"a": {"id": "cleopaws"}}, "actor")) Note that you need to pass ``"actor"`` as the namespace to :ref:`datasette_sign`. @@ -412,12 +408,16 @@ To include an expiry, add a ``"e"`` key to the cookie value containing a `base62 expires_at = int(time.time()) + (24 * 60 * 60) response = Response.redirect("/") - response.set_cookie("ds_actor", datasette.sign({ - "a": { - "id": "cleopaws" - }, - "e": baseconv.base62.encode(expires_at), - }, "actor")) + response.set_cookie( + "ds_actor", + datasette.sign( + { + "a": {"id": "cleopaws"}, + "e": baseconv.base62.encode(expires_at), + }, + "actor", + ), + ) The resulting cookie will encode data that looks something like this: diff --git a/docs/spatialite.rst b/docs/spatialite.rst index d1b300b..556bad8 100644 --- a/docs/spatialite.rst +++ b/docs/spatialite.rst @@ -58,19 +58,22 @@ Here's a recipe for taking a table with existing latitude and longitude columns, .. code-block:: python import sqlite3 - conn = sqlite3.connect('museums.db') + + conn = sqlite3.connect("museums.db") # Lead the spatialite extension: conn.enable_load_extension(True) - conn.load_extension('/usr/local/lib/mod_spatialite.dylib') + conn.load_extension("/usr/local/lib/mod_spatialite.dylib") # Initialize spatial metadata for this database: - conn.execute('select InitSpatialMetadata(1)') + conn.execute("select InitSpatialMetadata(1)") # Add a geometry column called point_geom to our museums table: conn.execute("SELECT AddGeometryColumn('museums', 'point_geom', 4326, 'POINT', 2);") # Now update that geometry column with the lat/lon points - conn.execute(''' + conn.execute( + """ UPDATE museums SET point_geom = GeomFromText('POINT('||"longitude"||' '||"latitude"||')',4326); - ''') + """ + ) # Now add a spatial index to that column conn.execute('select CreateSpatialIndex("museums", "point_geom");') # If you don't commit your changes will not be persisted: @@ -186,13 +189,14 @@ Here's Python code to create a SQLite database, enable SpatiaLite, create a plac .. code-block:: python import sqlite3 - conn = sqlite3.connect('places.db') + + conn = sqlite3.connect("places.db") # Enable SpatialLite extension conn.enable_load_extension(True) - conn.load_extension('/usr/local/lib/mod_spatialite.dylib') + conn.load_extension("/usr/local/lib/mod_spatialite.dylib") # Create the masic countries table - conn.execute('select InitSpatialMetadata(1)') - conn.execute('create table places (id integer primary key, name text);') + conn.execute("select InitSpatialMetadata(1)") + conn.execute("create table places (id integer primary key, name text);") # Add a MULTIPOLYGON Geometry column conn.execute("SELECT AddGeometryColumn('places', 'geom', 4326, 'MULTIPOLYGON', 2);") # Add a spatial index against the new column @@ -201,13 +205,17 @@ Here's Python code to create a SQLite database, enable SpatiaLite, create a plac from shapely.geometry.multipolygon import MultiPolygon from shapely.geometry import shape import requests - geojson = requests.get('https://data.whosonfirst.org/404/227/475/404227475.geojson').json() + + geojson = requests.get( + "https://data.whosonfirst.org/404/227/475/404227475.geojson" + ).json() # Convert to "Well Known Text" format - wkt = shape(geojson['geometry']).wkt + wkt = shape(geojson["geometry"]).wkt # Insert and commit the record - conn.execute("INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))", ( - "Wales", wkt - )) + conn.execute( + "INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))", + ("Wales", wkt), + ) conn.commit() Querying polygons using within() diff --git a/docs/writing_plugins.rst b/docs/writing_plugins.rst index bd60a4b..5af01f6 100644 --- a/docs/writing_plugins.rst +++ b/docs/writing_plugins.rst @@ -18,9 +18,10 @@ The quickest way to start writing a plugin is to create a ``my_plugin.py`` file from datasette import hookimpl + @hookimpl def prepare_connection(conn): - conn.create_function('hello_world', 0, lambda: 'Hello world!') + conn.create_function("hello_world", 0, lambda: "Hello world!") If you save this in ``plugins/my_plugin.py`` you can then start Datasette like this:: @@ -60,22 +61,18 @@ The example consists of two files: a ``setup.py`` file that defines the plugin: from setuptools import setup - VERSION = '0.1' + VERSION = "0.1" setup( - name='datasette-plugin-demos', - description='Examples of plugins for Datasette', - author='Simon Willison', - url='https://github.com/simonw/datasette-plugin-demos', - license='Apache License, Version 2.0', + name="datasette-plugin-demos", + description="Examples of plugins for Datasette", + author="Simon Willison", + url="https://github.com/simonw/datasette-plugin-demos", + license="Apache License, Version 2.0", version=VERSION, - py_modules=['datasette_plugin_demos'], - entry_points={ - 'datasette': [ - 'plugin_demos = datasette_plugin_demos' - ] - }, - install_requires=['datasette'] + py_modules=["datasette_plugin_demos"], + entry_points={"datasette": ["plugin_demos = datasette_plugin_demos"]}, + install_requires=["datasette"], ) And a Python module file, ``datasette_plugin_demos.py``, that implements the plugin: @@ -88,12 +85,12 @@ And a Python module file, ``datasette_plugin_demos.py``, that implements the plu @hookimpl def prepare_jinja2_environment(env): - env.filters['uppercase'] = lambda u: u.upper() + env.filters["uppercase"] = lambda u: u.upper() @hookimpl def prepare_connection(conn): - conn.create_function('random_integer', 2, random.randint) + conn.create_function("random_integer", 2, random.randint) Having built a plugin in this way you can turn it into an installable package using the following command:: @@ -123,11 +120,13 @@ To bundle the static assets for a plugin in the package that you publish to PyPI .. code-block:: python - package_data={ - 'datasette_plugin_name': [ - 'static/plugin.js', - ], - }, + package_data = ( + { + "datasette_plugin_name": [ + "static/plugin.js", + ], + }, + ) Where ``datasette_plugin_name`` is the name of the plugin package (note that it uses underscores, not hyphens) and ``static/plugin.js`` is the path within that package to the static file. @@ -152,11 +151,13 @@ Templates should be bundled for distribution using the same ``package_data`` mec .. code-block:: python - package_data={ - 'datasette_plugin_name': [ - 'templates/my_template.html', - ], - }, + package_data = ( + { + "datasette_plugin_name": [ + "templates/my_template.html", + ], + }, + ) You can also use wildcards here such as ``templates/*.html``. See `datasette-edit-schema <https://github.com/simonw/datasette-edit-schema>`__ for an example of this pattern. ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
989986586 | |
| https://github.com/simonw/datasette/issues/1461#issuecomment-914439356 | https://api.github.com/repos/simonw/datasette/issues/1461 | 914439356 | IC_kwDOBm6k_c42gTy8 | 9599 | 2021-09-07T16:11:37Z | 2021-09-07T16:11:37Z | OWNER | ``` (datasette) datasette % blacken-docs docs/*.rst docs/authentication.rst: Rewriting... docs/internals.rst:169: code block parse error Cannot parse: 14:0: <line number missing in source> docs/plugin_hooks.rst:251: code block parse error Cannot parse: 6:4: ] docs/plugin_hooks.rst:312: code block parse error Cannot parse: 38:0: <line number missing in source> docs/spatialite.rst: Rewriting... docs/testing_plugins.rst:135: code block parse error Cannot parse: 5:0: <line number missing in source> docs/writing_plugins.rst: Rewriting... ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
989986586 | |
| https://github.com/simonw/datasette/issues/1459#issuecomment-913218494 | https://api.github.com/repos/simonw/datasette/issues/1459 | 913218494 | IC_kwDOBm6k_c42bpu- | 9599 | 2021-09-05T19:58:51Z | 2021-09-05T19:59:15Z | OWNER | This idea makes sense to me. There's actually an existing option that takes a path, called `--get` - it returns the HTML or JSON for that oath directly to the console, eg `datasette my.db --get /mydb/mytable.json` So... one option would be to allow combining that with `-o` to open that URL in the browser: datasette my.db -o --get /mydb So some options here are: - `datasette my.db --open-url /mydb` - `datasette my.db --open-path /mydb` - `datasette my.db --open --get /mydb` I quite like that last combination option, mainly to avoid adding even more command options. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
988556488 | |
| https://github.com/simonw/datasette/pull/1455#issuecomment-913001416 | https://api.github.com/repos/simonw/datasette/issues/1455 | 913001416 | IC_kwDOBm6k_c42a0vI | 9599 | 2021-09-04T16:32:21Z | 2021-09-04T16:32:21Z | OWNER | I'll add researchers too. | {
"total_count": 1,
"+1": 1,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
988325628 | |
| https://github.com/simonw/datasette/pull/1455#issuecomment-913001298 | https://api.github.com/repos/simonw/datasette/issues/1455 | 913001298 | IC_kwDOBm6k_c42a0tS | 9599 | 2021-09-04T16:31:32Z | 2021-09-04T16:31:32Z | OWNER | Great idea! | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
988325628 | |
| https://github.com/simonw/datasette/pull/1455#issuecomment-913001282 | https://api.github.com/repos/simonw/datasette/issues/1455 | 913001282 | IC_kwDOBm6k_c42a0tC | 51016 | 2021-09-04T16:31:24Z | 2021-09-04T16:31:24Z | NONE | I love it! maybe 'researchers' instead? Or 'scientists and researchers'? | {
"total_count": 1,
"+1": 1,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
988325628 | |
| https://github.com/dogsheep/evernote-to-sqlite/issues/14#issuecomment-911772943 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/14 | 911772943 | IC_kwDOEhK-wc42WI0P | 46968 | 2021-09-02T14:53:11Z | 2021-09-02T14:53:11Z | NONE | Additionally, assuming the line numbers match up with the provided enenx file, the mentioned line plus one before and after is as follows: ``` <![CDATA[>]]> </span></div> <div style="padding: 0px; font-family: Arial, sans-serif; font-size: 12px; line-height: 16px; white-space: pre-wrap;"><br style=" padding: 0px;"/> ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
986829194 | |
| https://github.com/dogsheep/twitter-to-sqlite/issues/58#issuecomment-910121331 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/58 | 910121331 | IC_kwDODEm0Qs42P1lz | 42904 | 2021-09-01T09:49:33Z | 2021-09-01T09:49:33Z | CONTRIBUTOR | Found the cause, it's the other commands. PR #59 submitted. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
984939366 | |
| https://github.com/simonw/datasette/issues/1446#issuecomment-908832938 | https://api.github.com/repos/simonw/datasette/issues/1446 | 908832938 | IC_kwDOBm6k_c42K7Cq | 9599 | 2021-08-31T01:54:59Z | 2021-08-31T01:54:59Z | OWNER | I used the sticky footer mechanism in `datasette.app`: https://github.com/simonw/datasette.app/issues/3 | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978357984 | |
| https://github.com/simonw/datasette/pull/1453#issuecomment-908337614 | https://api.github.com/repos/simonw/datasette/issues/1453 | 908337614 | IC_kwDOBm6k_c42JCHO | 22429695 | 2021-08-30T13:20:26Z | 2021-08-30T13:20:26Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#1453](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (4f492a7) into [main](https://codecov.io/gh/simonw/datasette/commit/67cbf0ae7243431bf13702e6e3ba466b619c4d6f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (67cbf0a) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #1453 +/- ## ======================================= Coverage 91.83% 91.83% ======================================= Files 34 34 Lines 4421 4421 ======================================= Hits 4060 4060 Misses 361 361 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [67cbf0a...4f492a7](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
982780906 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907547624 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907547624 | IC_kwDOBm6k_c42GBPo | 9599 | 2021-08-28T01:44:57Z | 2021-08-28T01:58:35Z | OWNER | Documentation: https://docs.datasette.io/en/latest/plugin_hooks.html#register-commands-cli | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907547736 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907547736 | IC_kwDOBm6k_c42GBRY | 9599 | 2021-08-28T01:45:36Z | 2021-08-28T01:45:36Z | OWNER | I need to push this out as an alpha so I can release a demo plugin that uses it. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907543982 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907543982 | IC_kwDOBm6k_c42GAWu | 9599 | 2021-08-28T01:14:27Z | 2021-08-28T01:14:27Z | OWNER | Writing the test for this is proving difficult, because the `cli` module has already been imported when I attempt to register a new plugin - so it doesn't pick up on the additional command registrations. Trying to work around that with `importlib.reload(cli)`. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907542214 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907542214 | IC_kwDOBm6k_c42F_7G | 9599 | 2021-08-28T01:02:38Z | 2021-08-28T01:02:38Z | OWNER | Partial prototype of `datasette-verify`: ```python from datasette import hookimpl import click @hookimpl def register_commands(cli): from datasette.cli import sqlite_extensions @cli.command() @click.argument("files", type=click.Path(exists=True), nargs=-1) @sqlite_extensions def verify(files, sqlite_extensions): "Verify that files can be opened by Datasette" for file in files: print(file) ``` I had to move the `from datasette.cli import sqlite_extensions` inside the hook function to avoid a circular import. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907540928 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907540928 | IC_kwDOBm6k_c42F_nA | 9599 | 2021-08-28T00:53:37Z | 2021-08-28T00:53:37Z | OWNER | I'll probably have to use this mechanism for the tests then: https://til.simonwillison.net/pytest/registering-plugins-in-tests | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907540790 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907540790 | IC_kwDOBm6k_c42F_k2 | 9599 | 2021-08-28T00:52:32Z | 2021-08-28T00:52:32Z | OWNER | I don't think I can get this new hook to support the handy [--plugins-dir= mechanism](https://docs.datasette.io/en/stable/plugins.html#one-off-plugins-using-plugins-dir) for loading plugins from Python files as opposed to registering them with setuptools - that mechanism is itself implemented inside of code called by `datasette serve` so I don't have a way of taking advantage of it from outside that command. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1450#issuecomment-907540240 | https://api.github.com/repos/simonw/datasette/issues/1450 | 907540240 | IC_kwDOBm6k_c42F_cQ | 9599 | 2021-08-28T00:48:30Z | 2021-08-28T00:48:30Z | OWNER | I'll go with this: ``` % datasette --help Usage: datasette [OPTIONS] COMMAND [ARGS]... Datasette is an open source multi-tool for exploring and publishing data About Datasette: https://datasette.io/ Full documentation: https://docs.datasette.io/ Options: --version Show the version and exit. --help Show this message and exit. ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981681138 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907539668 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907539668 | IC_kwDOBm6k_c42F_TU | 9599 | 2021-08-28T00:44:16Z | 2021-08-28T00:44:16Z | OWNER | Considering this piece of code: https://github.com/simonw/datasette/blob/a1a33bb5822214be1cebd98cd858b2058d91a4aa/datasette/cli.py#L122-L142 I think the hook itself gets called with a single argument, `cli`, which it can then use in the standard Click way to register extra stuff. I can't pass it `datasette` (like I do with `register_routes()`) because the Datasette object itself is instantiated by the `serve` command, which will not have been called. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907539251 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907539251 | IC_kwDOBm6k_c42F_Mz | 9599 | 2021-08-28T00:41:37Z | 2021-08-28T00:41:50Z | OWNER | The first example plugin I'm going to build for this will be `datasette verify file.db file2.db` - it will take one or more paths to SQLite files and verify if they can be opened by Datasette or not. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907539065 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907539065 | IC_kwDOBm6k_c42F_J5 | 9599 | 2021-08-28T00:40:24Z | 2021-08-28T00:40:24Z | OWNER | I'm going to call the new hook `register_commands` - since it will allow ambitious plugins to register more than one command if they want to. That's also pleasingly similar to `register_routes`. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907538940 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907538940 | IC_kwDOBm6k_c42F_H8 | 9599 | 2021-08-28T00:39:28Z | 2021-08-28T00:39:28Z | OWNER | Here's the answer to that: ``` ~ % datasette --help Usage: datasette [OPTIONS] COMMAND [ARGS]... Datasette! Options: --version Show the version and exit. --help Show this message and exit. Commands: serve* Serve up specified SQLite database files with a web UI inspect install Install Python packages - e.g. package Package specified SQLite files into a new datasette Docker... plugins List currently available plugins publish Publish specified SQLite database files to the internet... uninstall Uninstall Python packages (e.g. ``` Since it's adding extra things that show up in `--help` under the "Commands:" heading, I should call them commands. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907537693 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907537693 | IC_kwDOBm6k_c42F-0d | 9599 | 2021-08-28T00:31:26Z | 2021-08-28T00:31:26Z | OWNER | Terminology question: is it correct to call these subcommands or should they be commands? `publish_subcommand()` adds subcommands of the format `datasette publish X` - but are we instead adding commands with this new one? | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907537610 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907537610 | IC_kwDOBm6k_c42F-zK | 9599 | 2021-08-28T00:30:51Z | 2021-08-28T00:30:51Z | OWNER | There's also the option for plugins to muck around with existing registered commands - this could get a bit untidy if multiple plugins try to do it, but being able to replace `serve` with a fresh implementation that adds an additional command-line option before calling back to the original might open up some interesting possibilities. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/simonw/datasette/issues/1449#issuecomment-907537366 | https://api.github.com/repos/simonw/datasette/issues/1449 | 907537366 | IC_kwDOBm6k_c42F-vW | 9599 | 2021-08-28T00:29:16Z | 2021-08-28T00:29:29Z | OWNER | The closest plugin hook to this right now is [publish_subcommand](https://docs.datasette.io/en/stable/plugin_hooks.html#publish-subcommand-publish) - which looks like this: ```python @hookimpl def publish_subcommand(publish): @publish.command() @add_common_publish_arguments_and_options @click.option( "-k", "--api_key", help="API key for talking to my hosting provider", ) def my_hosting_provider(...): ``` But there are also several plugin hooks with `register_` prefixes, which may be a good naming convention to stick to here: `register_output_renderer(datasette)`, `register_routes(datasette)`, `register_facet_classes()`, `register_magic_parameters(datasette)`. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
981676832 | |
| https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-906646452 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13 | 906646452 | IC_kwDOEhK-wc42ClO0 | 9599 | 2021-08-26T18:34:34Z | 2021-08-26T18:35:20Z | MEMBER | I tried this ampersand fix: https://regex101.com/r/ojU2H9/1 ```python # https://regex101.com/r/ojU2H9/1 _invalid_ampersand_re = re.compile(r'&(?![a-z0-9]+;)') def fix_bad_xml(xml): # More fixes for things like '&' not as part of an entity return _invalid_ampersand_re.sub('&', xml) ``` Even with that I'm still getting total garbage in the `<en-note>` content - it's just HTML, not even trying to be XML. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978743426 | |
| https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-906635938 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13 | 906635938 | IC_kwDOEhK-wc42Ciqi | 9599 | 2021-08-26T18:18:27Z | 2021-08-26T18:18:27Z | MEMBER | It looks like I was using the round-trip to dump the `<?xml version="1.0" encoding="UTF-8" standalone="no"?>` and `<!DOCTYPE` prefixes. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978743426 | |
| https://github.com/simonw/datasette/pull/1448#issuecomment-906414811 | https://api.github.com/repos/simonw/datasette/issues/1448 | 906414811 | IC_kwDOBm6k_c42Bsrb | 22429695 | 2021-08-26T13:29:02Z | 2021-08-26T13:29:02Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#1448](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a211747) into [main](https://codecov.io/gh/simonw/datasette/commit/a1a33bb5822214be1cebd98cd858b2058d91a4aa?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a1a33bb) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #1448 +/- ## ======================================= Coverage 91.82% 91.82% ======================================= Files 34 34 Lines 4418 4418 ======================================= Hits 4057 4057 Misses 361 361 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [a1a33bb...a211747](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
980228553 | |
| https://github.com/dogsheep/dogsheep-photos/issues/7#issuecomment-906015471 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/7 | 906015471 | IC_kwDOD079W842ALLv | 18232 | 2021-08-26T02:01:01Z | 2021-08-26T02:01:01Z | NONE | Perceptual hashes might be what you're after : http://phash.org | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
602585497 | |
| https://github.com/simonw/datasette/issues/859#issuecomment-905904540 | https://api.github.com/repos/simonw/datasette/issues/859 | 905904540 | IC_kwDOBm6k_c41_wGc | 2670795 | 2021-08-25T21:59:14Z | 2021-08-25T21:59:55Z | CONTRIBUTOR | I did two tests: one with 1000 5-30mb DBs and a second with 20 multi gig DBs. For the second, I created them like so: `for i in {1..20}; do sqlite-generate db$i.db --tables ${i}00 --rows 100,2000 --columns 5,100 --pks 0 --fks 0; done` This was for deciding whether to use lots of small DBs or to group things into a smaller number of bigger DBs. The second strategy wins. By simply persisting the `_internal` DB to disk, I was able to avoid most of the performance issues I was experiencing previously. (To do this, I changed the `datasette/internal_db.py:init_internal_db` creates to if not exists, and changed the `_internal` DB instantiation in `datasette/app.py:Datasette.__init__` to a path with `is_mutable=True`.) Super rough, but the pages now load so I can continue testing ideas. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
642572841 | |
| https://github.com/simonw/datasette/issues/859#issuecomment-905900807 | https://api.github.com/repos/simonw/datasette/issues/859 | 905900807 | IC_kwDOBm6k_c41_vMH | 9599 | 2021-08-25T21:51:10Z | 2021-08-25T21:51:10Z | OWNER | 10-20 minutes to populate `_internal`! How many databases and tables is that for? I may have to rethink the `_internal` mechanism entirely. One possible alternative would be for the Datasette homepage to just show a list of available databases (maybe only if there are more than X connected) and then load in their metadata only the first time they are accessed. I need to get my own stress testing rig setup for this. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
642572841 | |
| https://github.com/simonw/datasette/issues/859#issuecomment-905899177 | https://api.github.com/repos/simonw/datasette/issues/859 | 905899177 | IC_kwDOBm6k_c41_uyp | 2670795 | 2021-08-25T21:48:00Z | 2021-08-25T21:48:00Z | CONTRIBUTOR | Upon first stab, there's two issues here: - DB/table/row counts (as discussed above). This isn't too bad if the DBs are actually above the MAX limit check. - Populating the internal DB. On first load of a giant set of DBs, it can take 10-20 mins to populate. By altering datasette and persisting the internal DB to disk, this problem is vastly improved, but I'm sure this will cause problems elsewhere. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
642572841 | |
| https://github.com/simonw/sqlite-utils/issues/323#issuecomment-905886797 | https://api.github.com/repos/simonw/sqlite-utils/issues/323 | 905886797 | IC_kwDOCGYnMM41_rxN | 9599 | 2021-08-25T21:25:18Z | 2021-08-25T21:25:18Z | OWNER | As far as I can tell the Python `sqlite3` module doesn't actually have a mechanism for de-registering a custom SQL function. This means that if I implement a mechanism whereby each call to `.convert()` registers a new SQL function with a random suffix (`convert_value_23424()` for example) those functions will stay registered - and if `.convert()` is called a large number of times the number of obsolete custom function registrations will grow without bounds. For that reason, I'm going to `wontfix` this issue. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
979627285 | |
| https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-905206234 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13 | 905206234 | IC_kwDOEhK-wc419Fna | 9599 | 2021-08-25T05:58:42Z | 2021-08-25T05:58:42Z | MEMBER | https://github.com/dogsheep/evernote-to-sqlite/blob/36a466f142e5bad52719851c2fbda0c05cd35b99/evernote_to_sqlite/utils.py#L34-L42 Not sure why I was round-tripping the `content_xml` like that - I will try not doing that. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978743426 | |
| https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-905203570 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13 | 905203570 | IC_kwDOEhK-wc419E9y | 9599 | 2021-08-25T05:51:22Z | 2021-08-25T05:53:27Z | MEMBER | The debugger showed me that it broke on a string that looked like this: ```xml <?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE en-note SYSTEM "http://xml.evernote.com/pub/enml2.dtd"> <en-note> <h1 title="Q3 2018 Reflection & Development"> <span title=Q3 2018 Reflection & Development"> Q3 2018 Reflection & Development </span> </h1> ... ``` Yeah that is not valid XML! | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978743426 | |
| https://github.com/simonw/datasette/pull/1447#issuecomment-905097468 | https://api.github.com/repos/simonw/datasette/issues/1447 | 905097468 | IC_kwDOBm6k_c418rD8 | 9599 | 2021-08-25T01:28:53Z | 2021-08-25T01:28:53Z | OWNER | Good catch, thanks! | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978614898 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905043974 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905043974 | IC_kwDOCGYnMM418eAG | 9599 | 2021-08-24T23:33:44Z | 2021-08-24T23:33:44Z | OWNER | Updated documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#inserting-data-from-files | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905022931 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | 905022931 | IC_kwDOCGYnMM418Y3T | 22429695 | 2021-08-24T22:38:38Z | 2021-08-24T23:27:26Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#321](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (db2dd6d) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/9258f4bd8450c951900de998a7bf81ca9b45a014?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (9258f4b) will **increase** coverage by `0.16%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #321 +/- ## ========================================== + Coverage 96.41% 96.58% +0.16% ========================================== Files 5 5 Lines 2206 2223 +17 ========================================== + Hits 2127 2147 +20 + Misses 79 76 -3 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/321/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.69% <100.00%> (+0.42%)` | :arrow_up: | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [9258f4b...db2dd6d](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978537855 | |
| https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905040902 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | 905040902 | IC_kwDOCGYnMM418dQG | 9599 | 2021-08-24T23:25:03Z | 2021-08-24T23:25:03Z | OWNER | I'm going to skip this test on windows. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978537855 | |
| https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905040307 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | 905040307 | IC_kwDOCGYnMM418dGz | 9599 | 2021-08-24T23:23:36Z | 2021-08-24T23:23:36Z | OWNER | https://discuss.python.org/t/pep-597-use-utf-8-for-default-text-file-encoding/1819 says: > Currently, `TextIOWrapper` uses `locale.getpreferredencoding(False)` (hereinafter called “locale encoding”) when encoding is not specified. > ... > Package authors using macOS or Linux may forget that the default encoding is not always UTF-8. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978537855 | |
| https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905039576 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | 905039576 | IC_kwDOCGYnMM418c7Y | 9599 | 2021-08-24T23:21:29Z | 2021-08-24T23:21:29Z | OWNER | Hah, the error here is actually: ``` > assert result.exit_code == 1, result.output E AssertionError: E E assert 0 == 1 E + where 0 = <Result okay>.exit_code ``` So I was expecting an error, but instead the command worked. I suspect this is because on Windows the default character set may not be UTF-8? | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978537855 | |
| https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905037323 | https://api.github.com/repos/simonw/sqlite-utils/issues/321 | 905037323 | IC_kwDOCGYnMM418cYL | 9599 | 2021-08-24T23:15:29Z | 2021-08-24T23:15:29Z | OWNER | Huh, tests are failing but only on Windows! | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978537855 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905024066 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905024066 | IC_kwDOCGYnMM418ZJC | 66709385 | 2021-08-24T22:41:39Z | 2021-08-24T22:41:39Z | NONE | I'm happy with this functionality left the way you describe. In my case the data is homogeneous but other cases would work just by being consistent on the encoding. Thanks a lot, Simon! | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905021933 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905021933 | IC_kwDOCGYnMM418Ynt | 9599 | 2021-08-24T22:36:04Z | 2021-08-24T22:36:04Z | OWNER | > Oh, I misread. Yes some files will not be valid UTF-8, I'd throw a warning and continue (not adding that file) but if you want to get more elaborate you could allow to define a policy on what to do. Not adding the file, index binary content or use a conversion policy like the ones available on Python's decode. I thought about supporting those different policies (with something like `--errors ignore`) but I feel like that's getting a little bit too deep into the weeds. Right now if you try to import an invalid file the behaviour is the same as for the `sqlite-utils insert` command (I added the same detailed error message): ``` Error: Could not read file '/Users/simon/Dropbox/Development/sqlite-utils/data.txt' as text 'utf-8' codec can't decode byte 0xe3 in position 83: invalid continuation byte The input you provided uses a character encoding other than utf-8. You can fix this by passing the --encoding= option with the encoding of the file. If you do not know the encoding, running 'file filename.csv' may tell you. It's often worth trying: --encoding=latin-1 ``` If someone has data that can't be translated to valid text using a known encoding, I'm happy leaving them to have to insert it into a `BLOB` column instead. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905021047 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905021047 | IC_kwDOCGYnMM418YZ3 | 9599 | 2021-08-24T22:33:48Z | 2021-08-24T22:33:48Z | OWNER | I had a few doubts about the design just now. Since `content_text` is supported as a special argument, an alternative way of handling the above would be: sqlite-utils insert-files /tmp/text.db files *.txt -c path -c content_text -c size This does exactly the same thing as just using `--text` and not specifying any columns, because the actual implementation of `--text` is as follows: https://github.com/simonw/sqlite-utils/blob/0c796cd945b146b7395ff5f553861400be503867/sqlite_utils/cli.py#L1851-L1855 But actually I think that's OK - ``--text`` is a useful shorthand that avoids you having to remember how to manually specify those columns with `-c`. So I'm going to leave the design as is. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905021010 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905021010 | IC_kwDOCGYnMM418YZS | 66709385 | 2021-08-24T22:33:42Z | 2021-08-24T22:33:42Z | NONE | Oh, I misread. Yes some files will not be valid UTF-8, I'd throw a warning and continue (not adding that file) but if you want to get more elaborate you could allow to define a policy on what to do. Not adding the file, index binary content or use a conversion policy like the ones available on Python's decode. From https://stackoverflow.com/questions/24616678/unicodedecodeerror-in-python-when-reading-a-file-how-to-ignore-the-error-and-ju : - 'ignore' ignores errors. Note that ignoring encoding errors can lead to data loss. - 'replace' causes a replacement marker (such as '?') to be inserted where there is malformed data. - 'surrogateescape' will represent any incorrect bytes as code points in the Unicode Private Use Area ranging from U+DC80 to U+DCFF. These private code points will then be turned back into the same bytes when the surrogateescape error handler is used when writing data. This is useful for processing files in an unknown encoding. - 'xmlcharrefreplace' is only supported when writing to a file. Characters not supported by the encoding are replaced with the appropriate XML character reference &#nnn;. - 'backslashreplace' (also only supported when writing) replaces unsupported characters with Python’s backslashed escape sequences. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905013183 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905013183 | IC_kwDOCGYnMM418We_ | 9599 | 2021-08-24T22:15:34Z | 2021-08-24T22:15:34Z | OWNER | Here's the error message I have working for invalid unicode: ``` sqlite-utils insert-files /tmp/text.db files *.txt --text [------------------------------------] 0% Error: Could not read file '/Users/simon/Dropbox/Development/sqlite-utils/data.txt' as text 'utf-8' codec can't decode byte 0xe3 in position 83: invalid continuation byte ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905013162 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905013162 | IC_kwDOCGYnMM418Weq | 9599 | 2021-08-24T22:15:31Z | 2021-08-24T22:15:31Z | OWNER | I'm going to assume utf-8 but allow `--encoding` to be used to specify something different, since that option is already supported by other commands. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905003381 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905003381 | IC_kwDOCGYnMM418UF1 | 66709385 | 2021-08-24T21:56:49Z | 2021-08-24T21:56:49Z | NONE | I was thinking that an approach could be making FILE_COLUMNS a generator (_get_file_columns(mode)) or you can just have a different set of columns (is there something else that makes sense to be changed on the text scenario?). About UTF-8 I was referring to the encoding to use when reading files. This can be difficult to auto-detect but I believe that UTF-8 is pretty much the standard for text files. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905001586 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 905001586 | IC_kwDOCGYnMM418Tpy | 9599 | 2021-08-24T21:52:50Z | 2021-08-24T21:52:50Z | OWNER | Will need to re-title this section of the documentation: https://sqlite-utils.datasette.io/en/3.16/cli.html#inserting-binary-data-from-files - "Inserting binary data from files" will become "Inserting data from files" I'm OK with keeping the default as `BLOB` but I could add a `--text` option which stores the content as text instead. If the text can't be stored as `utf-8` I'll probably raise an error. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/sqlite-utils/issues/319#issuecomment-904999850 | https://api.github.com/repos/simonw/sqlite-utils/issues/319 | 904999850 | IC_kwDOCGYnMM418TOq | 9599 | 2021-08-24T21:49:08Z | 2021-08-24T21:49:08Z | OWNER | This is a good idea. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
976399638 | |
| https://github.com/simonw/datasette/issues/859#issuecomment-904982056 | https://api.github.com/repos/simonw/datasette/issues/859 | 904982056 | IC_kwDOBm6k_c418O4o | 2670795 | 2021-08-24T21:15:04Z | 2021-08-24T21:15:30Z | CONTRIBUTOR | I'm running into issues with this as well. All other pages seem to work with lots of DBs except the home page, which absolutely tanks. Would be willing to put some work into this, if there's been any kind of progress on concepts on how this ought to work. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
642572841 | |
| https://github.com/simonw/datasette/issues/1446#issuecomment-904954530 | https://api.github.com/repos/simonw/datasette/issues/1446 | 904954530 | IC_kwDOBm6k_c418IKi | 9599 | 2021-08-24T20:32:47Z | 2021-08-24T20:32:47Z | OWNER | Pasting that CSS into the styles editor in the developer tools on https://latest.datasette.io/ has the desired effect: footer at the bottom of the window unless the page is too long, in which case the footer is at the bottom of the scroll. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978357984 | |
| https://github.com/simonw/datasette/issues/1446#issuecomment-904866495 | https://api.github.com/repos/simonw/datasette/issues/1446 | 904866495 | IC_kwDOBm6k_c417yq_ | 9599 | 2021-08-24T18:13:49Z | 2021-08-24T18:13:49Z | OWNER | OK, now the following optional CSS gives us a sticky footer: ```css html, body { height: 100%; } body { display: flex; flex-direction: column; } .not-footer { flex: 1 0 auto; } footer { flex-shrink: 0; } ``` | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978357984 | |
| https://github.com/dogsheep/healthkit-to-sqlite/pull/13#issuecomment-904642396 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/13 | 904642396 | IC_kwDOC8tyDs41679c | 32016596 | 2021-08-24T13:27:40Z | 2021-08-24T13:28:26Z | NONE | This would fix #21 and make #22 obsolete. | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
743071410 | |
| https://github.com/dogsheep/healthkit-to-sqlite/pull/22#issuecomment-904641261 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/22 | 904641261 | IC_kwDOC8tyDs4167rt | 32016596 | 2021-08-24T13:26:20Z | 2021-08-24T13:26:20Z | NONE | Did not see that #13 fixes the same issue in a similar way. You can decide which one to merge ;) | {
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
978086284 |