home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

34 rows where "created_at" is on date 2021-11-29 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, reactions, created_at (date), updated_at (date)

issue 8

  • Support STRICT tables 13
  • Test against pysqlite3 running SQLite 3.37 8
  • Way to test SQLite 3.37 (and potentially other versions) in CI 5
  • Research pattern for re-registering existing Click tools with register_commands 3
  • `table.strict` introspection boolean for identifying STRICT mode tables 2
  • Document how to send multiple values for "Named parameters" 1
  • Use datasette-table Web Component to guide the design of the JSON API for 1.0 1
  • Update aiofiles requirement from <0.8,>=0.4 to >=0.4,<0.9 1

user 3

  • simonw 30
  • 20after4 2
  • codecov[bot] 2

author_association 2

  • OWNER 30
  • NONE 4
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
982123183 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982123183 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46igKv codecov[bot] 22429695 2021-11-29T23:20:35Z 2021-12-11T01:02:19Z NONE

Codecov Report

Merging #347 (71b6c38) into main (213a0ff) will increase coverage by 0.00%. The diff coverage is 100.00%.

:exclamation: Current head 71b6c38 differs from pull request most recent head 1a7ef2f. Consider uploading reports for the commit 1a7ef2f to get more accurate results

```diff @@ Coverage Diff @@

main #347 +/-

======================================= Coverage 96.51% 96.52%
======================================= Files 5 5
Lines 2270 2271 +1
======================================= + Hits 2191 2192 +1
Misses 79 79
```

| Impacted Files | Coverage Δ | | |---|---|---| | sqlite_utils/cli.py | 95.73% <100.00%> (ø) | | | sqlite_utils/utils.py | 93.68% <100.00%> (+0.03%) | :arrow_up: |


Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 213a0ff...1a7ef2f. Read the comment docs.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982137888 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137888 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46ijwg simonw 9599 2021-11-29T23:50:54Z 2021-11-29T23:50:54Z OWNER

If I'm going to skipIf() those tests I need a way to check if pysqlite3 is being used.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982137293 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137293 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46ijnN simonw 9599 2021-11-29T23:49:29Z 2021-11-29T23:49:29Z OWNER

A short term fix would be to skip those tests against pysqlite3 - but longer term it would be good to address the underlying issue, particularly for the WAL ones (the FTS ones aren't too worrying since if you deliberately try and break the FTS table it's not hugely problematic if you corrupt your database).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982136747 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982136747 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46ijer simonw 9599 2021-11-29T23:48:05Z 2021-11-29T23:48:05Z OWNER

Some interesting test failures in the version that runs with pysqlite3: =========================== short test summary info ============================ FAILED tests/test_cli.py::test_enable_wal - assert 0 == 1 FAILED tests/test_cli.py::test_disable_wal - pysqlite3.dbapi2.OperationalErro... FAILED tests/test_fts.py::test_rebuild_fts[searchable] - pysqlite3.dbapi2.Dat... FAILED tests/test_fts.py::test_rebuild_fts[searchable_fts] - pysqlite3.dbapi2... FAILED tests/test_wal.py::test_enable_disable_wal - pysqlite3.dbapi2.Operatio... ================== 5 failed, 750 passed, 3 skipped in 15.20s =================== https://github.com/simonw/sqlite-utils/runs/4360759085

The WAL errors look like this: E pysqlite3.dbapi2.OperationalError: cannot change into wal mode from within a transaction Triggered by a call to db.enable_wal()

The FTS errors are caused by tests that try to deliberately corrupt the FTS index by running fresh_db["searchable_fts_data"].delete_where() - and then rebuilding it using rebuild_fts(): ``` @pytest.mark.parametrize("table_to_fix", ["searchable", "searchable_fts"]) def test_rebuild_fts(fresh_db, table_to_fix): table = fresh_db["searchable"] table.insert(search_records[0]) table.enable_fts(["text", "country"]) # Run a search rows = list(table.search("tanuki")) assert len(rows) == 1 assert { "rowid": 1, "text": "tanuki are running tricksters", "country": "Japan", "not_searchable": "foo", }.items() <= rows[0].items() # Delete from searchable_fts_data fresh_db["searchable_fts_data"].delete_where() # This should have broken the index with pytest.raises(sqlite3.DatabaseError): list(table.search("tanuki")) # Running rebuild_fts() should fix it

  fresh_db[table_to_fix].rebuild_fts()

tests/test_fts.py:277:


sqlite_utils/db.py:1947: in rebuild_fts self.db.execute(


self = <Database \<pysqlite3.dbapi2.Connection object at 0x7fd95d299c30>> sql = "INSERT INTO searchable_fts VALUES('rebuild');" parameters = None

def execute(
    self, sql: str, parameters: Optional[Union[Iterable, dict]] = None
) -> sqlite3.Cursor:
    "Execute SQL query and return a ``sqlite3.Cursor``."
    if self._tracer:
        self._tracer(sql, parameters)
    if parameters is not None:
        return self.conn.execute(sql, parameters)
    else:
      return self.conn.execute(sql)

E pysqlite3.dbapi2.DatabaseError: database disk image is malformed ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982133970 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982133970 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46iizS simonw 9599 2021-11-29T23:41:17Z 2021-11-29T23:41:17Z OWNER

Took a bit of experimenting to get both mypy AND flake8 to ignore the same line. The incantation that worked was this one:

https://github.com/simonw/sqlite-utils/blob/f990e134aa8219b687ff6c261330f36824b5df36/sqlite_utils/utils.py#L8

Order here matters - this did NOT work for both tools:

python from sqlite3.dump import _iterdump as iterdump # noqa: F401 # type: ignore

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982126665 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982126665 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46ihBJ simonw 9599 2021-11-29T23:26:01Z 2021-11-29T23:33:48Z OWNER

https://github.com/simonw/sqlite-utils/blob/93b059dd230eae9eaae472b7fbabd4a66feeb79d/.github/workflows/test.yml#L11-L20

This configuration means that the numpy=0, Python=3.10, os=Ubuntu build will additionally use pysqlite3 with the SQLite 3.37.0.

It's failing right now: https://github.com/simonw/sqlite-utils/runs/4360593156 - because pysqlite3 doesn't provide .iterdump().

I can use the workaround from this comment: https://github.com/coleifer/pysqlite3/issues/24#issuecomment-982081267

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982129727 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129727 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46ihw_ simonw 9599 2021-11-29T23:31:58Z 2021-11-29T23:31:58Z OWNER

It failed on other Python versions with mypy: sqlite_utils/utils.py:8: error: Cannot find implementation or library stub for module named "sqlite3.dump" sqlite_utils/utils.py:8: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982129218 https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129218 https://api.github.com/repos/simonw/sqlite-utils/issues/347 IC_kwDOCGYnMM46ihpC simonw 9599 2021-11-29T23:31:02Z 2021-11-29T23:31:02Z OWNER

Here's the test run that's installing pysqlite3 and that version of SQLite: https://github.com/simonw/sqlite-utils/runs/4360663292

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Test against pysqlite3 running SQLite 3.37 1066603133  
982111751 https://github.com/simonw/sqlite-utils/issues/346#issuecomment-982111751 https://api.github.com/repos/simonw/sqlite-utils/issues/346 IC_kwDOCGYnMM46idYH simonw 9599 2021-11-29T23:11:17Z 2021-11-29T23:12:49Z OWNER

To keep things simple for the moment I'm only going to add one extra thing to the matrix, and it will be a run of the tests against SQLite 3.37.0 using pysqlite3 on Linux only.

I can use this mechanism: https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-including-new-combinations

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Way to test SQLite 3.37 (and potentially other versions) in CI 1066563554  
982094370 https://github.com/simonw/sqlite-utils/issues/346#issuecomment-982094370 https://api.github.com/repos/simonw/sqlite-utils/issues/346 IC_kwDOCGYnMM46iZIi simonw 9599 2021-11-29T22:50:49Z 2021-11-29T22:50:49Z OWNER

I have a working recipe for compiling it for macOS here: https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982006544

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Way to test SQLite 3.37 (and potentially other versions) in CI 1066563554  
982094020 https://github.com/simonw/sqlite-utils/issues/346#issuecomment-982094020 https://api.github.com/repos/simonw/sqlite-utils/issues/346 IC_kwDOCGYnMM46iZDE simonw 9599 2021-11-29T22:50:11Z 2021-11-29T22:50:11Z OWNER

For the moment I think I'll combine two problems into one, and just add a single matrix alternative that uses pysqlite3 running SQLite 3.37.0 - only on macOS and Linux so I don't have to figure out how to compile it for Windows.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Way to test SQLite 3.37 (and potentially other versions) in CI 1066563554  
982091363 https://github.com/simonw/sqlite-utils/issues/345#issuecomment-982091363 https://api.github.com/repos/simonw/sqlite-utils/issues/345 IC_kwDOCGYnMM46iYZj simonw 9599 2021-11-29T22:45:26Z 2021-11-29T22:45:26Z OWNER

This is the implementation: https://github.com/simonw/sqlite-utils/blob/213a0ff177f23a35f3b235386366ff132eb879f1/sqlite_utils/db.py#L1236-L1241

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.strict` introspection boolean for identifying STRICT mode tables 1066501534  
982090895 https://github.com/simonw/sqlite-utils/issues/345#issuecomment-982090895 https://api.github.com/repos/simonw/sqlite-utils/issues/345 IC_kwDOCGYnMM46iYSP simonw 9599 2021-11-29T22:44:36Z 2021-11-29T22:44:36Z OWNER

Documented here: https://sqlite-utils.datasette.io/en/latest/python-api.html#strict

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.strict` introspection boolean for identifying STRICT mode tables 1066501534  
982078527 https://github.com/simonw/sqlite-utils/issues/346#issuecomment-982078527 https://api.github.com/repos/simonw/sqlite-utils/issues/346 IC_kwDOCGYnMM46iVQ_ simonw 9599 2021-11-29T22:23:03Z 2021-11-29T22:23:03Z OWNER

Here's a modified version of the dump command that works even with pysqlite3:

```python @cli.command() @click.argument( "path", type=click.Path(exists=True, file_okay=True, dir_okay=False, allow_dash=False), required=True, ) @load_extension_option def dump(path, load_extension): """Output a SQL dump of the schema and full contents of the database""" db = sqlite_utils.Database(path) _load_extensions(db, load_extension) # pysqlite3 doesn't implement .iterdump() from sqlite3.dump import _iterdump

for line in _iterdump(db.conn):
    click.echo(line)

```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Way to test SQLite 3.37 (and potentially other versions) in CI 1066563554  
982077873 https://github.com/simonw/sqlite-utils/issues/346#issuecomment-982077873 https://api.github.com/repos/simonw/sqlite-utils/issues/346 IC_kwDOCGYnMM46iVGx simonw 9599 2021-11-29T22:22:05Z 2021-11-29T22:22:05Z OWNER

Ideally I'd like an extra set of matrix options for different versions of SQLite.

I can use pysqlite3 for this, but it isn't a completely compatible drop-in replacement - turns out it doesn't support the iterdump() method for example, see https://github.com/coleifer/pysqlite3/issues/24

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Way to test SQLite 3.37 (and potentially other versions) in CI 1066563554  
982076924 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982076924 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iU38 simonw 9599 2021-11-29T22:20:44Z 2021-11-29T22:20:44Z OWNER

Need to figure out a good pattern for testing this in CI too - it will currently skip the new tests if it doesn't have SQLite 3.37 or higher.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982076702 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982076702 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iU0e simonw 9599 2021-11-29T22:20:22Z 2021-11-29T22:20:22Z OWNER

I haven't documented db.supports_strict yet (I documented table.strict) because there wasn't an obvious section of the documentation for it.

I need to remember to document it once I add documentation for the strict=True parameter.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982049148 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982049148 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iOF8 simonw 9599 2021-11-29T21:40:59Z 2021-11-29T21:40:59Z OWNER

I'm going to add that as db.supports_strict.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982048918 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982048918 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iOCW simonw 9599 2021-11-29T21:40:42Z 2021-11-29T21:40:42Z OWNER

Here's a function that detects if strict is supported or not: ```python import secrets import sqlite3

def supports_strict_tables(db = None): if db is None: db = sqlite3.connect(":memory:") try: table_name = 't{}'.format(secrets.token_hex(16)) with db: db.execute("create table {} (name text) strict".format(table_name)) db.execute("drop table {}".format(table_name)) return True except: return False ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982026918 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982026918 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iIqm simonw 9599 2021-11-29T21:11:42Z 2021-11-29T21:16:31Z OWNER

Made myself a test strict table like so:

```pycon

import pysqlite3 conn = pysqlite3.connect("/tmp/strict-table.db") conn.execute("create table foo (id integer primary key, name text, weight real) strict") <pysqlite3.dbapi2.Cursor object at 0x104317340> cursor = conn.cursor() with conn: ... cursor.execute("insert into foo (name, weight) values ('Lila', '2.31')") <pysqlite3.dbapi2.Cursor object at 0x10331e1f0> conn.close() ``` Uploaded that to: https://static.simonwillison.net/static/2021/strict-table.db

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982020757 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982020757 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iHKV simonw 9599 2021-11-29T21:03:34Z 2021-11-29T21:03:34Z OWNER

From the STRICT docs:

The SQLite parser accepts a comma-separated list of table options after the final close parenthesis in a CREATE TABLE statement. As of this writing (2021-08-23) only two options are recognized:

  • STRICT
  • WITHOUT ROWID

So I think I need to read the CREATE TABLE statement from the sqlite_master table, split on the last ), split those tokens on , and see if create is in there (case insensitive).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982018304 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982018304 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iGkA simonw 9599 2021-11-29T21:00:02Z 2021-11-29T21:00:02Z OWNER

Is there a need for an introspection function for telling if a SQLite table is in strict mode or not? How would that work?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982017994 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982017994 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iGfK simonw 9599 2021-11-29T20:59:37Z 2021-11-29T20:59:37Z OWNER

I'm leaning towards silently ignore if SQLite version doesn't support it. That means that the first time you attempt to use strict=True we will need to run a test against the database connection to see what version of SQLite it uses, then cache the result to avoid making the same call again for the same connection.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982016594 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982016594 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iGJS simonw 9599 2021-11-29T20:57:42Z 2021-11-29T20:57:42Z OWNER

What should happen if you attempt to use strict=True against a SQLite version prior to 3.37.0?

An obvious error would be best... but how about silently ignoring it on older versions instead? That would match how we handle deterministic=True for registering functions:

https://github.com/simonw/sqlite-utils/blob/126703706ea153f63e6134ad14e5712e4bbcb8ae/sqlite_utils/db.py#L372-L380

https://github.com/simonw/sqlite-utils/blob/93c7fd9868fed3193a1732b39bfac539e5812b0b/tests/test_register_function.py#L34-L37

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982014776 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iFs4 simonw 9599 2021-11-29T20:55:19Z 2021-11-29T20:55:19Z OWNER

There are a few places that the strict=True option could go:

  • table.create() and table.create_table_sql()
  • The Database() constructor, to turn it on for all created tables
  • The .insert() / .insert_all() etc family of methods that can implicitly create tables

I'll definitely implement the first one, and likely the second one too. I'm on the fence with regards to the third one.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
982006544 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982006544 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iDsQ simonw 9599 2021-11-29T20:44:37Z 2021-11-29T20:48:43Z OWNER

This worked: cd /tmp mkdir sqlite-3.37 cd sqlite-3.37 wget 'https://www.sqlite.org/2021/sqlite-amalgamation-3370000.zip' unzip sqlite-amalgamation-3370000.zip git clone https://github.com/coleifer/pysqlite3/ cp sqlite-amalgamation-3370000/sqlite3.[ch] pysqlite3 cd pysqlite3 python3 setup.py build_static build bdist_wheel This gave me a file here:

pysqlite3 % ls -l dist total 1872 -rw-r--r-- 1 simon wheel 956557 Nov 29 12:38 pysqlite3-0.4.6-cp39-cp39-macosx_10_15_x86_64.whl That wheel only works when installed for Python 3.9 (it failed to install in a Python 3.10 virtual environment) - but pip install /tmp/sqlite-3.37/pysqlite3/dist/pysqlite3-0.4.6-cp39-cp39-macosx_10_15_x86_64.whl gave me a working pysqlite3 - and the following worked:

```pycon

import pysqlite3 pysqlite3.connect(":memory:").execute("select sqlite_version()").fetchall() [('3.37.0',)] And if I install `sqlite-utils` in the same virtual environment this works: % sqlite-utils memory 'select sqlite_version()' [{"sqlite_version()": "3.37.0"}] ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
981999025 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-981999025 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iB2x simonw 9599 2021-11-29T20:34:38Z 2021-11-29T20:35:58Z OWNER

I'm going to build my own pysqlite3 wheel with the latest SQLite to try this out, following the instructions on https://github.com/coleifer/pysqlite3

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
981997973 https://github.com/simonw/sqlite-utils/issues/344#issuecomment-981997973 https://api.github.com/repos/simonw/sqlite-utils/issues/344 IC_kwDOCGYnMM46iBmV simonw 9599 2021-11-29T20:33:52Z 2021-11-29T20:33:52Z OWNER

From that page:

If you try to open a database containing the STRICT keyword in an earlier version of SQLite, it will not recognize the keyword and will report an error (except as noted below.

[...]

Because of a quirk in the SQL language parser, versions of SQLite prior to 3.37.0 can still read and write STRICT tables if they set "PRAGMA writable_schema=ON" immediately after opening the database file, prior to doing anything else that requires knowing the schema.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support STRICT tables 1066474200  
981980048 https://github.com/simonw/datasette/issues/1304#issuecomment-981980048 https://api.github.com/repos/simonw/datasette/issues/1304 IC_kwDOBm6k_c46h9OQ 20after4 30934 2021-11-29T20:13:53Z 2021-11-29T20:14:11Z NONE

There isn't any way to do this with sqlite as far as I know. The only option is to insert the right number of ? placeholders into the sql template and then provide an array of values.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Document how to send multiple values for "Named parameters"  863884805  
981966693 https://github.com/simonw/datasette/issues/1532#issuecomment-981966693 https://api.github.com/repos/simonw/datasette/issues/1532 IC_kwDOBm6k_c46h59l 20after4 30934 2021-11-29T19:56:52Z 2021-11-29T19:56:52Z NONE

FWIW I've written some web components that consume the json api and I think it's a really nice way to work with datasette. I like the combination with datasette+sqlite as a back-end feeding data to a front-end that's entirely javascript + html.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use datasette-table Web Component to guide the design of the JSON API for 1.0 1065429936  
981856895 https://github.com/simonw/datasette/issues/1538#issuecomment-981856895 https://api.github.com/repos/simonw/datasette/issues/1538 IC_kwDOBm6k_c46hfJ_ simonw 9599 2021-11-29T17:32:44Z 2021-11-29T17:32:44Z OWNER

TIL: https://til.simonwillison.net/datasette/reuse-click-for-register-commands

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Research pattern for re-registering existing Click tools with register_commands 1066288689  
981852280 https://github.com/simonw/datasette/issues/1538#issuecomment-981852280 https://api.github.com/repos/simonw/datasette/issues/1538 IC_kwDOBm6k_c46heB4 simonw 9599 2021-11-29T17:27:12Z 2021-11-29T17:27:12Z OWNER

Thanks to https://stackoverflow.com/a/45514541/6083 I found the right pattern: ```python from datasette import hookimpl from git_history.cli import cli as git_history_cli

@hookimpl def register_commands(cli): cli.add_command(git_history_cli, name="git-history") ``` I think this is a little bit too obscure to add to the Datasette documentation - I'll turn it into a TIL instead.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Research pattern for re-registering existing Click tools with register_commands 1066288689  
981849494 https://github.com/simonw/datasette/issues/1538#issuecomment-981849494 https://api.github.com/repos/simonw/datasette/issues/1538 IC_kwDOBm6k_c46hdWW simonw 9599 2021-11-29T17:23:52Z 2021-11-29T17:23:52Z OWNER

Just trying to use git_history_file produces this error:

TypeError: Attempted to convert a callback into a command twice.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Research pattern for re-registering existing Click tools with register_commands 1066288689  
981631026 https://github.com/simonw/datasette/pull/1537#issuecomment-981631026 https://api.github.com/repos/simonw/datasette/issues/1537 IC_kwDOBm6k_c46goAy codecov[bot] 22429695 2021-11-29T13:23:20Z 2021-11-29T13:23:20Z NONE

Codecov Report

Merging #1537 (fcfaec1) into main (48f1199) will decrease coverage by 0.06%. The diff coverage is n/a.

```diff @@ Coverage Diff @@

main #1537 +/-

========================================== - Coverage 91.90% 91.83% -0.07%
========================================== Files 34 34
Lines 4434 4434
========================================== - Hits 4075 4072 -3
- Misses 359 362 +3
```

| Impacted Files | Coverage Δ | | |---|---|---| | datasette/views/index.py | 96.42% <0.00%> (-1.79%) | :arrow_down: | | datasette/database.py | 92.93% <0.00%> (-0.75%) | :arrow_down: |


Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 48f1199...fcfaec1. Read the comment docs.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Update aiofiles requirement from <0.8,>=0.4 to >=0.4,<0.9 1066023866  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 1697.116ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows