home / github

Menu
  • Search all tables
  • GraphQL API

issues

Table actions
  • GraphQL API for issues

1,939 rows where user = 9599 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: author_association, draft, state_reason, created_at (date), updated_at (date), closed_at (date)

repo 14

  • datasette 1,372
  • sqlite-utils 361
  • github-to-sqlite 55
  • twitter-to-sqlite 46
  • dogsheep-beta 32
  • dogsheep-photos 27
  • healthkit-to-sqlite 12
  • apple-notes-to-sqlite 10
  • swarm-to-sqlite 7
  • evernote-to-sqlite 7
  • pocket-to-sqlite 5
  • hacker-news-to-sqlite 3
  • google-takeout-to-sqlite 1
  • genome-to-sqlite 1

type 2

  • issue 1,750
  • pull 189

state 2

  • closed 1,514
  • open 425
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1633077183 I_kwDOBm6k_c5hVse_ 2041 Remove obsolete table POST code simonw 9599 open 0   Datasette 1.0a3 8755003 1 2023-03-21T01:01:40Z 2023-03-21T01:02:13Z   OWNER  

Spotted this in: - #1999

POST /db/table currently executes obsolete code for inserting a row - I replaced that with /db/table/-/insert in https://github.com/simonw/datasette/commit/6e788b49edf4f842c0817f006eb9d865778eea5e but forgot to remove the old code.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2041/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1551694938 PR_kwDOBm6k_c5IQeKz 1999 ?_extra= support (draft) simonw 9599 open 0     42 2023-01-21T04:55:18Z 2023-03-21T00:07:53Z   OWNER simonw/datasette/pulls/1999

Refs: - #262


:books: Documentation preview :books:: https://datasette--1999.org.readthedocs.build/en/1999/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1999/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
1620516340 I_kwDOCGYnMM5glx30 533 ReadTheDocs error: not all arguments converted during string formatting simonw 9599 closed 0     2 2023-03-12T21:21:05Z 2023-03-12T21:25:33Z 2023-03-12T21:25:33Z OWNER  

This came up as a failure running tests for: - #531

Traceback on https://readthedocs.org/projects/sqlite-utils/builds/19749348/

``` File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 889, in interpreted nodes, messages2 = role_fn(role, rawsource, text, lineno, self) File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py", line 103, in role title = caption % part TypeError: not all arguments converted during string formatting

Exception occurred: File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py", line 103, in role title = caption % part TypeError: not all arguments converted during string formatting ```

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/533/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
317001500 MDU6SXNzdWUzMTcwMDE1MDA= 236 datasette publish lambda plugin simonw 9599 open 0     11 2018-04-23T22:10:30Z 2023-03-12T14:04:15Z   OWNER  

Refs #217 - create a publish plugin that can deploy to AWS Lambda.

https://docs.aws.amazon.com/lambda/latest/dg/limits.html says lambda packages can be up to 50 MB, so this would only work with smaller databases (the command can check the filesize before attempting to package and deploy it).

Lambdas do get a 512 MB /tmp directory too, so for larger databases the function could start and then download up to 512MB from an S3 bucket - so the plugin could take an optional S3 bucket to write to and know how to upload the .db file there and then have the lambda download it on startup.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/236/reactions",
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1618249044 I_kwDOBm6k_c5gdIVU 2038 Consider a `strict_templates` setting simonw 9599 open 0     2 2023-03-10T02:09:13Z 2023-03-10T02:11:06Z   OWNER  

A setting which turns on Jinja strict mode, so any templates that access undefined variables raise a hard error.

Prototype here: diff diff --git a/datasette/app.py b/datasette/app.py index 40416713..1428a3f0 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -200,6 +200,7 @@ SETTINGS = ( "Allow display of SQL trace debug information with ?_trace=1", ), Setting("base_url", "/", "Datasette URLs should use this base path"), + Setting("strict_templates", False, "Raise errors for undefined template variables"), ) _HASH_URLS_REMOVED = "The hash_urls setting has been removed, try the datasette-hashed-urls plugin instead" OBSOLETE_SETTINGS = { @@ -399,11 +400,14 @@ class Datasette: ), ] ) + env_extras = {} + if self.setting("strict_templates"): + env_extras["undefined"] = StrictUndefined self.jinja_env = Environment( loader=template_loader, autoescape=True, enable_async=True, - undefined=StrictUndefined, + **env_extras, ) self.jinja_env.filters["escape_css_string"] = escape_css_string self.jinja_env.filters["quote_plus"] = urllib.parse.quote_plus Explored this idea a bit in: - #1999

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2038/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1618130434 I_kwDOJHON9s5gcrYC 11 Implement a SQL view to make it easier to query files in a nested folder simonw 9599 open 0     3 2023-03-09T23:19:28Z 2023-03-09T23:24:01Z   MEMBER  

Working with nested data in SQL is tricky, can I make it easier with a view or canned query?

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/11/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1615891776 I_kwDOBm6k_c5gUI1A 2037 Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError simonw 9599 closed 0     3 2023-03-08T20:30:06Z 2023-03-09T22:33:39Z 2023-03-09T22:33:39Z OWNER  

FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError: [Errno 2] No such file or directory

From https://github.com/simonw/datasette/actions/runs/4348548218/jobs/7597208191

``` =================================== FAILURES =================================== ____ test_install_requirements _______

run_module = <MagicMock name='run_module' id='139768358191936'>

@mock.patch("datasette.cli.run_module")
def test_install_requirements(run_module):
    runner = CliRunner()
  with runner.isolated_filesystem():

/home/runner/work/datasette/datasette/tests/test_cli.py:184:


/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/contextlib.py:119: in enter return next(self.gen)


self = <click.testing.CliRunner object at 0x7f1e5bfb9490>, temp_dir = None

@contextlib.contextmanager
def isolated_filesystem(
    self, temp_dir: t.Optional[t.Union[str, os.PathLike]] = None
) -> t.Iterator[str]:
    """A context manager that creates a temporary directory and
    changes the current working directory to it. This isolates tests
    that affect the contents of the CWD to prevent them from
    interfering with each other.

    :param temp_dir: Create the temporary directory under this
        directory. If given, the created directory is not removed
        when exiting.

    .. versionchanged:: 8.0
        Added the ``temp_dir`` parameter.
    """
  cwd = os.getcwd()

E FileNotFoundError: [Errno 2] No such file or directory

/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/testing.py:466: FileNotFoundError ``` Not sure why it only affected the "Calculate test coverage" one.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2037/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1617769847 I_kwDOJHON9s5gbTV3 7 Folder support simonw 9599 closed 0     6 2023-03-09T18:21:33Z 2023-03-09T20:48:18Z 2023-03-09T20:48:18Z MEMBER  

Notes can live in folders. These relationships should be exported too.

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1617962395 I_kwDOJHON9s5gcCWb 10 Include schema in README simonw 9599 closed 0     0 2023-03-09T20:38:59Z 2023-03-09T20:48:18Z 2023-03-09T20:48:18Z MEMBER  

As seen in other tools like https://github.com/simonw/git-history

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/10/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1617938730 I_kwDOJHON9s5gb8kq 9 Default to just storing plaintext, store HTML if `--html` is passed simonw 9599 open 0     0 2023-03-09T20:19:06Z 2023-03-09T20:19:06Z   MEMBER  

The full body version of the notes can get HUGE, due to embedded images. It turns out for my own purposes I'm usually happy with just the plaintext version.

I'm tempted to say you don't get HTML unless you pass a --html option.

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/9/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1616429236 I_kwDOJHON9s5gWMC0 4 Support incremental updates simonw 9599 open 0     2 2023-03-09T05:14:00Z 2023-03-09T18:20:56Z   MEMBER  

Running this script can take several hours against a large notes database.

Would be neat if it could run against just the notes that have been modified since it last ran. Could pull the max updated date and then keep on looping until it finds one modified before then.

Problem is I don't actually know what order it iterates over the notes in.

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/4/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1617602868 I_kwDOJHON9s5gaqk0 6 Character encoding problem simonw 9599 open 0     0 2023-03-09T16:44:34Z 2023-03-09T16:44:34Z   MEMBER  

I ran against a recent note with this in it:

Or just "Actions ⚙️ "

And got back:

Actions ⚙️

Pasting that into https://ftfy.vercel.app/?s=Actions+%E2%80%9A%C3%B6%C3%B4%C3%94%E2%88%8F%C3%A8+ gives this:

python s = 'Actions â\x80\x9aöôÃ\x94â\x88\x8fè' s = s.encode('latin-1') s = s.decode('utf-8') s = s.encode('macroman') s = s.decode('utf-8') print(s)

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/6/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1616440856 I_kwDOJHON9s5gWO4Y 5 Configure full text search simonw 9599 open 0     0 2023-03-09T05:20:46Z 2023-03-09T05:20:46Z   MEMBER  

FTS would be useful.

Maybe even extract the plain text from the notes to make that index easier to create, rather than creating it against the HTML. Can use the plaintext property for that.

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/5/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1616354999 I_kwDOJHON9s5gV563 2 First working version simonw 9599 closed 0     7 2023-03-09T03:53:00Z 2023-03-09T05:10:22Z 2023-03-09T05:10:22Z MEMBER  

It's going to shell out to osascript as seen in: - #1

I'm going with that option because https://appscript.sourceforge.io/status.html warns against the other potential methods:

Apple eliminated its Mac Automation department in 2016. The future of AppleScript and its related technologies is unclear. Caveat emptor.

But osascript looks pretty stable to me.

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1616422013 I_kwDOJHON9s5gWKR9 3 `apple-notes-to-sqlite --dump` option simonw 9599 closed 0     0 2023-03-09T05:05:49Z 2023-03-09T05:06:14Z 2023-03-09T05:06:14Z MEMBER  

Option that doesn't write to the database at all, it just outputs all the notes to stdout as newline-delimited JSON.

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/3/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1616347574 I_kwDOJHON9s5gV4G2 1 Initial proof of concept with ChatGPT simonw 9599 closed 0     3 2023-03-09T03:44:39Z 2023-03-09T03:51:55Z 2023-03-09T03:51:55Z MEMBER  

I'm using ChatGPT to figure out enough AppleScript to get at my notes data.

apple-notes-to-sqlite 611552758 issue    
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/1/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1615862295 I_kwDOBm6k_c5gUBoX 2036 `publish cloudrun` reuses image tags, which can lead to very surprising deploy problems simonw 9599 closed 0     6 2023-03-08T20:11:44Z 2023-03-08T20:57:34Z 2023-03-08T20:57:34Z OWNER  

See this issue: - https://github.com/simonw/datasette.io/issues/141

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2036/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1615692818 I_kwDOBm6k_c5gTYQS 2035 Potential feature: special support for `?a=1&a=2` on the query page simonw 9599 open 0   Datasette 1.0 3268330 13 2023-03-08T18:05:03Z 2023-03-08T20:14:47Z   OWNER  

From a discussion on Discord: https://discord.com/channels/823971286308356157/996877076982415491/1082789517062320138

The key idea is to make it easier for people to implement where id in (...) that's populated from query string arguments.

What if you could add ?id=11&id=32&id=62 to the URL and have that made available as a list that can be used in the query?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2035/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1612296210 I_kwDOBm6k_c5gGbAS 2033 `datasette install -r requirements.txt` simonw 9599 closed 0     2 2023-03-06T22:17:17Z 2023-03-06T22:54:52Z 2023-03-06T22:27:34Z OWNER  

Would be useful for cases where you want to install a whole set of plugins in one go, e.g. when running tutorials in GitHub Codespaces.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2033/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1121583414 I_kwDOBm6k_c5C2gE2 1619 JSON link on row page is 404 if base_url setting is used simonw 9599 open 0     4 2022-02-02T07:09:53Z 2023-03-05T20:30:14Z   OWNER  

On my local environment:

datasette fixtures.db -p 3344 --setting base_url /foo/bar/

Then hit http://127.0.0.1:3344/foo/bar/fixtures/table%2Fwith%2Fslashes.csv/3

But... that json link goes here, which is a 404:

http://127.0.0.1:3344/foo/bar/foo/bar/fixtures/table%2Fwith%2Fslashes.csv/3?_format=json

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1619/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1579973223 I_kwDOBm6k_c5eLHpn 2024 Mention WAL mode in documentation simonw 9599 open 0     1 2023-02-10T16:11:10Z 2023-02-10T16:11:53Z   OWNER  

It's not currently obvious from the docs how you can ensure that Datasette runs well in situations where other processes may update the underlying SQLite files.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2024/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
323658641 MDU6SXNzdWUzMjM2NTg2NDE= 262 Add ?_extra= mechanism for requesting extra properties in JSON simonw 9599 open 0   Datasette 1.0 3268330 25 2018-05-16T14:55:42Z 2023-02-08T18:36:48Z   OWNER  

Datasette views currently work by creating a set of data that should be returned as JSON, then defining an additional, optional template_data() function which is called if the view is being rendered as HTML.

This template_data() function calculates extra template context variables which are necessary for the HTML view but should not be included in the JSON.

Example of how that is used today: https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/table.py#L672-L704

With features like Facets in #255 I'm beginning to want to move more items into the template_data() - in the case of facets it's the suggested_facets array. This saves that feature from being calculated (involving several SQL queries) for the JSON case where it is unlikely to be used.

But... as an API user, I want to still optionally be able to access that information.

Solution: Add a ?_extra=suggested_facets&_extra=table_metadata argument which can be used to optionally request additional blocks to be added to the JSON API.

Then redefine as many of the current template_data() features as extra arguments instead, and teach Datasette to return certain extras by default when rendering templates.

This could allow the JSON representation to be slimmed down further (removing e.g. the table_definition and view_definition keys) while still making that information available to API users who need it.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/262/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1573424830 I_kwDOBm6k_c5dyI6- 2019 Refactor out the keyset pagination code simonw 9599 open 0     14 2023-02-06T23:04:00Z 2023-02-08T01:40:46Z   OWNER  

While working on: - #1999

I noticed that some of the most complex code in the existing table view is the code that implements keyset pagination:

https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L417-L493

Extracting that into a utility function would simplify that code a lot.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2019/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1516644980 I_kwDOCGYnMM5aZip0 520 rows_from_file() raises confusing error if file-like object is not in binary mode simonw 9599 open 0     2 2023-01-02T19:00:14Z 2023-02-07T22:43:10Z   OWNER  

I got this error:

File "/Users/simon/Dropbox/Development/openai-to-sqlite/openai_to_sqlite/cli.py", line 27, in embeddings rows, _ = rows_from_file(input) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/simon/.local/share/virtualenvs/openai-to-sqlite-jt4obeb2/lib/python3.11/site-packages/sqlite_utils/utils.py", line 305, in rows_from_file first_bytes = buffered.peek(2048).strip() ^^^^^^^^^^^^^^^^^^^ From this code: ```python

@cli.command() @click.argument( "db_path", type=click.Path(file_okay=True, dir_okay=False, allow_dash=False), ) @click.option( "-i", "--input", type=click.File("r"), default="-", ) def embeddings(db_path, input): "Store embeddings for one or more text documents" click.echo("Here is some output") db = sqlite_utils.Database(db_path) rows, _ = rows_from_file(input) print(list(rows)) `` The error went away when I changed it totype=click.File("rb")`.

This should either be called out in the documentation or rows_from_file() should be fixed to handle text-mode files in addition to binary files.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/520/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
473288428 MDExOlB1bGxSZXF1ZXN0MzAxNDgzNjEz 564 First proof-of-concept of Datasette Library simonw 9599 open 0     1 2019-07-26T10:22:26Z 2023-02-07T15:14:11Z   OWNER simonw/datasette/pulls/564

Refs #417. Run it like this:

 datasette -d ~/Library

Uses a new plugin hook - available_databases()

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/564/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
1570375808 I_kwDODFdgUs5dmgiA 79 Deploy demo job is failing simonw 9599 open 0     0 2023-02-03T20:05:01Z 2023-02-03T20:05:01Z   MEMBER  

https://github.com/dogsheep/github-to-sqlite/actions/runs/4080058087/jobs/7032116511

github-to-sqlite 207052882 issue    
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/79/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1564774831 I_kwDOBm6k_c5dRJGv 2012 Missing space in database summary simonw 9599 open 0     0 2023-01-31T18:01:13Z 2023-01-31T18:01:13Z   OWNER  

Spotted this on an instance index page:

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2012/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1564769997 I_kwDOBm6k_c5dRH7N 2011 Applied facet did not result in an "x" icon to dismiss it simonw 9599 open 0     1 2023-01-31T17:57:44Z 2023-01-31T17:58:54Z   OWNER  

That's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata

It's for Contract Type of Non-Purchasing Contract (Rents, etc.) - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2011/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1563264257 I_kwDOBm6k_c5dLYUB 2010 Row page should default to card view simonw 9599 open 0   Datasette 1.0 3268330 1 2023-01-30T21:49:37Z 2023-01-30T21:52:06Z   OWNER  

Datasette currently uses the same table layout on the row pages as it does on the table pages:

https://datasette.io/content/pypi_packages?_sort=name&name__exact=datasette-column-inspect

https://datasette.io/content/pypi_packages/datasette-column-inspect

If you shrink down to mobile width you get this instead, on both of those pages:

I think that view, which I think of as the "card view", is plain better if you're looking at just a single row - and it (or a variant of it) should be the default presentation on the row page.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2010/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1186696202 I_kwDOBm6k_c5Gu4wK 1696 Show foreign key label when filtering simonw 9599 open 0     2 2022-03-30T16:18:54Z 2023-01-29T20:56:20Z   OWNER  

For example here:

3 corresponds to "Human Related: Other" - it would be neat to display this in this area of the page somehow.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1696/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1560662739 I_kwDOBm6k_c5dBdLT 2007 `render_cell()` hook should take an optional `request` argument simonw 9599 closed 0     0 2023-01-28T03:13:00Z 2023-01-28T03:34:26Z 2023-01-28T03:34:26Z OWNER  

From Discord: https://discordapp.com/channels/823971286308356157/996877076982415491/1068227071156965486

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2007/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1558644003 I_kwDOBm6k_c5c5wUj 2006 Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release simonw 9599 open 0   Datasette 1.0 3268330 2 2023-01-26T19:17:40Z 2023-01-26T19:20:53Z   OWNER  

I just realized that when I ship Datasette 1.0 there may be automated deployments out there which could deploy the 1.0 version by accident, potentially breaking any customizations that aren't compatible with the 1.0 changes.

I can hopefully help avoid that by shipping one last entry in the 0.x series that ensures datasette publish pins to <1.0 when it installs Datasette itself.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2006/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1557507274 I_kwDOBm6k_c5c1azK 2005 `extra_template_vars` should be OK to return `None` simonw 9599 open 0     1 2023-01-26T01:40:45Z 2023-01-26T01:41:50Z   OWNER  

Got this exception and had to make sure it always returned {}:

File ".../python3.11/site-packages/datasette/app.py", line 1049, in render_template assert isinstance(extra_vars, dict), "extra_vars is of type {}".format( AssertionError: extra_vars is of type <class 'NoneType'>

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2005/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
743371103 MDU6SXNzdWU3NDMzNzExMDM= 1099 Support linking to compound foreign keys simonw 9599 open 0     6 2020-11-15T23:23:17Z 2023-01-25T00:58:26Z   OWNER  

Reported as a bug in #1098 because they caused 500 errors - but it would be even better if Datasette could hyperlink to related rows via compound foreign keys.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1099/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1554032168 I_kwDOBm6k_c5coKYo 2002 Document how actors are displayed simonw 9599 open 0     0 2023-01-24T00:08:49Z 2023-01-24T00:08:49Z   OWNER  

https://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/utils/init.py#L1052-L1056

This logic should be reflected in the documentation on https://docs.datasette.io/en/stable/authentication.html#actors

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2002/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
749283032 MDU6SXNzdWU3NDkyODMwMzI= 1101 register_output_renderer() should support streaming data simonw 9599 open 0   Datasette 1.0 3268330 13 2020-11-24T02:17:09Z 2023-01-21T22:07:19Z   OWNER  

I'd like to implement this by first extending the register_output_renderer() hook to support streaming huge responses, then switching CSV to use the plugin hook in addition to TSV using it.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1096#issuecomment-732542285

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1101/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1551113681 I_kwDOBm6k_c5cdB3R 1998 `datasette --version` should also show the SQLite version simonw 9599 open 0     2 2023-01-20T16:11:30Z 2023-01-20T18:19:06Z   OWNER  

Idea came up here: https://discord.com/channels/823971286308356157/823971286941302908/1066026473003159783

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1998/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1034535001 I_kwDOBm6k_c49qcBZ 1497 Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file" simonw 9599 closed 0     18 2021-10-24T22:57:07Z 2023-01-18T17:13:45Z 2021-10-24T23:36:55Z OWNER  

This means the Datasette 0.59.1 release has not been published to Docker Hub.

Here's where that failed: https://github.com/simonw/datasette/runs/3991043374?check_suite_focus=true

Preparing to unpack .../libc6_2.32-4_amd64.deb ... debconf: unable to initialize frontend: Dialog debconf: (TERM is not set, so the dialog frontend is not usable.) debconf: falling back to frontend: Readline debconf: unable to initialize frontend: Readline debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.) debconf: falling back to frontend: Teletype Checking for services that may need to be restarted... Checking init scripts... Unpacking libc6:amd64 (2.32-4) over (2.28-10) ... Setting up libc6:amd64 (2.32-4) ... /usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory dpkg: error processing package libc6:amd64 (--configure): installed libc6:amd64 package post-installation script subprocess returned error exit status 127 Errors were encountered while processing: libc6:amd64 E: Sub-process /usr/bin/dpkg returned an error code (1) The command '/bin/sh -c apt-get update && apt-get -y --no-install-recommends install software-properties-common && add-apt-repository "deb http://httpredir.debian.org/debian sid main" && apt-get update && apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite && apt-get remove -y software-properties-common && apt clean && rm -rf /var/lib/apt && rm -rf /var/lib/dpkg/info/*' returned a non-zero code: 100 Same problem when I attempted to publish using the "Push specific Docker tag" workflow: https://github.com/simonw/datasette/runs/3991059912?check_suite_focus=true

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1497/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
627794879 MDU6SXNzdWU2Mjc3OTQ4Nzk= 782 Redesign default .json format simonw 9599 open 0   Datasette 1.0a3 8755003 54 2020-05-30T18:47:07Z 2023-01-17T02:05:45Z   OWNER  

The default JSON just isn't right. I find myself using ?_shape=array for almost everything I build against the API.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/782/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1529707837 I_kwDOBm6k_c5bLX09 1988 Reconsider pattern where plugins could break existing template context simonw 9599 open 0   Datasette 1.0 3268330 4 2023-01-11T21:13:43Z 2023-01-11T21:25:05Z   OWNER  

I hadn't run into an issue with plugins like datasette-template-sql interfering with the existing context for other features before! Definitely not a good thing.

Originally posted by @simonw in https://github.com/simonw/datasette-write/issues/6#issuecomment-1379490596

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1988/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1529452371 I_kwDOBm6k_c5bKZdT 1987 installpython3.com is now a spam website simonw 9599 closed 0     4 2023-01-11T17:55:12Z 2023-01-11T18:29:26Z 2023-01-11T18:29:25Z OWNER  

Need to stop linking to it from the docs.

I'll link to https://www.python.org/about/gettingstarted/ instead.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1987/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1528448642 I_kwDOBm6k_c5bGkaC 1985 Don't let Datasette(path) without a list cause weird errors simonw 9599 closed 0     1 2023-01-11T05:17:44Z 2023-01-11T18:25:04Z 2023-01-11T18:25:04Z OWNER  

I got a confusing sqlite3.OperationalError: disk I/O error error in my tests, it turned out it was because this: python ds = Datasette(path) Should have been this: python ds = Datasette([path])

Originally posted by @simonw in https://github.com/simonw/datasette-faiss/issues/1#issuecomment-1378252673

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1985/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1515185383 I_kwDOBm6k_c5aT-Tn 1971 Upgrade for Sphinx 6.0 (once Furo has support for it) simonw 9599 closed 0     3 2022-12-31T19:04:35Z 2023-01-10T02:02:34Z 2023-01-10T02:02:34Z OWNER  

A deployment of #1967 to ReadTheDocs just failed like this: https://readthedocs.org/projects/datasette/builds/19045460/

``` Running Sphinx v6.0.0 making output directory... done building [mo]: targets for 0 po files that are out of date building [html]: targets for 28 source files that are out of date updating environment: [new config] 28 added, 0 changed, 0 removed reading sources... [ 3%] authentication reading sources... [ 7%] binary_data reading sources... [ 10%] changelog

Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 299, in next_line self.line = self.input_lines[self.line_offset] File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 1136, in getitem return self.data[i] IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 226, in run self.next_line() File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 302, in next_line raise EOFError EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/cmd/build.py", line 281, in build_main app.build(args.force_all, args.filenames) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/application.py", line 344, in build self.builder.build_update() File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/init.py", line 310, in build_update self.build(to_build, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/init.py", line 326, in build updated_docnames = set(self.read()) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/init.py", line 433, in read self._read_serial(docnames) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/init.py", line 454, in _read_serial self.read_doc(docname) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/init.py", line 510, in read_doc publisher.publish() File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/core.py", line 224, in publish self.document = self.reader.read(self.source, self.parser, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/io.py", line 103, in read self.parse() File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/readers/init.py", line 76, in parse self.parser.parse(self.input, document) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/parsers.py", line 78, in parse self.statemachine.run(inputlines, document, inliner=self.inliner) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 169, in run results = StateMachineWS.run(self, input_lines, input_offset, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 233, in run context, next_state, result = self.check_line( File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 445, in check_line return method(match, context, next_state) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 3024, in text self.section(title.lstrip(), source, style, lineno + 1, messages) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 325, in section self.new_subsection(title, lineno, messages) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 391, in new_subsection newabsoffset = self.nested_parse( File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 279, in nested_parse state_machine.run(block, input_offset, memo=self.memo, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 195, in run results = StateMachineWS.run(self, input_lines, input_offset) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 233, in run context, next_state, result = self.check_line( File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 445, in check_line return method(match, context, next_state) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2785, in underline self.section(title, source, style, lineno - 1, messages) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 325, in section self.new_subsection(title, lineno, messages) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 391, in new_subsection newabsoffset = self.nested_parse( File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 279, in nested_parse state_machine.run(block, input_offset, memo=self.memo, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 195, in run results = StateMachineWS.run(self, input_lines, input_offset) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 233, in run context, next_state, result = self.check_line( File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 445, in check_line return method(match, context, next_state) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 1273, in bullet i, blank_finish = self.list_item(match.end()) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 1295, in list_item self.nested_parse(indented, input_offset=line_offset, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 279, in nested_parse state_machine.run(block, input_offset, memo=self.memo, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 195, in run results = StateMachineWS.run(self, input_lines, input_offset) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py", line 239, in run result = state.eof(context) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2725, in eof self.blank(None, context, None) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2716, in blank paragraph, literalnext = self.paragraph( File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 416, in paragraph textnodes, messages = self.inline_text(text, lineno) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 425, in inline_text nodes, messages = self.inliner.parse(text, lineno, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 649, in parse before, inlines, remaining, sysmessages = method(self, match, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 792, in interpreted_or_phrase_ref nodelist, messages = self.interpreted(rawsource, escaped, role, File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 889, in interpreted nodes, messages2 = role_fn(role, rawsource, text, lineno, self) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/ext/extlinks.py", line 101, in role title = caption % part TypeError: not all arguments converted during string formatting

Exception occurred: File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/ext/extlinks.py", line 101, in role title = caption % part TypeError: not all arguments converted during string formatting The full traceback has been saved in /tmp/sphinx-err-kq7ylgqo.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at https://github.com/sphinx-doc/sphinx/issues. Thanks! ```

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1971/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1526635374 PR_kwDOBm6k_c5HCCY2 1984 Upgrade Sphinx simonw 9599 closed 0     1 2023-01-10T02:00:40Z 2023-01-10T02:02:33Z 2023-01-10T02:02:33Z OWNER simonw/datasette/pulls/1984

Refs #1971


:books: Documentation preview :books:: https://datasette--1984.org.readthedocs.build/en/1984/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1984/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1525815985 I_kwDOBm6k_c5a8hqx 1983 Make CustomJSONEncoder a documented public API simonw 9599 open 0     3 2023-01-09T15:27:05Z 2023-01-09T15:35:58Z   OWNER  

It's used by datasette-geojson here: https://github.com/eyeseast/datasette-geojson/commit/902bf135a5a33a0dc8264673d00a59a67cb05152

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1983/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1426080014 I_kwDOBm6k_c5VAEEO 1867 /db/table/-/rename API (also allows atomic replace) simonw 9599 open 0   Datasette 1.0a3 8755003 1 2022-10-27T18:13:23Z 2023-01-09T15:34:12Z   OWNER  

There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first.

...

If people care about that kind of thing they could always push all of their inserts to a table called _tablename and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism).

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1867/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
710650633 MDU6SXNzdWU3MTA2NTA2MzM= 979 Default table view JSON should include CREATE TABLE simonw 9599 closed 0     3 2020-09-28T23:54:58Z 2023-01-09T15:32:39Z 2023-01-09T15:32:22Z OWNER  

https://latest.datasette.io/fixtures/facetable.json doesn't currently include the CREATE TABLE statement for the page, even though it's available on the HTML version at https://latest.datasette.io/fixtures/facetable

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/979/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1082584499 I_kwDOBm6k_c5Ahu2z 1558 Redesign `facet_results` JSON structure prior to Datasette 1.0 simonw 9599 open 0   Datasette 1.0 3268330 3 2021-12-16T19:45:10Z 2023-01-09T15:31:17Z   OWNER  

Decision: as an initial fix I'm going to de-duplicate those keys by using tags__array etc - with a _2 on the end if that key is already used.

I'll open a separate issue to redesign this better for Datasette 1.0.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/625#issuecomment-996130862

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1558/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1524983536 I_kwDOBm6k_c5a5Wbw 1981 Canned query field labels truncated simonw 9599 open 0     1 2023-01-09T06:04:24Z 2023-01-09T06:05:44Z   OWNER  

Eg here on mobile: https://timezones.datasette.io/timezones/by_point?longitude=-0.1406632&latitude=50.8246776

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1981/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1524867951 I_kwDOBm6k_c5a46Nv 1980 "Cannot sort table by id" when sortable_columns is used simonw 9599 open 0     2 2023-01-09T03:21:33Z 2023-01-09T03:23:53Z   OWNER  

I had an instance with this in metadata.yml:

yaml databases: timezones: tables: timezones: sortable_columns: - tzid When I clicked on the "Apply" button here:

It sent me to /timezones/timezones?_sort=id&id__exact=133 with the error message:

500: Cannot sort table by id

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1980/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1524076587 I_kwDOBm6k_c5a15Ar 1979 More useful error message if enable_load_extension is not available simonw 9599 closed 0     5 2023-01-07T19:13:19Z 2023-01-08T00:21:23Z 2023-01-08T00:21:23Z OWNER  

I get this from:

datasette --load-extension spatialite --get /-/versions.json

File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/datasette/app.py", line 614, in _prepare_connection conn.enable_load_extension(True) AttributeError: 'sqlite3.Connection' object has no attribute 'enable_load_extension' It would be useful if Datasette caught this error and output something more friendly.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1979/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
957310278 MDU6SXNzdWU5NTczMTAyNzg= 1409 `default_allow_sql` setting (a re-imagining of the old `allow_sql` setting) simonw 9599 closed 0   Datasette 1.0 3268330 10 2021-07-31T19:48:56Z 2023-01-07T18:06:01Z 2023-01-05T00:51:31Z OWNER  

In 49d6d2f7b0f6cb02e25022e1c9403811f1fa0a7c as part of #813 I removed the allow_sql setting - on the basis that users could disable the ability to execute custom SQL queries using the new permission system instead.

I don't think this was the right decision. Disabling custom SQL is an important security capability, and explaining how to do it using permissions is significantly more complex than letting people know they can add --setting allow_sql off.

So I want to bring that setting back - maybe with a different, better name - and have it modify the default for that option if the permissions system doesn't have an opinion.

That way people can still use the setting but then use permissions to allow specific signed-in users access to execute SQL.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1409/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1516815571 I_kwDOBm6k_c5aaMTT 1975 _col=id can cause id column to export twice in CSV export simonw 9599 open 0     0 2023-01-03T00:25:15Z 2023-01-03T00:25:21Z   OWNER  

https://datasette.simonwillison.net/simonwillisonblog/blog_entry.csv?_col=id&_col=title&_col=body&_labels=on&_size=1

csv id,id,title,body 1,1,WaSP Phase II,"<p>The <a href=""http://www.webstandards.org/"">Web Standards</a> project has launched Phase II.</p>" That should not have two id columns.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1975/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1515182998 I_kwDOBm6k_c5aT9uW 1970 Path "None" in _internal database table simonw 9599 closed 0     2 2022-12-31T18:51:05Z 2022-12-31T19:22:58Z 2022-12-31T18:52:49Z OWNER  

See https://latest.datasette.io/_internal/databases (after https://latest.datasette.io/login-as-root)

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1970/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1515186569 I_kwDOBm6k_c5aT-mJ 1972 Fix Sphinx warning about extlink extension simonw 9599 closed 0     0 2022-12-31T19:12:04Z 2022-12-31T19:13:26Z 2022-12-31T19:13:26Z OWNER  

[sphinx-autobuild] > sphinx-build -b html /Users/simon/Dropbox/Development/datasette/docs /Users/simon/Dropbox/Development/datasette/docs/_build Running Sphinx v5.3.0 loading pickled environment... done WARNING: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1971#issuecomment-1368266904

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1972/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1115435536 I_kwDOBm6k_c5CfDIQ 1614 Try again with SQLite codemirror support simonw 9599 open 0     1 2022-01-26T20:05:20Z 2022-12-23T21:27:10Z   OWNER  

I tried and failed to implement autocomplete a while ago. Relevant code:

https://github.com/codemirror/legacy-modes/blob/8f36abca5f55024258cd23d9cfb0203d8d244f0d/mode/sql.js#L335

Sounds like upgrading to CodeMirror 6 ASAP would be worthwhile since it has better accessibility and touch screen support: https://codemirror.net/6/

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1614/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1355148385 I_kwDOBm6k_c5Qxexh 1796 Research an upgrade to CodeMirror 6 simonw 9599 open 0     4 2022-08-30T04:27:46Z 2022-12-23T21:27:03Z   OWNER  

There are still a bunch of bugs in CodeMirror 5 that affect various mobile browsers - see Datasette Discord report here: https://discord.com/channels/823971286308356157/823971286941302908/1013878624992108645

https://user-images.githubusercontent.com/9599/187349269-7b7c0c8c-3894-4810-82f0-de7c1eb940b3.mp4

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1796/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1373224657 I_kwDOCGYnMM5R2b7R 488 `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float simonw 9599 open 0     5 2022-09-14T15:51:30Z 2022-12-23T17:38:55Z   OWNER  

/tmp % echo "id,age,weight\n1,3,2.5\n2,," | sqlite-utils insert test.db test - --csv /tmp % sqlite-utils schema test.db CREATE TABLE [test] ( [id] TEXT, [age] TEXT, [weight] TEXT ); /tmp % sqlite-utils transform test.db test --type age integer --type weight float /tmp % sqlite-utils schema test.db CREATE TABLE "test" ( [id] TEXT, [age] INTEGER, [weight] FLOAT ); /tmp % sqlite-utils rows test.db test [{"id": "1", "age": 3, "weight": 2.5}, {"id": "2", "age": "", "weight": ""}] It would be neat if this resulted in the following instead: {"id": "2", "age": null, "weight": null} Related Discord discussion: https://discord.com/channels/823971286308356157/823971286941302908/1019635490833567794

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/488/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1447050738 I_kwDOBm6k_c5WQD3y 1886 Call for birthday presents: if you're using Datasette, let us know how you're using it here simonw 9599 open 0     13 2022-11-13T19:25:51Z 2022-12-18T17:34:20Z   OWNER  

Datasette is 5 years old today. To celebrate, I'm asking the community for birthday presents:

https://simonwillison.net/2022/Nov/13/datasette-birthday/

To celebrate this open source project’s birthday, I’ve decided to try something new: I’m going to ask for birthday presents.

An aspect of Datastte’s marketing that I’ve so far neglected is social proof. I think it’s time to change that: I know people are using the software to do cool things, but this often happens behind closed doors.

For Datastte’s birthday, I’m looking for endorsements and case studies and just general demonstrations that show how people are using it do so cool stuff.

So: if you’ve used Datasette to solve a problem, and you’re willing to publicize it, please give us the gift of your endorsement!

[...]

Add a comment to this issue thread describing what you’re doing. Just a few sentences is fine—though a screenshot or even a link to a live instance would be even better

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1886/reactions",
    "total_count": 2,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 2,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1501778647 I_kwDOBm6k_c5Zg1LX 1964 Cog menu is not keyboard accessible (also no ARIA) simonw 9599 open 0     1 2022-12-18T06:36:28Z 2022-12-18T06:37:28Z   OWNER  

This menu here: https://latest.datasette.io/fixtures/attraction_characteristic

You can tab to it (see the outline) and hit space or enter to open it, but you can't then navigate the items in the open menu using the keyboard.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1964/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1501713288 I_kwDOBm6k_c5ZglOI 1963 0.63.3 bugfix release simonw 9599 closed 0     2 2022-12-18T02:48:15Z 2022-12-18T03:26:55Z 2022-12-18T03:26:55Z OWNER  

I'm going to ship a release which back-ports these two fixes:

  • https://github.com/simonw/datasette/issues/1958
  • https://github.com/simonw/datasette/issues/1955
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1963/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
806849424 MDU6SXNzdWU4MDY4NDk0MjQ= 1221 Support SSL/TLS directly simonw 9599 closed 0     4 2021-02-12T00:18:29Z 2022-12-18T02:39:04Z 2021-02-12T00:52:18Z OWNER  

This should be pretty easy because Uvicorn supports them already. Need a good mechanism for testing it - https://pypi.org/project/trustme/ looks ideal.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1221/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1408757705 I_kwDOBm6k_c5T9-_J 1843 Intermittent "Too many open files" error running tests simonw 9599 open 0     16 2022-10-14T04:45:01Z 2022-12-17T22:02:41Z   OWNER  

Partial stack trace from one of them: ``` /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/jinja2/loaders.py:200: in get_source f = open_if_exists(filename)


filename = '/Users/simon/Dropbox/Development/datasette/datasette/templates/error.html', mode = 'rb'

def open_if_exists(filename: str, mode: str = "rb") -> t.Optional[t.IO]:
    """Returns a file descriptor for the filename if that file exists,
    otherwise ``None``.
    """
    if not os.path.isfile(filename):
        return None
  return open(filename, mode)

E OSError: [Errno 24] Too many open files: '/Users/simon/Dropbox/Development/datasette/datasette/templates/error.html' ```

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1843/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  reopened
1499081664 I_kwDOBm6k_c5ZWivA 1959 Refactor test suite to use mostly `async def` tests simonw 9599 closed 0     9 2022-12-15T21:02:54Z 2022-12-17T21:49:37Z 2022-12-17T21:49:36Z OWNER  

I got blocked working on this issue due to weird and hard-to-debug test suite problems:

  • 1955

The test suite has needed a major upgrade for several years now. It has a LOT of def test_... synchronous functions that could be upgraded to async def for better performance and less test complexity - I've used the new async def pattern in plugins and new tests for a couple of years now.

Hopefully I can get more of the tests to use in-memory named databases too, ideally so I can fix this consistent problem:

  • 1843

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1959/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1499150951 PR_kwDOBm6k_c5FlZmG 1960 Port as many tests as possible to async def tests against ds_client simonw 9599 closed 0     29 2022-12-15T21:45:53Z 2022-12-17T21:47:56Z 2022-12-17T21:47:55Z OWNER simonw/datasette/pulls/1960

Refs: - #1959


:books: Documentation preview :books:: https://datasette--1960.org.readthedocs.build/en/1960/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1960/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1500636982 I_kwDOBm6k_c5Zcec2 1962 Alternative, async-friendly pattern for `make_app_client()` and similar - fully retire `TestClient` simonw 9599 open 0     1 2022-12-16T17:56:51Z 2022-12-16T21:55:29Z   OWNER  

In this issue I replaced a whole bunch of places that used the non-async app_client fixture with an async ds_client fixture instead: - #1959

But I didn't get everything, and a lot of tests are still using the old TestClient mechanism as a result.

The main work here is replacing all of the app_client_... fixtures which use variants on the default client - and changing the tests that call make_app_client() to do something else instead.

This requires some careful thought. I need to come up with a really nice pattern for creating variants on the ds_client default fixture - and do so in a way that minimizes the number of open files, refs:

  • 1843

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1962/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1499387139 PR_kwDOBm6k_c5FmNXs 1961 Use click.echo() instead of print() for --root option simonw 9599 closed 0     0 2022-12-16T00:54:56Z 2022-12-16T00:55:19Z 2022-12-16T00:55:18Z OWNER simonw/datasette/pulls/1961

This ensures the URL is output correctly when running under Docker.

Closes #1958


:books: Documentation preview :books:: https://datasette--1961.org.readthedocs.build/en/1961/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1961/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1495821607 I_kwDOBm6k_c5ZKG0n 1953 Release notes for Datasette 1.0a2 simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-14T06:26:40Z 2022-12-15T02:02:15Z 2022-12-15T02:01:08Z OWNER  

https://github.com/simonw/datasette/milestone/27?closed=1

https://github.com/simonw/datasette/compare/1.0a1...9ad76d279e2c3874ca5070626a25458ce129f126

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1953/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1473411197 I_kwDOBm6k_c5X0nh9 1927 ignore:true/replace:true options for /db/-/create API simonw 9599 closed 0   Datasette 1.0a2 8711695 5 2022-12-02T20:32:30Z 2022-12-15T01:47:01Z 2022-12-08T01:43:01Z OWNER  

See also: - #1924

It turns out I want to be able to call /db/-/create multiple times with the rows argument, so that I don't have to worry about creating the table first.

As such I find myself wanting support for the "insert": true and "replace": true options as well.

Still TODO:

  • [x] A test for the case where you call /-/create twice with rows without using these options
  • [x] pk should be required if you are using these options
  • [x] Error if you pass pk and the table exists already but has a different pk
  • [x] Documentation for insert and replace - and what happens if you repeat a /-/create with rows generally
  • [x] Documentation should explain that you are allowed to call /-/create more than once using rows.
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1927/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1493471221 I_kwDOBm6k_c5ZBI_1 1949 `.json` errors should be returned as JSON simonw 9599 open 0   Datasette 1.0a3 8755003 10 2022-12-13T06:14:12Z 2022-12-15T00:46:27Z   OWNER  

Eg the error in this issue: - #1945

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1949/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1490576818 I_kwDOBm6k_c5Y2GWy 1943 `/-/permissions` should list available permissions simonw 9599 open 0   Datasette 1.0a3 8755003 1 2022-12-11T23:38:03Z 2022-12-15T00:41:37Z   OWNER  

Idea: a /-/permissions introspection endpoint for listing registered permissions

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1939#issuecomment-1345691103

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1943/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1497577017 I_kwDOBm6k_c5ZQzY5 1957 Reconsider row value truncation on query page simonw 9599 open 0     1 2022-12-14T23:49:47Z 2022-12-14T23:50:50Z   OWNER  

Consider this example: https://ripgrep.datasette.io/repos?sql=select+json_group_array%28full_name%29+from+repos

sql select json_group_array(full_name) from repos

My intention here was to get a string of JSON I can copy and paste elsewhere - see: https://til.simonwillison.net/sqlite/compare-before-after-json

The truncation isn't helping here.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1957/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
855296937 MDU6SXNzdWU4NTUyOTY5Mzc= 1295 Errors should have links to further information simonw 9599 open 0     2 2021-04-11T12:39:12Z 2022-12-14T23:28:49Z   OWNER  

Inspired by this tweet: https://twitter.com/willmcgugan/status/1381186384510255104

While I am thinking about faqs. I’d also like to add short URLs to Rich exceptions.

I loath cryptic error messages, and I’ve created a fair few myself. In Rich I’ve tried to make them as plain English as possible. But...

would be great if every error message linked to a page that explains the error in detail and offers fixes.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1295/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1483320357 I_kwDOBm6k_c5Yaawl 1937 /db/-/create API should require insert-rows permission to use row: or rows: option simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-08T01:33:09Z 2022-12-14T20:21:26Z 2022-12-14T20:21:26Z OWNER  

Otherwise someone with create-table but noinsert-rows permission could abuse it to insert data.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1937/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1497288666 I_kwDOBm6k_c5ZPs_a 1956 Handle abbreviations properly in permission_allowed_actor_restrictions simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-14T19:54:21Z 2022-12-14T20:04:29Z 2022-12-14T20:04:28Z OWNER  

This code currently assumes abbreviations are:

pyton action_initials = "".join([word[0] for word in action.split("-")])

https://github.com/simonw/datasette/blob/1a3dcf494376e32f7cff110c86a88e5b0a3f3924/datasette/default_permissions.py#L182-L208

That's no longer correct, they are now registered by the new plugin hook: - #1939

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1956/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1495716243 I_kwDOBm6k_c5ZJtGT 1952 Improvements to /-/create-token restrictions interface simonw 9599 open 0   Datasette 1.0a3 8755003 1 2022-12-14T05:22:39Z 2022-12-14T05:23:13Z   OWNER  

It would be neat not to show write permissions against immutable databases too - and not hard from a performance perspective since it doesn't involve hundreds more permission checks.

That will need permissions to grow a flag for if they need a mutable database though, which is a bigger job.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1947#issuecomment-1350414402

Also, DO show the _memory database there if Datasette was started in --crossdb mode.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1952/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1493390939 I_kwDOBm6k_c5ZA1Zb 1947 UI to create reduced scope tokens from the `/-/create-token` page simonw 9599 closed 0   Datasette 1.0a2 8711695 22 2022-12-13T05:10:48Z 2022-12-14T05:22:00Z 2022-12-14T05:13:24Z OWNER  

Split from: - #1855

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1947/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1495431932 I_kwDOBm6k_c5ZInr8 1951 `datasette.create_token(...)` method for creating signed API tokens simonw 9599 closed 0   Datasette 1.0a2 8711695 6 2022-12-14T01:25:34Z 2022-12-14T02:43:45Z 2022-12-14T02:42:05Z OWNER  

I need this for: - #1947

And I can refactor this to use it too: - #1855

By making this a documented internal API it can be used by other plugins too. It's also going to be really useful for writing tests.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1951/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1423336089 I_kwDOBm6k_c5U1mKZ 1855 `datasette create-token` ability to create tokens with a reduced set of permissions simonw 9599 closed 0   Datasette 1.0a2 8711695 19 2022-10-26T02:20:52Z 2022-12-14T01:24:49Z 2022-12-13T05:20:24Z OWNER  

Initial design ideas: https://github.com/simonw/datasette/issues/1852#issuecomment-1289733483

Token design concept:

json { "t": { "a": ["ir", "ur", "dr"], "d": { "fixtures": ["ir", "ur", "dr"] }, "t": { "fixtures": { "searchable": ["ir"] } } } }

That JSON would be minified and signed.

Minified version of the above looks like this (101 characters):

{"t":{"a":["ir","ur","dr"],"d":{"fixtures":["ir","ur","dr"]},"t":{"fixtures":{"searchable":["ir"]}}}}

The "t" key shows this is a token that as a default API key.

"a" means "all" - these are permissions that have been granted on all tables and databases.

"d" means "databases" - this is a way to set permissions for all tables in a specific database.

"t" means "tables" - this lets you set permissions at a finely grained table level.

Then the permissions themselves are two character codes which are shortened versions - so:

  • ir = insert-row
  • ur = update-row
  • dr = delete-row

Remaining tasks

  • [x] Add these options to the datasette create-token command
  • [x] Tests for datasette create-token options
  • [x] Documentation for those options at https://docs.datasette.io/en/latest/authentication.html#datasette-create-token
  • [x] A way to handle permissions that don't have known abbreviations (permissions added by plugins). Probably need to solve the plugin permission registration problem as part of that
  • [x] Stop hard-coding names of actions in the permission_allowed_actor_restrictions function
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1855/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1495241162 I_kwDOBm6k_c5ZH5HK 1950 Bad ?_sort returns a 500 error, should be a 400 simonw 9599 closed 0     2 2022-12-13T22:08:16Z 2022-12-13T22:23:22Z 2022-12-13T22:23:22Z OWNER  

https://latest.datasette.io/fixtures/facetable?_sort=bad

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1950/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1468689139 I_kwDOBm6k_c5Ximrz 1914 Finalize design of JSON for Datasette 1.0 simonw 9599 open 0   Datasette 1.0a3 8755003 1 2022-11-29T20:59:10Z 2022-12-13T06:15:54Z   OWNER  

Tracking issue.

  • [ ] #1709
  • [ ] #1729
  • [ ] #1875
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1914/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1468495358 I_kwDOBm6k_c5Xh3X- 1910 Check incoming column types on various write APIs simonw 9599 open 0   Datasette 1.0a3 8755003 0 2022-11-29T18:09:10Z 2022-12-13T05:29:09Z   OWNER  

I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1910/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1428630253 I_kwDOBm6k_c5VJyrt 1873 Ensure insert API has good tests for rowid and compound primark key tables simonw 9599 open 0   Datasette 1.0a3 8755003 11 2022-10-30T06:22:17Z 2022-12-13T05:29:08Z   OWNER  

Following: - #1866

I need to design and implement various edge-cases or primary keys:

  • Table without an auto-incrementing primary key
  • Table with compound primary keys
  • Table with just a rowid
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1873/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  reopened
1430797211 I_kwDOBm6k_c5VSDub 1875 Figure out design for JSON errors (consider RFC 7807) simonw 9599 open 0   Datasette 1.0a3 8755003 7 2022-11-01T03:14:15Z 2022-12-13T05:29:08Z   OWNER  

https://datatracker.ietf.org/doc/draft-ietf-httpapi-rfc7807bis/ is a brand new standard.

Since I need a neat, predictable format for my JSON errors, maybe I should use this one?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1875/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1447465004 I_kwDOBm6k_c5WRpAs 1889 Ability to create new tokens via the API simonw 9599 open 0   Datasette 1.0a3 8755003 0 2022-11-14T06:21:36Z 2022-12-13T05:29:08Z   OWNER  

Refs: - #1850

Initially I decided that the API shouldn't be able to create new tokens at all - I don't like the idea of an API token holder creating themselves additional tokens.

Then I realized that two of the API features are specifically more useful if you can generate fresh tokens via the API:

  • Tokes that expire after a time limit are MUCH more useful if they can be automatically generated
  • Likewise, tokens that are restricted to a subset of permissions (see #1855) make more sense to be generated like this, especially in conjunction with expiry times
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1889/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1216436131 I_kwDOBm6k_c5IgVej 1721 Implement plugin hooks: `register_table_extras`, `register_row_extras`, `register_query_extras` simonw 9599 open 0   Datasette 1.0a3 8755003 0 2022-04-26T20:21:49Z 2022-12-13T05:29:07Z   OWNER  

Designed in: - #1720

Part of: - #262 - #1709

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1721/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1219385669 I_kwDOBm6k_c5IrllF 1729 Implement ?_extra and new API design for TableView simonw 9599 open 0   Datasette 1.0a3 8755003 12 2022-04-28T22:28:14Z 2022-12-13T05:29:07Z   OWNER  

Part of: - #262 - #1518

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1729/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1200649124 I_kwDOBm6k_c5HkHOk 1708 Datasette 1.0 alpha upcoming release notes simonw 9599 open 0   Datasette 1.0a3 8755003 2 2022-04-11T22:57:12Z 2022-12-13T05:29:06Z   OWNER  

I'm going to try writing the release notes first, to see if that helps unblock me.

⚠️ Any release notes in this issue are a draft, and should not be treated as the real thing ⚠️

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1708/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1200649502 I_kwDOBm6k_c5HkHUe 1709 Redesigned JSON API with ?_extra= parameters simonw 9599 open 0   Datasette 1.0a3 8755003 1 2022-04-11T22:57:49Z 2022-12-13T05:29:06Z   OWNER  

This will be the single biggest breaking change for the 1.0 release.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1709/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1200650491 I_kwDOBm6k_c5HkHj7 1711 Template context powered entirely by the JSON API format simonw 9599 open 0   Datasette 1.0a3 8755003 1 2022-04-11T22:59:27Z 2022-12-13T05:29:06Z   OWNER  

Datasette 1.0 will have a stable template context. I'm going to achieve this by refactoring the templates to work only with keys returned by the API (or some of its extras) - then the API documentation will double up as template documentation.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1711/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1197926598 I_kwDOBm6k_c5HZujG 1705 How to upgrade your plugin for 1.0 documentation simonw 9599 open 0   Datasette 1.0a3 8755003 1 2022-04-08T23:16:47Z 2022-12-13T05:29:05Z   OWNER  

Among other things, needed by: - #1704

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1705/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1485488236 PR_kwDOBm6k_c5E1iJG 1938 "permissions" blocks in metadata.json/yaml simonw 9599 closed 0   Datasette 1.0a2 8711695 3 2022-12-08T22:07:36Z 2022-12-13T05:23:18Z 2022-12-13T05:23:18Z OWNER simonw/datasette/pulls/1938

Refs #1636

  • [x] Documentation
  • [ ] Implementation
  • [ ] Validate metadata to check there are no nonsensical permissions (like debug-menu set at the table level)
  • [ ] Tests

:books: Documentation preview :books:: https://datasette--1938.org.readthedocs.build/en/1938/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1938/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1493404423 I_kwDOBm6k_c5ZA4sH 1948 500 error on permission debug page when testing actors with _r simonw 9599 open 0     1 2022-12-13T05:22:03Z 2022-12-13T05:22:19Z   OWNER  

The 500 error is silent unless you are looking at the DevTools network pane.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1948/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1493339206 I_kwDOBm6k_c5ZAoxG 1946 `datasette --get` mechanism for sending tokens simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-13T04:25:05Z 2022-12-13T04:36:57Z 2022-12-13T04:36:57Z OWNER  

For the tests for datasette create-token it would be useful if datasette --get had a mechanism for sending an Authorization: Bearer X header.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1855#issuecomment-1347731288

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1946/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1493306655 I_kwDOBm6k_c5ZAg0f 1945 `view-instance` should not be checked for /-/actor.json simonw 9599 closed 0   Datasette 1.0a2 8711695 0 2022-12-13T04:01:46Z 2022-12-13T04:11:56Z 2022-12-13T04:11:56Z OWNER  

Spotted this while testing:

  • 1855

    export TOKEN=$(datasette create-token root --secret s -a foo) curl -H "Authorization: Bearer $TOKEN" http://localhost:8002/-/actor.json Returned a Forbidden error (and not in JSON either).
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1945/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1138008042 I_kwDOBm6k_c5D1J_q 1636 "permissions" propery in metadata for configuring arbitrary permissions simonw 9599 closed 0   Datasette 1.0a2 8711695 14 2022-02-15T00:25:59Z 2022-12-13T02:40:50Z 2022-12-13T02:40:50Z OWNER  

The "allow" block mechanism can already be used to configure various default permissions. When adding permissions to datasette-tiddlywiki I realized it would be good to be able to configure arbitrary permissions such as edit-tiddlywiki there too.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1636/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1485757511 I_kwDOBm6k_c5YjtxH 1939 register_permissions(datasette) plugin hook simonw 9599 closed 0   Datasette 1.0a2 8711695 20 2022-12-09T01:33:25Z 2022-12-13T02:07:50Z 2022-12-13T02:05:56Z OWNER  

A plugin hook that adds more named permissions to the list which is initially populated here:

https://github.com/simonw/datasette/blob/e539c1c024bc62d88df91d9107cbe37e7f0fe55f/datasette/permissions.py#L1-L19

Originally imagined this hook in this comment:

  • https://github.com/simonw/datasette/issues/1881#issuecomment-1301639370

I need this for a few reasons:

  • https://github.com/simonw/datasette/issues/1636
  • Needs it in order to validate that permissions defined in metadata.json are set in the right place (don't set an instance permissions at table level for example)
  • https://github.com/simonw/datasette/issues/1855
  • Needs it to be able to register additional abbreviations for use in signed cookies
  • And for validation when you use datasette create-token and pass in extra permissions
  • The https://latest.datasette.io/-/permissions debug interface needs it to add extra debug options to the <select>
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1939/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1486011362 PR_kwDOBm6k_c5E3XqB 1940 register_permissions() plugin hook simonw 9599 closed 0   Datasette 1.0a2 8711695 6 2022-12-09T05:09:28Z 2022-12-13T02:05:55Z 2022-12-13T02:05:54Z OWNER simonw/datasette/pulls/1940

Refs #1939

From this comment: https://github.com/simonw/datasette/issues/1939#issuecomment-1343872168

  • [x] Unit test for the registration plugin hook itself
  • [x] Use them in check_permission_actions_are_documented test in conftest.py
  • [x] Add description field to Permissions (and update tests and docs)
  • [x] Documentation for datasette.permissions dictionary
  • [x] If no default= provided in call to permission_allowed() then use default from datasette.permissions list
  • [x] Remove default= from a bunch of places
  • [x] Throw an error if two permissions are registered with the same name or abbreviation (but other attributes differ)
  • [x] Update authentication and permissions documentation to explain that permissions are now registered and have a registered default

:books: Documentation preview :books:: https://datasette--1940.org.readthedocs.build/en/1940/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1940/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1384549993 I_kwDOBm6k_c5Sho5p 1818 Setting to turn off table row counts entirely simonw 9599 open 0     4 2022-09-24T06:39:22Z 2022-12-11T02:03:09Z   OWNER  

There are situations - such as loading SQLite files remotely using HTTP range headers - where counting all of the rows in a table should be avoided entirely.

Also, this chunked inefficiency means that I have to hack the URL to not load tables of a database as it seems to try to load the whole database when I click on a database.

I bet that's because Datasette tries to show a count of all of the rows in each table when it shows the list on that page, which triggers a full table scan.

Would be great to have a setting that turns that feature off, which could then be exposed as a query string option for Datasette Lite.

Originally posted by @simonw in https://github.com/simonw/datasette-lite/issues/49#issuecomment-1256880715

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1818/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1487764628 I_kwDOCGYnMM5YrXyU 518 flake8 ValueError: Error code '#' supplied to 'extend-ignore' option... simonw 9599 closed 0     0 2022-12-10T01:30:24Z 2022-12-10T01:36:46Z 2022-12-10T01:36:46Z OWNER  

Error code '#' supplied to 'extend-ignore' option does not match '^[A-Z]{1,3}[0-9]{0,3}$'

https://github.com/simonw/sqlite-utils/actions/runs/3662011265/jobs/6190770361

I think from this:

https://github.com/simonw/sqlite-utils/blob/e660635cea6c32f4022818380b1e1ee88e7c93a6/setup.cfg#L1-L3

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/518/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);
Powered by Datasette · Queries took 423.24ms · About: github-to-sqlite