home / github

Menu
  • Search all tables
  • GraphQL API

issues

Table actions
  • GraphQL API for issues

48 rows where comments = 7 and repo = 107914493 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: user, milestone, author_association, draft, state_reason, created_at (date), updated_at (date), closed_at (date)

type 2

  • issue 43
  • pull 5

state 2

  • closed 31
  • open 17

repo 1

  • datasette · 48 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1940346034 I_kwDOBm6k_c5zp1Sy 2199 Detailed upgrade instructions for metadata.yaml -> datasette.yaml simonw 9599 open 0   Datasette 1.0 3268330 7 2023-10-12T16:21:25Z 2023-10-12T22:08:42Z   OWNER  

Exception: Datasette no longer accepts plugin configuration in --metadata. Move your "plugins" configuration blocks to a separate file - we suggest calling that datasette..json - and start Datasette with datasette -c datasette..json. See https://docs.datasette.io/en/latest/configuration.html for more details.

I think we should link directly to documentation that tells people how to perform this upgrade.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/2190#issuecomment-1759947021

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2199/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1426379903 PR_kwDOBm6k_c5BtJNn 1870 don't use immutable=1, only mode=ro fgregg 536941 open 0     7 2022-10-27T23:33:04Z 2023-10-03T19:12:37Z   CONTRIBUTOR simonw/datasette/pulls/1870

Opening db files in immutable mode sometimes leads to the file being mutated, which causes duplication in the docker image layers: see #1836, #1480

That this happens in "immutable" mode is surprising, because the sqlite docs say that setting this should open the database as read only.

https://www.sqlite.org/c3ref/open.html

immutable: The immutable parameter is a boolean query parameter that indicates that the database file is stored on read-only media. When immutable is set, SQLite assumes that the database file cannot be changed, even by a process with higher privilege, and so the database is opened read-only and all locking and change detection is disabled. Caution: Setting the immutable property on a database file that does in fact change can result in incorrect query results and/or SQLITE_CORRUPT errors. See also: SQLITE_IOCAP_IMMUTABLE.

Perhaps this is a bug in sqlite?


:books: Documentation preview :books:: https://datasette--1870.org.readthedocs.build/en/1870/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1870/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1010112818 I_kwDOBm6k_c48NRky 1479 Win32 "used by another process" error with datasette publish kirajano 76450761 open 0     7 2021-09-28T19:12:00Z 2023-09-07T02:14:16Z   NONE  

I unfortunately was not successful to deploy to fly.io. Please see the details above of the three scenarios that I took. I am also new to datasette.

Failed to deploy. Attaching logs: 1. Tried with an app created via flyctl apps create frosty-fog-8565 and the ran datasette publish fly covid.db --app frosty-fog-8565 ``` Deploying frosty-fog-8565 ==> Validating app configuration --> Validating app configuration done Services TCP 80/443 ⇢ 8080

Error error connecting to docker: An unknown error occured.

Traceback (most recent call last): File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\grott\Anaconda3\Scripts\datasette.exe__main__.py", line 7, in <module> File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in call return self.main(args, kwargs) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(args, **kwargs) File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly__init__.py", line 156, in fly "--remote-only", File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in exit next(self.gen) File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils__init__.py", line 451, in temporary_docker_directory tmp.cleanup() File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup _shutil.rmtree(self.name) File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree return _rmtree_unsafe(path, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe onerror(os.rmdir, path, sys.exc_info()) File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe os.rmdir(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\grott\AppData\Local\Temp\tmpgcm8cz66\frosty-fog-8565' ```

  1. Tried also with an app that gets autogenerate when running flyctl launch. This also generates the .toml file. Ran then datasette publish fly covid.db --app dark-feather-168 but different error now ```Deploying dark-feather-168 ==> Validating app configuration

Error not possible to validate configuration: server returned Post "https://api.fly.io/graphql": unexpected EOF

Traceback (most recent call last): File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec) exec(code, run_globals) File "C:\Users\grott\Anaconda3\Scripts\datasette.exe__main__.py", line 7, in <module> File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in call return self.main(args, kwargs) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(args, **kwargs) File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly__init__.py", line 156, in fly "--remote-only", File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in exit next(self.gen) File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils__init__.py", line 451, in temporary_docker_directory tmp.cleanup() File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup _shutil.rmtree(self.name) File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree return _rmtree_unsafe(path, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe onerror(os.rmdir, path, sys.exc_info()) File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe os.rmdir(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\grott\AppData\Local\Temp\tmpnoyewcre\dark-feather-168' ```

These are also the contents of the generated .toml file in 2 scenario:

```

fly.toml file generated for dark-feather-168 on 2021-09-28T20:35:44+02:00

app = "dark-feather-168"

kill_signal = "SIGINT" kill_timeout = 5 processes = []

[env]

[experimental] allowed_public_ports = [] auto_rollback = true

[[services]] http_checks = [] internal_port = 8080 processes = ["app"] protocol = "tcp" script_checks = []

[services.concurrency] hard_limit = 25 soft_limit = 20 type = "connections"

[[services.ports]] handlers = ["http"] port = 80

[[services.ports]] handlers = ["tls", "http"] port = 443

[[services.tcp_checks]] grace_period = "1s" interval = "15s" restart_limit = 6 timeout = "2s" ```

  1. But also trying datasette package covid.db to create a local DOCKERFILE to later try to push it via flyctl deploy fails as well.

```[+] Building 147.3s (11/11) FINISHED => [internal] load build definition from Dockerfile 0.2s => => transferring dockerfile: 396B 0.0s => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/python:3.8 4.7s => [auth] library/python:pull token for registry-1.docker.io 0.0s => [internal] load build context 0.1s => => transferring context: 82.37kB 0.0s => [1/5] FROM docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3 108.3s => => resolve docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5 0.0s => => sha256:56182bcdf4d4283aa1f46944b4ef7ac881e28b4d5526720a4e9ba03a4730846a 2.22kB / 2.22kB 0.0s => => sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 54.93MB / 54.93MB 21.6s => => sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 10.87MB / 10.87MB 15.5s => => sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5c5b6 1.86kB / 1.86kB 0.0s => => sha256:ff08f08727e50193dcf499afc30594c47e70cc96f6fcfd1a01240524624264d0 8.65kB / 8.65kB 0.0s => => sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 5.15MB / 5.15MB 6.4s => => sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 54.57MB / 54.57MB 37.7s => => sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 196.45MB / 196.45MB 82.3s => => sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 6.29MB / 6.29MB 26.6s => => extracting sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 5.4s => => sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 16.81MB / 16.81MB 40.5s => => extracting sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 0.6s => => extracting sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 0.6s => => sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 232B / 232B 40.1s => => extracting sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 5.8s => => sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 2.35MB / 2.35MB 41.8s => => extracting sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 18.8s => => extracting sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 1.2s => => extracting sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 2.9s => => extracting sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 0.0s => => extracting sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 0.8s => [2/5] COPY . /app 2.3s => [3/5] WORKDIR /app 0.2s => [4/5] RUN pip install -U datasette 26.9s => [5/5] RUN datasette inspect covid.db --inspect-file inspect-data.json 3.1s => exporting to image 1.2s => => exporting layers 1.2s => => writing image sha256:b5db0c205cd3454c21fbb00ecf6043f261540bcf91c2dfc36d418f1a23a75d7a 0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them Traceback (most recent call last): "main", mod_spec) File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\grott\Anaconda3\Scripts\datasette.exe__main__.py", line 7, in <module> File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in call return self.main(args, kwargs) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(args, **kwargs) File "c:\users\grott\anaconda3\lib\site-packages\datasette\cli.py", line 283, in package call(args) File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in exit next(self.gen) File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils__init__.py", line 451, in temporary_docker_directory tmp.cleanup() File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup _shutil.rmtree(self.name) File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree return _rmtree_unsafe(path, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe onerror(os.rmdir, path, sys.exc_info()) File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe os.rmdir(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\grott\AppData\Local\Temp\tmpkb27qid3\datasette'```

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1479/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
449886319 MDU6SXNzdWU0NDk4ODYzMTk= 493 Rename metadata.json to config.json simonw 9599 closed 0   Datasette 1.0 3268330 7 2019-05-29T15:48:03Z 2023-08-23T01:29:21Z 2023-08-23T01:29:20Z OWNER  

It is increasingly being useful configuration options, when it started out as purely metadata.

Could cause confusion with the --config mechanism though - maybe that should be called "settings" instead?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/493/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
323223872 MDU6SXNzdWUzMjMyMjM4NzI= 260 Validate metadata.json on startup simonw 9599 open 0     7 2018-05-15T13:42:56Z 2023-06-21T12:51:22Z   OWNER  

It's easy to misspell the name of a database or table and then be puzzled when the metadata settings silently fail.

To avoid this, let's sanity check the provided metadata.json on startup and quit with a useful error message if we find any obvious mistakes.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/260/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1430797211 I_kwDOBm6k_c5VSDub 1875 Figure out design for JSON errors (consider RFC 7807) simonw 9599 open 0   Datasette 1.0a-next 8755003 7 2022-11-01T03:14:15Z 2022-12-13T05:29:08Z   OWNER  

https://datatracker.ietf.org/doc/draft-ietf-httpapi-rfc7807bis/ is a brand new standard.

Since I need a neat, predictable format for my JSON errors, maybe I should use this one?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1875/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1473481262 I_kwDOBm6k_c5X04ou 1928 Hacker News Datasette write demo simonw 9599 closed 0     7 2022-12-02T21:17:41Z 2022-12-02T23:47:11Z 2022-12-02T21:43:19Z OWNER  

Idea is to have my existing scraper at https://github.com/simonw/scrape-hacker-news-by-domain also write to my private Datasette Cloud account, then create an atom feed from it.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1928/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1422915587 I_kwDOBm6k_c5Uz_gD 1853 Upgrade Datasette Docker to Python 3.11 simonw 9599 closed 0     7 2022-10-25T18:44:31Z 2022-10-25T19:28:56Z 2022-10-25T19:05:16Z OWNER  

Related: - #1768

I think this base image looks right: 3.11.0-slim-bullseye

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1853/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1363552780 I_kwDOBm6k_c5RRioM 1805 truncate_cells_html does not work for links? CharlesNepote 562352 open 0     7 2022-09-06T16:41:29Z 2022-10-03T09:18:06Z   NONE  

We have many links inside our dataset (please don't blame us ;-).

When I use --settings truncate_cells_html 60 it is not working for the links.

Eg. https://images.openfoodfacts.org/images/products/000/000/000/088/nutrition_fr.5.200.jpg (87 chars) is not truncated:

IMHO It would make sense that links should be treated as HTML. The link should work of course, but Datasette could truncate it: https://images.openfoodfacts.org/images/products/00[...].jpg

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1805/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  reopened
1384273985 I_kwDOBm6k_c5SglhB 1817 Expose `sql` and `params` arguments to various plugin hooks simonw 9599 open 0     7 2022-09-23T20:34:45Z 2022-09-27T00:27:53Z   OWNER  

On Discord: https://discord.com/channels/823971286308356157/996877076982415491/1022784534363787305

Hi! I'm attempting to write a plugin that would provide some statistics on text fields (most common words, etc). I would want this information displayed in the table pages, and (ideally) also updated when users make custom queries from the table pages.

It seems one way to do this would be to use the extra_template_vars hook, and make the appropriate SQL query there. So extra_template_vars would create a variable that is a list of most common words, and this is displayed on the page, possibly above the regular table view.

Is there a way that the plugin code can access the SQL query (or even the data) that was used to produce the table view? I can see that TableView class constructs the SQL query, but I can't seem to find a way to access that information from the objects that are available to extra_template_vars.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1817/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
728905098 MDU6SXNzdWU3Mjg5MDUwOTg= 1048 Documentation and unit tests for urls.row() urls.row_blob() methods simonw 9599 open 0     7 2020-10-25T00:13:53Z 2022-07-10T16:23:57Z   OWNER  

https://github.com/simonw/datasette/blob/5db7ae3ce165ded57c7fb1cfbdb3258b1cf06c10/datasette/app.py#L1307-L1313

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1048/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1239008850 I_kwDOBm6k_c5J2cZS 1744 `--nolock` feature for opening locked databases simonw 9599 closed 0     7 2022-05-17T18:25:16Z 2022-05-17T19:46:38Z 2022-05-17T19:40:30Z OWNER  

The getting started docs currently suggest you try this to browse your Chrome history:

datasette ~/Library/Application\ Support/Google/Chrome/Default/History

But if Chrome is running you will likely get this error:

sqlite3.OperationalError: database is locked

Turns out there's a workaround for this which I just spotted on the SQLite forum:

You can do this using a URI filename: sqlite3 'file:places.sqlite?mode=ro&nolock=1' That opens the file places.sqlite in read-only mode with locking disabled. This isn't safe, in that changes to the database made by other corrections are likely to cause this connection to return incorrect results or crash. Read-only mode should at least mean that you don't corrupt the database in the process.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1744/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1175648453 I_kwDOBm6k_c5GEvjF 1675 Extract out `check_permissions()` from `BaseView simonw 9599 closed 0     7 2022-03-21T16:39:46Z 2022-03-21T17:14:31Z 2022-03-21T17:13:21Z OWNER  

I'm going to refactor this stuff out and document it so it can be easily used by plugins:

https://github.com/simonw/datasette/blob/4a4164b81191dec35e423486a208b05a9edc65e4/datasette/views/base.py#L69-L103

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1074136176

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1675/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1173017980 PR_kwDOBm6k_c40oRq- 1664 Remove hashed URL mode simonw 9599 closed 0     7 2022-03-17T23:19:10Z 2022-03-19T00:12:04Z 2022-03-19T00:12:04Z OWNER simonw/datasette/pulls/1664

Refs #1661.

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1664/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1160432941 PR_kwDOBm6k_c4z_p6S 1648 Use dash encoding for table names and row primary keys in URLs simonw 9599 closed 0     7 2022-03-05T19:50:45Z 2022-03-07T15:38:30Z 2022-03-07T15:38:30Z OWNER simonw/datasette/pulls/1648

Refs #1439.

  • [x] Build dash_encode / dash_decode functions
  • [x] Use dash encoding for row primary keys
  • [x] Use dash encoding for ?_next= pagination tokens
  • [x] Use dash encoding for table names in URLs
  • [x] Use dash encoding for database name
  • ~~Implement redirects from previous % URLs that replace those with -~~ - separate issue: #1650
datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1648/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
688622148 MDU6SXNzdWU2ODg2MjIxNDg= 957 Simplify imports of common classes simonw 9599 closed 0   Datasette 1.0 3268330 7 2020-08-29T23:44:04Z 2022-02-06T06:36:41Z 2022-02-06T06:34:37Z OWNER  

There are only a few classes that plugins need to import. It would be nice if these imports were as short and memorable as possible.

For example: python from datasette.app import Datasette from datasette.utils.asgi import Response Could both become: python from datasette import Datasette from datasette import Response

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/957/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
734777631 MDU6SXNzdWU3MzQ3Nzc2MzE= 1080 "View all" option for facets, to provide a (paginated) list of ALL of the facet counts plus a link to view them simonw 9599 open 0   Datasette 1.0 3268330 7 2020-11-02T19:55:06Z 2022-02-04T06:25:18Z   OWNER  

Can use /database/-/... namespace from #296

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1080/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1100015398 I_kwDOBm6k_c5BkOcm 1591 Maybe let plugins define custom serve options? simonw 9599 open 0     7 2022-01-12T08:18:47Z 2022-01-15T11:56:59Z   OWNER  

https://twitter.com/psychemedia/status/1481171650934714370

can extensions be passed their own cli args? eg --ext-tiddlywiki-dbname tiddlywiki2.sqlite ?

I've thought something like this might be useful for other plugins in the past, too.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1591/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1059555791 I_kwDOBm6k_c4_J4nP 1527 Columns starting with an underscore behave poorly in filters simonw 9599 closed 0   Datasette 0.60 7571612 7 2021-11-22T01:01:36Z 2022-01-14T00:57:08Z 2022-01-14T00:57:08Z OWNER  

Similar bug to #1525 (and #1506 before it). Start on https://latest.datasette.io/fixtures/facetable?_facet=_neighborhood - then select a neighborhood - then try to remove that filter using the little "x" and submitting the form again.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1527/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1087931918 I_kwDOBm6k_c5A2IYO 1579 `.execute_write(... block=True)` should be the default behaviour simonw 9599 closed 0   Datasette 0.60 7571612 7 2021-12-23T18:54:28Z 2022-01-13T22:28:08Z 2021-12-23T19:18:26Z OWNER  

Every single piece of code I've written against the write APIs has used the block=True option to wait for the result.

Without that, it instead fires the write into the queue but then continues even before it has finished executing.

block=True should clearly be the default behaviour here!

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1579/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
520681725 MDU6SXNzdWU1MjA2ODE3MjU= 621 Syntax for ?_through= that works as a form field simonw 9599 open 0     7 2019-11-11T00:19:03Z 2021-12-18T01:42:33Z   OWNER  

The current syntax for ?_through= uses JSON to avoid any risk of confusion with table or column names that contain special characters.

This means you can't target a form field at it.

We should be able to support both - ?x.y.z=value for tables and columns with "regular" names, falling back to the current JSON syntax for columns or tables that won't work with the key/value syntax.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/621/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1055402144 PR_kwDOBm6k_c4unfnq 1512 New pattern for async view classes simonw 9599 closed 0     7 2021-11-16T21:55:44Z 2021-11-17T01:39:54Z 2021-11-17T01:39:44Z OWNER simonw/datasette/pulls/1512

Refs #878 - starting out with the new AsyncBase class implementing a pytest-inspired asyncio parallel execution mechanism.

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1512/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
972918533 MDU6SXNzdWU5NzI5MTg1MzM= 1438 Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions simonw 9599 open 0     7 2021-08-17T17:35:36Z 2021-08-18T00:22:23Z   OWNER  

Confirmed: https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A is a page where the URL correctly encoded % as %25 - but then in the HTML on that page that links to the CSV and JSON versions we get this:

html <p class="export-links">This data as <a href="/thesession.json?sql=select * from tunes where name like "%wise maid%"">json</a>, <a href="/thesession.csv?sql=select * from tunes where name like "%wise maid%"&_size=max">CSV</a> </p> Those CSV and JSON links are incorrect.

Originally posted by @simonw in https://github.com/simonw/datasette-publish-vercel/issues/48#issuecomment-900497579

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1438/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
642388564 MDU6SXNzdWU2NDIzODg1NjQ= 858 publish heroku does not work on Windows 10 simonlau 870912 open 0     7 2020-06-20T14:40:28Z 2021-06-10T17:44:09Z   NONE  

When executing "datasette publish heroku schools.db" on Windows 10, I get the following error

shell File "c:\users\dell\.virtualenvs\sec-schools-jn-cwk8z\lib\site-packages\datasette\publish\heroku.py", line 54, in heroku line.split()[0] for line in check_output(["heroku", "plugins"]).splitlines() File "c:\python38\lib\subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "c:\python38\lib\subprocess.py", line 489, in run with Popen(*popenargs, **kwargs) as process: File "c:\python38\lib\subprocess.py", line 854, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "c:\python38\lib\subprocess.py", line 1307, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file specified

Changing https://github.com/simonw/datasette/blob/55a6ffb93c57680e71a070416baae1129a0243b8/datasette/publish/heroku.py#L54

to

python line.split()[0] for line in check_output(["heroku", "plugins"], shell=True).splitlines()

as well as the other check_output() and call() within the same file leads me to another recursive error about temp files

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/858/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
904071938 MDU6SXNzdWU5MDQwNzE5Mzg= 1345 ?_nocol= does not interact well with default facets simonw 9599 closed 0     7 2021-05-27T18:39:55Z 2021-05-31T02:40:44Z 2021-05-31T02:31:21Z OWNER  

Clicking "Hide this column" on fips on https://covid-19.datasettes.com/covid/ny_times_us_counties shows this error:

https://covid-19.datasettes.com/covid/ny_times_us_counties?_nocol=fips

Invalid SQL

no such column: fips

The reason is that https://covid-19.datasettes.com/-/metadata sets up the following:

json "ny_times_us_counties": { "sort_desc": "date", "facets": [ "state", "county", "fips" ], It's setting fips as a default facet, which breaks if you attempt to remove the column using ?_nocol.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1345/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
898904402 MDU6SXNzdWU4OTg5MDQ0MDI= 1337 "More" link for facets that shows _facet_size=max results simonw 9599 closed 0     7 2021-05-23T00:08:51Z 2021-05-27T16:14:14Z 2021-05-27T16:01:03Z OWNER  

Original title: "More" link for facets that shows the full set of results

The simplest way to do this will be to have it link to a generated SQL query.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1332#issuecomment-846479062

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1337/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
672421411 MDU6SXNzdWU2NzI0MjE0MTE= 916 Support reverse pagination (previous page, has-previous-items) simonw 9599 open 0     7 2020-08-04T00:32:06Z 2021-04-03T23:43:11Z   OWNER  

I need this for datasette-graphql for full compatibility with the way Relay likes to paginate - using cursors for paginating backwards as well as for paginating forwards.

This may be the kick I need to get Datasette pagination to work in reverse too. Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-668305853

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/916/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
841456306 MDU6SXNzdWU4NDE0NTYzMDY= 1276 Invalid SQL: "no such table: pragma_database_list" on database page justinallen 1314318 closed 0     7 2021-03-26T00:03:53Z 2021-03-31T16:27:27Z 2021-03-28T23:52:31Z NONE  

Don't think this has been covered here yet. I'm a little stumped with this one and can't tell if it's a bug or I have something misconfigured.

Oddly, when running locally the usual list of tables populates (i.e. at /charts a list of tables in charts.db). But when on the web server it throws an Invalid SQL error and "no such table: pragma_database_list" below.

All the url endpoints seem to work fine aside from this - individual tables (/charts/chart_one), as well as stored queries (/charts/query_one).

Not sure if this has anything to do with upgrading to Datasette 0.55, or something to do with our setup, which uses a metadata build script similar to the one for the 538 server, or something else.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1276/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
842881221 MDU6SXNzdWU4NDI4ODEyMjE= 1281 Latest Datasette tags missing from Docker Hub simonw 9599 closed 0     7 2021-03-29T00:58:30Z 2021-03-29T01:41:48Z 2021-03-29T01:41:48Z OWNER  

Spotted this while testing https://github.com/simonw/datasette/issues/1249#issuecomment-808998719_

https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated isn't showing the tags for any version more recent than 0.54.1 - we are up to 0.56 now.

But the :latest tag is for the new 0.56 release.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1281/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
714377268 MDU6SXNzdWU3MTQzNzcyNjg= 991 Redesign application homepage simonw 9599 open 0     7 2020-10-04T18:48:45Z 2021-01-26T19:06:36Z   OWNER  

Most Datasette instances only host a single database, but the current homepage design assumes that it should leave plenty of space for multiple databases:

Reconsider this design - should the default show more information?

The Covid-19 Datasette homepage looks particularly sparse I think: https://covid-19.datasettes.com/

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/991/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
712984738 MDU6SXNzdWU3MTI5ODQ3Mzg= 987 Documented HTML hooks for JavaScript plugin authors simonw 9599 open 0     7 2020-10-01T16:10:14Z 2021-01-25T04:00:03Z   OWNER  

In #981 I added data-column= attributes to the <th> on the table page. These should become part of Datasette's documented API so JavaScript plugin authors can use them to derive things about the tables shown on a page (`datasette-cluster-map uses them as-of https://github.com/simonw/datasette-cluster-map/issues/18).

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/987/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
721050815 MDU6SXNzdWU3MjEwNTA4MTU= 1019 "Edit SQL" button on canned queries jsfenfen 639012 closed 0   0.51 6026070 7 2020-10-14T00:51:39Z 2020-10-23T19:44:06Z 2020-10-14T03:44:23Z CONTRIBUTOR  

Feature request: Would it be possible to add an "edit this query" button on canned queries? Clicking it would open the canned query as an editable sql query. I think the intent is to have named parameters to allow this, but sometimes you just gotta rewrite it?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1019/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
722724086 MDU6SXNzdWU3MjI3MjQwODY= 1025 Fix last remaining links to "/" that do not respect base_url simonw 9599 closed 0   0.51 6026070 7 2020-10-15T22:46:38Z 2020-10-23T19:44:06Z 2020-10-20T05:21:29Z OWNER  

Refs #1023 datasette % git grep '"/"' -- '*.html' datasette/templates/error.html: <a href="/">home</a> datasette/templates/patterns.html: <a href="/">home</a> / datasette/templates/query.html: <a href="/">home</a> /

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1025/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
440134714 MDU6SXNzdWU0NDAxMzQ3MTQ= 446 Define mechanism for plugins to return structured data simonw 9599 closed 0   Datasette 1.0 3268330 7 2019-05-03T17:00:16Z 2020-10-02T00:08:54Z 2020-10-02T00:08:47Z OWNER  

Several plugin hooks now expect plugins to return data in a specific shape - notably the new output format hook and the custom facet hook.

These use Python dictionaries right now but that's quite error prone: it would be good to have a mechanism that supported a more structured format.

Full list of current hooks is here: https://datasette.readthedocs.io/en/latest/plugins.html#plugin-hooks

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/446/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
705827457 MDU6SXNzdWU3MDU4Mjc0NTc= 971 Support the dbstat table simonw 9599 closed 0     7 2020-09-21T18:38:53Z 2020-09-21T19:00:02Z 2020-09-21T18:59:52Z OWNER  

dbstat is a table that is usually available on SQLite giving statistics about the database. For example:

https://fivethirtyeight.datasettes.com/fivethirtyeight?sql=SELECT+*+FROM+%22dbstat%22+WHERE+name%3D%27bachelorette%2Fbachelorette%27%3B

| name | path | pageno | pagetype | ncell | payload | unused | mx_payload | pgoffset | pgsize | |---------------------------|--------|----------|------------|---------|-----------|----------|--------------|------------|----------| | bachelorette/bachelorette | / | 89 | internal | 13 | 0 | 3981 | 0 | 360448 | 4096 | | bachelorette/bachelorette | /000/ | 91 | leaf | 66 | 3792 | 32 | 74 | 368640 | 4096 | | bachelorette/bachelorette | /001/ | 92 | leaf | 67 | 3800 | 14 | 74 | 372736 | 4096 | | bachelorette/bachelorette | /002/ | 93 | leaf | 65 | 3717 | 46 | 70 | 376832 | 4096 | | bachelorette/bachelorette | /003/ | 94 | leaf | 68 | 3742 | 6 | 71 | 380928 | 4096 | | bachelorette/bachelorette | /004/ | 95 | leaf | 70 | 3696 | 42 | 66 | 385024 | 4096 | | bachelorette/bachelorette | /005/ | 96 | leaf | 69 | 3721 | 22 | 71 | 389120 | 4096 | | bachelorette/bachelorette | /006/ | 97 | leaf | 70 | 3737 | 1 | 72 | 393216 | 4096 | | bachelorette/bachelorette | /007/ | 98 | leaf | 69 | 3728 | 15 | 69 | 397312 | 4096 | | bachelorette/bachelorette | /008/ | 99 | leaf | 73 | 3715 | 8 | 64 | 401408 | 4096 | | bachelorette/bachelorette | /009/ | 100 | leaf | 73 | 3705 | 18 | 62 | 405504 | 4096 | | bachelorette/bachelorette | /00a/ | 101 | leaf | 75 | 3681 | 32 | 62 | 409600 | 4096 | | bachelorette/bachelorette | /00b/ | 102 | leaf | 77 | 3694 | 9 | 62 | 413696 | 4096 | | bachelorette/bachelorette | /00c/ | 103 | leaf | 74 | 3673 | 45 | 62 | 417792 | 4096 | | bachelorette/bachelorette | /00d/ | 104 | leaf | 5 | 228 | 3835 | 48 | 421888 | 4096 |

Other than direct select * from dbsat queries it is completely invisible.

It would be cool if https://fivethirtyeight.datasettes.com/fivethirtyeight/dbstat didn't 404 (on databases for which that table was available).

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/971/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
688427751 MDU6SXNzdWU2ODg0Mjc3NTE= 956 Push to Docker Hub failed - but it shouldn't run for alpha releases anyway simonw 9599 closed 0     7 2020-08-29T01:09:12Z 2020-09-15T20:46:41Z 2020-09-15T20:36:34Z OWNER  

https://github.com/simonw/datasette/runs/1043709494?check_suite_focus=true

  • [x] This step should not run if a release is an alpha or beta
  • [x] When it DOES run it should work
  • [x] See it work for both an alpha and a non-alpha release, then close this ticket
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/956/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
677926613 MDU6SXNzdWU2Nzc5MjY2MTM= 931 Docker container is no longer being pushed (it's stuck on 0.45) simonw 9599 closed 0     7 2020-08-12T19:33:03Z 2020-08-12T21:36:20Z 2020-08-12T21:36:20Z OWNER  

e.g. https://travis-ci.org/github/simonw/datasette/jobs/717123725

Here's how it broke: --2020-08-12 03:08:17-- https://www.gaia-gis.it/gaia-sins/freexl-1.0.5.tar.gz Resolving www.gaia-gis.it (www.gaia-gis.it)... 212.83.162.51 Connecting to www.gaia-gis.it (www.gaia-gis.it)|212.83.162.51|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2020-08-12 03:08:18 ERROR 404: Not Found. The command '/bin/sh -c wget "https://www.gaia-gis.it/gaia-sins/freexl-1.0.5.tar.gz" && tar zxf freexl-1.0.5.tar.gz && cd freexl-1.0.5 && ./configure && make && make install' returned a non-zero code: 8 The command "docker build -f Dockerfile -t $REPO:$TRAVIS_TAG ." exited with 8. 0.07s$ docker tag $REPO:$TRAVIS_TAG $REPO:latest Error response from daemon: No such image: [secure]/datasette:0.47.1 The command "docker tag $REPO:$TRAVIS_TAG $REPO:latest" exited with 1. 0.08s$ docker push $REPO The push refers to repository [docker.io/[secure]/datasette] An image does not exist locally with the tag: [secure]/datasette The command "docker push $REPO" exited with 1. cache.2 store build cache

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/931/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
577578306 MDU6SXNzdWU1Nzc1NzgzMDY= 697 index.html is not reliably loaded from a plugin simonw 9599 closed 0     7 2020-03-08T22:37:55Z 2020-03-08T23:33:28Z 2020-03-08T23:11:27Z OWNER  

Lots of detail in https://github.com/simonw/datasette-search-all/issues/2 - short version is that I have a plugin with its own index.html template and Datasette intermittently fails to load it and uses the default index.html that ships with Datasette instead.

Related:

  • 689: "Templates considered" comment broken in >=0.35

  • 693: Variables from extra_template_vars() not exposed in _context=1 (may as well fix this while I'm in there)

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/697/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
576711589 MDU6SXNzdWU1NzY3MTE1ODk= 695 Update SQLite bundled with Docker container simonw 9599 closed 0     7 2020-03-06T05:42:12Z 2020-03-08T23:33:23Z 2020-03-06T06:15:27Z OWNER  

It's 3.26.0 at the moment: https://github.com/simonw/datasette/blob/af9cd4ca64652fae262e6f7b5d201f6e0adc989b/Dockerfile#L9-L11 Most recent release is 3.31.1: https://www.sqlite.org/releaselog/3_31_1.html

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/695/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
497171390 MDU6SXNzdWU0OTcxNzEzOTA= 577 Utility mechanism for plugins to render templates simonw 9599 closed 0   Datasette 1.0 3268330 7 2019-09-23T15:30:36Z 2020-02-04T20:26:20Z 2020-02-04T20:26:19Z OWNER  

Sometimes a plugin will need to render a template for some custom UI. We need a documented API for doing this, which ensures that everything will work correctly if you extend base.html etc.

See also #576. This could be a .render() method on the Datasette class, but that feels a bit weird - should that class also take responsibility for rendering?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/577/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
289425975 MDExOlB1bGxSZXF1ZXN0MTYzNTYxODMw 181 add "format sql" button to query page, uses sql-formatter bsmithgall 1957344 closed 0     7 2018-01-17T21:50:04Z 2019-11-11T03:08:25Z 2019-11-11T03:08:25Z NONE simonw/datasette/pulls/181

Cool project!

This fixes #136 using the suggested sql formatter library. I included the minified version in the bundle and added the relevant scripts to the codemirror includes instead of adding new files, though I could also add new files. I wanted to keep it all together, since the result of the format needs access to the editor in order to properly update the codemirror instance.

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/181/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
327365110 MDU6SXNzdWUzMjczNjUxMTA= 294 inspect should record column types simonw 9599 open 0     7 2018-05-29T15:10:41Z 2019-06-28T16:45:28Z   OWNER  

For each table we want to know the columns, their order and what type they are.

I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a geometry type for columns that are detected as SpatiaLite geometries.

Possible JSON design:

"columns": [{
    "name": "title",
    "type": "text"
}, ...]

Refs #276

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/294/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
309033998 MDU6SXNzdWUzMDkwMzM5OTg= 187 Windows installation error robmarkcole 11855322 closed 0     7 2018-03-27T16:04:37Z 2019-06-15T21:44:23Z 2019-06-15T21:44:23Z NONE  

On attempting install on a Win 7 PC with py 3.6.2 (Anaconda dist) I get the error:

Collecting uvloop>=0.5.3 (from Sanic==0.7.0->datasette) Downloading uvloop-0.9.1.tar.gz (1.8MB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 1.8MB 12.8MB/s Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\RCole\AppData\Local\Temp\pip-build-juakfqt8\uvloop\setup.py ", line 10, in <module> raise RuntimeError('uvloop does not support Windows at the moment') RuntimeError: uvloop does not support Windows at the moment

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/187/reactions",
    "total_count": 4,
    "+1": 4,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
432636432 MDU6SXNzdWU0MzI2MzY0MzI= 429 ?_where=sql-fragment parameter for table views simonw 9599 closed 0     7 2019-04-12T15:58:51Z 2019-04-15T10:48:01Z 2019-04-13T01:37:25Z OWNER  

Only available if arbitrary SQL is enabled (the default).

?_where=id in (1,2,3)&_where=id in (select tag_id from tags)

Allows any table (or view) page to have arbitrary additional extra_where clauses defined using the URL!

This would be extremely useful for building JavaScript applications against the Datasette API that only need on extra tiny bit of SQL but still want to benefit from other table view features like faceting.

Would be nice if this could take :named parameters and have them filled in via querystring as well.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/429/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
336465018 MDU6SXNzdWUzMzY0NjUwMTg= 329 Travis should push tagged images to Docker Hub for each release simonw 9599 closed 0     7 2018-06-28T04:01:31Z 2018-11-05T06:54:10Z 2018-11-05T06:53:28Z OWNER  

https://sebest.github.io/post/using-travis-ci-to-build-docker-images/

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/329/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
273127694 MDU6SXNzdWUyNzMxMjc2OTQ= 57 Ship a Docker image of the whole thing simonw 9599 closed 0     7 2017-11-11T07:51:28Z 2018-06-28T04:01:51Z 2018-06-28T04:01:38Z OWNER  

The generated Docker images can then just inherit from that. This will speed up deploys as no need to pip install anything.

  • [x] Ship that image to Docker Hub
  • [ ] Update the generated Dockerfile to use it
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/57/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
269731374 MDU6SXNzdWUyNjk3MzEzNzQ= 44 ?_group_count=country - return counts by specific column(s) simonw 9599 closed 0     7 2017-10-30T19:50:32Z 2018-04-26T15:09:58Z 2018-04-26T15:09:58Z OWNER  

Imagine if this:

https://stateless-datasets-jykibytogk.now.sh/flights-07d1283/airports.jsono?country__contains=gu&_group_count=country

Turned into this:

https://stateless-datasets-jykibytogk.now.sh/flights-07d1283?sql=select%20country,%20count(*)%20as%20group_count_country%20from%20airports%20where%20country%20like%20%27%gu%%27%20group%20by%20country%20order%20by%20group_count_country%20desc

This would involve introducing a new precedent of query string arguments that start with an _ having special meanings. While we're at it, could try adding _fields=x,y,z

Tasks:

  • [x] Get initial version working
  • [ ] Refactor code to not just "pretend to be a view"
  • [ ] Get foreign key relationships expanded
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/44/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
309558826 MDU6SXNzdWUzMDk1NTg4MjY= 190 Keyset pagination doesn't work correctly for compound primary keys simonw 9599 closed 0     7 2018-03-28T22:45:06Z 2018-03-30T06:31:15Z 2018-03-30T06:26:28Z OWNER  

Consider https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key

The next= link is to d,v:

https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key?_next=d%2Cv

But that page starts with:

The next key in the sequence should be d,w. Also we should return the full a-z of the ones that start with the letter e - in this example we only return e-w, e-x, e-y and e-z

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/190/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);
Powered by Datasette · Queries took 539.507ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows