github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1886#issuecomment-1313252879 | https://api.github.com/repos/simonw/datasette/issues/1886 | 1313252879 | IC_kwDOBm6k_c5ORqYP | 883348 | 2022-11-14T08:10:23Z | 2022-11-14T08:10:23Z | CONTRIBUTOR | Hi @simonw and thanks for the great tools you're publishing, your dedication is inspiring! I work for the French Ministry of Culture on a surveying tool for objects protected for their historical value. It is part of a program building modern public services called [beta.gouv.fr](https://beta.gouv.fr/). In that context I'm using data published by the Ministry that I have ingested into datasette and published on a free Fly instance : https://collectif-objets-datasette.fly.dev . I have also ingested another data set with infos about french cities on this instance so that I can perform joined queries. The surveying tool synchronizes its data regularly from this datasette instance, and I also use it to perform queries when asked generic questions about the distribution of objects. (The data is not very accessible as it's undocumented and for internal usage mostly) | { "total_count": 3, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 3, "rocket": 0, "eyes": 0 } |
1447050738 | |
https://github.com/simonw/sqlite-utils/issues/448#issuecomment-1297703307 | https://api.github.com/repos/simonw/sqlite-utils/issues/448 | 1297703307 | IC_kwDOCGYnMM5NWWGL | 167893 | 2022-10-31T21:23:51Z | 2022-10-31T21:27:32Z | CONTRIBUTOR | The Windows aspect is a red herring: OP's sample above produces the same error on Linux. (Though I don't know what's going on with the CI). The same error can also be obtained by passing an `io` from a file opened in non-binary mode (`'r'` as opposed to `'rb'`) to `rows_from_file()`. This is how I got here. The fix for my case is easy: open the file in mode `'rb'`. The analagous fix for OP's problem also works: use `BytesIO` in place of `StringIO`. Minimal test case (derived from [utils.py](https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/utils.py#L304)): ``` python import io from typing import cast #fp = io.StringIO("id,name\n1,Cleo") # error fp = io.BytesIO(bytes("id,name\n1,Cleo", encoding='utf-8')) # okay reader = io.BufferedReader(cast(io.RawIOBase, fp)) reader.peek(1) # exception thrown here ``` I see the signature of `rows_from_file()` correctly has `fp: BinaryIO` but I guess you'd need either a runtime type check for that (not all `io`s have `mode()`), or to catch the `AttributeError` on `peek()` to produce a better error for users. Neither option is ideal. Some thoughts on testing binary-ness of `io`s in this SO question: https://stackoverflow.com/questions/44584829/how-to-determine-if-file-is-opened-in-binary-or-text-mode | { "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1279144769 | |
https://github.com/simonw/datasette/pull/2052#issuecomment-1530822437 | https://api.github.com/repos/simonw/datasette/issues/2052 | 1530822437 | IC_kwDOBm6k_c5bPn8l | 193185 | 2023-05-02T03:35:30Z | 2023-05-02T16:02:38Z | CONTRIBUTOR | Also, just checking - is this how I'd write bulletproof plugin registration code that is robust against the order in which the script tags load (eg if both my code and the Datasette code are loaded via a `<script async src='...'/>` tag)? ```js if (window.__DATASETTE__) go(window.__DATASETTE__); else document.addEventListener("datasette_init", (evt) => go(evt.detail)); function go(manager) { manager.registerPlugin(...) } ``` I don't know if it'd make sense, but you could also consider the asynchronous queuing pattern that Google Analytics uses (see [this Stack Overflow post](https://stackoverflow.com/questions/6963779/whats-the-name-of-google-analytics-async-design-pattern-and-where-is-it-used) for more details): ```js __DATASETTE__ = __DATASETTE__ || []; __DATASETTE__.push(go); function go(manager) { manager.registerPlugin(...); } ``` | { "total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
1651082214 | |
https://github.com/simonw/datasette/issues/394#issuecomment-602907207 | https://api.github.com/repos/simonw/datasette/issues/394 | 602907207 | MDEyOklzc3VlQ29tbWVudDYwMjkwNzIwNw== | 127565 | 2020-03-23T23:12:18Z | 2020-03-23T23:12:18Z | CONTRIBUTOR | This would also be useful for running Datasette in Jupyter notebooks on [Binder](https://mybinder.org/). While you can use [Jupyter-server-proxy](https://github.com/jupyterhub/jupyter-server-proxy) to access Datasette on Binder, the links are broken. Why run Datasette on Binder? I'm developing a [range of Jupyter notebooks](https://glam-workbench.github.io/) that are aimed at getting humanities researchers to explore data from libraries, archives, and museums. Many of them are aimed at researchers with limited digital skills, so being able to run examples in Binder without them installing anything is fantastic. For example, there are a [series of notebooks](https://glam-workbench.github.io/trove-harvester/) that help researchers harvest digitised historical newspaper articles from Trove. The metadata from this harvest is saved as a CSV file that users can download. I've also provided some extra notebooks that use Pandas etc to demonstrate ways of analysing and visualising the harvested data. But it would be really nice if, after completing a harvest, the user could spin up Datasette for some initial exploration of their harvested data without ever leaving their browser. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
396212021 | |
https://github.com/simonw/sqlite-utils/issues/121#issuecomment-655652679 | https://api.github.com/repos/simonw/sqlite-utils/issues/121 | 655652679 | MDEyOklzc3VlQ29tbWVudDY1NTY1MjY3OQ== | 79913 | 2020-07-08T17:24:46Z | 2020-07-08T17:24:46Z | CONTRIBUTOR | Better transaction handling would be really great. Some of my thoughts on implementing better transaction discipline are in https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655239728. My preferences: - Each CLI command should operate in a single transaction so that either the whole thing succeeds or the whole thing is rolled back. This avoids partially completed operations when an error occurs part way through processing. Partially completed operations are typically much harder to recovery from gracefully and may cause inconsistent data states. - The Python API should be transaction-agnostic and rely on the caller to coordinate transactions. Only the caller knows how individual insert, create, update, etc operations/methods should be bundled conceptually into transactions. When the caller is the CLI, for example, that bundling would be at the CLI command-level. Other callers might want to break up operations into multiple transactions. Transactions are usually most useful when controlled at the application-level (like logging configuration) instead of the library level. The library needs to provide an API that's conducive to transaction use, though. - The Python API should provide a context manager to provide consistent transactions handling with more useful defaults than Python's `sqlite3` module. The latter issues implicit `BEGIN` statements by default for most DML (`INSERT`, `UPDATE`, `DELETE`, … but not `SELECT`, I believe), but **not** DDL (`CREATE TABLE`, `DROP TABLE`, `CREATE VIEW`, …). Notably, the `sqlite3` module doesn't issue the implicit `BEGIN` until the first DML statement. It _does not_ issue it when entering the `with conn` block, like other DBAPI2-compatible modules do. The `with conn` block for `sqlite3` only arranges to commit or rollback an existing transaction when exiting. Including DDL and `SELECT`s in transactions is important for operation consistency, though. There are several existing bugs.python.org tickets about this and future changes are in the works, but sql… | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
652961907 | |
https://github.com/simonw/sqlite-utils/pull/189#issuecomment-717359145 | https://api.github.com/repos/simonw/sqlite-utils/issues/189 | 717359145 | MDEyOklzc3VlQ29tbWVudDcxNzM1OTE0NQ== | 35681 | 2020-10-27T16:20:32Z | 2020-10-27T16:20:32Z | CONTRIBUTOR | No problem. I added a test. Let me know if it looks sufficient or if you want me to to tweak something! If you don't mind, would you tag this PR as "hacktoberfest-accepted"? If you do mind, no problem and I'm sorry for asking :) My kiddos like the shirts. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
729818242 | |
https://github.com/simonw/datasette/issues/1274#issuecomment-805214307 | https://api.github.com/repos/simonw/datasette/issues/1274 | 805214307 | MDEyOklzc3VlQ29tbWVudDgwNTIxNDMwNw== | 7476523 | 2021-03-23T20:12:29Z | 2021-03-23T20:12:29Z | CONTRIBUTOR | One issue I could see with adding first class support for metadata in hjson format is that this would require adding an additional dependency to handle this, for a feature that would be unused by many users. I wonder if this could fit in as a plugin instead; if a hook existed for loading metadata (maybe as part of https://github.com/simonw/datasette/issues/860) the metadata could then come from any source, as specified by plugins, e.g. hjson, toml, XML, a database table etc. Until/unless this exists, a few ideas for how you could add comments: - Using YAML as you suggest. - A common pattern is adding a `"comment"` key for comments to any object in JSON - I don't think including an unnecessary key like this would break anything in Datasette, but not certain. - You could use another tool as a preprocessor for your JSON metadata - e.g. hjson or Jsonnet. You'd write the metadata in that format, and then convert that into JSON to actually use as your final metadata. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
839008371 | |
https://github.com/simonw/datasette/issues/1286#issuecomment-815978405 | https://api.github.com/repos/simonw/datasette/issues/1286 | 815978405 | MDEyOklzc3VlQ29tbWVudDgxNTk3ODQwNQ== | 192568 | 2021-04-08T16:47:29Z | 2021-04-10T03:59:00Z | CONTRIBUTOR | This worked for me: `<td class="col-{{ cell.column|to_css_class }} type-{{ cell.value_type }}">{{ cell.value | replace('", "','; ') | replace('[\"','') | replace('\"]','')}}</td>` I'm sure there is a prettier (and more flexible) way, but for now, this is ever-so-much more pleasant to look at. ------ AFTER: <img width="778" alt="Screen Shot 2021-04-08 at 12 27 36 PM" src="https://user-images.githubusercontent.com/192568/114062829-f2f20c00-9865-11eb-891d-b0c348a9b433.png"> ------ BEFORE: <img width="795" alt="Screen Shot 2021-04-08 at 12 25 54 PM" src="https://user-images.githubusercontent.com/192568/114062871-01402800-9866-11eb-91ce-91efe4ee45cd.png"> (Note: I didn't figure out how to have one item have no semicolon, while multi-items close with a semicolon, but this is good enough for now. I also didn't figure out how to set up a new jinja filter. I don't want to add to /datasette/utils/__init__.py as I assume that would get overwritten when upgrading datasette. Having a starter guide on creating jinja filters in datasette would be helpful. (The jinja documentation isn't datasette-specific enough for me to quite nail it.) | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849220154 | |
https://github.com/simonw/datasette/pull/1455#issuecomment-913001282 | https://api.github.com/repos/simonw/datasette/issues/1455 | 913001282 | IC_kwDOBm6k_c42a0tC | 51016 | 2021-09-04T16:31:24Z | 2021-09-04T16:31:24Z | CONTRIBUTOR | I love it! maybe 'researchers' instead? Or 'scientists and researchers'? | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
988325628 | |
https://github.com/simonw/datasette/issues/1284#issuecomment-949604763 | https://api.github.com/repos/simonw/datasette/issues/1284 | 949604763 | IC_kwDOBm6k_c44mdGb | 536941 | 2021-10-22T12:54:34Z | 2021-10-22T12:54:34Z | CONTRIBUTOR | i'm going to take a swing at this today. we'll see. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
845794436 | |
https://github.com/simonw/datasette/issues/1522#issuecomment-976117989 | https://api.github.com/repos/simonw/datasette/issues/1522 | 976117989 | IC_kwDOBm6k_c46LmDl | 813732 | 2021-11-23T03:00:34Z | 2021-11-23T03:00:34Z | CONTRIBUTOR | I tried deploying the most recent version of the Dockerfile in this thread ([link to comment](https://github.com/simonw/datasette/issues/1522#issuecomment-974605128)), and after trying a few different different combinations, I was only successful when I used `--no-cpu-throttling` ("CPU Is always allocated" in the UI) Using this method, I got a very similar issue to you: The first time I'd load the site I'd get a 503. But after that first load, I didn't get the issue again. It would re-occur if the service started from cold boot. I suspect this is a race condition in the supervisord configuration. The errors I got were the same `Connection refused: AH00957: http: attempt to connect to 127.0.0.1:8001 (127.0.0.1) failed`, and that seems to indicate that `datasette` hadn't yet started. Looking at the order of logs getting back, the processes reported successfully completing loading after the first 503 was returned, so that makes me think race condition. I can replicate this locally, if I `docker run` and request `localhost:5000/prefix` _before_ I get the `datasette entered RUNNING state` message. Cloud Run wakes up when requests are received, so this test would semi-replicate that, but local docker would be the equivalent of a persistent process, hence it doesn't normally exhibit the same issues. Unfortunately supervisor/supervisor issue 122 (not linking as to prevent cross-project link spam) seems to say that dependency chaining is a feature that's been asked for for a long time, but hasn't been implemented. You could try some suggestions in that thread. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1058896236 | |
https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030740653 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | 1030740653 | IC_kwDOCGYnMM49b9qt | 25778 | 2022-02-06T02:57:17Z | 2022-02-06T02:57:17Z | CONTRIBUTOR | I like the idea of having stock conversions you could import. I'd actually move them to a dedicated module (call it `sqlite_utils.conversions` or something), because it's different from other utilities. Maybe they even take configuration, or they're composable. ```python from sqlite_utils.conversions import LongitudeLatitude db["places"].insert( { "name": "London", "lng": -0.118092, "lat": 51.509865, }, conversions={"point": LongitudeLatitude("lng", "lat")}, ) ``` I would definitely use that for every CSV I get with lat/lng columns where I actually need GeoJSON. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1124731464 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-1066222323 | https://api.github.com/repos/simonw/datasette/issues/1384 | 1066222323 | IC_kwDOBm6k_c4_jULz | 2670795 | 2022-03-14T00:36:42Z | 2022-03-14T00:36:42Z | CONTRIBUTOR | > Ah, sorry, I didn't get what you were saying you the first time. Using _metadata_local in that way makes total sense -- I agree, refreshing metadata each cell was seeming quite excessive. Now I'm on the same page! :) All good. Report back any issues you find with this stuff. Metadata/dynamic config hasn't been tested widely outside of what I've done AFAIK. If you find a strong use case for async meta, it's going to be better to know sooner rather than later! | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317681193 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317681193 | IC_kwDOBm6k_c5Oijgp | 95570 | 2022-11-16T21:19:13Z | 2022-11-16T21:19:13Z | CONTRIBUTOR | Alright, added Cmd+Enter to submit (Ctrl+Enter on Windows as well bc of using Meta-Enter on codemirror). We can make that MacOS only by changing the combo to Cmd+Enter specifically but I think it's probably fine to have both. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1159#issuecomment-1399589414 | https://api.github.com/repos/simonw/datasette/issues/1159 | 1399589414 | IC_kwDOBm6k_c5TbAom | 193185 | 2023-01-22T19:48:41Z | 2023-01-22T19:48:41Z | CONTRIBUTOR | Hey @lovasoa, I hope you don't mind - I pulled this PR into [datasette-ui-extras](https://github.com/cldellow/datasette-ui-extras), a plugin I'm making that collects UI tweaks to Datasette. You can apply it to your own Datasette instance by running `datasette install datasette-ui-extras` | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
774332247 | |
https://github.com/simonw/datasette/issues/2001#issuecomment-1403084856 | https://api.github.com/repos/simonw/datasette/issues/2001 | 1403084856 | IC_kwDOBm6k_c5ToWA4 | 193185 | 2023-01-25T04:31:02Z | 2023-01-25T04:31:02Z | CONTRIBUTOR | Aha, it's user error on my part. Adding ``` sqlite3_db_config.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int, ctypes.c_int] ``` makes it work reliably both on the CLI and from datasette, and now I can reproduce the errors you mentioned in the issue description. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1553615704 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-1404070841 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 1404070841 | IC_kwDOCGYnMM5TsGu5 | 536941 | 2023-01-25T18:47:18Z | 2023-01-25T18:47:18Z | CONTRIBUTOR | i'll adopt this PR to make the changes @simonw suggested https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567932 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/datasette/issues/2023#issuecomment-1425974877 | https://api.github.com/repos/simonw/datasette/issues/2023 | 1425974877 | IC_kwDOBm6k_c5U_qZd | 193185 | 2023-02-10T15:32:41Z | 2023-02-10T15:32:41Z | CONTRIBUTOR | I think this feature was removed in Datasette 0.61 and moved to a plugin. People who want hashed URLs can use the [datasette-hashed-urls](https://docs.datasette.io/en/stable/performance.html#performance-hashed-urls) plugin to achieve the same affect. It looks like you're trying to disable hashed urls, so I think you can just remove that config setting and things will work. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1579695809 | |
https://github.com/simonw/datasette/issues/1258#issuecomment-1437671409 | https://api.github.com/repos/simonw/datasette/issues/1258 | 1437671409 | IC_kwDOBm6k_c5VsR_x | 2670795 | 2023-02-20T23:39:58Z | 2023-02-20T23:39:58Z | CONTRIBUTOR | This is pretty annoying for FTS because sqlite throws an error instead of just doing something like returning all or no results. This makes users who are unfamiliar with SQL and Datasette think the canned query page is broken and is a frequent source of confusion. To anyone dealing with this: My solution is to modify the canned query so that it returns no results which cues people to fill in the blank parameters. So instead of `emails_fts match escape_fts(:search))` My canned queries now look like this: `emails_fts match escape_fts(iif(:search=="", "*", :search))` There are no asterisks in my data so the result is always blank. Ultimately it would be nice to be able to handle this in the metadata. Either making some named parameters required or setting some default values. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
828858421 | |
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1444474487 | https://api.github.com/repos/simonw/sqlite-utils/issues/433 | 1444474487 | IC_kwDOCGYnMM5WGO53 | 167893 | 2023-02-24T20:57:43Z | 2023-02-24T22:22:18Z | CONTRIBUTOR | I think I see what is happening here, although I haven't quite work out a fix yet. Usually: * `click.progressbar.render_progress()` renders the cursor invisible on each invocation (update of the bar) * When the progress bar goes out of scope, the `__exit()__` method is invoked, which calls `render_finish()` to make the cursor re-appear. (See terminal escape sequences `BEFORE_BAR` and `AFTER_BAR` in click). However the sqlite-utils `utils.file_progress` context manager wraps `click.progressbar` and yields an instance of a helper class: ``` python @contextlib.contextmanager def file_progress(file, silent=False, **kwargs): ... with click.progressbar(length=file_length, **kwargs) as bar: yield UpdateWrapper(file, bar.update) ``` The yielded `UpdateWrapper` goes out of scope quickly and `click.progressbar.__exit__()` is called. The cursor is made un-invisible. Hoewever `bar` is still live and so when the caller iterates on the yielded wrapper this invokes the bar's update method, calling `render_progress()`, each time printing the "make cursor invisible" escape code. The `progressbar.__exit__` function is not called again, so the cursor doesn't re-appear. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1239034903 | |
https://github.com/simonw/datasette/pull/2052#issuecomment-1548617257 | https://api.github.com/repos/simonw/datasette/issues/2052 | 1548617257 | IC_kwDOBm6k_c5cTgYp | 193185 | 2023-05-15T21:32:20Z | 2023-05-15T21:32:20Z | CONTRIBUTOR | > Were you picturing that the whole plugin config object could be returned as a promise, or that the individual hooks (like makeColumnActions or makeAboveTablePanelConfigs supported returning a promise of arrays instead only returning plain arrays? The latter - that you could return a promise of arrays, so it parallels the ["await me maybe" pattern in Datasette](https://simonwillison.net/2020/Sep/2/await-me-maybe/), where you can return either a value, a callable or an awaitable. > I have a hunch that what you're describing might be achievable without adding Promises to the API with something Oops, I did a poor job explaining. Yes, this would work - but it requires me to continue to communicate the column names out of band (in order to fetch the facet data per-column before registering my plugin), vs being able to re-use them from the plugin implementation. This isn't that big of a deal - it'd be a nice ergonomic improvement, but nowhere near as a big of an improvement as having an officially sanctioned way to add stuff to the column menus in the first place. This could also be layered on in a future commit without breaking v1 users, too, so it's not at all urgent. > especially if those lines are encapsulated by a function we provide (maybe something that's available on the window provided by Datasette as an inline script tag Ah, this is maybe the the key point. Since it's all hosted inside Datasette, Datasette can provide some arbitrary sugar to make it easier to work with. My experience with async scripts in JS is that people sometimes don't understand the race conditions inherent to them. If they copy/paste from a tutorial, it does just work. But then they'll delete half the code, and by chance it still works on their machine/Datasette templates, and now someone's headed for an annoying debugging session -- maybe them, maybe someone else who tries to re-use their plugin. Again, a fairly minor thing, though. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1651082214 | |
https://github.com/simonw/datasette/issues/2093#issuecomment-1613895188 | https://api.github.com/repos/simonw/datasette/issues/2093 | 1613895188 | IC_kwDOBm6k_c5gMhYU | 15178711 | 2023-06-29T22:51:53Z | 2023-06-29T22:51:53Z | CONTRIBUTOR | I agree with not liking `metadata.json` stuff in a `datasette.*` config file. Editing description of a table/column in a file like `datasette.*` seems odd to me. Though since plugin configuration currently lives in `metadata.json`, I think it should be removed from there and placed in `datasette.*`, at least for top-level config like `datasette-auth-github`'s config. Keeping `metadata.json` strictly for documentation/licensing/column units makes sense to me, but anything plugin related should be in some config file, like `datasette.*`. And ya, supporting both `datasette.*` and CLI flags makes a lot of sense to me. Any `--setting` flag should override anything in `datasette.*` for easier debugging, with possibly a warning message so people don't get confused. Same with `--port` and a port defined in `datasette.*` | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1781530343 | |
https://github.com/simonw/datasette/pull/2052#issuecomment-1616095810 | https://api.github.com/repos/simonw/datasette/issues/2052 | 1616095810 | IC_kwDOBm6k_c5gU6pC | 15178711 | 2023-07-01T20:31:31Z | 2023-07-01T20:31:31Z | CONTRIBUTOR | > Just curious, is there a query that can be used to compile this programmatically, or did you identify these through memory? I just did a github search for `user:simonw "def extra_js_urls("` ! Though I'm sure other plugins made by people other than Simon also exist out there https://github.com/search?q=user%3Asimonw+%22def+extra_js_urls%28%22&type=code | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1651082214 | |
https://github.com/simonw/datasette/issues/46#issuecomment-344810525 | https://api.github.com/repos/simonw/datasette/issues/46 | 344810525 | MDEyOklzc3VlQ29tbWVudDM0NDgxMDUyNQ== | 54999 | 2017-11-16T04:11:25Z | 2017-11-16T04:11:25Z | CONTRIBUTOR | @simonw On the spatialite support, here is some info to make it work and a screenshot: <img width="1230" alt="screen shot 2017-11-15 at 11 08 14 pm" src="https://user-images.githubusercontent.com/54999/32873420-f8a6d5a0-ca59-11e7-8a73-7d58d467e413.png"> I used the following Dockerfile: ``` FROM prolocutor/python3-sqlite-ext:3.5.1-spatialite as build RUN mkdir /code ADD . /code/ RUN pip install /code/ EXPOSE 8001 CMD ["datasette", "serve", "/code/ne.sqlite", "--host", "0.0.0.0"] ``` and added this to `prepare_connection`: ``` conn.enable_load_extension(True) conn.execute("SELECT load_extension('/usr/local/lib/mod_spatialite.so')") ``` | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
271301468 | |
https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-748562330 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31 | 748562330 | MDEyOklzc3VlQ29tbWVudDc0ODU2MjMzMA== | 41546558 | 2020-12-20T04:45:08Z | 2020-12-20T04:45:08Z | CONTRIBUTOR | Fixes the issue mentioned here: https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-748436115 | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771511344 | |
https://github.com/simonw/datasette/pull/2052#issuecomment-1613778296 | https://api.github.com/repos/simonw/datasette/issues/2052 | 1613778296 | IC_kwDOBm6k_c5gME14 | 15178711 | 2023-06-29T20:36:09Z | 2023-06-29T20:36:09Z | CONTRIBUTOR | Ok @hydrosquall a couple things before this PR should be good to go: - Can we move `datasette/static/table-example-plugins.js` into `demos/plugins/static`? - For `datasetteManager.VERSION`, can we fill that in or just comment it out for now? Not sure how difficult it'll be to inject it server-side. I imagine we could also have a small build process with esbuild/rollup that just injects a version string into `manager.js` directly, so we don't have to worry about server-rendering (but that can be a future PR) In terms of how to integrate this into Datasette, a few options I can see working: - Push this as-is and figure it out before the next release - Hide this feature behind a settings flag (`--setting unstable-js-plugins on`) and use that setting to hide/show `<script src="{{ urls.static('datasette-manager.js') }}" defer></script>` in `base.html` I'll let @simonw decide which one to work with. I kindof like the idea of having an "unstable" opt-in process to enable JS plugins, to give us time to try it out with a wide variety of plugins until we feel its ready. I'm also curious to see how "plugins for a plugin' would work, like #1542. For example, if the leaflet plugin showed default markers, but also included its own hook for other plugins to add more markers/styling. I'm imagine that the individual plugin would re-create their own plugin system compared to this, since handling "plugins of plugins" at the top with Datasette seems really convoluted. Also for posterity, here's a list of Simon's Datasette plugins that use "extra_js_urls()", which probably means they can be ported/re-written to use this new plugin system: - [`datasette-vega`](https://github.com/simonw/datasette-vega/blob/00de059ab1ef77394ba9f9547abfacf966c479c4/datasette_vega/__init__.py#L25) - [`datasette-cluster-map`](https://github.com/simonw/datasette-cluster-map/blob/795d25ad9ff6cba0307191f44fecc8f8070bef5c/datasette_cluster_map/__init__.py#L14) - [`datasette-leaflet-geojson`](https://github.com/simonw/datasette-leaflet… | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
1651082214 | |
https://github.com/simonw/datasette/issues/2104#issuecomment-1641082395 | https://api.github.com/repos/simonw/datasette/issues/2104 | 1641082395 | IC_kwDOBm6k_c5h0O4b | 15178711 | 2023-07-18T22:41:37Z | 2023-07-18T22:41:37Z | CONTRIBUTOR | For filtering virtual table's "shadow tables" (ex the FTS5 _content and most the spatialite tables), you can use `pragma_table_list` (first appeared in SQLite 3.37 (2021-11-27), which has a `type` column that calls out `type="shadow"` tables https://www.sqlite.org/pragma.html#pragma_table_list | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
1808215339 | |
https://github.com/simonw/datasette/pull/81#issuecomment-344125441 | https://api.github.com/repos/simonw/datasette/issues/81 | 344125441 | MDEyOklzc3VlQ29tbWVudDM0NDEyNTQ0MQ== | 50527 | 2017-11-14T02:24:54Z | 2017-11-14T02:24:54Z | CONTRIBUTOR | Oops, if I jumped the gun. I saw the project in my github activity feed and saw some low hanging fruit :) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273595473 | |
https://github.com/simonw/datasette/issues/57#issuecomment-344145265 | https://api.github.com/repos/simonw/datasette/issues/57 | 344145265 | MDEyOklzc3VlQ29tbWVudDM0NDE0NTI2NQ== | 247192 | 2017-11-14T04:45:38Z | 2017-11-14T04:45:38Z | CONTRIBUTOR | I'm happy to contribute this. Just let me know if you want a Dockerfile for development or production purposes, or both. If it's prod then we can just pip install the source from pypi, otherwise for dev we'll need a `requirements.txt` to speed up rebuilds. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273127694 | |
https://github.com/simonw/datasette/issues/57#issuecomment-344147583 | https://api.github.com/repos/simonw/datasette/issues/57 | 344147583 | MDEyOklzc3VlQ29tbWVudDM0NDE0NzU4Mw== | 247192 | 2017-11-14T05:03:47Z | 2017-11-14T05:03:47Z | CONTRIBUTOR | Let me know if you'd like a PR. The image is usable as `docker run --rm -t -i -p 9000:8001 -v $(pwd)/db:/db datasette datasette serve /db/chinook.db` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273127694 | |
https://github.com/simonw/datasette/issues/57#issuecomment-344151223 | https://api.github.com/repos/simonw/datasette/issues/57 | 344151223 | MDEyOklzc3VlQ29tbWVudDM0NDE1MTIyMw== | 247192 | 2017-11-14T05:32:28Z | 2017-11-14T05:33:03Z | CONTRIBUTOR | The pattern is called "multi-stage builds". And the result is a svelte 226MB image (201MB for 3.6-slim) vs 700MB+ for the full image. It's possible to get it even smaller, but that takes a lot more work. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273127694 | |
https://github.com/simonw/datasette/issues/88#issuecomment-344430689 | https://api.github.com/repos/simonw/datasette/issues/88 | 344430689 | MDEyOklzc3VlQ29tbWVudDM0NDQzMDY4OQ== | 15543 | 2017-11-14T23:08:22Z | 2017-11-14T23:08:22Z | CONTRIBUTOR | > I'm getting an internal server error on http://run.plnkr.co/preview/cj9zlf1qc0003414y90ajkwpk/ at the moment Sorry about that - here's a working version on Netlify: https://nhs-england-map.netlify.com | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273775212 | |
https://github.com/simonw/datasette/pull/104#issuecomment-344710204 | https://api.github.com/repos/simonw/datasette/issues/104 | 344710204 | MDEyOklzc3VlQ29tbWVudDM0NDcxMDIwNA== | 21148 | 2017-11-15T19:57:50Z | 2017-11-15T19:57:50Z | CONTRIBUTOR | A first basic stab at making this work, just to prove the approach. Right now this requires [a Heroku CLI plugin](https://github.com/heroku/heroku-builds), which seems pretty unreasonable. I think this can be replaced with direct API calls, which could clean up a lot of things. But I wanted to prove it worked first, and it does. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274284246 | |
https://github.com/simonw/datasette/pull/107#issuecomment-344811268 | https://api.github.com/repos/simonw/datasette/issues/107 | 344811268 | MDEyOklzc3VlQ29tbWVudDM0NDgxMTI2OA== | 3433657 | 2017-11-16T04:17:45Z | 2017-11-16T04:17:45Z | CONTRIBUTOR | Thanks for the guidance. I added a unit test and made a slight change to utils.py. I didn't realize this, but evidently string.format only complains if you supply less arguments than there are format placeholders, so the original commit worked, but was adding a superfluous named param. I added a conditional that prevents the named param from being created and ensures the correct number of args are passed to sting.format. It has the side effect of hiding the SQL query in /templates/table.html when there are no other where clauses--not sure if that's the desired outcome here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274343647 | |
https://github.com/simonw/datasette/issues/46#issuecomment-345002908 | https://api.github.com/repos/simonw/datasette/issues/46 | 345002908 | MDEyOklzc3VlQ29tbWVudDM0NTAwMjkwOA== | 54999 | 2017-11-16T17:47:49Z | 2017-11-16T17:47:49Z | CONTRIBUTOR | I'll try to find alternatives to the Dockerfile option - I also think we should not use that old one without sources or license. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
271301468 | |
https://github.com/simonw/datasette/pull/107#issuecomment-345117690 | https://api.github.com/repos/simonw/datasette/issues/107 | 345117690 | MDEyOklzc3VlQ29tbWVudDM0NTExNzY5MA== | 3433657 | 2017-11-17T01:29:41Z | 2017-11-17T01:29:41Z | CONTRIBUTOR | Thanks for bearing with me. I was getting a message about my branch diverging when I tried to push after rebasing, so I merged master into isnull, seems like that did the trick. Let me know if I should make any corrections. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274343647 | |
https://github.com/simonw/datasette/pull/104#issuecomment-345452669 | https://api.github.com/repos/simonw/datasette/issues/104 | 345452669 | MDEyOklzc3VlQ29tbWVudDM0NTQ1MjY2OQ== | 21148 | 2017-11-18T16:18:45Z | 2017-11-18T16:18:45Z | CONTRIBUTOR | I'd like to do a bit of cleanup, and some error checking in case heroku/heroku-builds isn't installed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274284246 | |
https://github.com/simonw/datasette/issues/105#issuecomment-345503897 | https://api.github.com/repos/simonw/datasette/issues/105 | 345503897 | MDEyOklzc3VlQ29tbWVudDM0NTUwMzg5Nw== | 198537 | 2017-11-19T09:38:08Z | 2017-11-19T09:38:08Z | CONTRIBUTOR | Thanks, I wrote this very simple reader because the default approach as described on the Datahub pages seemed to complicated. I had metadata from the `datapackage.json` attached to the returned DataFrames but removed this due to some attribute handling change in the latest Pandas version. This could also be useful for getting from Data Package to SQL db: https://github.com/frictionlessdata/tableschema-sql-py I maintain a few climate science related dataset at https://github.com/openclimatedata/ The Data Retriever (mainly ecological data) by @ethanwhite et al. is also using the Data Package format for metadata and has some tooling for different dbs: https://frictionlessdata.io/articles/the-data-retriever/ https://github.com/weecology/retriever The Open Power System Data project also has a couple of datasets that show nicely how CSV is great for assembling and then already make SQLite files available. It's one of the first data sets I tried with Datasette, perfect for the use case of getting an API for putting power stations on a map ... https://data.open-power-system-data.org/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274314940 | |
https://github.com/simonw/datasette/issues/27#issuecomment-345652450 | https://api.github.com/repos/simonw/datasette/issues/27 | 345652450 | MDEyOklzc3VlQ29tbWVudDM0NTY1MjQ1MA== | 198537 | 2017-11-20T10:19:39Z | 2017-11-20T10:19:39Z | CONTRIBUTOR | If Data Package metadata gets adopted (#105) the views spec work might also be worth a look: http://frictionlessdata.io/specs/views/ http://datahub.io/docs/features/views | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
267886330 | |
https://github.com/simonw/datasette/pull/104#issuecomment-346116745 | https://api.github.com/repos/simonw/datasette/issues/104 | 346116745 | MDEyOklzc3VlQ29tbWVudDM0NjExNjc0NQ== | 21148 | 2017-11-21T18:23:25Z | 2017-11-21T18:23:25Z | CONTRIBUTOR | @simonw ready for a review and merge if you want. There's still some nasty duplicated code in cli.py and utils.py, which is just going to get worse if/when we start adding any other deploy targets (and I want to do one for cloud.gov, at least). I think there's an opportunity for some refactoring here. I'm happy to do that now as part of this PR, or if you merge this first I'll do it in a different one. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274284246 | |
https://github.com/simonw/datasette/pull/104#issuecomment-346124073 | https://api.github.com/repos/simonw/datasette/issues/104 | 346124073 | MDEyOklzc3VlQ29tbWVudDM0NjEyNDA3Mw== | 21148 | 2017-11-21T18:49:55Z | 2017-11-21T18:49:55Z | CONTRIBUTOR | Actually hang on, don't merge - there are some bugs that #141 masked when I tested this out elsewhere. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274284246 | |
https://github.com/simonw/datasette/pull/104#issuecomment-346124764 | https://api.github.com/repos/simonw/datasette/issues/104 | 346124764 | MDEyOklzc3VlQ29tbWVudDM0NjEyNDc2NA== | 21148 | 2017-11-21T18:52:14Z | 2017-11-21T18:52:14Z | CONTRIBUTOR | OK, now this should work. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274284246 | |
https://github.com/simonw/datasette/issues/14#issuecomment-346244871 | https://api.github.com/repos/simonw/datasette/issues/14 | 346244871 | MDEyOklzc3VlQ29tbWVudDM0NjI0NDg3MQ== | 21148 | 2017-11-22T05:06:30Z | 2017-11-22T05:06:30Z | CONTRIBUTOR | I'd also suggest taking a look at [stevedore](https://docs.openstack.org/stevedore/latest/), which has a ton of tools for doing plugin stuff. I've had good luck with it in the past. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
267707940 | |
https://github.com/simonw/datasette/issues/179#issuecomment-360535979 | https://api.github.com/repos/simonw/datasette/issues/179 | 360535979 | MDEyOklzc3VlQ29tbWVudDM2MDUzNTk3OQ== | 82988 | 2018-01-25T17:18:24Z | 2018-01-25T17:18:24Z | CONTRIBUTOR | To summarise that thread: - expose full `metadata.json` object to the index page template, eg to allow tables to be referred to by name; - ability to import multiple `metadata.json` files, eg to allow metadata files created for a specific SQLite db to be reused in a datasette referring to several database files; It could also be useful to allow users to import a python file containing custom functions that can that be loaded into scope and made available to custom templates. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
288438570 | |
https://github.com/simonw/datasette/pull/200#issuecomment-380608372 | https://api.github.com/repos/simonw/datasette/issues/200 | 380608372 | MDEyOklzc3VlQ29tbWVudDM4MDYwODM3Mg== | 45057 | 2018-04-11T21:55:46Z | 2018-04-11T21:55:46Z | CONTRIBUTOR | > I think the most reliable way to detect spatialite is to run `SELECT AddGeometryColumn(1, 2, 3, 4, 5);` against a `:memory:` database and see if it throws an exception Or just see if there's a `geometry_columns` table? I think that's quite unlikely to be added by accident (and it's an OGC standard). It also tells you if Spatialite is installed in the database rather than just loaded. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
313494458 | |
https://github.com/simonw/datasette/issues/203#issuecomment-380966565 | https://api.github.com/repos/simonw/datasette/issues/203 | 380966565 | MDEyOklzc3VlQ29tbWVudDM4MDk2NjU2NQ== | 45057 | 2018-04-12T22:43:08Z | 2018-04-12T22:43:08Z | CONTRIBUTOR | Looks like [pint](https://pint.readthedocs.io/en/latest/tutorial.html) is pretty good at this. ```python In [1]: import pint In [2]: ureg = pint.UnitRegistry() In [3]: q = 3e6 * ureg('Hz') In [4]: '{:~P}'.format(q.to_compact()) Out[4]: '3.0 MHz' In [5]: q = 0.3 * ureg('m') In [5]: '{:~P}'.format(q.to_compact()) Out[5]: '300.0 mm' In [6]: q = 5 * ureg('') In [7]: '{:~P}'.format(q.to_compact()) Out[7]: '5' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
313837303 | |
https://github.com/simonw/datasette/pull/202#issuecomment-381237440 | https://api.github.com/repos/simonw/datasette/issues/202 | 381237440 | MDEyOklzc3VlQ29tbWVudDM4MTIzNzQ0MA== | 45057 | 2018-04-13T19:22:53Z | 2018-04-13T19:22:53Z | CONTRIBUTOR | I spotted you'd mentioned that in #184 but only after I'd written the patch! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
313785206 | |
https://github.com/simonw/datasette/issues/203#issuecomment-381315675 | https://api.github.com/repos/simonw/datasette/issues/203 | 381315675 | MDEyOklzc3VlQ29tbWVudDM4MTMxNTY3NQ== | 45057 | 2018-04-14T09:14:45Z | 2018-04-14T09:27:30Z | CONTRIBUTOR | > I'd like to figure out a sensible opt-in way to expose this in the JSON output as well. Maybe with a &_units=true parameter? <s>From a machine-readable perspective I'm not sure why it would be useful to decorate the values with units</s>. Edit: Should have had some coffee first. It's clearly useful for stuff like map rendering! I agree that the unit metadata should definitely be exposed in the JSON. > In #204 you said "I'd like to add support for using units when querying but this is PR is pretty usable as-is." - I'm fascinated to hear more about how this could work. I'm thinking about a couple of approaches here. I think the simplest one is: if the column has a unit attached, optionally accept units in query fields: ```python column_units = ureg("Hz") # Create a unit object for the column's unit query_variable = ureg("4 GHz") # Supplied query variable # Now we can convert the query units into column units before querying supplied_value.to(column_units).magnitude > 4000000000.0 # If the user doesn't supply units, pint just returns the plain # number and we can query as usual assuming it's the base unit query_variable = ureg("50") query_variable > 50 isinstance(query_variable, numbers.Number) > True ``` This also lets us do some nice unit conversion on querying: ```python column_units = ureg("m") query_variable = ureg("50 ft") supplied_value.to(column_units) > <Quantity(15.239999999999998, 'meter')> ``` The alternative would be to provide a dropdown of units next to the query field (so a "Hz" field would give you "kHz", "MHz", "GHz"). Although this would be clearer to the user, it isn't so easy - we'd need to know more about the context of the field to give you sensible SI prefixes (I'm not so interested in nanoHertz, for example). You also lose the bonus of being able to convert - although pint will happily show you all the compatible units, it again suffers from a lack of context: ```python ureg("m").compatible_units() > frozenset({<Unit('angstrom')>, … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
313837303 | |
https://github.com/simonw/datasette/pull/205#issuecomment-381332222 | https://api.github.com/repos/simonw/datasette/issues/205 | 381332222 | MDEyOklzc3VlQ29tbWVudDM4MTMzMjIyMg== | 45057 | 2018-04-14T14:16:35Z | 2018-04-14T14:16:35Z | CONTRIBUTOR | I've added some tests and that docs link. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
314319372 | |
https://github.com/simonw/datasette/issues/125#issuecomment-381361734 | https://api.github.com/repos/simonw/datasette/issues/125 | 381361734 | MDEyOklzc3VlQ29tbWVudDM4MTM2MTczNA== | 45057 | 2018-04-14T21:26:30Z | 2018-04-14T21:26:30Z | CONTRIBUTOR | FWIW I am now doing this on my WTR app (instead of silently limiting maps to 1000). [Telefonica](https://wtr-api.herokuapp.com/wtr-663ea99/licensee/18325) now has about 4000 markers and good old [BT](https://wtr-api.herokuapp.com/wtr-663ea99/licensee/8412) has 22,000 or so. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
275135393 | |
https://github.com/simonw/datasette/pull/209#issuecomment-381441392 | https://api.github.com/repos/simonw/datasette/issues/209 | 381441392 | MDEyOklzc3VlQ29tbWVudDM4MTQ0MTM5Mg== | 45057 | 2018-04-15T21:59:15Z | 2018-04-15T21:59:15Z | CONTRIBUTOR | I suspected this would cause some test failures, but I'll wait for opinions before attempting to fix them. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
314455877 | |
https://github.com/simonw/datasette/pull/209#issuecomment-381738137 | https://api.github.com/repos/simonw/datasette/issues/209 | 381738137 | MDEyOklzc3VlQ29tbWVudDM4MTczODEzNw== | 45057 | 2018-04-16T20:27:43Z | 2018-04-16T20:27:43Z | CONTRIBUTOR | Tests now fixed, honest. The failing test on Travis looks like an intermittent sqlite failure which should resolve itself on a retry... | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
314455877 | |
https://github.com/simonw/datasette/issues/203#issuecomment-381763651 | https://api.github.com/repos/simonw/datasette/issues/203 | 381763651 | MDEyOklzc3VlQ29tbWVudDM4MTc2MzY1MQ== | 45057 | 2018-04-16T21:59:17Z | 2018-04-16T21:59:17Z | CONTRIBUTOR | Ah, I had no idea you could bind python functions into sqlite! I think the primary purpose of this issue has been served now - I'm going to close this and create a new issue for the only bit of this that hasn't been touched yet, which is (optionally) exposing units in the JSON API. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
313837303 | |
https://github.com/simonw/datasette/pull/209#issuecomment-381905593 | https://api.github.com/repos/simonw/datasette/issues/209 | 381905593 | MDEyOklzc3VlQ29tbWVudDM4MTkwNTU5Mw== | 45057 | 2018-04-17T08:50:28Z | 2018-04-17T08:50:28Z | CONTRIBUTOR | I've added another commit which puts classes a class on each `<td>` by default with its column name, and I've also made the PK column bold. Unfortunately the tests are still failing on 3.6, which is weird. I can't reproduce locally... | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
314455877 | |
https://github.com/simonw/datasette/issues/273#issuecomment-390250253 | https://api.github.com/repos/simonw/datasette/issues/273 | 390250253 | MDEyOklzc3VlQ29tbWVudDM5MDI1MDI1Mw== | 198537 | 2018-05-18T15:49:52Z | 2018-05-18T15:49:52Z | CONTRIBUTOR | Shouldn't [versioneer](https://github.com/warner/python-versioneer) do that? E.g. 0.21+2.g1076c97 You'd need to install via `pip install git+https://github.com/simow/datasette.git` though, this does a temp git clone. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324451322 | |
https://github.com/simonw/datasette/issues/276#issuecomment-390795067 | https://api.github.com/repos/simonw/datasette/issues/276 | 390795067 | MDEyOklzc3VlQ29tbWVudDM5MDc5NTA2Nw== | 45057 | 2018-05-21T21:55:57Z | 2018-05-21T21:55:57Z | CONTRIBUTOR | Well, we do have the capability to detect spatialite so my intention certainly wasn't to require it. I can see the advantage of having it as a plugin but it does touch a number of points in the code. I think I'm going to attack this by refactoring the necessary bits and seeing where that leads (which was my plan anyway). I think my main concern is - if I add certain plugin hooks for this, is anything else ever going to use them? I'm not sure I have an answer to that question yet, either way. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324835838 | |
https://github.com/simonw/datasette/issues/276#issuecomment-391050113 | https://api.github.com/repos/simonw/datasette/issues/276 | 391050113 | MDEyOklzc3VlQ29tbWVudDM5MTA1MDExMw== | 45057 | 2018-05-22T16:13:00Z | 2018-05-22T16:13:00Z | CONTRIBUTOR | Yup, I'll have a think about it. My current thoughts are for spatialite we'll need to hook into the following places: * Inspection, so we can detect which columns are geometry columns. (We also currently ignore spatialite tables during inspection, it may be worth moving that to the plugin as well.) * After data load, so we can convert WKB into the correct intermediate format for display. The alternative here is to alter the select SQL itself and get spatialite to do this conversion, but that strikes me as a bit more complex and possibly not as useful. * HTML rendering. * Querying? The rendering and querying hooks could also potentially be used to move the units support into a plugin. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324835838 | |
https://github.com/simonw/datasette/pull/280#issuecomment-391059008 | https://api.github.com/repos/simonw/datasette/issues/280 | 391059008 | MDEyOklzc3VlQ29tbWVudDM5MTA1OTAwOA== | 565628 | 2018-05-22T16:40:27Z | 2018-05-22T16:40:27Z | CONTRIBUTOR | ```python >>> import sqlite3 >>> sqlite3.sqlite_version '3.23.1' >>> ``` running the above in the container seems to show 3.23.1 too so maybe we don't need pysqlite3 at all? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
325373747 | |
https://github.com/simonw/datasette/pull/279#issuecomment-391073009 | https://api.github.com/repos/simonw/datasette/issues/279 | 391073009 | MDEyOklzc3VlQ29tbWVudDM5MTA3MzAwOQ== | 198537 | 2018-05-22T17:23:26Z | 2018-05-22T17:23:26Z | CONTRIBUTOR | > I think I prefer the aesthetics of just "0.22" for the version string if it's a tagged release with no additional changes - does that work? Yes! That's the default versioneer behaviour. > I'd like to continue to provide a tuple that can be imported from the version.py module as well, as seen here: Should work now, it can be a two (for a tagged version), three or four items tuple. ``` In [2]: datasette.__version__ Out[2]: '0.12+292.ga70c2a8.dirty' In [3]: datasette.__version_info__ Out[3]: ('0', '12+292', 'ga70c2a8', 'dirty') ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
325352370 | |
https://github.com/simonw/datasette/pull/279#issuecomment-391073267 | https://api.github.com/repos/simonw/datasette/issues/279 | 391073267 | MDEyOklzc3VlQ29tbWVudDM5MTA3MzI2Nw== | 198537 | 2018-05-22T17:24:16Z | 2018-05-22T17:24:16Z | CONTRIBUTOR | Sorry, just realised you rely on `version` being a module ... | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
325352370 | |
https://github.com/simonw/datasette/pull/279#issuecomment-391077700 | https://api.github.com/repos/simonw/datasette/issues/279 | 391077700 | MDEyOklzc3VlQ29tbWVudDM5MTA3NzcwMA== | 198537 | 2018-05-22T17:38:17Z | 2018-05-22T17:38:17Z | CONTRIBUTOR | Alright, that should work now -- let me know if you would prefer any different behaviour. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
325352370 | |
https://github.com/simonw/datasette/pull/280#issuecomment-391141391 | https://api.github.com/repos/simonw/datasette/issues/280 | 391141391 | MDEyOklzc3VlQ29tbWVudDM5MTE0MTM5MQ== | 565628 | 2018-05-22T21:08:39Z | 2018-05-22T21:08:39Z | CONTRIBUTOR | I'm going to clean this up for consistency tomorrow morning so hold off merging until then please On Tue, May 22, 2018 at 6:34 PM, Simon Willison <notifications@github.com> wrote: > Yeah let's try this without pysqlite3 and see if we still get the correct > version. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <https://github.com/simonw/datasette/pull/280#issuecomment-391076458>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAihfMI-H6CBt-Py0xdBbH2xDK0KsjT2ks5t1EwYgaJpZM4UI_2m> > . > | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
325373747 | |
https://github.com/simonw/datasette/pull/280#issuecomment-391290271 | https://api.github.com/repos/simonw/datasette/issues/280 | 391290271 | MDEyOklzc3VlQ29tbWVudDM5MTI5MDI3MQ== | 565628 | 2018-05-23T09:53:38Z | 2018-05-23T09:53:38Z | CONTRIBUTOR | Running: ```bash docker run -p 8001:8001 -v `pwd`:/mnt datasette \ datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db \ --load-extension=/usr/local/lib/mod_spatialite.so ``` is now returning FTS5 enabled in the versions output: ```json { "datasette": { "version": "0.22" }, "python": { "full": "3.6.5 (default, May 5 2018, 03:07:21) \n[GCC 6.3.0 20170516]", "version": "3.6.5" }, "sqlite": { "extensions": { "json1": null, "spatialite": "4.4.0-RC0" }, "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "version": "3.23.1" } } ``` The old query didn't work because specifying `(t TEXT)` caused an error | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
325373747 | |
https://github.com/simonw/datasette/pull/280#issuecomment-391355030 | https://api.github.com/repos/simonw/datasette/issues/280 | 391355030 | MDEyOklzc3VlQ29tbWVudDM5MTM1NTAzMA== | 565628 | 2018-05-23T13:53:27Z | 2018-05-23T15:22:45Z | CONTRIBUTOR | No objections; It's good to go @simonw On Wed, 23 May 2018, 14:51 Simon Willison, <notifications@github.com> wrote: > @r4vi <https://github.com/r4vi> any objections to me merging this? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/simonw/datasette/pull/280#issuecomment-391354237>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAihfM_2DN5WR2mkO-VK6ozDmkUQ4IMjks5t1WlcgaJpZM4UI_2m> > . > | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
325373747 | |
https://github.com/simonw/datasette/issues/276#issuecomment-391505930 | https://api.github.com/repos/simonw/datasette/issues/276 | 391505930 | MDEyOklzc3VlQ29tbWVudDM5MTUwNTkzMA== | 45057 | 2018-05-23T21:41:37Z | 2018-05-23T21:41:37Z | CONTRIBUTOR | > I'm not keen on anything that modifies the SQLite file itself on startup Ah I didn't mean that - I meant altering the SELECT query to fetch the data so that it ran a spatialite function to transform that specific column. I think that's less useful as a general-purpose plugin hook though, and it's not that hard to parse the WKB in Python (my default approach would be to use [shapely](https://github.com/Toblerity/Shapely), which is great, but geomet looks like an interesting pure-python alternative). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324835838 | |
https://github.com/simonw/datasette/issues/276#issuecomment-392825746 | https://api.github.com/repos/simonw/datasette/issues/276 | 392825746 | MDEyOklzc3VlQ29tbWVudDM5MjgyNTc0Ng== | 45057 | 2018-05-29T15:42:53Z | 2018-05-29T15:42:53Z | CONTRIBUTOR | I haven't had time to look further into this, but if doing this as a plugin results in useful hooks then I think we should do it that way. We could always require the plugin as a standard dependency. I think this is going to result in quite a bit of refactoring anyway so it's a good time to add hooks regardless. On the other hand, if we have to add lots of specialist hooks for it then maybe it's worth integrating into the core. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324835838 | |
https://github.com/simonw/datasette/issues/276#issuecomment-393106520 | https://api.github.com/repos/simonw/datasette/issues/276 | 393106520 | MDEyOklzc3VlQ29tbWVudDM5MzEwNjUyMA== | 45057 | 2018-05-30T10:09:25Z | 2018-05-30T10:09:25Z | CONTRIBUTOR | I don't think it's unreasonable to only support spatialite geometries in a coordinate reference system which is at least transformable to WGS84. It would be nice to support different CRSes in the database so conversion to spatialite from the source data is lossless. I think the working CRS for datasette should be WGS84 though (leaflet requires it, for example) - it's just a case of calling `ST_Transform(geom, 4326)` on the column while we're loading the data. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324835838 | |
https://github.com/simonw/datasette/issues/276#issuecomment-401310732 | https://api.github.com/repos/simonw/datasette/issues/276 | 401310732 | MDEyOklzc3VlQ29tbWVudDQwMTMxMDczMg== | 82988 | 2018-06-29T10:05:04Z | 2018-06-29T10:07:25Z | CONTRIBUTOR | @russs Different map projections can presumably be handled on the client side using a leaflet plugin to transform the geometry (eg [kartena/Proj4Leaflet](https://kartena.github.io/Proj4Leaflet/)) although the leaflet side would need to detect or be informed of the original projection? Another possibility would be to provide an easy way/guidance for users to create an FK'd table containing the WGS84 projection of a non-WGS84 geometry in the original/principle table? This could then as a proxy for serving GeoJSON to the leaflet map? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324835838 | |
https://github.com/simonw/datasette/issues/276#issuecomment-401312981 | https://api.github.com/repos/simonw/datasette/issues/276 | 401312981 | MDEyOklzc3VlQ29tbWVudDQwMTMxMjk4MQ== | 45057 | 2018-06-29T10:14:54Z | 2018-06-29T10:14:54Z | CONTRIBUTOR | > @RusSs Different map projections can presumably be handled on the client side using a leaflet plugin to transform the geometry (eg kartena/Proj4Leaflet) although the leaflet side would need to detect or be informed of the original projection? Well, as @simonw mentioned, GeoJSON only supports WGS84, and GeoJSON (and/or TopoJSON) is the standard we probably want to aim for. On-the-fly reprojection in spatialite is not an issue anyway, and in general I think you want to be serving stuff to web maps in WGS84 or Web Mercator. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324835838 | |
https://github.com/simonw/datasette/issues/344#issuecomment-405022335 | https://api.github.com/repos/simonw/datasette/issues/344 | 405022335 | MDEyOklzc3VlQ29tbWVudDQwNTAyMjMzNQ== | 45057 | 2018-07-14T13:00:48Z | 2018-07-14T13:00:48Z | CONTRIBUTOR | Looks like this was a red herring actually, and heroku had a blip when I was testing it... | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
341229113 | |
https://github.com/simonw/datasette/issues/343#issuecomment-405026441 | https://api.github.com/repos/simonw/datasette/issues/343 | 405026441 | MDEyOklzc3VlQ29tbWVudDQwNTAyNjQ0MQ== | 45057 | 2018-07-14T14:17:14Z | 2018-07-14T14:17:14Z | CONTRIBUTOR | This probably depends on #294. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
341228846 | |
https://github.com/simonw/datasette/issues/294#issuecomment-405026800 | https://api.github.com/repos/simonw/datasette/issues/294 | 405026800 | MDEyOklzc3VlQ29tbWVudDQwNTAyNjgwMA== | 45057 | 2018-07-14T14:24:31Z | 2018-07-14T14:24:31Z | CONTRIBUTOR | I had a quick look at this in relation to #343 and I feel like it might be worth modelling the inspected table metadata internally as an object rather than a dict. (We'd still have to serialise it back to JSON.) There are a few places where we rely on the structure of this metadata dict for various reasons, including in templates (and potentially also in user templates). It would be nice to have a reasonably well defined API for accessing metadata internally so that it's clearer what we're breaking. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
327365110 | |
https://github.com/simonw/datasette/issues/329#issuecomment-422821483 | https://api.github.com/repos/simonw/datasette/issues/329 | 422821483 | MDEyOklzc3VlQ29tbWVudDQyMjgyMTQ4Mw== | 418191 | 2018-09-19T14:17:42Z | 2018-09-19T14:17:42Z | CONTRIBUTOR | I'm using the docker image (0.23.2) and notice some differences/bugs between the docs and the published version with canned queries. (submitted a tiny doc fix also) I was able to build the docker container locally using `master` and I'm using that for now. Would it be possible to manually push 0.24 to DockerHub until the TravisCI stuff is fixed? I would like to run this in our Kubernetes cluster but don't want to publish a version in our internal registry if I don't have to. Thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
336465018 | |
https://github.com/simonw/datasette/issues/329#issuecomment-422915450 | https://api.github.com/repos/simonw/datasette/issues/329 | 422915450 | MDEyOklzc3VlQ29tbWVudDQyMjkxNTQ1MA== | 418191 | 2018-09-19T18:45:02Z | 2018-09-20T10:50:50Z | CONTRIBUTOR | That works for me. Was able to pull the public image and no errors on my canned query. (~although a small rendering bug. I'll create an issue and if I have time today, a PR to fix~ this turned out to be my error.) Thanks for the quick response! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
336465018 | |
https://github.com/simonw/datasette/issues/366#issuecomment-429737929 | https://api.github.com/repos/simonw/datasette/issues/366 | 429737929 | MDEyOklzc3VlQ29tbWVudDQyOTczNzkyOQ== | 416374 | 2018-10-15T07:32:57Z | 2018-10-15T07:32:57Z | CONTRIBUTOR | Very hacky solution is to write now.json file forcing the usage of v1 of Zeit cloud, see https://github.com/slygent/datasette/commit/3ab824793ec6534b6dd87078aa46b11c4fa78ea3 This does work, at least. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
369716228 | |
https://github.com/simonw/datasette/issues/369#issuecomment-435768450 | https://api.github.com/repos/simonw/datasette/issues/369 | 435768450 | MDEyOklzc3VlQ29tbWVudDQzNTc2ODQ1MA== | 416374 | 2018-11-05T06:31:59Z | 2018-11-05T06:31:59Z | CONTRIBUTOR | That would be ideal, but you know better than me whether the CSV streaming trick works for custom SQL queries. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
374953006 | |
https://github.com/simonw/datasette/issues/371#issuecomment-435862009 | https://api.github.com/repos/simonw/datasette/issues/371 | 435862009 | MDEyOklzc3VlQ29tbWVudDQzNTg2MjAwOQ== | 82988 | 2018-11-05T12:48:35Z | 2018-11-05T12:48:35Z | CONTRIBUTOR | I think you need to register a domain name you own separately in order to get a non-IP address address? https://www.digitalocean.com/docs/networking/dns/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
377156339 | |
https://github.com/simonw/datasette/issues/370#issuecomment-436037692 | https://api.github.com/repos/simonw/datasette/issues/370 | 436037692 | MDEyOklzc3VlQ29tbWVudDQzNjAzNzY5Mg== | 82988 | 2018-11-05T21:15:47Z | 2018-11-05T21:18:37Z | CONTRIBUTOR | In terms of integration with `pandas`, I was pondering two different ways `datasette`/`csvs_to_sqlite` integration may work: - like [`pandasql`](https://github.com/yhat/pandasql), to provide a SQL query layer either by a direct connection to the sqlite db or via `datasette` API; - as an improvement of `pandas.to_sql()`, which is a bit ropey (e.g. `pandas.to_sql_from_csvs()`, routing the dataframe to sqlite via `csvs_tosqlite` rather than the dodgy mapping that `pandas` supports). The `pandas.publish_*` idea could be quite interesting though... Would it be useful/fruitful to think about `publish_` as a complement to [`pandas.to_`](https://pandas.pydata.org/pandas-docs/stable/api.html#id12)? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
377155320 | |
https://github.com/simonw/datasette/issues/370#issuecomment-436042445 | https://api.github.com/repos/simonw/datasette/issues/370 | 436042445 | MDEyOklzc3VlQ29tbWVudDQzNjA0MjQ0NQ== | 82988 | 2018-11-05T21:30:42Z | 2018-11-05T21:31:48Z | CONTRIBUTOR | Another route would be something like creating a `datasette` IPython magic for notebooks to take a dataframe and easily render it as a `datasette`. You'd need to run the app in the background rather than block execution in the notebook. Related to that, or to publishing a dataframe in notebook cell for use in other cells in a non-blocking way, there may be cribs in something like https://github.com/micahscopes/nbmultitask . | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
377155320 | |
https://github.com/simonw/datasette/issues/160#issuecomment-459915995 | https://api.github.com/repos/simonw/datasette/issues/160 | 459915995 | MDEyOklzc3VlQ29tbWVudDQ1OTkxNTk5NQ== | 82988 | 2019-02-02T00:43:16Z | 2019-02-02T00:58:20Z | CONTRIBUTOR | Do you have any simple working examples of how to use `--static`? Inspection of default served files suggests locations such as `http://example.com/-/static/app.css?0e06ee`. If `datasette` is being proxied to `http://example.com/foo/datasette`, what form should arguments to `--static` take so that static files are correctly referenced? Use case is here: https://github.com/psychemedia/jupyterserverproxy-datasette-demo Trying to do a really simple `datasette` demo in MyBinder using jupyter-server-proxy. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
278208011 | |
https://github.com/simonw/datasette/issues/417#issuecomment-474280581 | https://api.github.com/repos/simonw/datasette/issues/417 | 474280581 | MDEyOklzc3VlQ29tbWVudDQ3NDI4MDU4MQ== | 82988 | 2019-03-19T10:06:42Z | 2019-03-19T10:06:42Z | CONTRIBUTOR | This would be really interesting but several possibilities in use arise, I think? For example: - I put a new CSV file into the import dir and a new table is created therefrom - I put a CSV file into the import dir that replaces a previous file / table of the same name as a pre-existing table (eg files that contain monthly data in year to date). The data may also patch previous months, so a full replace / DROP on the original table may well be in order. - I put a CSV file into the import dir that updates a table of the same name as a pre-existing table (eg files that contain last month's data) CSV files may also have messy names compared to the table you want. Or for an update CSV, may have the form `MYTABLENAME-February2019.csv` etc | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
421546944 | |
https://github.com/simonw/datasette/issues/412#issuecomment-474282321 | https://api.github.com/repos/simonw/datasette/issues/412 | 474282321 | MDEyOklzc3VlQ29tbWVudDQ3NDI4MjMyMQ== | 82988 | 2019-03-19T10:09:46Z | 2019-03-19T10:09:46Z | CONTRIBUTOR | Does this also relate to https://github.com/simonw/datasette/issues/283 and the ability to `ATTACH DATABASE`? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
411257981 | |
https://github.com/simonw/datasette/issues/431#issuecomment-483017176 | https://api.github.com/repos/simonw/datasette/issues/431 | 483017176 | MDEyOklzc3VlQ29tbWVudDQ4MzAxNzE3Ng== | 82988 | 2019-04-14T16:58:37Z | 2019-04-14T16:58:37Z | CONTRIBUTOR | Hmm... nope... I see an updated timestamp from `ls -al` on the db but no reload? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
432870248 | |
https://github.com/simonw/datasette/issues/429#issuecomment-483202658 | https://api.github.com/repos/simonw/datasette/issues/429 | 483202658 | MDEyOklzc3VlQ29tbWVudDQ4MzIwMjY1OA== | 82988 | 2019-04-15T10:48:01Z | 2019-04-15T10:48:01Z | CONTRIBUTOR | Minor UI observation: ![image](https://user-images.githubusercontent.com/82988/56127017-2bf78e80-5f74-11e9-9120-9393eb5d4988.png) `_where=` renders a `[remove]` link whereas `_facet=` gets a cross to remove it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
432636432 | |
https://github.com/simonw/datasette/pull/437#issuecomment-487537452 | https://api.github.com/repos/simonw/datasette/issues/437 | 487537452 | MDEyOklzc3VlQ29tbWVudDQ4NzUzNzQ1Mg== | 45057 | 2019-04-29T10:58:49Z | 2019-04-29T10:58:49Z | CONTRIBUTOR | I've just spotted that this implements #215. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438048318 | |
https://github.com/simonw/datasette/pull/439#issuecomment-487542486 | https://api.github.com/repos/simonw/datasette/issues/439 | 487542486 | MDEyOklzc3VlQ29tbWVudDQ4NzU0MjQ4Ng== | 45057 | 2019-04-29T11:20:30Z | 2019-04-29T11:20:30Z | CONTRIBUTOR | Actually I think this is not the whole story because of the rowid issue. I'm going to think about this one a bit more. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438240541 | |
https://github.com/simonw/datasette/pull/441#issuecomment-487686655 | https://api.github.com/repos/simonw/datasette/issues/441 | 487686655 | MDEyOklzc3VlQ29tbWVudDQ4NzY4NjY1NQ== | 45057 | 2019-04-29T18:14:25Z | 2019-04-29T18:14:25Z | CONTRIBUTOR | Subsidiary note which I forgot in the commit message: I've decided to give each view a short string name to aid in differentiating which view a hook is being called from. Since hooks are functions and not subclasses, and can get called from different places in the URL hierarchy, it's sometimes difficult to distinguish what data you're actually operating on. I think this will come in handy for other hooks as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438437973 | |
https://github.com/simonw/datasette/pull/424#issuecomment-487689477 | https://api.github.com/repos/simonw/datasette/issues/424 | 487689477 | MDEyOklzc3VlQ29tbWVudDQ4NzY4OTQ3Nw== | 45057 | 2019-04-29T18:22:40Z | 2019-04-29T18:22:40Z | CONTRIBUTOR | This is pretty conflicty because I forgot how to use git fetch. If you're interested in merging this I'll rewrite it against an actual modern checkout... | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
427429265 | |
https://github.com/simonw/datasette/pull/424#issuecomment-487692377 | https://api.github.com/repos/simonw/datasette/issues/424 | 487692377 | MDEyOklzc3VlQ29tbWVudDQ4NzY5MjM3Nw== | 45057 | 2019-04-29T18:30:46Z | 2019-04-29T18:30:46Z | CONTRIBUTOR | Actually no, I ended up not using the inspected column types in my plugin, and the binary column issue can be solved a lot more simply, so I'll close this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
427429265 | |
https://github.com/simonw/datasette/pull/441#issuecomment-487723476 | https://api.github.com/repos/simonw/datasette/issues/441 | 487723476 | MDEyOklzc3VlQ29tbWVudDQ4NzcyMzQ3Ng== | 45057 | 2019-04-29T20:05:23Z | 2019-04-29T20:05:23Z | CONTRIBUTOR | This is the minimal example (I also included it in the docs): ```python from datasette import hookimpl def render_test(args, data, view_name): return { 'body': 'Hello World', 'content_type': 'text/plain' } @hookimpl def register_output_renderer(): return { 'extension': 'test', 'callback': render_test } ``` I'm working on the GeoJSON one now and it should be ready soon. (I forgot I was going to run into the same problem as before - that Spatialite's stupid binary format isn't WKB and I have no way of altering the query to change that - but I've just managed to write some code to rearrange the bytes from Spatialite blob-geometry into WKB...) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438437973 | |
https://github.com/simonw/datasette/pull/441#issuecomment-487724539 | https://api.github.com/repos/simonw/datasette/issues/441 | 487724539 | MDEyOklzc3VlQ29tbWVudDQ4NzcyNDUzOQ== | 45057 | 2019-04-29T20:08:32Z | 2019-04-29T20:08:32Z | CONTRIBUTOR | I also just realised that I should be passing the datasette object into the hook function...as I just found I need it. So hold off merging until I've fixed that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438437973 | |
https://github.com/simonw/datasette/pull/441#issuecomment-487735247 | https://api.github.com/repos/simonw/datasette/issues/441 | 487735247 | MDEyOklzc3VlQ29tbWVudDQ4NzczNTI0Nw== | 45057 | 2019-04-29T20:39:43Z | 2019-04-29T20:39:43Z | CONTRIBUTOR | I updated the hook to pass the datasette object through now. You can see the working [GeoJSON render function here](https://github.com/russss/datasette-geo/blob/master/datasette_plugin_geo/geojson.py) - the [hook function is here](https://github.com/russss/datasette-geo/blob/master/datasette_plugin_geo/__init__.py#L65-L70). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438437973 | |
https://github.com/simonw/datasette/pull/441#issuecomment-487748271 | https://api.github.com/repos/simonw/datasette/issues/441 | 487748271 | MDEyOklzc3VlQ29tbWVudDQ4Nzc0ODI3MQ== | 45057 | 2019-04-29T21:20:17Z | 2019-04-29T21:20:17Z | CONTRIBUTOR | Also I just pushed a change to add registered output renderers to the templates: ![image](https://user-images.githubusercontent.com/45057/56927799-f18e0580-6acc-11e9-8ea9-a0ee961323ec.png) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438437973 | |
https://github.com/simonw/datasette/pull/439#issuecomment-487859345 | https://api.github.com/repos/simonw/datasette/issues/439 | 487859345 | MDEyOklzc3VlQ29tbWVudDQ4Nzg1OTM0NQ== | 45057 | 2019-04-30T08:21:19Z | 2019-04-30T08:21:19Z | CONTRIBUTOR | I think the best approach to this is to pass through the `view_name` parameter I added in #441. It's then simple enough for me to add `.geojson` to the URL in JS - I don't need the pkey. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438240541 | |
https://github.com/simonw/datasette/pull/441#issuecomment-488247617 | https://api.github.com/repos/simonw/datasette/issues/441 | 488247617 | MDEyOklzc3VlQ29tbWVudDQ4ODI0NzYxNw== | 45057 | 2019-05-01T09:57:50Z | 2019-05-01T09:57:50Z | CONTRIBUTOR | Just for the record, this PR is now finished and ready to merge from my perspective. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
438437973 | |
https://github.com/simonw/datasette/pull/432#issuecomment-488595724 | https://api.github.com/repos/simonw/datasette/issues/432 | 488595724 | MDEyOklzc3VlQ29tbWVudDQ4ODU5NTcyNA== | 45057 | 2019-05-02T08:50:53Z | 2019-05-02T08:50:53Z | CONTRIBUTOR | > Can I pull those needs out of the Facet class somehow? I was thinking that it might be handy for datasette to have a request object which wraps the Sanic Request. This could include the datasette-specific querystring decoding and the `special_args` parsing from TableView.data. This would mean that we could expose the request object to plugin hooks without coupling them to Sanic. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
432893491 | |
https://github.com/simonw/datasette/issues/419#issuecomment-489060765 | https://api.github.com/repos/simonw/datasette/issues/419 | 489060765 | MDEyOklzc3VlQ29tbWVudDQ4OTA2MDc2NQ== | 45057 | 2019-05-03T11:07:42Z | 2019-05-03T11:07:42Z | CONTRIBUTOR | Are you planning on removing inspect entirely? I didn't spot this work before I started on datasette-geo, but ironically I think it has a use case which really needs the inspect functionality (or some replacement). Datasette-geo uses it to store the bounding box of all the geographic features in the table. This is needed when rendering the map because it avoids having to send loads of tile requests for areas which are empty. Even with relatively small datasets, calculating the bounding box seems to take around 5 seconds, so I don't think it's really feasible to do this on page load. One possible fix would be to do this on startup, and then in a thread which watches the database for changes. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
421551434 | |
https://github.com/simonw/datasette/pull/434#issuecomment-489105665 | https://api.github.com/repos/simonw/datasette/issues/434 | 489105665 | MDEyOklzc3VlQ29tbWVudDQ4OTEwNTY2NQ== | 25778 | 2019-05-03T14:01:30Z | 2019-05-03T14:01:30Z | CONTRIBUTOR | This is exactly what I needed. Thank you. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
434321685 | |
https://github.com/simonw/datasette/pull/434#issuecomment-489163939 | https://api.github.com/repos/simonw/datasette/issues/434 | 489163939 | MDEyOklzc3VlQ29tbWVudDQ4OTE2MzkzOQ== | 10352819 | 2019-05-03T16:49:45Z | 2019-05-03T16:50:03Z | CONTRIBUTOR | > The second time I ran the command I got an error: > > ERROR: (gcloud.beta.run.deploy) Deployment endpoint was not found. Perhaps the > provided region was invalid. Set the `run/region` property to a valid region and > retry. Ex: `gcloud config set run/region us-central1` > Yes, I was able to reproduce this; I used to get prompted for a run region interactively by the `gcloud` tool before, but maybe this is changing? (the [documentation](https://cloud.google.com/run/docs/deploying) now assumes `run/region` is set). Not sure which course of action is best: making `datasette` ensure that `run/region` is set beforehand or wait a bit until the gcloud CLI stabilizes? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
434321685 | |
https://github.com/simonw/datasette/issues/446#issuecomment-489221481 | https://api.github.com/repos/simonw/datasette/issues/446 | 489221481 | MDEyOklzc3VlQ29tbWVudDQ4OTIyMTQ4MQ== | 45057 | 2019-05-03T19:58:31Z | 2019-05-03T19:58:31Z | CONTRIBUTOR | In this particular case I don't think there's an issue making all those required. However, I suspect we might have to allow optional values at some point - my preferred solution to russss/datasette-geo#2 would need one. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
440134714 | |
https://github.com/simonw/datasette/issues/446#issuecomment-489222223 | https://api.github.com/repos/simonw/datasette/issues/446 | 489222223 | MDEyOklzc3VlQ29tbWVudDQ4OTIyMjIyMw== | 45057 | 2019-05-03T20:01:19Z | 2019-05-03T20:01:29Z | CONTRIBUTOR | Also I have a slight preference against (ab)using `__slots__` to enforce fields, although I have done it myself in the past. It would be possible to do this with `__setattr__` instead, although that's an implementation detail and I'm not too fussed about it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
440134714 |