github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/268#issuecomment-790257263 | https://api.github.com/repos/simonw/datasette/issues/268 | 790257263 | MDEyOklzc3VlQ29tbWVudDc5MDI1NzI2Mw== | 649467 | 2021-03-04T03:20:23Z | 2021-03-04T03:20:23Z | NONE | It's kind of an ugly hack, but you can try out what using the fts5 table as an actual datasette-accessible table looks like without changing any datasette code by creating yet another view on top of the fts5 table: `create view proxyview as select *, rank, table_fts as fts from table_fts;` That's now visible from datasette, just like any other view, but you can use `fts match escape_fts(search_string) order by rank`. This is only good as a proof of concept because you're inefficiently going from view -> fts5 external content table -> view -> data table. However, it does show it works. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323718842 | |
https://github.com/simonw/datasette/issues/268#issuecomment-789409126 | https://api.github.com/repos/simonw/datasette/issues/268 | 789409126 | MDEyOklzc3VlQ29tbWVudDc4OTQwOTEyNg== | 649467 | 2021-03-03T03:57:15Z | 2021-03-03T03:58:40Z | NONE | In FTS5, I think doing an FTS search is actually much easier than doing a join against the main table like datasette does now. In fact, FTS5 external content tables provide a transparent interface back to the original table or view. Here's what I'm currently doing: * build a view that joins whatever tables I want and rename the columns to non-joiny names (e.g, `chapter.name AS chapter_name` in the view where needed) * Create an FTS5 table with `content="viewname"` * As described in the "external content tables" section (https://www.sqlite.org/fts5.html#external_content_tables), sql queries can be made directly to the FTS table, which behind the covers makes select calls to the content table when the content of the original columns are needed. * In addition, you get "rank" and "bm25()" available to you when you select on the _fts table. Unfortunately, datasette doesn't currently seem happy being coerced into doing a real query on an fts5 table. This works: ```select col1, col2, col3 from table_fts where coll1="value" and table_fts match escape_fts("search term") order by rank``` But this doesn't work in the datasette SQL query interface: ```select col1, col2, col3 from table_fts where coll1="value" and table_fts match escape_fts(:search) order by rank``` (the "search" input text field doesn't show up) For what datasette is doing right now, I think you could just use contentless fts5 tables (`content=""`), since all you care about is the rowid since all you're doing a subselect to get the rowid anyway. In fts5, that's just a contentless table. I guess if you want to follow this suggestion, you'd need a somewhat different code path for fts5. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323718842 | |
https://github.com/simonw/sqlite-utils/issues/220#issuecomment-783662968 | https://api.github.com/repos/simonw/sqlite-utils/issues/220 | 783662968 | MDEyOklzc3VlQ29tbWVudDc4MzY2Mjk2OA== | 649467 | 2021-02-22T20:44:51Z | 2021-02-22T20:44:51Z | NONE | Actually, coming back to this, I have a clearer use case for enabling fts generation for views: making it easier to bring in text from lookup tables and other joins. The datasette documentation describes populating an fts table like so: ``` INSERT INTO "items_fts" (rowid, name, description, category_name) SELECT items. rowid, items.name, items.description, categories.name FROM items JOIN categories ON items.category_id=categories.id; ``` Alternatively if you have fts support in sqlite_utils for views (which sqlite and fts5 support), you can do the same thing just by creating a view that captures the above joins as columns, then creating an fts table from that view. Such an fts table can be created using sqlite_utils, where one created with your method can't. The resulting fts table can then be used by a whole family of related tables and views in the manner you described earlier in this issue. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
783778672 | |
https://github.com/simonw/sqlite-utils/issues/171#issuecomment-714219725 | https://api.github.com/repos/simonw/sqlite-utils/issues/171 | 714219725 | MDEyOklzc3VlQ29tbWVudDcxNDIxOTcyNQ== | 649467 | 2020-10-22T04:38:35Z | 2020-10-22T04:38:35Z | NONE | Thanks. As I said, I think the result (being able to query tree structures like ancestors and descendants) is more important than the implementation, and I agree that this particular sqlite extension is too obscure. Just providing an sqlite utility to build or rebuild a transitive closure table might be more generically useful. I find that hierarchical data shows up pretty frequently in some data science problems. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707407567 | |
https://github.com/simonw/datasette/issues/1003#issuecomment-706302863 | https://api.github.com/repos/simonw/datasette/issues/1003 | 706302863 | MDEyOklzc3VlQ29tbWVudDcwNjMwMjg2Mw== | 649467 | 2020-10-09T17:17:06Z | 2020-10-09T17:17:06Z | NONE | I agree on the descriptive and python-consistent naming. There is already a tojson, but frankly i find the "to" and "from" confusing in a text templating language where what's a string and what's data isn't 100% transparent. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
718238967 | |
https://github.com/simonw/datasette/issues/144#issuecomment-346427794 | https://api.github.com/repos/simonw/datasette/issues/144 | 346427794 | MDEyOklzc3VlQ29tbWVudDM0NjQyNzc5NA== | 649467 | 2017-11-22T17:55:45Z | 2017-11-22T17:55:45Z | NONE | Thanks. There is a way to use pip to grab apsw, which also let's you configure it (flags to build extensions, use an internal sqlite, etc). Don't know how that works as a dependency for another package, though. On November 22, 2017 11:38:06 AM EST, Simon Willison <notifications@github.com> wrote: >I have a solution for FTS already, but I'm interested in apsw as a >mechanism for allowing custom virtual tables to be written in Python >(pysqlite only lets you write custom functions) > >Not having PyPI support is pretty tough though. I'm planning a >plugin/extension system which would be ideal for things like an >optional apsw mode, but that's a lot harder if apsw isn't in PyPI. > >-- >You are receiving this because you authored the thread. >Reply to this email directly or view it on GitHub: >https://github.com/simonw/datasette/issues/144#issuecomment-346405660 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
276091279 |