issues

In: - https://github.com/simonw/datasette.io/issues/152#issuecomment-1712323347

This turned out to be the fix:

bash dogsheep-beta index dogsheep-index.db templates/dogsheep-beta.yml sqlite-utils rebuild-fts dogsheep-index.db

When creating a dogsheep index from config.yml file on pocket.db (created using pocket-to-sqlite), I am getting this error

Traceback (most recent call last): File "/Users/khushmeeet/.pyenv/versions/3.11.2/bin/dogsheep-beta", line 8, in <module> sys.exit(cli()) ^^^^^ File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/dogsheep_beta/cli.py", line 36, in index run_indexer( File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/dogsheep_beta/utils.py", line 32, in run_indexer ensure_table_and_indexes(db, tokenize) File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/dogsheep_beta/utils.py", line 91, in ensure_table_and_indexes table.add_foreign_key(*fk) File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/sqlite_utils/db.py", line 2155, in add_foreign_key self.db.add_foreign_keys([(self.name, column, other_table, other_column)]) File "/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/sqlite_utils/db.py", line 1116, in add_foreign_keys cursor.execute( sqlite3.OperationalError: table sqlite_master may not be modified

Command I ran to get this error dogsheep-beta index pocket.db config.yml

Dogsheep version dogsheep-beta, version 0.10.2

Python version Python 3.11.2

This makes it so you can use Dogsheep if you're using Datasette with the --base-url /some-path/ setting.

Hi,

First of all, thank you for this wonderful project :smile:

I started to use dogsheep to make my personal data searchable, and by using the project I noticed an issue with the index command.

It always expects you are running it from the root folder from where the data is located, so I got some errors while trying to make it work on my setup.

I separate all databases inside a data folder (I published my setup to be easier to follow: https://github.com/humrochagf/my-dogsheep)

Before, I configured dogsheep.yml to add the data folder to its path like this:

yml data/twitter.db: tweets: sql: |- ...

And running the index command like this:

dogsheep-beta index data/dogsheep.db dogsheep.yml

It worked to the normal search feature with no problem this way, but when I started adding display_sql rules the app started to crash, because at datasette get_database it was looking for data/twitter and it only had a db called twitter there.

So my workaround to that was to cd into the data folder and run the indexer. You can check the way I'm doing it at this line of the makefile: https://github.com/humrochagf/my-dogsheep/blob/main/makefile#L3

It works but it would be nice to have an option to pass the path where the data is located to the index function.

https://datasette.io/-/beta?q=+ returns a 500

fts5: syntax error near ""

Refs https://github.com/simonw/datasette/issues/878

``` results = beta_db["search_index"].search("run") if use_porter: assert results == [ ( "dogs.db/dogs", "1", "Cleo", "2020-08-22 04:41:33", 1, 0, "running", None, None, ) ] else:

      assert results == []

E assert <generator ob...x7f3c080a3e08> == [] E +<generator object Table.search at 0x7f3c080a3e08> E -[] E Full diff: E - [] E + <generator object Table.search at 0x7f3c080a3e08> ``` This was caused by a backwards incompatible change in sqlite-utils 3.0: https://sqlite-utils.readthedocs.io/en/stable/changelog.html#v3-0

A link to /-/beta?category=1&timestamp__date=2018-08-01&q=swedish should be to /-/beta?category=1&timestamp__date=2018-08-01&q=swedish&sort=newest

Following #21

In regular timeline view sort defaults to newest, not relevance - so this UI is incorrect:

Sometimes I want to sort by date, not by relevance.

quantified-self habit-tracking google-fit time-tracking wearables quantifiedself for example

Searching for eg #dogfest currently throws an FTS exception - but I want to support advanced FTS query tricks as seen in #13.

https://dogsheep.simonwillison.net/-/beta?q=%23dogfest

fts5: syntax error near "#"

Idea: catch that error and re-run the query with FTS escaping applied!

It would be cool if the display_sql could return a column populated with GeoJSON which would the automatically be displayed on a map in the results (or maybe default JS would look for a class="geojson" element output by the display template) - ala https://github.com/simonw/datasette-leaflet-geojson

Then I could render workout routes on a map, or Swarm checkin points.

Once I've got the Datasette plugin to a state where it's worth building a demo: #3

I can use data from my public https://github-to-sqlite.dogsheep.net/ demo plus the Pocket data subset I use for the demo in https://github.com/dogsheep/pocket-to-sqlite/issues/5 - I could pull in the https://dogsheep-photos.dogsheep.net/ photos data too.

Since default view will likely be ordered by timestamp descending.