issues

Table actions

1 row where repo = 140912432, state = "open" and user = 82988 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

type 1

state 1

  • open · 1 ✖

repo 1

  • sqlite-utils · 1 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1128466114 I_kwDOCGYnMM5DQwbC 406 Creating tables with custom datatypes psychemedia 82988 open 0     5 2022-02-09T12:16:31Z 2022-09-15T18:13:50Z   NONE  

Via https://stackoverflow.com/a/18622264/454773 I note the ability to register custom handlers for novel datatypes that can map into and out of things like sqlite BLOBs.

From a quick look and a quick play, I didn't spot a way to do this in sqlite_utils?

For example:

```python

Via https://stackoverflow.com/a/18622264/454773

import sqlite3 import numpy as np import io

def adapt_array(arr): """ http://stackoverflow.com/a/31312102/190597 (SoulNibbler) """ out = io.BytesIO() np.save(out, arr) out.seek(0) return sqlite3.Binary(out.read())

def convert_array(text): out = io.BytesIO(text) out.seek(0) return np.load(out)

Converts np.array to TEXT when inserting

sqlite3.register_adapter(np.ndarray, adapt_array)

Converts TEXT to np.array when selecting

sqlite3.register_converter("array", convert_array) ```

```python from sqlite_utils import Database db = Database('test.db')

Reset the database connection to used the parsed datatype

sqlite_utils doesn't seem to support eg:

Database('test.db', detect_types=sqlite3.PARSE_DECLTYPES)

db.conn = sqlite3.connect(db_name, detect_types=sqlite3.PARSE_DECLTYPES)

Create a table the old fashioned way

but using the new custom data type

vector_table_create = """ CREATE TABLE dummy (title TEXT, vector array ); """

cur = db.conn.cursor() cur.execute(vector_table_create)

sqlite_utils doesn't appear to support custom types (yet?!)

The following errors on the "array" datatype

""" db["dummy"].create({ "title": str, "vector": "array", }) """ ```

We can then add / retrieve records from the database where the datatype of the vector field is a custom registered array type (which is to say, a numpy array):

```python import numpy as np

db["dummy"].insert({'title':"test1", 'vector':np.array([1,2,3])})

for row in db.query("SELECT * FROM dummy"): print(row['title'], row['vector'], type(row['vector']))

""" test1 [1 2 3] <class 'numpy.ndarray'> """ ```

It would be handy to be able to do this idiomatically in sqlite_utils.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/406/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);