Compare commits

..

181 Commits

Author SHA1 Message Date
Nuno Cruces
a75b8887db Updated dependencies. 2024-12-11 19:02:03 +00:00
Nuno Cruces
9f456fecb9 Updated dependencies. 2024-12-11 18:38:57 +00:00
Nuno Cruces
36bbd674c2 Add ColumnTypeScanType to driver (#199). 2024-12-11 18:35:50 +00:00
Nuno Cruces
7f5ea54009 Windows blocking locks. (#200) 2024-12-11 15:05:22 +00:00
Nuno Cruces
5f1d5727cd SQLite 3.47.2. 2024-12-10 18:38:49 +00:00
Nuno Cruces
6fb259e2b9 Fix. 2024-12-10 18:25:49 +00:00
Nuno Cruces
301f6bc2bd Fix. 2024-12-09 19:26:47 +00:00
Nuno Cruces
9e112c54b0 Fix, see #197. 2024-12-08 11:47:11 +00:00
Nuno Cruces
270efcb4af Updated dependencies. 2024-12-05 12:46:46 +00:00
dependabot[bot]
8252198dd2 Bump actions/attest-build-provenance from 1 to 2 (#196)
Bumps [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) from 1 to 2.
- [Release notes](https://github.com/actions/attest-build-provenance/releases)
- [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md)
- [Commits](https://github.com/actions/attest-build-provenance/compare/v1...v2)

---
updated-dependencies:
- dependency-name: actions/attest-build-provenance
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-05 13:04:47 +00:00
dependabot[bot]
dff825ae81 Bump golang.org/x/crypto from 0.29.0 to 0.30.0 (#193)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.29.0 to 0.30.0.
- [Commits](https://github.com/golang/crypto/compare/v0.29.0...v0.30.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-05 03:03:36 +00:00
Nuno Cruces
aae732e530 API fix. 2024-12-04 17:09:22 +00:00
Nuno Cruces
2e3ba3949e Remove sqlite3_nosys. 2024-11-29 15:51:51 +00:00
Nuno Cruces
a44690035f Refactor. 2024-11-28 00:15:55 +00:00
Nuno Cruces
7e12105b22 Split module. 2024-11-27 14:25:01 +00:00
Nuno Cruces
987db177ad Split module. 2024-11-27 14:22:31 +00:00
Nuno Cruces
1469cb9f1a Imports. 2024-11-27 11:42:25 +00:00
Nuno Cruces
4ede2c7216 Updated dependencies. 2024-11-26 14:18:53 +00:00
Nuno Cruces
cf14f190b2 SQLite 3.47.1. 2024-11-26 11:40:11 +00:00
dependabot[bot]
6ca92b035d Bump github.com/tetratelabs/wazero (#189)
Bumps [github.com/tetratelabs/wazero](https://github.com/tetratelabs/wazero) from 1.8.2-0.20241115151925-0a207958052e to 1.8.2.
- [Release notes](https://github.com/tetratelabs/wazero/releases)
- [Commits](https://github.com/tetratelabs/wazero/commits/v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/tetratelabs/wazero
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-25 22:34:34 +00:00
Nuno Cruces
2912adf226 Update README.md 2024-11-25 14:17:08 +00:00
Nuno Cruces
b3f83a4392 Enable compiler. 2024-11-18 12:52:48 +00:00
Nuno Cruces
1223c4fc80 Flags. 2024-11-18 12:52:04 +00:00
Nuno Cruces
e9d6509577 Tweak. 2024-11-15 18:41:31 +00:00
Nuno Cruces
81dd786af7 Test more. 2024-11-12 14:14:45 +00:00
Nuno Cruces
a2253558ef Use lock file. 2024-11-11 16:55:37 +00:00
Nuno Cruces
466d14a9e0 Fix, speed. 2024-11-08 13:02:19 +00:00
Nuno Cruces
ada7b3a906 Updated dependencies. 2024-11-08 11:48:45 +00:00
Nuno Cruces
eba73a87d9 Refactor. 2024-11-07 12:18:42 +00:00
Nuno Cruces
38da27b5d1 Fix. 2024-11-06 12:10:48 +00:00
Nuno Cruces
23737a61ba Log. 2024-11-06 12:10:47 +00:00
Nuno Cruces
af473c7519 Update README.md 2024-11-06 01:34:21 +00:00
Nuno Cruces
a946c00f8e Refactor, speed. 2024-11-06 00:30:37 +00:00
Nuno Cruces
32153763a3 Update README.md 2024-11-05 17:39:36 +00:00
Nuno Cruces
a57ce87157 Windows shared memory. (#181) 2024-11-05 17:30:10 +00:00
Nuno Cruces
81e7a94ca4 Optimize regexp. 2024-11-04 19:30:10 +00:00
Nuno Cruces
034b9a3b4d More regexp. 2024-11-04 18:53:03 +00:00
Nuno Cruces
363b12ee4c Fix #178. 2024-11-03 13:14:38 +00:00
Nuno Cruces
90d6ec31b9 Refactor, speed. 2024-11-02 15:32:41 +00:00
Nuno Cruces
17f7840a83 Share memory by copying. (#180) 2024-10-31 15:21:15 +00:00
Nuno Cruces
b2e8636227 Dot file locking. (#179) 2024-10-30 12:20:21 +00:00
Nuno Cruces
1ad1608228 Refactor. 2024-10-30 00:50:28 +00:00
Nuno Cruces
9f284f0b26 Fix tests. 2024-10-30 00:44:50 +00:00
Nuno Cruces
df4e144e89 Prevent overflow. 2024-10-28 14:16:43 +00:00
Nuno Cruces
96074b24bf binaryen-version_120. 2024-10-28 10:56:10 +00:00
Nuno Cruces
4d68f8976c Dependencies. 2024-10-25 14:29:03 +01:00
Nuno Cruces
f2545534af Rename. 2024-10-25 14:20:52 +01:00
Nuno Cruces
69e5cf706b Checksums in default VFS. (#177) 2024-10-25 13:49:06 +01:00
Nuno Cruces
75c1dbb052 Checksum VFS. (#176) 2024-10-25 00:12:29 +01:00
Nuno Cruces
64e2500ca8 SQLite 3.47.0. 2024-10-24 00:34:54 +01:00
Nuno Cruces
0cd0f48365 Rename WAL, fixes. 2024-10-24 00:23:13 +01:00
Nuno Cruces
c69ee0fe8d Rename sql3util. 2024-10-22 23:36:38 +01:00
Nuno Cruces
b9b489aae9 Fixes, internal docs. 2024-10-22 13:08:31 +01:00
Nuno Cruces
21de004779 VFS API refactor. 2024-10-21 15:07:30 +01:00
Nuno Cruces
9eec439d35 More tests. 2024-10-19 08:58:55 +01:00
Nuno Cruces
fefee692db VFS utilities. 2024-10-18 18:07:48 +01:00
Nuno Cruces
f18561ee11 Lerp. 2024-10-18 13:07:39 +01:00
Nuno Cruces
ace01b2927 Split local imports. 2024-10-18 12:20:32 +01:00
Nuno Cruces
89f750a6e9 Wrap FilePersistentWAL. 2024-10-18 12:14:31 +01:00
Nuno Cruces
d6aebe67cc AES-XTS VFS (#171)
Co-authored-by: Ben Krieger <ben.krieger@intel.com>
2024-10-17 23:53:39 +01:00
Nuno Cruces
714ea0e779 Blocking locks improvements. 2024-10-17 15:39:01 +01:00
Nuno Cruces
c900889848 Configure memory, 32-bit WAL. (#170) 2024-10-17 13:04:23 +01:00
Nuno Cruces
50c8517603 Window percentil. 2024-10-17 08:15:44 +01:00
Nuno Cruces
c78d00dca0 Better percentile compatibility. 2024-10-16 14:00:22 +01:00
Nuno Cruces
ddfaf12cd8 Reorg READMEs. 2024-10-15 14:51:11 +01:00
Nuno Cruces
368c900db8 Test latest snapshot. 2024-10-10 12:58:32 +01:00
Nuno Cruces
e524fb185d More consistent interrupts. 2024-10-08 16:38:57 +01:00
Nuno Cruces
cc7bacfb9c Remove rangefunc experiment. 2024-10-07 15:13:19 +01:00
Nuno Cruces
911e497891 Improved quoting. 2024-10-07 14:42:09 +01:00
Nuno Cruces
3469460635 Improved error handling. 2024-10-07 13:22:31 +01:00
Nuno Cruces
b5adcacec4 Don't panic on memory commit failure. (#154) 2024-10-06 00:06:43 +01:00
dependabot[bot]
62d6712f82 Bump github.com/tetratelabs/wazero from 1.8.0 to 1.8.1 (#166)
Bumps [github.com/tetratelabs/wazero](https://github.com/tetratelabs/wazero) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/tetratelabs/wazero/releases)
- [Commits](https://github.com/tetratelabs/wazero/compare/v1.8.0...v1.8.1)

---
updated-dependencies:
- dependency-name: github.com/tetratelabs/wazero
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-06 00:02:33 +01:00
dependabot[bot]
d34e6197a8 Bump golang.org/x/crypto from 0.27.0 to 0.28.0 (#163)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.27.0 to 0.28.0.
- [Commits](https://github.com/golang/crypto/compare/v0.27.0...v0.28.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-04 23:39:43 +01:00
Nuno Cruces
f9e867be60 Time fix. 2024-10-04 18:29:35 +01:00
Nuno Cruces
96c61a2f55 Improved interrupts. 2024-10-04 16:18:24 +01:00
Nuno Cruces
ac94a5406e Clear bindings. 2024-10-04 00:35:04 +01:00
Nuno Cruces
34617e15f0 Savepoint quoting. 2024-10-03 12:01:54 +01:00
Nuno Cruces
83b3f6ce0a Time fix. 2024-10-02 12:59:59 +01:00
Nuno Cruces
f2c8aa0ddf Scanning improvements. 2024-10-02 12:59:49 +01:00
Nuno Cruces
63ea13e41e Fix examples. 2024-10-02 12:01:22 +01:00
Nuno Cruces
b1508bface Faster stats. 2024-10-01 15:16:06 +01:00
Nuno Cruces
1c6897c8e2 Multi-value Wasm. 2024-09-30 23:45:33 +01:00
Nuno Cruces
170e1dbebd Don't override LIKE. 2024-09-30 14:04:23 +01:00
Nuno Cruces
25fc5a606a Delete functions, modules, hooks. 2024-09-30 13:11:04 +01:00
Nuno Cruces
8b6c2b28fb More BCE. 2024-09-30 11:03:50 +01:00
kim
e59e2ed2a2 use "len(buf)-1" to access final slice index to eliminate boundary-checks (#160) 2024-09-29 20:01:51 +01:00
Nuno Cruces
505c6640c2 Unaccent, title case. 2024-09-29 20:01:14 +01:00
Nuno Cruces
5b0a063bfe More BCE. 2024-09-28 10:37:47 +01:00
kim
32931032d3 add compiler hints to reduce number of slice bounds checks (#158)
* add compiler hints to reduce number of slice bounds checks

* panic on assert instead of returning error for code coverage
2024-09-28 01:26:46 +01:00
Nuno Cruces
b7055ef04b Spellfix1, fix #157. 2024-09-27 16:28:17 +01:00
Nuno Cruces
167025f47a On demand collations. 2024-09-27 16:28:17 +01:00
Nuno Cruces
b4b50fc547 Context aware busy handler. (#146) 2024-09-27 12:33:57 +01:00
Nuno Cruces
08f7764fe0 Improved OPEN_NOFOLLOW. 2024-09-27 12:32:11 +01:00
Nuno Cruces
4e0b8aeaa8 Updated dependencies. 2024-09-24 14:07:45 +01:00
Nuno Cruces
d694e9718e Test ppc64le. 2024-09-24 13:20:16 +01:00
Nuno Cruces
90218c0d79 Check interfaces. 2024-09-24 12:49:53 +01:00
Nuno Cruces
44c3f9b4e7 Coverage, docs. 2024-09-22 17:34:35 +01:00
Nuno Cruces
2526fc8444 Transitive closure virtual table. 2024-09-21 11:44:17 +01:00
Nuno Cruces
d7376209ee Fix. 2024-09-21 00:51:03 +01:00
Nuno Cruces
83e2587596 Gorm v1.25.12. 2024-09-20 23:07:20 +01:00
Nuno Cruces
6101debe28 Locking improvements. 2024-09-18 11:24:46 +01:00
Nuno Cruces
06eaf41c4f Fix #151. 2024-09-16 12:05:37 +01:00
Michael Lynch
9638976991 Fix HardHeapLimit name in comment (#152) 2024-09-14 00:34:58 +01:00
Nuno Cruces
b631ff1add Configure heap limits. 2024-09-13 16:21:20 +01:00
Nuno Cruces
fdfaaa8cec Fix memory allocator issue.
Windows and unix allocators would panic when asked to allocate the maximum size.
2024-09-12 16:05:31 +01:00
Nuno Cruces
6a2827f989 Reuse blob buffer. 2024-09-10 11:48:19 +01:00
Nuno Cruces
9d77322d50 Memory management. 2024-09-09 13:21:33 +01:00
Michael Lynch
c1915feb2e Expand explanation of openblob semantics (#149)
Per @ncruces' comments in https://github.com/ncruces/go-sqlite3/issues/148#issuecomment-2334155468, this change clarifies the documentation for openblob to help clients understand the lifecycle of the blob handle within the callback.
2024-09-06 18:15:20 +01:00
Nuno Cruces
52f9af3ca0 binaryen-version_119. 2024-09-06 02:13:48 +01:00
dependabot[bot]
2f90277165 Bump golang.org/x/crypto from 0.26.0 to 0.27.0 (#147)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.26.0 to 0.27.0.
- [Commits](https://github.com/golang/crypto/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-05 23:44:46 +01:00
Nuno Cruces
356dd56e5f Improved blobio extension. 2024-09-05 12:09:21 +01:00
Nuno Cruces
e2a2d447ce Updated binaries. 2024-09-04 19:38:10 +01:00
Nuno Cruces
75190a6f98 os.Executable rather than os.Args[0] 2024-09-04 18:48:42 +01:00
Nuno Cruces
35c5619880 Tweak. 2024-09-04 18:26:57 +01:00
Nuno Cruces
b51234cc82 Reduce allocs. 2024-09-03 17:32:06 +01:00
Nuno Cruces
cf7b89d3c4 Issue #145. 2024-09-03 12:24:03 +01:00
Nuno Cruces
ff9f27a778 Fix #141. 2024-09-03 11:22:24 +01:00
Nuno Cruces
f26f1a17a9 Blocking locks (#144) 2024-09-02 23:59:26 +01:00
Nuno Cruces
b9b2ff13da Stricter test. 2024-08-30 09:32:30 +01:00
Nuno Cruces
78473b4b37 Fix BSD locks. 2024-08-30 01:27:57 +01:00
Nuno Cruces
3806c1cc23 Test tweaks. 2024-08-30 01:27:22 +01:00
Nuno Cruces
1660c41f8c Typo. 2024-08-29 10:32:10 +01:00
Nuno Cruces
62b67c937e Tweak. 2024-08-27 01:55:39 +01:00
Nuno Cruces
9e9971c292 Fix enconding. 2024-08-27 01:45:44 +01:00
Nuno Cruces
d13bf1afaa Readability. 2024-08-26 19:47:46 +01:00
Nuno Cruces
f7c9551d66 Update README.md 2024-08-15 12:42:38 +01:00
Nuno Cruces
22beef91d2 Updated dependencies. 2024-08-14 17:56:06 +01:00
Nuno Cruces
c97bbc7dab vet fix. 2024-08-14 17:20:50 +01:00
Nuno Cruces
800eb107f9 wazero v1.8.0. 2024-08-14 16:54:24 +01:00
Nuno Cruces
6a1973f530 SQLite 3.46.1. 2024-08-13 15:27:00 +01:00
Nuno Cruces
bd141fec92 Tests. 2024-08-12 17:50:23 +01:00
Nuno Cruces
e92999bfe3 Avoid alloc. 2024-08-12 17:36:06 +01:00
Nuno Cruces
d5583b6ec9 Try to fix flaky test. 2024-08-11 17:35:27 +01:00
Nuno Cruces
3649c1098e Remove unneeded check. 2024-08-11 16:18:44 +01:00
Nuno Cruces
f743639c8f Docs. 2024-08-09 14:05:49 +01:00
Nuno Cruces
7cb974fd9a Windows CI. 2024-08-09 12:16:46 +01:00
Nuno Cruces
eea6aa7493 Docs. 2024-08-09 10:06:05 +01:00
Nuno Cruces
9a610888f9 BEGIN CONCURRENT, wal2. (#138) 2024-08-09 00:48:25 +01:00
dependabot[bot]
dc4113073c Bump golang.org/x/sys from 0.23.0 to 0.24.0 (#139)
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.23.0 to 0.24.0.
- [Commits](https://github.com/golang/sys/compare/v0.23.0...v0.24.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-09 00:47:32 +01:00
Daenney
38cab3202a Add example for custom time with database/sql (#133) 2024-08-07 18:34:34 +01:00
Nuno Cruces
2068b97116 Updated dependencies. 2024-08-07 12:15:03 +01:00
Nuno Cruces
8f835eda79 Use memdb for tests. (#131) 2024-08-05 21:25:47 +01:00
Nuno Cruces
40db26c1dd wasi-sdk-24. 2024-08-03 00:13:06 +01:00
Nuno Cruces
a6815531e0 Copy blobs. 2024-08-02 13:27:01 +01:00
Daenney
6c12a8c1fa More JSON/B examples. (#127) 2024-07-31 23:31:56 +01:00
Nuno Cruces
e9de84a87f Testing. 2024-07-31 13:18:52 +01:00
Nuno Cruces
3bb1898335 More APIs. (#125)
sqlite3_db_cacheflush
sqlite3_db_status
sqlite3_expanded_sql
sqlite3_next_stmt
sqlite3_sql
sqlite3_table_column_metadata
sqlite3_trace_v2
sqlite3_value_frombind
2024-07-31 12:15:08 +01:00
Nuno Cruces
22132620b8 API tweaks, tests. 2024-07-30 14:49:58 +01:00
Nuno Cruces
c766a4fed2 Testing. 2024-07-26 23:51:35 +01:00
Nuno Cruces
73125945f8 Fix API inconsistency. 2024-07-26 12:25:15 +01:00
Nuno Cruces
32d998c84b Filenames. 2024-07-26 01:23:35 +01:00
Nuno Cruces
8d450f82fc Remove init. 2024-07-25 13:01:00 +01:00
Nuno Cruces
64b77f1a79 Concurrent transactions. 2024-07-25 01:00:31 +01:00
Nuno Cruces
19639be9f9 Gorm v1.25.11. 2024-07-24 14:25:14 +01:00
Nuno Cruces
2996e77420 Implement file control. (#123) 2024-07-24 12:37:35 +01:00
Nuno Cruces
24288c0e26 Tests. 2024-07-23 13:28:09 +01:00
Nuno Cruces
06f58c35e3 Fix Context.ResultPointer. 2024-07-20 12:52:25 +01:00
Nuno Cruces
28f225b32e Testing. 2024-07-20 01:42:50 +01:00
Nuno Cruces
b289fca3ca Everything changes, stays the same. 2024-07-20 00:43:34 +01:00
dependabot[bot]
21de85e849 Bump cross-platform-actions/action from 0.24.0 to 0.25.0 (#119)
* Bump cross-platform-actions/action from 0.24.0 to 0.25.0

Bumps [cross-platform-actions/action](https://github.com/cross-platform-actions/action) from 0.24.0 to 0.25.0.
- [Release notes](https://github.com/cross-platform-actions/action/releases)
- [Changelog](https://github.com/cross-platform-actions/action/blob/master/changelog.md)
- [Commits](https://github.com/cross-platform-actions/action/compare/v0.24.0...v0.25.0)

---
updated-dependencies:
- dependency-name: cross-platform-actions/action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nuno Cruces <ncruces@users.noreply.github.com>
2024-07-12 00:54:51 +01:00
Nuno Cruces
4498f35a39 wasi-sdk-23.0. 2024-07-11 15:03:16 +01:00
Nuno Cruces
0c7d0a097d Allow SQLite to use atomic operations. (#118) 2024-07-11 13:35:41 +01:00
Nuno Cruces
f537ab9a94 Tests. 2024-07-10 15:41:28 +01:00
Nuno Cruces
88b5b409df Use courtney. 2024-07-10 11:12:31 +01:00
Nuno Cruces
51c325bc5b Optimization. 2024-07-10 00:08:59 +01:00
Nuno Cruces
5872224f77 Tests. 2024-07-09 22:13:14 +01:00
Nuno Cruces
7b56989489 Use iter. 2024-07-09 15:54:45 +01:00
Nuno Cruces
bd5be4cde6 Tests. 2024-07-09 14:52:01 +01:00
Nuno Cruces
c19fec1e83 binaryen-version_118. 2024-07-09 00:51:54 +01:00
Nuno Cruces
b5f746aadf Automatically load extensions. (#115) 2024-07-08 12:06:57 +01:00
dependabot[bot]
fff8b1c74f Bump golang.org/x/crypto from 0.24.0 to 0.25.0 (#116)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.24.0 to 0.25.0.
- [Commits](https://github.com/golang/crypto/compare/v0.24.0...v0.25.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-06 00:33:50 +01:00
Nuno Cruces
d27da3f390 Fix flaky test. 2024-07-05 00:49:22 +01:00
Nuno Cruces
a1fae26b66 Regular expression extension. (#114) 2024-07-05 00:12:26 +01:00
Nuno Cruces
806cc6677d Updated dependencies. 2024-07-04 19:38:26 +01:00
Nuno Cruces
da6e4d8b86 UUID extension (#113) 2024-07-04 15:28:49 +01:00
Nuno Cruces
72f8ad0f14 Toolchain. 2024-07-03 15:02:58 +01:00
Nuno Cruces
5a4c7a58c4 Refactor CREATE parser. (#111) 2024-07-03 14:06:07 +01:00
Nuno Cruces
90f7e502be Tweaks. 2024-07-02 15:42:20 +01:00
Nuno Cruces
c0b289d000 More BSDs. 2024-06-26 14:56:36 +01:00
Nuno Cruces
a84d905d8c Fix go:linkname for mmap (#107) 2024-06-25 10:31:11 +01:00
Nuno Cruces
aa7edb1848 Tests. 2024-06-21 16:23:56 +01:00
250 changed files with 8519 additions and 2459 deletions

16
.github/actions/vmactions/template.yml vendored Normal file
View File

@@ -0,0 +1,16 @@
name: VM Actions matrix
description: VM Actions matrix template
inputs:
run:
description: The CI command to run
required: true
runs:
using: composite
steps:
- uses: ${VMACTIONS}
with:
usesh: true
copyback: false
run: ${{inputs.run}}

View File

@@ -6,6 +6,10 @@ echo 'set -eu' > test.sh
for p in $(go list ./...); do
dir=".${p#github.com/ncruces/go-sqlite3}"
name="$(basename "$p").test"
(cd ${dir}; go test -c)
[ -f "${dir}/${name}" ] && echo "(cd ${dir}; ./${name} ${TESTFLAGS})" >> test.sh
done
(cd ${dir}; go test -c ${BUILDFLAGS:-})
[ -f "${dir}/${name}" ] && echo "(cd ${dir}; ./${name} ${TESTFLAGS:-})" >> test.sh
done
if [[ -v VMACTIONS ]]; then
envsubst < .github/actions/vmactions/template.yml > .github/actions/vmactions/action.yml
fi

View File

@@ -17,11 +17,9 @@ echo aix ; GOOS=aix GOARCH=ppc64 go build .
echo js ; GOOS=js GOARCH=wasm go build .
echo wasip1 ; GOOS=wasip1 GOARCH=wasm go build .
echo linux-flock ; GOOS=linux GOARCH=amd64 go build -tags sqlite3_flock .
echo linux-noshm ; GOOS=linux GOARCH=amd64 go build -tags sqlite3_noshm .
echo linux-nosys ; GOOS=linux GOARCH=amd64 go build -tags sqlite3_nosys .
echo linux-dotlk ; GOOS=linux GOARCH=amd64 go build -tags sqlite3_dotlk .
echo darwin-flock ; GOOS=darwin GOARCH=amd64 go build -tags sqlite3_flock .
echo darwin-noshm ; GOOS=darwin GOARCH=amd64 go build -tags sqlite3_noshm .
echo darwin-nosys ; GOOS=darwin GOARCH=amd64 go build -tags sqlite3_nosys .
echo windows-nosys ; GOOS=windows GOARCH=amd64 go build -tags sqlite3_nosys .
echo freebsd-nosys ; GOOS=freebsd GOARCH=amd64 go build -tags sqlite3_nosys .
echo solaris-flock ; GOOS=solaris GOARCH=amd64 go build -tags sqlite3_flock .
echo darwin-dotlk ; GOOS=darwin GOARCH=amd64 go build -tags sqlite3_dotlk .
echo windows-dotlk ; GOOS=windows GOARCH=amd64 go build -tags sqlite3_dotlk .
echo freebsd-dotlk ; GOOS=freebsd GOARCH=amd64 go build -tags sqlite3_dotlk .
echo solaris-dotlk ; GOOS=solaris GOARCH=amd64 go build -tags sqlite3_dotlk .

View File

@@ -13,4 +13,4 @@ jobs:
with: { go-version: stable }
- name: Build
run: .github/workflows/cross.sh
run: .github/workflows/cross.sh

View File

@@ -2,29 +2,33 @@
set -euo pipefail
if [[ "$OSTYPE" == "linux"* ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-22/wasi-sdk-22.0-linux.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_117/binaryen-version_117-x86_64-linux.tar.gz"
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-24/wasi-sdk-24.0-x86_64-linux.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_120/binaryen-version_120-x86_64-linux.tar.gz"
elif [[ "$OSTYPE" == "darwin"* ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-22/wasi-sdk-22.0-macos.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_117/binaryen-version_117-x86_64-macos.tar.gz"
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-24/wasi-sdk-24.0-arm64-macos.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_120/binaryen-version_120-arm64-macos.tar.gz"
elif [[ "$OSTYPE" == "msys" || "$OSTYPE" == "cygwin" ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-22/wasi-sdk-22.0.m-mingw.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_117/binaryen-version_117-x86_64-windows.tar.gz"
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-24/wasi-sdk-24.0-x86_64-windows.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_120/binaryen-version_120-x86_64-windows.tar.gz"
fi
# Download tools
mkdir -p tools/
[ -d "tools/wasi-sdk"* ] || curl -#L "$WASI_SDK" | tar xzC tools &
[ -d "tools/binaryen-version"* ] || curl -#L "$BINARYEN" | tar xzC tools &
[ -d "tools/wasi-sdk" ] || curl -#L "$WASI_SDK" | tar xzC tools &
[ -d "tools/binaryen" ] || curl -#L "$BINARYEN" | tar xzC tools &
wait
[ -d "tools/wasi-sdk" ] || mv "tools/wasi-sdk"* "tools/wasi-sdk"
[ -d "tools/binaryen" ] || mv "tools/binaryen"* "tools/binaryen"
# Download and build SQLite
sqlite3/download.sh
embed/build.sh
embed/bcw2/build.sh
# Download and build sqlite-createtable-parser
util/vtabutil/parse/download.sh
util/vtabutil/parse/build.sh
util/sql3util/parse/download.sh
util/sql3util/parse/build.sh
# Check diffs
git diff --exit-code
git diff --exit-code

View File

@@ -16,16 +16,19 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- uses: ilammy/msvc-dev-cmd@v1
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Build
shell: bash
run: .github/workflows/repro.sh
- uses: actions/attest-build-provenance@v1
- uses: actions/attest-build-provenance@v2
if: matrix.os == 'ubuntu-latest'
with:
subject-path: |
embed/sqlite3.wasm
util/vtabutil/parse/sql3parse_table.wasm
embed/bcw2/bcw2.wasm
util/sql3util/parse/sql3parse_table.wasm

View File

@@ -3,8 +3,18 @@ name: Test
on:
push:
branches: [ "main" ]
paths:
- '**.go'
- '**.mod'
- '**.wasm'
- '**.wasm.bz2'
pull_request:
branches: [ "main" ]
paths:
- '**.go'
- '**.mod'
- '**.wasm'
- '**.wasm.bz2'
workflow_dispatch:
jobs:
@@ -48,67 +58,104 @@ jobs:
run: go test -v -tags sqlite3_flock ./...
if: matrix.os == 'macos-latest'
- name: Test no shared memory
run: go test -v -tags sqlite3_noshm ./...
if: matrix.os == 'ubuntu-latest'
- name: Test no locks
run: go test -v -tags sqlite3_nosys ./...
if: matrix.os == 'ubuntu-latest'
- name: Test dot locks
run: go test -v -tags sqlite3_dotlk ./...
if: matrix.os != 'windows-latest'
- name: Test GORM
shell: bash
run: gormlite/test.sh
- name: Collect coverage
run: go run github.com/dave/courtney@latest
if: |
github.event_name == 'push' &&
matrix.os == 'ubuntu-latest'
- uses: ncruces/go-coverage-report@v0
with:
coverage-file: coverage.out
chart: true
amend: true
if: |
github.event_name == 'push' &&
matrix.os == 'ubuntu-latest'
test-intel:
runs-on: macos-13
needs: test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Git LFS pull
uses: ./.github/actions/lfs
- name: Test
run: go test -v ./...
test-bsd:
strategy:
matrix:
os:
- name: freebsd
version: '14.2'
flags: '-test.v'
- name: netbsd
version: '10.0'
flags: '-test.v'
- name: openbsd
version: '7.6'
flags: '-test.v -test.short'
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Git LFS pull
uses: ./.github/actions/lfs
- name: Build
env:
GOOS: freebsd
TESTFLAGS: '-test.v'
GOOS: ${{ matrix.os.name }}
TESTFLAGS: ${{ matrix.os.flags }}
run: .github/workflows/build-test.sh
- name: Test
uses: cross-platform-actions/action@v0.24.0
uses: cross-platform-actions/action@v0.26.0
with:
operating_system: freebsd
version: '14.0'
operating_system: ${{ matrix.os.name }}
version: ${{ matrix.os.version }}
shell: bash
run: . ./test.sh
sync_files: runner-to-vm
test-vm:
strategy:
matrix:
os:
- name: dragonfly
action: 'vmactions/dragonflybsd-vm@v1'
tflags: '-test.v'
- name: illumos
action: 'vmactions/omnios-vm@v1'
tflags: '-test.v'
- name: solaris
action: 'vmactions/solaris-vm@v1'
bflags: '-tags sqlite3_dotlk'
tflags: '-test.v'
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- name: Git LFS pull
uses: ./.github/actions/lfs
- name: Build
env:
GOOS: ${{ matrix.os.name }}
BUILDFLAGS: ${{ matrix.os.bflags }}
TESTFLAGS: ${{ matrix.os.tflags }}
VMACTIONS: ${{ matrix.os.action }}
run: .github/workflows/build-test.sh
- name: Test
uses: ./.github/actions/vmactions
with:
usesh: true
copyback: false
run: . ./test.sh
test-qemu:
runs-on: ubuntu-latest
needs: test
@@ -131,11 +178,14 @@ jobs:
- name: Test riscv64 (interpreter)
run: GOARCH=riscv64 go test -v -short ./...
- name: Test s390x (big-endian, z/OS demo)
run: GOARCH=s390x go test -v -short -tags sqlite3_flock ./...
- name: Test ppc64le (interpreter)
run: GOARCH=ppc64le go test -v -short ./...
test-vm:
runs-on: ubuntu-latest
- name: Test s390x (big-endian)
run: GOARCH=s390x go test -v -short -tags sqlite3_dotlk ./...
test-macintel:
runs-on: macos-13
needs: test
steps:
@@ -146,29 +196,5 @@ jobs:
- name: Git LFS pull
uses: ./.github/actions/lfs
- name: Build illumos
env:
GOOS: illumos
TESTFLAGS: '-test.v -test.short'
run: .github/workflows/build-test.sh
- name: Test illumos
uses: vmactions/omnios-vm@v1
with:
usesh: true
copyback: false
run: . ./test.sh
- name: Build Solaris
env:
GOOS: solaris
TESTFLAGS: '-test.v -test.short'
run: .github/workflows/build-test.sh
- name: Test Solaris
uses: vmactions/solaris-vm@v1
with:
usesh: true
copyback: false
run: . ./test.sh
continue-on-error: true
- name: Test
run: go test -v ./...

View File

@@ -1,4 +1,4 @@
# Go bindings to SQLite using Wazero
# Go bindings to SQLite using wazero
[![Go Reference](https://pkg.go.dev/badge/image)](https://pkg.go.dev/github.com/ncruces/go-sqlite3)
[![Go Report](https://goreportcard.com/badge/github.com/ncruces/go-sqlite3)](https://goreportcard.com/report/github.com/ncruces/go-sqlite3)
@@ -10,7 +10,21 @@ as well as direct access to most of the [C SQLite API](https://sqlite.org/cintro
It wraps a [Wasm](https://webassembly.org/) [build](embed/) of SQLite,
and uses [wazero](https://wazero.io/) as the runtime.\
Go, wazero and [`x/sys`](https://pkg.go.dev/golang.org/x/sys) are the _only_ runtime dependencies [^1].
Go, wazero and [`x/sys`](https://pkg.go.dev/golang.org/x/sys) are the _only_ direct dependencies.
### Getting started
Using the [`database/sql`](https://pkg.go.dev/database/sql) driver:
```go
import "database/sql"
import _ "github.com/ncruces/go-sqlite3/driver"
import _ "github.com/ncruces/go-sqlite3/embed"
var version string
db, _ := sql.Open("sqlite3", "file:demo.db")
db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
```
### Packages
@@ -27,39 +41,6 @@ Go, wazero and [`x/sys`](https://pkg.go.dev/golang.org/x/sys) are the _only_ run
- [`github.com/ncruces/go-sqlite3/gormlite`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/gormlite)
provides a [GORM](https://gorm.io) driver.
### Extensions
- [`github.com/ncruces/go-sqlite3/ext/array`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/array)
provides the [`array`](https://sqlite.org/carray.html) table-valued function.
- [`github.com/ncruces/go-sqlite3/ext/blobio`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/blobio)
simplifies [incremental BLOB I/O](https://sqlite.org/c3ref/blob_open.html).
- [`github.com/ncruces/go-sqlite3/ext/bloom`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/bloom)
provides a [Bloom filter](https://github.com/nalgeon/sqlean/issues/27#issuecomment-1002267134) virtual table.
- [`github.com/ncruces/go-sqlite3/ext/csv`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/csv)
reads [comma-separated values](https://sqlite.org/csv.html).
- [`github.com/ncruces/go-sqlite3/ext/fileio`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/fileio)
reads, writes and lists files.
- [`github.com/ncruces/go-sqlite3/ext/hash`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/hash)
provides cryptographic hash functions.
- [`github.com/ncruces/go-sqlite3/ext/lines`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/lines)
reads data [line-by-line](https://github.com/asg017/sqlite-lines).
- [`github.com/ncruces/go-sqlite3/ext/pivot`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/pivot)
creates [pivot tables](https://github.com/jakethaw/pivot_vtab).
- [`github.com/ncruces/go-sqlite3/ext/statement`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/statement)
creates [parameterized views](https://github.com/0x09/sqlite-statement-vtab).
- [`github.com/ncruces/go-sqlite3/ext/stats`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/stats)
provides [statistics](https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html) functions.
- [`github.com/ncruces/go-sqlite3/ext/unicode`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/unicode)
provides [Unicode aware](https://sqlite.org/src/dir/ext/icu) functions.
- [`github.com/ncruces/go-sqlite3/ext/zorder`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/zorder)
maps multidimensional data to one dimension.
- [`github.com/ncruces/go-sqlite3/vfs/adiantum`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/adiantum)
wraps a VFS to offer encryption at rest.
- [`github.com/ncruces/go-sqlite3/vfs/memdb`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/memdb)
implements an in-memory VFS.
- [`github.com/ncruces/go-sqlite3/vfs/readervfs`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readervfs)
implements a VFS for immutable databases.
### Advanced features
- [incremental BLOB I/O](https://sqlite.org/c3ref/blob_open.html)
@@ -72,7 +53,11 @@ Go, wazero and [`x/sys`](https://pkg.go.dev/golang.org/x/sys) are the _only_ run
- [math functions](https://sqlite.org/lang_mathfunc.html)
- [full-text search](https://sqlite.org/fts5.html)
- [geospatial search](https://sqlite.org/geopoly.html)
- [Unicode support](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/unicode)
- [statistics functions](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/stats)
- [encryption at rest](vfs/adiantum/README.md)
- [many extensions](ext/README.md)
- [custom VFSes](vfs/README.md#custom-vfses)
- [and more…](embed/README.md)
### Caveats
@@ -89,9 +74,10 @@ This project aims for [high test coverage](https://github.com/ncruces/go-sqlite3
It also benefits greatly from [SQLite's](https://sqlite.org/testing.html) and
[wazero's](https://tetrate.io/blog/introducing-wazero-from-tetrate/#:~:text=Rock%2Dsolid%20test%20approach) thorough testing.
Every commit is [tested](.github/workflows/test.yml) on
Linux (amd64/arm64/386/riscv64/s390x), macOS (amd64/arm64),
Windows (amd64), FreeBSD (amd64), illumos (amd64), and Solaris (amd64).
Every commit is [tested](https://github.com/ncruces/go-sqlite3/wiki/Test-matrix) on
Linux (amd64/arm64/386/riscv64/ppc64le/s390x), macOS (amd64/arm64),
Windows (amd64), FreeBSD (amd64), OpenBSD (amd64), NetBSD (amd64),
DragonFly BSD (amd64), illumos (amd64), and Solaris (amd64).
The Go VFS is tested by running SQLite's
[mptest](https://github.com/sqlite/sqlite/blob/master/mptest/mptest.c).
@@ -104,12 +90,20 @@ Perfomance of the [`database/sql`](https://pkg.go.dev/database/sql) driver is
The Wasm and VFS layers are also tested by running SQLite's
[speedtest1](https://github.com/sqlite/sqlite/blob/master/test/speedtest1.c).
### FAQ, issues, new features
For questions, please see [Discussions](https://github.com/ncruces/go-sqlite3/discussions/categories/q-a).
Also, post there if you used this driver for something interesting
([_"Show and tell"_](https://github.com/ncruces/go-sqlite3/discussions/categories/show-and-tell)),
have an [idea](https://github.com/ncruces/go-sqlite3/discussions/categories/ideas)…
The [Issue](https://github.com/ncruces/go-sqlite3/issues) tracker is for bugs we want fixed,
and features we're working on, planning to work on, or asking for help with.
### Alternatives
- [`modernc.org/sqlite`](https://pkg.go.dev/modernc.org/sqlite)
- [`crawshaw.io/sqlite`](https://pkg.go.dev/crawshaw.io/sqlite)
- [`github.com/mattn/go-sqlite3`](https://pkg.go.dev/github.com/mattn/go-sqlite3)
- [`github.com/zombiezen/go-sqlite`](https://pkg.go.dev/github.com/zombiezen/go-sqlite)
[^1]: anything else you find in `go.mod` is either a test dependency,
or needed by one of the extensions.

53
blob.go
View File

@@ -21,6 +21,8 @@ type Blob struct {
bytes int64
offset int64
handle uint32
bufptr uint32
buflen int64
}
var _ io.ReadWriteSeeker = &Blob{}
@@ -29,7 +31,6 @@ var _ io.ReadWriteSeeker = &Blob{}
//
// https://sqlite.org/c3ref/blob_open.html
func (c *Conn) OpenBlob(db, table, column string, row int64, write bool) (*Blob, error) {
c.checkInterrupt()
defer c.arena.mark()()
blobPtr := c.arena.new(ptrlen)
dbPtr := c.arena.string(db)
@@ -41,6 +42,7 @@ func (c *Conn) OpenBlob(db, table, column string, row int64, write bool) (*Blob,
flags = 1
}
c.checkInterrupt(c.handle)
r := c.call("sqlite3_blob_open", uint64(c.handle),
uint64(dbPtr), uint64(tablePtr), uint64(columnPtr),
uint64(row), flags, uint64(blobPtr))
@@ -66,7 +68,7 @@ func (b *Blob) Close() error {
}
r := b.c.call("sqlite3_blob_close", uint64(b.handle))
b.c.free(b.bufptr)
b.handle = 0
return b.c.error(r)
}
@@ -86,17 +88,18 @@ func (b *Blob) Read(p []byte) (n int, err error) {
return 0, io.EOF
}
avail := b.bytes - b.offset
want := int64(len(p))
avail := b.bytes - b.offset
if want > avail {
want = avail
}
defer b.c.arena.mark()()
ptr := b.c.arena.new(uint64(want))
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, uint64(want))
b.buflen = want
}
r := b.c.call("sqlite3_blob_read", uint64(b.handle),
uint64(ptr), uint64(want), uint64(b.offset))
uint64(b.bufptr), uint64(want), uint64(b.offset))
err = b.c.error(r)
if err != nil {
return 0, err
@@ -106,7 +109,7 @@ func (b *Blob) Read(p []byte) (n int, err error) {
err = io.EOF
}
copy(p, util.View(b.c.mod, ptr, uint64(want)))
copy(p, util.View(b.c.mod, b.bufptr, uint64(want)))
return int(want), err
}
@@ -123,19 +126,20 @@ func (b *Blob) WriteTo(w io.Writer) (n int64, err error) {
if want > avail {
want = avail
}
defer b.c.arena.mark()()
ptr := b.c.arena.new(uint64(want))
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, uint64(want))
b.buflen = want
}
for want > 0 {
r := b.c.call("sqlite3_blob_read", uint64(b.handle),
uint64(ptr), uint64(want), uint64(b.offset))
uint64(b.bufptr), uint64(want), uint64(b.offset))
err = b.c.error(r)
if err != nil {
return n, err
}
mem := util.View(b.c.mod, ptr, uint64(want))
mem := util.View(b.c.mod, b.bufptr, uint64(want))
m, err := w.Write(mem[:want])
b.offset += int64(m)
n += int64(m)
@@ -143,6 +147,7 @@ func (b *Blob) WriteTo(w io.Writer) (n int64, err error) {
return n, err
}
if int64(m) != want {
// notest // Write misbehaving
return n, io.ErrShortWrite
}
@@ -158,11 +163,15 @@ func (b *Blob) WriteTo(w io.Writer) (n int64, err error) {
//
// https://sqlite.org/c3ref/blob_write.html
func (b *Blob) Write(p []byte) (n int, err error) {
defer b.c.arena.mark()()
ptr := b.c.arena.bytes(p)
want := int64(len(p))
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, uint64(want))
b.buflen = want
}
util.WriteBytes(b.c.mod, b.bufptr, p)
r := b.c.call("sqlite3_blob_write", uint64(b.handle),
uint64(ptr), uint64(len(p)), uint64(b.offset))
uint64(b.bufptr), uint64(want), uint64(b.offset))
err = b.c.error(r)
if err != nil {
return 0, err
@@ -186,16 +195,17 @@ func (b *Blob) ReadFrom(r io.Reader) (n int64, err error) {
if want < 1 {
want = 1
}
defer b.c.arena.mark()()
ptr := b.c.arena.new(uint64(want))
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, uint64(want))
b.buflen = want
}
for {
mem := util.View(b.c.mod, ptr, uint64(want))
mem := util.View(b.c.mod, b.bufptr, uint64(want))
m, err := r.Read(mem[:want])
if m > 0 {
r := b.c.call("sqlite3_blob_write", uint64(b.handle),
uint64(ptr), uint64(m), uint64(b.offset))
uint64(b.bufptr), uint64(m), uint64(b.offset))
err := b.c.error(r)
if err != nil {
return n, err
@@ -243,6 +253,7 @@ func (b *Blob) Seek(offset int64, whence int) (int64, error) {
//
// https://sqlite.org/c3ref/blob_reopen.html
func (b *Blob) Reopen(row int64) error {
b.c.checkInterrupt(b.c.handle)
err := b.c.error(b.c.call("sqlite3_blob_reopen", uint64(b.handle), uint64(row)))
b.bytes = int64(b.c.call("sqlite3_blob_bytes", uint64(b.handle)))
b.offset = 0

249
config.go
View File

@@ -2,9 +2,13 @@ package sqlite3
import (
"context"
"fmt"
"strconv"
"github.com/tetratelabs/wazero/api"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/tetratelabs/wazero/api"
"github.com/ncruces/go-sqlite3/vfs"
)
// Config makes configuration changes to a database connection.
@@ -14,8 +18,19 @@ import (
//
// https://sqlite.org/c3ref/db_config.html
func (c *Conn) Config(op DBConfig, arg ...bool) (bool, error) {
if op < DBCONFIG_ENABLE_FKEY || op > DBCONFIG_REVERSE_SCANORDER {
return false, MISUSE
}
// We need to call sqlite3_db_config, a variadic function.
// We only support the `int int*` variants.
// The int is a three-valued bool: -1 queries, 0/1 sets false/true.
// The int* points to where new state will be written to.
// The vararg is a pointer to an array containing these arguments:
// an int and an int* pointing to that int.
defer c.arena.mark()()
argsPtr := c.arena.new(2 * ptrlen)
argsPtr := c.arena.new(intlen + ptrlen)
var flag int
switch {
@@ -56,6 +71,112 @@ func logCallback(ctx context.Context, mod api.Module, _, iCode, zMsg uint32) {
}
}
// Log writes a message into the error log established by [Conn.ConfigLog].
//
// https://sqlite.org/c3ref/log.html
func (c *Conn) Log(code ExtendedErrorCode, format string, a ...any) {
if c.log != nil {
c.log(code, fmt.Sprintf(format, a...))
}
}
// FileControl allows low-level control of database files.
// Only a subset of opcodes are supported.
//
// https://sqlite.org/c3ref/file_control.html
func (c *Conn) FileControl(schema string, op FcntlOpcode, arg ...any) (any, error) {
defer c.arena.mark()()
ptr := c.arena.new(max(ptrlen, intlen))
var schemaPtr uint32
if schema != "" {
schemaPtr = c.arena.string(schema)
}
var rc uint64
var res any
switch op {
default:
return nil, MISUSE
case FCNTL_RESET_CACHE:
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), 0)
case FCNTL_PERSIST_WAL, FCNTL_POWERSAFE_OVERWRITE:
var flag int
switch {
case len(arg) == 0:
flag = -1
case arg[0]:
flag = 1
}
util.WriteUint32(c.mod, ptr, uint32(flag))
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), uint64(ptr))
res = util.ReadUint32(c.mod, ptr) != 0
case FCNTL_CHUNK_SIZE:
util.WriteUint32(c.mod, ptr, uint32(arg[0].(int)))
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), uint64(ptr))
case FCNTL_RESERVE_BYTES:
bytes := -1
if len(arg) > 0 {
bytes = arg[0].(int)
}
util.WriteUint32(c.mod, ptr, uint32(bytes))
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), uint64(ptr))
res = int(util.ReadUint32(c.mod, ptr))
case FCNTL_DATA_VERSION:
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), uint64(ptr))
res = util.ReadUint32(c.mod, ptr)
case FCNTL_LOCKSTATE:
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), uint64(ptr))
res = vfs.LockLevel(util.ReadUint32(c.mod, ptr))
case FCNTL_VFS_POINTER:
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), uint64(ptr))
if rc == _OK {
const zNameOffset = 16
ptr = util.ReadUint32(c.mod, ptr)
ptr = util.ReadUint32(c.mod, ptr+zNameOffset)
name := util.ReadString(c.mod, ptr, _MAX_NAME)
res = vfs.Find(name)
}
case FCNTL_FILE_POINTER, FCNTL_JOURNAL_POINTER:
rc = c.call("sqlite3_file_control",
uint64(c.handle), uint64(schemaPtr),
uint64(op), uint64(ptr))
if rc == _OK {
const fileHandleOffset = 4
ptr = util.ReadUint32(c.mod, ptr)
ptr = util.ReadUint32(c.mod, ptr+fileHandleOffset)
res = util.GetHandle(c.ctx, ptr)
}
}
if err := c.error(rc); err != nil {
return nil, err
}
return res, nil
}
// Limit allows the size of various constructs to be
// limited on a connection by connection basis.
//
@@ -68,7 +189,7 @@ func (c *Conn) Limit(id LimitCategory, value int) int {
// SetAuthorizer registers an authorizer callback with the database connection.
//
// https://sqlite.org/c3ref/set_authorizer.html
func (c *Conn) SetAuthorizer(cb func(action AuthorizerActionCode, name3rd, name4th, schema, nameInner string) AuthorizerReturnCode) error {
func (c *Conn) SetAuthorizer(cb func(action AuthorizerActionCode, name3rd, name4th, schema, inner string) AuthorizerReturnCode) error {
var enable uint64
if cb != nil {
enable = 1
@@ -82,9 +203,9 @@ func (c *Conn) SetAuthorizer(cb func(action AuthorizerActionCode, name3rd, name4
}
func authorizerCallback(ctx context.Context, mod api.Module, pDB uint32, action AuthorizerActionCode, zName3rd, zName4th, zSchema, zNameInner uint32) (rc AuthorizerReturnCode) {
func authorizerCallback(ctx context.Context, mod api.Module, pDB uint32, action AuthorizerActionCode, zName3rd, zName4th, zSchema, zInner uint32) (rc AuthorizerReturnCode) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.authorizer != nil {
var name3rd, name4th, schema, nameInner string
var name3rd, name4th, schema, inner string
if zName3rd != 0 {
name3rd = util.ReadString(mod, zName3rd, _MAX_NAME)
}
@@ -94,18 +215,56 @@ func authorizerCallback(ctx context.Context, mod api.Module, pDB uint32, action
if zSchema != 0 {
schema = util.ReadString(mod, zSchema, _MAX_NAME)
}
if zNameInner != 0 {
nameInner = util.ReadString(mod, zNameInner, _MAX_NAME)
if zInner != 0 {
inner = util.ReadString(mod, zInner, _MAX_NAME)
}
rc = c.authorizer(action, name3rd, name4th, schema, nameInner)
rc = c.authorizer(action, name3rd, name4th, schema, inner)
}
return rc
}
// WalCheckpoint checkpoints a WAL database.
// Trace registers a trace callback function against the database connection.
//
// https://sqlite.org/c3ref/trace_v2.html
func (c *Conn) Trace(mask TraceEvent, cb func(evt TraceEvent, arg1 any, arg2 any) error) error {
r := c.call("sqlite3_trace_go", uint64(c.handle), uint64(mask))
if err := c.error(r); err != nil {
return err
}
c.trace = cb
return nil
}
func traceCallback(ctx context.Context, mod api.Module, evt TraceEvent, pDB, pArg1, pArg2 uint32) (rc uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.trace != nil {
var arg1, arg2 any
if evt == TRACE_CLOSE {
arg1 = c
} else {
for _, s := range c.stmts {
if pArg1 == s.handle {
arg1 = s
switch evt {
case TRACE_STMT:
arg2 = s.SQL()
case TRACE_PROFILE:
arg2 = int64(util.ReadUint64(mod, pArg2))
}
break
}
}
}
if arg1 != nil {
_, rc = errorCode(c.trace(evt, arg1, arg2), ERROR)
}
}
return rc
}
// WALCheckpoint checkpoints a WAL database.
//
// https://sqlite.org/c3ref/wal_checkpoint_v2.html
func (c *Conn) WalCheckpoint(schema string, mode CheckpointMode) (nLog, nCkpt int, err error) {
func (c *Conn) WALCheckpoint(schema string, mode CheckpointMode) (nLog, nCkpt int, err error) {
defer c.arena.mark()()
nLogPtr := c.arena.new(ptrlen)
nCkptPtr := c.arena.new(ptrlen)
@@ -118,19 +277,19 @@ func (c *Conn) WalCheckpoint(schema string, mode CheckpointMode) (nLog, nCkpt in
return nLog, nCkpt, c.error(r)
}
// WalAutoCheckpoint configures WAL auto-checkpoints.
// WALAutoCheckpoint configures WAL auto-checkpoints.
//
// https://sqlite.org/c3ref/wal_autocheckpoint.html
func (c *Conn) WalAutoCheckpoint(pages int) error {
func (c *Conn) WALAutoCheckpoint(pages int) error {
r := c.call("sqlite3_wal_autocheckpoint", uint64(c.handle), uint64(pages))
return c.error(r)
}
// WalHook registers a callback function to be invoked
// WALHook registers a callback function to be invoked
// each time data is committed to a database in WAL mode.
//
// https://sqlite.org/c3ref/wal_hook.html
func (c *Conn) WalHook(cb func(db *Conn, schema string, pages int) error) {
func (c *Conn) WALHook(cb func(db *Conn, schema string, pages int) error) {
var enable uint64
if cb != nil {
enable = 1
@@ -152,7 +311,10 @@ func walCallback(ctx context.Context, mod api.Module, _, pDB, zSchema uint32, pa
//
// https://sqlite.org/c3ref/autovacuum_pages.html
func (c *Conn) AutoVacuumPages(cb func(schema string, dbPages, freePages, bytesPerPage uint) uint) error {
funcPtr := util.AddHandle(c.ctx, cb)
var funcPtr uint32
if cb != nil {
funcPtr = util.AddHandle(c.ctx, cb)
}
r := c.call("sqlite3_autovacuum_pages_go", uint64(c.handle), uint64(funcPtr))
return c.error(r)
}
@@ -162,3 +324,60 @@ func autoVacuumCallback(ctx context.Context, mod api.Module, pApp, zSchema, nDbP
schema := util.ReadString(mod, zSchema, _MAX_NAME)
return uint32(fn(schema, uint(nDbPage), uint(nFreePage), uint(nBytePerPage)))
}
// SoftHeapLimit imposes a soft limit on heap size.
//
// https://sqlite.org/c3ref/hard_heap_limit64.html
func (c *Conn) SoftHeapLimit(n int64) int64 {
return int64(c.call("sqlite3_soft_heap_limit64", uint64(n)))
}
// HardHeapLimit imposes a hard limit on heap size.
//
// https://sqlite.org/c3ref/hard_heap_limit64.html
func (c *Conn) HardHeapLimit(n int64) int64 {
return int64(c.call("sqlite3_hard_heap_limit64", uint64(n)))
}
// EnableChecksums enables checksums on a database.
//
// https://sqlite.org/cksumvfs.html
func (c *Conn) EnableChecksums(schema string) error {
r, err := c.FileControl(schema, FCNTL_RESERVE_BYTES)
if err != nil {
return err
}
if r == 8 {
// Correct value, enabled.
return nil
}
if r == 0 {
// Default value, enable.
_, err = c.FileControl(schema, FCNTL_RESERVE_BYTES, 8)
if err != nil {
return err
}
r, err = c.FileControl(schema, FCNTL_RESERVE_BYTES)
if err != nil {
return err
}
}
if r != 8 {
// Invalid value.
return util.ErrorString("sqlite3: reserve bytes must be 8, is: " + strconv.Itoa(r.(int)))
}
// VACUUM the database.
if schema != "" {
err = c.Exec(`VACUUM ` + QuoteIdentifier(schema))
} else {
err = c.Exec(`VACUUM`)
}
if err != nil {
return err
}
// Checkpoint the WAL.
_, _, err = c.WALCheckpoint(schema, CHECKPOINT_RESTART)
return err
}

203
conn.go
View File

@@ -8,9 +8,10 @@ import (
"strings"
"time"
"github.com/tetratelabs/wazero/api"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/vfs"
"github.com/tetratelabs/wazero/api"
)
// Conn is a database connection handle.
@@ -22,27 +23,36 @@ type Conn struct {
interrupt context.Context
pending *Stmt
busy func(int) bool
stmts []*Stmt
timer *time.Timer
busy func(context.Context, int) bool
log func(xErrorCode, string)
collation func(*Conn, string)
wal func(*Conn, string, int) error
trace func(TraceEvent, any, any) error
authorizer func(AuthorizerActionCode, string, string, string, string) AuthorizerReturnCode
update func(AuthorizerActionCode, string, string, int64)
commit func() bool
rollback func()
wal func(*Conn, string, int) error
arena arena
handle uint32
}
// Open calls [OpenFlags] with [OPEN_READWRITE], [OPEN_CREATE], [OPEN_URI] and [OPEN_NOFOLLOW].
// Open calls [OpenFlags] with [OPEN_READWRITE], [OPEN_CREATE] and [OPEN_URI].
func Open(filename string) (*Conn, error) {
return newConn(filename, OPEN_READWRITE|OPEN_CREATE|OPEN_URI|OPEN_NOFOLLOW)
return newConn(context.Background(), filename, OPEN_READWRITE|OPEN_CREATE|OPEN_URI)
}
// OpenContext is like [Open] but includes a context,
// which is used to interrupt the process of opening the connectiton.
func OpenContext(ctx context.Context, filename string) (*Conn, error) {
return newConn(ctx, filename, OPEN_READWRITE|OPEN_CREATE|OPEN_URI)
}
// OpenFlags opens an SQLite database file as specified by the filename argument.
//
// If none of the required flags is used, a combination of [OPEN_READWRITE] and [OPEN_CREATE] is used.
// If none of the required flags are used, a combination of [OPEN_READWRITE] and [OPEN_CREATE] is used.
// If a URI filename is used, PRAGMA statements to execute can be specified using "_pragma":
//
// sqlite3.Open("file:demo.db?_pragma=busy_timeout(10000)")
@@ -52,26 +62,37 @@ func OpenFlags(filename string, flags OpenFlag) (*Conn, error) {
if flags&(OPEN_READONLY|OPEN_READWRITE|OPEN_CREATE) == 0 {
flags |= OPEN_READWRITE | OPEN_CREATE
}
return newConn(filename, flags)
return newConn(context.Background(), filename, flags)
}
type connKey struct{}
func newConn(filename string, flags OpenFlag) (conn *Conn, err error) {
sqlite, err := instantiateSQLite()
func newConn(ctx context.Context, filename string, flags OpenFlag) (res *Conn, _ error) {
err := ctx.Err()
if err != nil {
return nil, err
}
c := &Conn{interrupt: ctx}
c.sqlite, err = instantiateSQLite()
if err != nil {
return nil, err
}
defer func() {
if conn == nil {
sqlite.close()
if res == nil {
c.Close()
c.sqlite.close()
} else {
c.interrupt = context.Background()
}
}()
c := &Conn{sqlite: sqlite}
c.arena = c.newArena(1024)
c.ctx = context.WithValue(c.ctx, connKey{}, c)
c.arena = c.newArena(1024)
c.handle, err = c.openDB(filename, flags)
if err == nil {
err = initExtensions(c)
}
if err != nil {
return nil, err
}
@@ -92,6 +113,7 @@ func (c *Conn) openDB(filename string, flags OpenFlag) (uint32, error) {
return 0, err
}
c.call("sqlite3_progress_handler_go", uint64(handle), 100)
if flags|OPEN_URI != 0 && strings.HasPrefix(filename, "file:") {
var pragmas strings.Builder
if _, after, ok := strings.Cut(filename, "?"); ok {
@@ -103,6 +125,7 @@ func (c *Conn) openDB(filename string, flags OpenFlag) (uint32, error) {
}
}
if pragmas.Len() != 0 {
c.checkInterrupt(handle)
pragmaPtr := c.arena.string(pragmas.String())
r := c.call("sqlite3_exec", uint64(handle), uint64(pragmaPtr), 0, 0, 0)
if err := c.sqlite.error(r, handle, pragmas.String()); err != nil {
@@ -112,7 +135,6 @@ func (c *Conn) openDB(filename string, flags OpenFlag) (uint32, error) {
}
}
}
c.call("sqlite3_progress_handler_go", uint64(handle), 100)
return handle, nil
}
@@ -154,10 +176,10 @@ func (c *Conn) Close() error {
//
// https://sqlite.org/c3ref/exec.html
func (c *Conn) Exec(sql string) error {
c.checkInterrupt()
defer c.arena.mark()()
sqlPtr := c.arena.string(sql)
c.checkInterrupt(c.handle)
r := c.call("sqlite3_exec", uint64(c.handle), uint64(sqlPtr), 0, 0, 0)
return c.error(r, sql)
}
@@ -183,6 +205,7 @@ func (c *Conn) PrepareFlags(sql string, flags PrepareFlag) (stmt *Stmt, tail str
tailPtr := c.arena.new(ptrlen)
sqlPtr := c.arena.string(sql)
c.checkInterrupt(c.handle)
r := c.call("sqlite3_prepare_v3", uint64(c.handle),
uint64(sqlPtr), uint64(len(sql)+1), uint64(flags),
uint64(stmtPtr), uint64(tailPtr))
@@ -199,6 +222,7 @@ func (c *Conn) PrepareFlags(sql string, flags PrepareFlag) (stmt *Stmt, tail str
if stmt.handle == 0 {
return nil, "", nil
}
c.stmts = append(c.stmts, stmt)
return stmt, tail, nil
}
@@ -224,9 +248,8 @@ func (c *Conn) Filename(schema string) *vfs.Filename {
defer c.arena.mark()()
ptr = c.arena.string(schema)
}
r := c.call("sqlite3_db_filename", uint64(c.handle), uint64(ptr))
return vfs.OpenFilename(c.ctx, c.mod, uint32(r), vfs.OPEN_MAIN_DB)
return vfs.GetFilename(c.ctx, c.mod, uint32(r), vfs.OPEN_MAIN_DB)
}
// ReadOnly determines if a database is read-only.
@@ -295,8 +318,7 @@ func (c *Conn) ReleaseMemory() error {
return c.error(r)
}
// GetInterrupt gets the context set with [Conn.SetInterrupt],
// or nil if none was set.
// GetInterrupt gets the context set with [Conn.SetInterrupt].
func (c *Conn) GetInterrupt() context.Context {
return c.interrupt
}
@@ -316,38 +338,42 @@ func (c *Conn) GetInterrupt() context.Context {
//
// https://sqlite.org/c3ref/interrupt.html
func (c *Conn) SetInterrupt(ctx context.Context) (old context.Context) {
// Is it the same context?
if ctx == c.interrupt {
return ctx
old = c.interrupt
c.interrupt = ctx
if ctx == old || ctx.Done() == old.Done() {
return old
}
// A busy SQL statement prevents SQLite from ignoring an interrupt
// that comes before any other statements are started.
if c.pending == nil {
c.pending, _, _ = c.Prepare(`WITH RECURSIVE c(x) AS (VALUES(0) UNION ALL SELECT x FROM c) SELECT x FROM c`)
defer c.arena.mark()()
stmtPtr := c.arena.new(ptrlen)
loopPtr := c.arena.string(`WITH RECURSIVE c(x) AS (VALUES(0) UNION ALL SELECT x FROM c) SELECT x FROM c`)
c.call("sqlite3_prepare_v3", uint64(c.handle), uint64(loopPtr), math.MaxUint64,
uint64(PREPARE_PERSISTENT), uint64(stmtPtr), 0)
c.pending = &Stmt{c: c}
c.pending.handle = util.ReadUint32(c.mod, stmtPtr)
}
old = c.interrupt
c.interrupt = ctx
if old != nil && old.Done() != nil && (ctx == nil || ctx.Err() == nil) {
if old.Done() != nil && ctx.Err() == nil {
c.pending.Reset()
}
if ctx != nil && ctx.Done() != nil {
if ctx.Done() != nil {
c.pending.Step()
}
return old
}
func (c *Conn) checkInterrupt() {
if c.interrupt != nil && c.interrupt.Err() != nil {
c.call("sqlite3_interrupt", uint64(c.handle))
func (c *Conn) checkInterrupt(handle uint32) {
if c.interrupt.Err() != nil {
c.call("sqlite3_interrupt", uint64(handle))
}
}
func progressCallback(ctx context.Context, mod api.Module, pDB uint32) (interrupt uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB &&
c.interrupt != nil && c.interrupt.Err() != nil {
func progressCallback(ctx context.Context, mod api.Module, _ uint32) (interrupt uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.interrupt.Err() != nil {
interrupt = 1
}
return interrupt
@@ -362,9 +388,8 @@ func (c *Conn) BusyTimeout(timeout time.Duration) error {
return c.error(r)
}
func timeoutCallback(ctx context.Context, mod api.Module, pDB uint32, count, tmout int32) (retry uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok &&
(c.interrupt == nil || c.interrupt.Err() == nil) {
func timeoutCallback(ctx context.Context, mod api.Module, count, tmout int32) (retry uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.interrupt.Err() == nil {
const delays = "\x01\x02\x05\x0a\x0f\x14\x19\x19\x19\x32\x32\x64"
const totals = "\x00\x01\x03\x08\x12\x21\x35\x4e\x67\x80\xb2\xe4"
const ndelay = int32(len(delays) - 1)
@@ -379,17 +404,31 @@ func timeoutCallback(ctx context.Context, mod api.Module, pDB uint32, count, tmo
}
if delay = min(delay, tmout-prior); delay > 0 {
time.Sleep(time.Duration(delay) * time.Millisecond)
retry = 1
delay := time.Duration(delay) * time.Millisecond
if c.interrupt.Done() == nil {
time.Sleep(delay)
return 1
}
if c.timer == nil {
c.timer = time.NewTimer(delay)
} else {
c.timer.Reset(delay)
}
select {
case <-c.interrupt.Done():
c.timer.Stop()
case <-c.timer.C:
return 1
}
}
}
return retry
return 0
}
// BusyHandler registers a callback to handle [BUSY] errors.
//
// https://sqlite.org/c3ref/busy_handler.html
func (c *Conn) BusyHandler(cb func(count int) (retry bool)) error {
func (c *Conn) BusyHandler(cb func(ctx context.Context, count int) (retry bool)) error {
var enable uint64
if cb != nil {
enable = 1
@@ -403,24 +442,86 @@ func (c *Conn) BusyHandler(cb func(count int) (retry bool)) error {
}
func busyCallback(ctx context.Context, mod api.Module, pDB uint32, count int32) (retry uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.busy != nil &&
(c.interrupt == nil || c.interrupt.Err() == nil) {
if c.busy(int(count)) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.busy != nil {
interrupt := c.interrupt
if interrupt == nil {
interrupt = context.Background()
}
if interrupt.Err() == nil && c.busy(interrupt, int(count)) {
retry = 1
}
}
return retry
}
// Status retrieves runtime status information about a database connection.
//
// https://sqlite.org/c3ref/db_status.html
func (c *Conn) Status(op DBStatus, reset bool) (current, highwater int, err error) {
defer c.arena.mark()()
hiPtr := c.arena.new(intlen)
curPtr := c.arena.new(intlen)
var i uint64
if reset {
i = 1
}
r := c.call("sqlite3_db_status", uint64(c.handle),
uint64(op), uint64(curPtr), uint64(hiPtr), i)
if err = c.error(r); err == nil {
current = int(util.ReadUint32(c.mod, curPtr))
highwater = int(util.ReadUint32(c.mod, hiPtr))
}
return
}
// TableColumnMetadata extracts metadata about a column of a table.
//
// https://sqlite.org/c3ref/table_column_metadata.html
func (c *Conn) TableColumnMetadata(schema, table, column string) (declType, collSeq string, notNull, primaryKey, autoInc bool, err error) {
defer c.arena.mark()()
var schemaPtr, columnPtr uint32
declTypePtr := c.arena.new(ptrlen)
collSeqPtr := c.arena.new(ptrlen)
notNullPtr := c.arena.new(ptrlen)
autoIncPtr := c.arena.new(ptrlen)
primaryKeyPtr := c.arena.new(ptrlen)
if schema != "" {
schemaPtr = c.arena.string(schema)
}
tablePtr := c.arena.string(table)
if column != "" {
columnPtr = c.arena.string(column)
}
r := c.call("sqlite3_table_column_metadata", uint64(c.handle),
uint64(schemaPtr), uint64(tablePtr), uint64(columnPtr),
uint64(declTypePtr), uint64(collSeqPtr),
uint64(notNullPtr), uint64(primaryKeyPtr), uint64(autoIncPtr))
if err = c.error(r); err == nil && column != "" {
if ptr := util.ReadUint32(c.mod, declTypePtr); ptr != 0 {
declType = util.ReadString(c.mod, ptr, _MAX_NAME)
}
if ptr := util.ReadUint32(c.mod, collSeqPtr); ptr != 0 {
collSeq = util.ReadString(c.mod, ptr, _MAX_NAME)
}
notNull = util.ReadUint32(c.mod, notNullPtr) != 0
autoInc = util.ReadUint32(c.mod, autoIncPtr) != 0
primaryKey = util.ReadUint32(c.mod, primaryKeyPtr) != 0
}
return
}
func (c *Conn) error(rc uint64, sql ...string) error {
return c.sqlite.error(rc, c.handle, sql...)
}
// DriverConn is implemented by the SQLite [database/sql] driver connection.
//
// It can be used to access SQLite features like [online backup].
//
// [online backup]: https://sqlite.org/backup.html
type DriverConn interface {
Raw() *Conn
func (c *Conn) stmtsIter(yield func(*Stmt) bool) {
for _, s := range c.stmts {
if !yield(s) {
break
}
}
}

11
conn_iter.go Normal file
View File

@@ -0,0 +1,11 @@
//go:build go1.23
package sqlite3
import "iter"
// Stmts returns an iterator for the prepared statements
// associated with the database connection.
//
// https://sqlite.org/c3ref/next_stmt.html
func (c *Conn) Stmts() iter.Seq[*Stmt] { return c.stmtsIter }

9
conn_old.go Normal file
View File

@@ -0,0 +1,9 @@
//go:build !go1.23
package sqlite3
// Stmts returns an iterator for the prepared statements
// associated with the database connection.
//
// https://sqlite.org/c3ref/next_stmt.html
func (c *Conn) Stmts() func(func(*Stmt) bool) { return c.stmtsIter }

View File

@@ -7,15 +7,13 @@ const (
_ROW = 100 /* sqlite3_step() has another row ready */
_DONE = 101 /* sqlite3_step() has finished executing */
_UTF8 = 1
_MAX_NAME = 1e6 // Self-imposed limit for most NUL terminated strings.
_MAX_LENGTH = 1e9
_MAX_SQL_LENGTH = 1e9
_MAX_ALLOCATION_SIZE = 0x7ffffeff
_MAX_FUNCTION_ARG = 100
_MAX_NAME = 1e6 // Self-imposed limit for most NUL terminated strings.
_MAX_LENGTH = 1e9
_MAX_SQL_LENGTH = 1e9
_MAX_FUNCTION_ARG = 100
ptrlen = 4
intlen = 4
)
// ErrorCode is a result code that [Error.Code] might return.
@@ -109,7 +107,7 @@ const (
CANTOPEN_ISDIR ExtendedErrorCode = xErrorCode(CANTOPEN) | (2 << 8)
CANTOPEN_FULLPATH ExtendedErrorCode = xErrorCode(CANTOPEN) | (3 << 8)
CANTOPEN_CONVPATH ExtendedErrorCode = xErrorCode(CANTOPEN) | (4 << 8)
CANTOPEN_DIRTYWAL ExtendedErrorCode = xErrorCode(CANTOPEN) | (5 << 8) /* Not Used */
// CANTOPEN_DIRTYWAL ExtendedErrorCode = xErrorCode(CANTOPEN) | (5 << 8) /* Not Used */
CANTOPEN_SYMLINK ExtendedErrorCode = xErrorCode(CANTOPEN) | (6 << 8)
CORRUPT_VTAB ExtendedErrorCode = xErrorCode(CORRUPT) | (1 << 8)
CORRUPT_SEQUENCE ExtendedErrorCode = xErrorCode(CORRUPT) | (2 << 8)
@@ -177,11 +175,12 @@ const (
type FunctionFlag uint32
const (
DETERMINISTIC FunctionFlag = 0x000000800
DIRECTONLY FunctionFlag = 0x000080000
SUBTYPE FunctionFlag = 0x000100000
INNOCUOUS FunctionFlag = 0x000200000
RESULT_SUBTYPE FunctionFlag = 0x001000000
DETERMINISTIC FunctionFlag = 0x000000800
DIRECTONLY FunctionFlag = 0x000080000
INNOCUOUS FunctionFlag = 0x000200000
SELFORDER1 FunctionFlag = 0x002000000
// SUBTYPE FunctionFlag = 0x000100000
// RESULT_SUBTYPE FunctionFlag = 0x001000000
)
// StmtStatus name counter values associated with the [Stmt.Status] method.
@@ -201,6 +200,27 @@ const (
STMTSTATUS_MEMUSED StmtStatus = 99
)
// DBStatus are the available "verbs" that can be passed to the [Conn.Status] method.
//
// https://sqlite.org/c3ref/c_dbstatus_options.html
type DBStatus uint32
const (
DBSTATUS_LOOKASIDE_USED DBStatus = 0
DBSTATUS_CACHE_USED DBStatus = 1
DBSTATUS_SCHEMA_USED DBStatus = 2
DBSTATUS_STMT_USED DBStatus = 3
DBSTATUS_LOOKASIDE_HIT DBStatus = 4
DBSTATUS_LOOKASIDE_MISS_SIZE DBStatus = 5
DBSTATUS_LOOKASIDE_MISS_FULL DBStatus = 6
DBSTATUS_CACHE_HIT DBStatus = 7
DBSTATUS_CACHE_MISS DBStatus = 8
DBSTATUS_CACHE_WRITE DBStatus = 9
DBSTATUS_DEFERRED_FKS DBStatus = 10
DBSTATUS_CACHE_USED_SHARED DBStatus = 11
DBSTATUS_CACHE_SPILL DBStatus = 12
)
// DBConfig are the available database connection configuration options.
//
// https://sqlite.org/c3ref/c_dbconfig_defensive.html
@@ -227,6 +247,25 @@ const (
DBCONFIG_TRUSTED_SCHEMA DBConfig = 1017
DBCONFIG_STMT_SCANSTATUS DBConfig = 1018
DBCONFIG_REVERSE_SCANORDER DBConfig = 1019
// DBCONFIG_MAX DBConfig = 1019
)
// FcntlOpcode are the available opcodes for [Conn.FileControl].
//
// https://sqlite.org/c3ref/c_fcntl_begin_atomic_write.html
type FcntlOpcode uint32
const (
FCNTL_LOCKSTATE FcntlOpcode = 1
FCNTL_CHUNK_SIZE FcntlOpcode = 6
FCNTL_FILE_POINTER FcntlOpcode = 7
FCNTL_PERSIST_WAL FcntlOpcode = 10
FCNTL_POWERSAFE_OVERWRITE FcntlOpcode = 13
FCNTL_VFS_POINTER FcntlOpcode = 27
FCNTL_JOURNAL_POINTER FcntlOpcode = 28
FCNTL_DATA_VERSION FcntlOpcode = 35
FCNTL_RESERVE_BYTES FcntlOpcode = 38
FCNTL_RESET_CACHE FcntlOpcode = 42
)
// LimitCategory are the available run-time limit categories.
@@ -289,8 +328,8 @@ const (
AUTH_DROP_VTABLE AuthorizerActionCode = 30 /* Table Name Module Name */
AUTH_FUNCTION AuthorizerActionCode = 31 /* NULL Function Name */
AUTH_SAVEPOINT AuthorizerActionCode = 32 /* Operation Savepoint Name */
AUTH_COPY AuthorizerActionCode = 0 /* No longer used */
AUTH_RECURSIVE AuthorizerActionCode = 33 /* NULL NULL */
// AUTH_COPY AuthorizerActionCode = 0 /* No longer used */
)
// AuthorizerReturnCode are the integer codes
@@ -328,6 +367,18 @@ const (
TXN_WRITE TxnState = 2
)
// TraceEvent identify classes of events that can be monitored with [Conn.Trace].
//
// https://sqlite.org/c3ref/c_trace.html
type TraceEvent uint32
const (
TRACE_STMT TraceEvent = 0x01
TRACE_PROFILE TraceEvent = 0x02
TRACE_ROW TraceEvent = 0x04
TRACE_CLOSE TraceEvent = 0x08
)
// Datatype is a fundamental datatype of SQLite.
//
// https://sqlite.org/c3ref/c_blob.html

View File

@@ -84,19 +84,18 @@ func (ctx Context) ResultFloat(value float64) {
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultText(value string) {
ptr := ctx.c.newString(value)
ctx.c.call("sqlite3_result_text64",
uint64(ctx.handle), uint64(ptr), uint64(len(value)),
uint64(ctx.c.freer), _UTF8)
ctx.c.call("sqlite3_result_text_go",
uint64(ctx.handle), uint64(ptr), uint64(len(value)))
}
// ResultRawText sets the text result of the function to a []byte.
// Returning a nil slice is the same as calling [Context.ResultNull].
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultRawText(value []byte) {
ptr := ctx.c.newBytes(value)
ctx.c.call("sqlite3_result_text64",
uint64(ctx.handle), uint64(ptr), uint64(len(value)),
uint64(ctx.c.freer), _UTF8)
ctx.c.call("sqlite3_result_text_go",
uint64(ctx.handle), uint64(ptr), uint64(len(value)))
}
// ResultBlob sets the result of the function to a []byte.
@@ -105,9 +104,8 @@ func (ctx Context) ResultRawText(value []byte) {
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultBlob(value []byte) {
ptr := ctx.c.newBytes(value)
ctx.c.call("sqlite3_result_blob64",
uint64(ctx.handle), uint64(ptr), uint64(len(value)),
uint64(ctx.c.freer))
ctx.c.call("sqlite3_result_blob_go",
uint64(ctx.handle), uint64(ptr), uint64(len(value)))
}
// ResultZeroBlob sets the result of the function to a zero-filled, length n BLOB.
@@ -130,7 +128,8 @@ func (ctx Context) ResultNull() {
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultTime(value time.Time, format TimeFormat) {
if format == TimeFormatDefault {
switch format {
case TimeFormatDefault, TimeFormatAuto, time.RFC3339Nano:
ctx.resultRFC3339Nano(value)
return
}
@@ -153,9 +152,8 @@ func (ctx Context) resultRFC3339Nano(value time.Time) {
buf := util.View(ctx.c.mod, ptr, maxlen)
buf = value.AppendFormat(buf[:0], time.RFC3339Nano)
ctx.c.call("sqlite3_result_text64",
uint64(ctx.handle), uint64(ptr), uint64(len(buf)),
uint64(ctx.c.freer), _UTF8)
ctx.c.call("sqlite3_result_text_go",
uint64(ctx.handle), uint64(ptr), uint64(len(buf)))
}
// ResultPointer sets the result of the function to NULL, just like [Context.ResultNull],
@@ -165,7 +163,8 @@ func (ctx Context) resultRFC3339Nano(value time.Time) {
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultPointer(ptr any) {
valPtr := util.AddHandle(ctx.c.ctx, ptr)
ctx.c.call("sqlite3_result_pointer_go", uint64(valPtr))
ctx.c.call("sqlite3_result_pointer_go",
uint64(ctx.handle), uint64(valPtr))
}
// ResultJSON sets the result of the function to the JSON encoding of value.
@@ -175,7 +174,7 @@ func (ctx Context) ResultJSON(value any) {
data, err := json.Marshal(value)
if err != nil {
ctx.ResultError(err)
return
return // notest
}
ctx.ResultRawText(data)
}

View File

@@ -8,21 +8,50 @@
//
// The data source name for "sqlite3" databases can be a filename or a "file:" [URI].
//
// # Default transaction mode
//
// The [TRANSACTION] mode can be specified using "_txlock":
//
// sql.Open("sqlite3", "file:demo.db?_txlock=immediate")
//
// Possible values are: "deferred", "immediate", "exclusive".
// A [read-only] transaction is always "deferred", regardless of "_txlock".
// Possible values are: "deferred" (the default), "immediate", "exclusive".
// Regardless of "_txlock":
// - a [linearizable] transaction is always "exclusive";
// - a [serializable] transaction is always "immediate";
// - a [read-only] transaction is always "deferred".
//
// # Working with time
//
// The time encoding/decoding format can be specified using "_timefmt":
//
// sql.Open("sqlite3", "file:demo.db?_timefmt=sqlite")
//
// Possible values are: "auto" (the default), "sqlite", "rfc3339";
// "auto" encodes as RFC 3339 and decodes any [format] supported by SQLite;
// "sqlite" encodes as SQLite and decodes any [format] supported by SQLite;
// "rfc3339" encodes and decodes RFC 3339 only.
// - "auto" encodes as RFC 3339 and decodes any [format] supported by SQLite;
// - "sqlite" encodes as SQLite and decodes any [format] supported by SQLite;
// - "rfc3339" encodes and decodes RFC 3339 only.
//
// If you encode as RFC 3339 (the default),
// consider using the TIME [collating sequence] to produce a time-ordered sequence.
//
// To scan values in other formats, [sqlite3.TimeFormat.Scanner] may be helpful.
// To bind values in other formats, [sqlite3.TimeFormat.Encode] them before binding.
//
// When using a custom time struct, you'll have to implement
// [database/sql/driver.Valuer] and [database/sql.Scanner].
//
// The Value method should ideally encode to a time [format] supported by SQLite.
// This ensures SQL date and time functions work as they should,
// and that your schema works with other SQLite tools.
// [sqlite3.TimeFormat.Encode] may help.
//
// The Scan method needs to take into account that the value it receives can be of differing types.
// It can already be a [time.Time], if the driver decoded the value according to "_timefmt" rules.
// Or it can be a: string, int64, float64, []byte, or nil,
// depending on the column type and what whoever wrote the value.
// [sqlite3.TimeFormat.Decode] may help.
//
// # Setting PRAGMAs
//
// [PRAGMA] statements can be specified using "_pragma":
//
@@ -31,13 +60,17 @@
// If no PRAGMAs are specified, a busy timeout of 1 minute is set.
//
// Order matters:
// busy timeout and locking mode should be the first PRAGMAs set, in that order.
// encryption keys, busy timeout and locking mode should be the first PRAGMAs set,
// in that order.
//
// [URI]: https://sqlite.org/uri.html
// [PRAGMA]: https://sqlite.org/pragma.html
// [format]: https://sqlite.org/lang_datefunc.html#time_values
// [TRANSACTION]: https://sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions
// [linearizable]: https://pkg.go.dev/database/sql#TxOptions
// [serializable]: https://pkg.go.dev/database/sql#TxOptions
// [read-only]: https://pkg.go.dev/database/sql#TxOptions
// [format]: https://sqlite.org/lang_datefunc.html#time_values
// [collating sequence]: https://sqlite.org/datatype3.html#collating_sequences
package driver
import (
@@ -48,6 +81,7 @@ import (
"fmt"
"io"
"net/url"
"reflect"
"strings"
"time"
"unsafe"
@@ -69,11 +103,22 @@ func init() {
// Open opens the SQLite database specified by dataSourceName as a [database/sql.DB].
//
// The init function is called by the driver on new connections.
// Open accepts zero, one, or two callbacks (nil callbacks are ignored).
// The first callback is called when the driver opens a new connection.
// The second callback is called before the driver closes a connection.
// The [sqlite3.Conn] can be used to execute queries, register functions, etc.
// Any error returned closes the connection and is returned to [database/sql].
func Open(dataSourceName string, init func(*sqlite3.Conn) error) (*sql.DB, error) {
c, err := (&SQLite{Init: init}).OpenConnector(dataSourceName)
func Open(dataSourceName string, fn ...func(*sqlite3.Conn) error) (*sql.DB, error) {
if len(fn) > 2 {
return nil, sqlite3.MISUSE
}
var init, term func(*sqlite3.Conn) error
if len(fn) > 1 {
term = fn[1]
}
if len(fn) > 0 {
init = fn[0]
}
c, err := newConnector(dataSourceName, init, term)
if err != nil {
return nil, err
}
@@ -81,16 +126,16 @@ func Open(dataSourceName string, init func(*sqlite3.Conn) error) (*sql.DB, error
}
// SQLite implements [database/sql/driver.Driver].
type SQLite struct {
// Init function is called by the driver on new connections.
// The [sqlite3.Conn] can be used to execute queries, register functions, etc.
// Any error returned closes the connection and is returned to [database/sql].
Init func(*sqlite3.Conn) error
}
type SQLite struct{}
var (
// Ensure these interfaces are implemented:
_ driver.DriverContext = &SQLite{}
)
// Open implements [database/sql/driver.Driver].
func (d *SQLite) Open(name string) (driver.Conn, error) {
c, err := d.newConnector(name)
c, err := newConnector(name, nil, nil)
if err != nil {
return nil, err
}
@@ -99,11 +144,11 @@ func (d *SQLite) Open(name string) (driver.Conn, error) {
// OpenConnector implements [database/sql/driver.DriverContext].
func (d *SQLite) OpenConnector(name string) (driver.Connector, error) {
return d.newConnector(name)
return newConnector(name, nil, nil)
}
func (d *SQLite) newConnector(name string) (*connector, error) {
c := connector{driver: d, name: name}
func newConnector(name string, init, term func(*sqlite3.Conn) error) (*connector, error) {
c := connector{name: name, init: init, term: term}
var txlock, timefmt string
if strings.HasPrefix(name, "file:") {
@@ -119,10 +164,8 @@ func (d *SQLite) newConnector(name string) (*connector, error) {
}
switch txlock {
case "":
c.txBegin = "BEGIN"
case "deferred", "immediate", "exclusive":
c.txBegin = "BEGIN " + txlock
case "", "deferred", "concurrent", "immediate", "exclusive":
c.txLock = txlock
default:
return nil, fmt.Errorf("sqlite3: invalid _txlock: %s", txlock)
}
@@ -145,31 +188,32 @@ func (d *SQLite) newConnector(name string) (*connector, error) {
}
type connector struct {
driver *SQLite
init func(*sqlite3.Conn) error
term func(*sqlite3.Conn) error
name string
txBegin string
txLock string
tmRead sqlite3.TimeFormat
tmWrite sqlite3.TimeFormat
pragmas bool
}
func (n *connector) Driver() driver.Driver {
return n.driver
return &SQLite{}
}
func (n *connector) Connect(ctx context.Context) (_ driver.Conn, err error) {
func (n *connector) Connect(ctx context.Context) (res driver.Conn, err error) {
c := &conn{
txBegin: n.txBegin,
txLock: n.txLock,
tmRead: n.tmRead,
tmWrite: n.tmWrite,
}
c.Conn, err = sqlite3.Open(n.name)
c.Conn, err = sqlite3.OpenContext(ctx, n.name)
if err != nil {
return nil, err
}
defer func() {
if err != nil {
if res == nil {
c.Close()
}
}()
@@ -178,22 +222,23 @@ func (n *connector) Connect(ctx context.Context) (_ driver.Conn, err error) {
defer c.Conn.SetInterrupt(old)
if !n.pragmas {
err = c.Conn.BusyTimeout(60 * time.Second)
err = c.Conn.BusyTimeout(time.Minute)
if err != nil {
return nil, err
}
}
if n.driver.Init != nil {
err = n.driver.Init(c.Conn)
if n.init != nil {
err = n.init(c.Conn)
if err != nil {
return nil, err
}
}
if n.pragmas || n.driver.Init != nil {
if n.pragmas || n.init != nil {
s, _, err := c.Conn.Prepare(`PRAGMA query_only`)
if err != nil {
return nil, err
}
defer s.Close()
if s.Step() && s.ColumnBool(0) {
c.readOnly = '1'
} else {
@@ -204,25 +249,61 @@ func (n *connector) Connect(ctx context.Context) (_ driver.Conn, err error) {
return nil, err
}
}
if n.term != nil {
err = c.Conn.Trace(sqlite3.TRACE_CLOSE, func(sqlite3.TraceEvent, any, any) error {
return n.term(c.Conn)
})
if err != nil {
return nil, err
}
}
return c, nil
}
// Conn is implemented by the SQLite [database/sql] driver connections.
//
// It can be used to access SQLite features like [online backup]:
//
// db, err := driver.Open("temp.db")
// if err != nil {
// log.Fatal(err)
// }
// defer db.Close()
//
// conn, err := db.Conn(context.TODO())
// if err != nil {
// log.Fatal(err)
// }
//
// err = conn.Raw(func(driverConn any) error {
// conn := driverConn.(driver.Conn)
// return conn.Raw().Backup("main", "backup.db")
// })
// if err != nil {
// log.Fatal(err)
// }
//
// [online backup]: https://sqlite.org/backup.html
type Conn interface {
Raw() *sqlite3.Conn
driver.Conn
driver.ConnBeginTx
driver.ConnPrepareContext
}
type conn struct {
*sqlite3.Conn
txBegin string
txCommit string
txRollback string
tmRead sqlite3.TimeFormat
tmWrite sqlite3.TimeFormat
readOnly byte
txLock string
txReset string
tmRead sqlite3.TimeFormat
tmWrite sqlite3.TimeFormat
readOnly byte
}
var (
// Ensure these interfaces are implemented:
_ driver.ConnPrepareContext = &conn{}
_ driver.ExecerContext = &conn{}
_ driver.ConnBeginTx = &conn{}
_ sqlite3.DriverConn = &conn{}
_ Conn = &conn{}
_ driver.ExecerContext = &conn{}
)
func (c *conn) Raw() *sqlite3.Conn {
@@ -231,31 +312,30 @@ func (c *conn) Raw() *sqlite3.Conn {
// Deprecated: use BeginTx instead.
func (c *conn) Begin() (driver.Tx, error) {
// notest
return c.BeginTx(context.Background(), driver.TxOptions{})
}
func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {
txBegin := c.txBegin
c.txCommit = `COMMIT`
c.txRollback = `ROLLBACK`
if opts.ReadOnly {
txBegin = `
BEGIN deferred;
PRAGMA query_only=on`
c.txRollback = `
ROLLBACK;
PRAGMA query_only=` + string(c.readOnly)
c.txCommit = c.txRollback
}
var txLock string
switch opts.Isolation {
default:
return nil, util.IsolationErr
case
driver.IsolationLevel(sql.LevelDefault),
driver.IsolationLevel(sql.LevelSerializable):
break
case driver.IsolationLevel(sql.LevelLinearizable):
txLock = "exclusive"
case driver.IsolationLevel(sql.LevelSerializable):
txLock = "immediate"
case driver.IsolationLevel(sql.LevelDefault):
if !opts.ReadOnly {
txLock = c.txLock
}
}
c.txReset = ``
txBegin := `BEGIN ` + txLock
if opts.ReadOnly {
txBegin += ` ; PRAGMA query_only=on`
c.txReset = `; PRAGMA query_only=` + string(c.readOnly)
}
old := c.Conn.SetInterrupt(ctx)
@@ -269,7 +349,7 @@ func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, e
}
func (c *conn) Commit() error {
err := c.Conn.Exec(c.txCommit)
err := c.Conn.Exec(`COMMIT` + c.txReset)
if err != nil && !c.Conn.GetAutocommit() {
c.Rollback()
}
@@ -277,16 +357,17 @@ func (c *conn) Commit() error {
}
func (c *conn) Rollback() error {
err := c.Conn.Exec(c.txRollback)
err := c.Conn.Exec(`ROLLBACK` + c.txReset)
if errors.Is(err, sqlite3.INTERRUPT) {
old := c.Conn.SetInterrupt(context.Background())
defer c.Conn.SetInterrupt(old)
err = c.Conn.Exec(c.txRollback)
err = c.Conn.Exec(`ROLLBACK` + c.txReset)
}
return err
}
func (c *conn) Prepare(query string) (driver.Stmt, error) {
// notest
return c.PrepareContext(context.Background(), query)
}
@@ -329,6 +410,8 @@ func (c *conn) ExecContext(ctx context.Context, query string, args []driver.Name
}
func (c *conn) CheckNamedValue(arg *driver.NamedValue) error {
// Fast path: short circuit argument verification.
// Arguments will be rejected by conn.ExecContext.
return nil
}
@@ -363,11 +446,13 @@ func (s *stmt) NumInput() int {
// Deprecated: use ExecContext instead.
func (s *stmt) Exec(args []driver.Value) (driver.Result, error) {
// notest
return s.ExecContext(context.Background(), namedValues(args))
}
// Deprecated: use QueryContext instead.
func (s *stmt) Query(args []driver.Value) (driver.Rows, error) {
// notest
return s.QueryContext(context.Background(), namedValues(args))
}
@@ -381,6 +466,7 @@ func (s *stmt) ExecContext(ctx context.Context, args []driver.NamedValue) (drive
defer s.Stmt.Conn().SetInterrupt(old)
err = s.Stmt.Exec()
s.Stmt.ClearBindings()
if err != nil {
return nil, err
}
@@ -403,7 +489,7 @@ func (s *stmt) setupBindings(args []driver.NamedValue) (err error) {
if arg.Name == "" {
ids = append(ids, arg.Ordinal)
} else {
for _, prefix := range []string{":", "@", "$"} {
for _, prefix := range [...]string{":", "@", "$"} {
if id := s.Stmt.BindIndex(prefix + arg.Name); id != 0 {
ids = append(ids, id)
}
@@ -437,9 +523,9 @@ func (s *stmt) setupBindings(args []driver.NamedValue) (err error) {
default:
panic(util.AssertErr())
}
}
if err != nil {
return err
if err != nil {
return err
}
}
}
return nil
@@ -493,8 +579,29 @@ type rows struct {
*stmt
names []string
types []string
nulls []bool
scans []scantype
}
type scantype byte
const (
_ANY scantype = iota
_INT scantype = scantype(sqlite3.INTEGER)
_REAL scantype = scantype(sqlite3.FLOAT)
_TEXT scantype = scantype(sqlite3.TEXT)
_BLOB scantype = scantype(sqlite3.BLOB)
_NULL scantype = scantype(sqlite3.NULL)
_BOOL scantype = iota
_TIME
)
var (
// Ensure these interfaces are implemented:
_ driver.RowsColumnTypeDatabaseTypeName = &rows{}
_ driver.RowsColumnTypeNullable = &rows{}
)
func (r *rows) Close() error {
r.Stmt.ClearBindings()
return r.Stmt.Reset()
@@ -503,27 +610,69 @@ func (r *rows) Close() error {
func (r *rows) Columns() []string {
if r.names == nil {
count := r.Stmt.ColumnCount()
r.names = make([]string, count)
for i := range r.names {
r.names[i] = r.Stmt.ColumnName(i)
names := make([]string, count)
for i := range names {
names[i] = r.Stmt.ColumnName(i)
}
r.names = names
}
return r.names
}
func (r *rows) loadColumnMetadata() {
if r.nulls == nil {
count := r.Stmt.ColumnCount()
nulls := make([]bool, count)
types := make([]string, count)
scans := make([]scantype, count)
for i := range nulls {
if col := r.Stmt.ColumnOriginName(i); col != "" {
types[i], _, nulls[i], _, _, _ = r.Stmt.Conn().TableColumnMetadata(
r.Stmt.ColumnDatabaseName(i),
r.Stmt.ColumnTableName(i),
col)
types[i] = strings.ToUpper(types[i])
// These types are only used before we have rows,
// and otherwise as type hints.
// The first few ensure STRICT tables are strictly typed.
// The other two are type hints for booleans and time.
switch types[i] {
case "INT", "INTEGER":
scans[i] = _INT
case "REAL":
scans[i] = _REAL
case "TEXT":
scans[i] = _TEXT
case "BLOB":
scans[i] = _BLOB
case "BOOLEAN":
scans[i] = _BOOL
case "DATE", "TIME", "DATETIME", "TIMESTAMP":
scans[i] = _TIME
}
}
}
r.nulls = nulls
r.types = types
r.scans = scans
}
}
func (r *rows) declType(index int) string {
if r.types == nil {
count := r.Stmt.ColumnCount()
r.types = make([]string, count)
for i := range r.types {
r.types[i] = strings.ToUpper(r.Stmt.ColumnDeclType(i))
types := make([]string, count)
for i := range types {
types[i] = strings.ToUpper(r.Stmt.ColumnDeclType(i))
}
r.types = types
}
return r.types[index]
}
func (r *rows) ColumnTypeDatabaseTypeName(index int) string {
decltype := r.declType(index)
r.loadColumnMetadata()
decltype := r.types[index]
if len := len(decltype); len > 0 && decltype[len-1] == ')' {
if i := strings.LastIndexByte(decltype, '('); i >= 0 {
decltype = decltype[:i]
@@ -532,6 +681,58 @@ func (r *rows) ColumnTypeDatabaseTypeName(index int) string {
return strings.TrimSpace(decltype)
}
func (r *rows) ColumnTypeNullable(index int) (nullable, ok bool) {
r.loadColumnMetadata()
if r.nulls[index] {
return false, true
}
return true, false
}
func (r *rows) ColumnTypeScanType(index int) (typ reflect.Type) {
r.loadColumnMetadata()
scan := r.scans[index]
if r.Stmt.Busy() {
// SQLite is dynamically typed and we now have a row.
// Always use the type of the value itself,
// unless the scan type is more specific
// and can scan the actual value.
val := scantype(r.Stmt.ColumnType(index))
useValType := true
switch {
case scan == _TIME && val != _BLOB && val != _NULL:
t := r.Stmt.ColumnTime(index, r.tmRead)
useValType = t == time.Time{}
case scan == _BOOL && val == _INT:
i := r.Stmt.ColumnInt64(index)
useValType = i != 0 && i != 1
case scan == _BLOB && val == _NULL:
useValType = false
}
if useValType {
scan = val
}
}
switch scan {
case _INT:
return reflect.TypeOf(int64(0))
case _REAL:
return reflect.TypeOf(float64(0))
case _TEXT:
return reflect.TypeOf("")
case _BLOB:
return reflect.TypeOf([]byte{})
case _BOOL:
return reflect.TypeOf(false)
case _TIME:
return reflect.TypeOf(time.Time{})
default:
return reflect.TypeOf((*any)(nil)).Elem()
}
}
func (r *rows) Next(dest []driver.Value) error {
old := r.Stmt.Conn().SetInterrupt(r.ctx)
defer r.Stmt.Conn().SetInterrupt(old)
@@ -544,30 +745,27 @@ func (r *rows) Next(dest []driver.Value) error {
}
data := unsafe.Slice((*any)(unsafe.SliceData(dest)), len(dest))
err := r.Stmt.Columns(data)
err := r.Stmt.Columns(data...)
for i := range dest {
if t, ok := r.decodeTime(i, dest[i]); ok {
dest[i] = t
continue
}
if s, ok := dest[i].(string); ok {
t, ok := maybeTime(s)
if ok {
dest[i] = t
}
}
}
return err
}
func (r *rows) decodeTime(i int, v any) (_ time.Time, ok bool) {
if r.tmRead == sqlite3.TimeFormatDefault {
// handled by maybeTime
return
}
switch v.(type) {
case int64, float64, string:
switch v := v.(type) {
case int64, float64:
// could be a time value
case string:
if r.tmWrite != "" && r.tmWrite != time.RFC3339 && r.tmWrite != time.RFC3339Nano {
break
}
t, ok := maybeTime(v)
if ok {
return t, true
}
default:
return
}

View File

@@ -7,7 +7,7 @@ import (
"errors"
"math"
"net/url"
"path/filepath"
"reflect"
"testing"
"time"
@@ -15,9 +15,21 @@ import (
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/vfs"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Test_Open_error(t *testing.T) {
t.Parallel()
_, err := Open("", nil, nil, nil)
if err == nil {
t.Error("want error")
}
if !errors.Is(err, sqlite3.MISUSE) {
t.Errorf("got %v, want sqlite3.MISUSE", err)
}
}
func Test_Open_dir(t *testing.T) {
t.Parallel()
@@ -38,8 +50,11 @@ func Test_Open_dir(t *testing.T) {
func Test_Open_pragma(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
"_pragma": {"busy_timeout(1000)"},
})
db, err := sql.Open("sqlite3", "file::memory:?_pragma=busy_timeout(1000)")
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -57,8 +72,11 @@ func Test_Open_pragma(t *testing.T) {
func Test_Open_pragma_invalid(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
"_pragma": {"busy_timeout 1000"},
})
db, err := sql.Open("sqlite3", "file::memory:?_pragma=busy_timeout+1000")
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -81,14 +99,13 @@ func Test_Open_pragma_invalid(t *testing.T) {
}
func Test_Open_txLock(t *testing.T) {
if !vfs.SupportsFileLocking {
t.Skip("skipping without locks")
}
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
"_txlock": {"exclusive"},
"_pragma": {"busy_timeout(1000)"},
})
db, err := sql.Open("sqlite3", "file:"+
filepath.ToSlash(filepath.Join(t.TempDir(), "test.db"))+
"?_txlock=exclusive&_pragma=busy_timeout(0)")
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -119,8 +136,11 @@ func Test_Open_txLock(t *testing.T) {
func Test_Open_txLock_invalid(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
"_txlock": {"xclusive"},
})
_, err := sql.Open("sqlite3", "file::memory:?_txlock=xclusive")
_, err := sql.Open("sqlite3", tmp+"_txlock=xclusive")
if err == nil {
t.Fatal("want error")
}
@@ -130,17 +150,16 @@ func Test_Open_txLock_invalid(t *testing.T) {
}
func Test_BeginTx(t *testing.T) {
if !vfs.SupportsFileLocking {
t.Skip("skipping without locks")
}
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
"_txlock": {"exclusive"},
"_pragma": {"busy_timeout(0)"},
})
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
db, err := sql.Open("sqlite3", "file:"+
filepath.ToSlash(filepath.Join(t.TempDir(), "test.db"))+
"?_txlock=exclusive&_pragma=busy_timeout(0)")
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -182,8 +201,9 @@ func Test_BeginTx(t *testing.T) {
func Test_Prepare(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := sql.Open("sqlite3", ":memory:")
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -222,11 +242,12 @@ func Test_Prepare(t *testing.T) {
func Test_QueryRow_named(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
db, err := sql.Open("sqlite3", ":memory:")
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -274,8 +295,9 @@ func Test_QueryRow_named(t *testing.T) {
func Test_QueryRow_blob_null(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := sql.Open("sqlite3", ":memory:")
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -310,7 +332,11 @@ func Test_time(t *testing.T) {
for _, fmt := range []string{"auto", "sqlite", "rfc3339", time.ANSIC} {
t.Run(fmt, func(t *testing.T) {
db, err := sql.Open("sqlite3", "file::memory:?_timefmt="+url.QueryEscape(fmt))
tmp := memdb.TestDB(t, url.Values{
"_timefmt": {fmt},
})
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
@@ -340,3 +366,104 @@ func Test_time(t *testing.T) {
})
}
}
func Test_ColumnType_ScanType(t *testing.T) {
var (
INT = reflect.TypeOf(int64(0))
REAL = reflect.TypeOf(float64(0))
TEXT = reflect.TypeOf("")
BLOB = reflect.TypeOf([]byte{})
BOOL = reflect.TypeOf(false)
TIME = reflect.TypeOf(time.Time{})
ANY = reflect.TypeOf((*any)(nil)).Elem()
)
t.Parallel()
tmp := memdb.TestDB(t)
db, err := sql.Open("sqlite3", tmp)
if err != nil {
t.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`
CREATE TABLE test (
col_int INTEGER,
col_real REAL,
col_text TEXT,
col_blob BLOB,
col_bool BOOLEAN,
col_time DATETIME,
col_decimal DECIMAL
);
INSERT INTO test VALUES
(1, 1, 1, 1, 1, 1, 1),
(2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0),
('1', '1', '1', '1', '1', '1', '1'),
('x', 'x', 'x', 'x', 'x', 'x', 'x'),
(x'', x'', x'', x'', x'', x'', x''),
('2006-01-02T15:04:05Z', '2006-01-02T15:04:05Z', '2006-01-02T15:04:05Z', '2006-01-02T15:04:05Z',
'2006-01-02T15:04:05Z', '2006-01-02T15:04:05Z', '2006-01-02T15:04:05Z'),
(TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE),
(NULL, NULL, NULL, NULL, NULL, NULL, NULL);
`)
if err != nil {
t.Fatal(err)
}
rows, err := db.Query(`SELECT * FROM test`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
cols, err := rows.ColumnTypes()
if err != nil {
t.Fatal(err)
}
want := [][]reflect.Type{
{INT, REAL, TEXT, BLOB, BOOL, TIME, ANY},
{INT, REAL, TEXT, INT, BOOL, TIME, INT},
{INT, REAL, TEXT, REAL, INT, TIME, INT},
{INT, REAL, TEXT, TEXT, BOOL, TIME, INT},
{TEXT, TEXT, TEXT, TEXT, TEXT, TEXT, TEXT},
{BLOB, BLOB, BLOB, BLOB, BLOB, BLOB, BLOB},
{TEXT, TEXT, TEXT, TEXT, TEXT, TIME, TEXT},
{INT, REAL, TEXT, INT, BOOL, TIME, INT},
{ANY, ANY, ANY, BLOB, ANY, ANY, ANY},
}
for j, c := range cols {
got := c.ScanType()
if got != want[0][j] {
t.Errorf("want %v, got %v, at column %d", want[0][j], got, j)
}
}
dest := make([]any, len(cols))
for i := 1; rows.Next(); i++ {
cols, err := rows.ColumnTypes()
if err != nil {
t.Fatal(err)
}
for j, c := range cols {
got := c.ScanType()
if got != want[i][j] {
t.Errorf("want %v, got %v, at row %d column %d", want[i][j], got, i, j)
}
dest[j] = reflect.New(got).Interface()
}
err = rows.Scan(dest...)
if err != nil {
t.Error(err)
}
}
err = rows.Err()
if err != nil {
t.Fatal(err)
}
}

144
driver/example2_test.go Normal file
View File

@@ -0,0 +1,144 @@
//go:build linux || darwin || windows || freebsd || openbsd || netbsd || dragonfly || illumos || sqlite3_flock || sqlite3_dotlk
package driver_test
// Adapted from: https://go.dev/doc/tutorial/database-access
import (
"database/sql"
"database/sql/driver"
"fmt"
"log"
"time"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Example_customTime() {
db, err := sql.Open("sqlite3", "file:/time.db?vfs=memdb")
if err != nil {
log.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`
CREATE TABLE data (
id INTEGER PRIMARY KEY,
date_time TEXT
) STRICT;
`)
if err != nil {
log.Fatal(err)
}
// This one will be returned as string to [sql.Scanner] because it doesn't
// pass the driver's round-trip test when it tries to figure out if it's
// a time. 2009-11-17T20:34:58.650Z goes in, but parsing and formatting
// it with [time.RFC3338Nano] results in 2009-11-17T20:34:58.65Z. Though
// the times are identical, the trailing zero is lost in the string
// representation so the driver considers the conversion unsuccessful.
c1 := CustomTime{time.Date(
2009, 11, 17, 20, 34, 58, 650000000, time.UTC)}
// Store our custom time in the database.
_, err = db.Exec(`INSERT INTO data (date_time) VALUES(?)`, c1)
if err != nil {
log.Fatal(err)
}
var strc1 string
// Retrieve it as a string, the result of Value().
err = db.QueryRow(`
SELECT date_time
FROM data
WHERE id = last_insert_rowid()
`).Scan(&strc1)
if err != nil {
log.Fatal(err)
}
fmt.Println("in db:", strc1)
var resc1 CustomTime
// Retrieve it as our custom time type, going through Scan().
err = db.QueryRow(`
SELECT date_time
FROM data
WHERE id = last_insert_rowid()
`).Scan(&resc1)
if err != nil {
log.Fatal(err)
}
fmt.Println("custom time:", resc1)
// This one will be returned as [time.Time] to [sql.Scanner] because it does
// pass the driver's round-trip test when it tries to figure out if it's
// a time. 2009-11-17T20:34:58.651Z goes in, and parsing and formatting
// it with [time.RFC3339Nano] results in 2009-11-17T20:34:58.651Z.
c2 := CustomTime{time.Date(
2009, 11, 17, 20, 34, 58, 651000000, time.UTC)}
// Store our custom time in the database.
_, err = db.Exec(`INSERT INTO data (date_time) VALUES(?)`, c2)
if err != nil {
log.Fatal(err)
}
var strc2 string
// Retrieve it as a string, the result of Value().
err = db.QueryRow(`
SELECT date_time
FROM data
WHERE id = last_insert_rowid()
`).Scan(&strc2)
if err != nil {
log.Fatal(err)
}
fmt.Println("in db:", strc2)
var resc2 CustomTime
// Retrieve it as our custom time type, going through Scan().
err = db.QueryRow(`
SELECT date_time
FROM data
WHERE id = last_insert_rowid()
`).Scan(&resc2)
if err != nil {
log.Fatal(err)
}
fmt.Println("custom time:", resc2)
// Output:
// in db: 2009-11-17T20:34:58.650Z
// scan type string: 2009-11-17T20:34:58.650Z
// custom time: 2009-11-17 20:34:58.65 +0000 UTC
// in db: 2009-11-17T20:34:58.651Z
// scan type time: 2009-11-17 20:34:58.651 +0000 UTC
// custom time: 2009-11-17 20:34:58.651 +0000 UTC
}
type CustomTime struct{ time.Time }
func (c CustomTime) Value() (driver.Value, error) {
return sqlite3.TimeFormat7TZ.Encode(c.UTC()), nil
}
func (c *CustomTime) Scan(value any) error {
switch v := value.(type) {
case nil:
*c = CustomTime{time.Time{}}
case time.Time:
fmt.Println("scan type time:", v)
*c = CustomTime{v}
case string:
fmt.Println("scan type string:", v)
t, err := sqlite3.TimeFormat7TZ.Decode(v)
if err != nil {
return err
}
*c = CustomTime{t}
default:
panic("unsupported value type")
}
return nil
}

View File

@@ -1,4 +1,4 @@
//go:build (linux || darwin || windows || freebsd || illumos) && !sqlite3_nosys
//go:build linux || darwin || windows || freebsd || openbsd || netbsd || dragonfly || illumos || sqlite3_flock || sqlite3_dotlk
package driver_test
@@ -12,6 +12,7 @@ import (
_ "github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
)
var db *sql.DB

View File

@@ -11,7 +11,7 @@ import (
)
func Example_json() {
db, err := driver.Open("file:/test.db?vfs=memdb", nil)
db, err := driver.Open("file:/json.db?vfs=memdb")
if err != nil {
log.Fatal(err)
}
@@ -21,8 +21,8 @@ func Example_json() {
CREATE TABLE orders (
cart_id INTEGER PRIMARY KEY,
user_id INTEGER NOT NULL,
cart TEXT
);
cart BLOB -- stored as JSONB
) STRICT;
`)
if err != nil {
log.Fatal(err)
@@ -39,7 +39,8 @@ func Example_json() {
Items []CartItem `json:"items"`
}
_, err = db.Exec(`INSERT INTO orders (user_id, cart) VALUES (?, ?)`, 123, sqlite3.JSON(Cart{
// convert to JSONB on insertion
_, err = db.Exec(`INSERT INTO orders (user_id, cart) VALUES (?, jsonb(?))`, 123, sqlite3.JSON(Cart{
[]CartItem{
{ItemID: "111", Name: "T-shirt", Quantity: 1, Price: 250},
{ItemID: "222", Name: "Trousers", Quantity: 1, Price: 600},
@@ -60,6 +61,24 @@ func Example_json() {
}
fmt.Println("total:", total)
var cart Cart
err = db.QueryRow(`
SELECT json(cart) -- convert to JSON on retrieval
FROM orders
WHERE cart_id = last_insert_rowid()
`).Scan(sqlite3.JSON(&cart))
if err != nil {
log.Fatal(err)
}
for _, item := range cart.Items {
fmt.Printf("id: %s, name: %s, quantity: %d, price: %d\n",
item.ItemID, item.Name, item.Quantity, item.Price)
}
// Output:
// total: 850
// id: 111, name: T-shirt, quantity: 1, price: 250
// id: 222, name: Trousers, quantity: 1, price: 600
}

View File

@@ -16,12 +16,25 @@ func Savepoint(tx *sql.Tx) sqlite3.Savepoint {
return ctx.Savepoint
}
// A saveptCtx is never canceled, has no values, and has no deadline.
type saveptCtx struct{ sqlite3.Savepoint }
func (*saveptCtx) Deadline() (deadline time.Time, ok bool) { return }
func (*saveptCtx) Deadline() (deadline time.Time, ok bool) {
// notest
return
}
func (*saveptCtx) Done() <-chan struct{} { return nil }
func (*saveptCtx) Done() <-chan struct{} {
// notest
return nil
}
func (*saveptCtx) Err() error { return nil }
func (*saveptCtx) Err() error {
// notest
return nil
}
func (*saveptCtx) Value(key any) any { return nil }
func (*saveptCtx) Value(key any) any {
// notest
return nil
}

View File

@@ -10,7 +10,7 @@ import (
)
func ExampleSavepoint() {
db, err := driver.Open("file:/test.db?vfs=memdb", nil)
db, err := driver.Open("file:/svpt.db?vfs=memdb")
if err != nil {
log.Fatal(err)
}

View File

@@ -1,8 +1,6 @@
package driver
import (
"time"
)
import "time"
// Convert a string in [time.RFC3339Nano] format into a [time.Time]
// if it roundtrips back to the same string.

View File

@@ -1,6 +1,6 @@
# Embeddable Wasm build of SQLite
This folder includes an embeddable Wasm build of SQLite 3.46.0 for use with
This folder includes an embeddable Wasm build of SQLite 3.47.2 for use with
[`github.com/ncruces/go-sqlite3`](https://pkg.go.dev/github.com/ncruces/go-sqlite3).
The following optional features are compiled in:
@@ -9,6 +9,7 @@ The following optional features are compiled in:
- [JSON](https://sqlite.org/json1.html)
- [R*Tree](https://sqlite.org/rtree.html)
- [GeoPoly](https://sqlite.org/geopoly.html)
- [Spellfix1](https://sqlite.org/spellfix1.html)
- [soundex](https://sqlite.org/lang_corefunc.html#soundex)
- [stat4](https://sqlite.org/compile.html#enable_stat4)
- [base64](https://github.com/sqlite/sqlite/blob/master/ext/misc/base64.c)
@@ -17,14 +18,24 @@ The following optional features are compiled in:
- [regexp](https://github.com/sqlite/sqlite/blob/master/ext/misc/regexp.c)
- [series](https://github.com/sqlite/sqlite/blob/master/ext/misc/series.c)
- [uint](https://github.com/sqlite/sqlite/blob/master/ext/misc/uint.c)
- [uuid](https://github.com/sqlite/sqlite/blob/master/ext/misc/uuid.c)
- [time](../sqlite3/time.c)
See the [configuration options](../sqlite3/sqlite_cfg.h),
See the [configuration options](../sqlite3/sqlite_opt.h),
and [patches](../sqlite3) applied.
Built using [`wasi-sdk`](https://github.com/WebAssembly/wasi-sdk),
and [`binaryen`](https://github.com/WebAssembly/binaryen).
The build is easily reproducible, and verifiable, using
[Artifact Attestations](https://github.com/ncruces/go-sqlite3/attestations).
[Artifact Attestations](https://github.com/ncruces/go-sqlite3/attestations).
### Customizing the build
You can use your own custom build of SQLite.
Examples of custom builds of SQLite are:
- [`github.com/ncruces/go-sqlite3/embed/bcw2`](https://github.com/ncruces/go-sqlite3/tree/main/embed/bcw2)
built from a branch supporting [`BEGIN CONCURRENT`](https://sqlite.org/src/doc/begin-concurrent/doc/begin_concurrent.md)
and [Wal2](https://sqlite.org/cgi/src/doc/wal2/doc/wal2.md).
- [`github.com/asg017/sqlite-vec-go-bindings/ncruces`](https://github.com/asg017/sqlite-vec-go-bindings)
which includes the [`sqlite-vec`](https://github.com/asg017/sqlite-vec) vector search extension.

2
embed/bcw2/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
build/
sqlite/

16
embed/bcw2/README.md Normal file
View File

@@ -0,0 +1,16 @@
# Embeddable Wasm build of SQLite
This folder includes an embeddable Wasm build of SQLite, including the experimental
[`BEGIN CONCURRENT`](https://sqlite.org/src/doc/begin-concurrent/doc/begin_concurrent.md) and
[Wal2](https://sqlite.org/cgi/src/doc/wal2/doc/wal2.md) patches.
> [!IMPORTANT]
> This package is experimental.
> It is built from the `bedrock` branch of SQLite,
> since that is _currently_ the most stable, maintained branch to include both features.
> [!CAUTION]
> The Wal2 journaling mode creates databases that other versions of SQLite cannot access.
The build is easily reproducible, and verifiable, using
[Artifact Attestations](https://github.com/ncruces/go-sqlite3/attestations).

BIN
embed/bcw2/bcw2.wasm Executable file

Binary file not shown.

48
embed/bcw2/bcw2_test.go Normal file
View File

@@ -0,0 +1,48 @@
package bcw2
import (
"path/filepath"
"testing"
"github.com/ncruces/go-sqlite3/driver"
"github.com/ncruces/go-sqlite3/vfs"
)
func Test_bcw2(t *testing.T) {
if !vfs.SupportsSharedMemory {
t.Skip("skipping without shared memory")
}
tmp := filepath.ToSlash(filepath.Join(t.TempDir(), "test.db"))
db, err := driver.Open("file:" + tmp + "?_pragma=journal_mode(wal2)&_txlock=concurrent")
if err != nil {
t.Fatal(err)
}
defer db.Close()
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
_, err = tx.Exec(`CREATE TABLE test (col)`)
if err != nil {
t.Fatal(err)
}
err = tx.Commit()
if err != nil {
t.Fatal(err)
}
var version string
err = db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
if err != nil {
t.Fatal(err)
}
if version != "3.48.0" {
t.Error(version)
}
}

65
embed/bcw2/build.sh Executable file
View File

@@ -0,0 +1,65 @@
#!/usr/bin/env bash
set -euo pipefail
cd -P -- "$(dirname -- "$0")"
ROOT=../../
BINARYEN="$ROOT/tools/binaryen/bin"
WASI_SDK="$ROOT/tools/wasi-sdk/bin"
trap 'rm -rf build/ sqlite/ bcw2.tmp' EXIT
mkdir -p build/ext/
cp "$ROOT"/sqlite3/*.[ch] build/
cp "$ROOT"/sqlite3/*.patch build/
# https://sqlite.org/src/info/08cfa7e8b3090151
curl -# https://sqlite.org/src/tarball/sqlite.tar.gz?r=08cfa7e8 | tar xz
cd sqlite
if [[ "$OSTYPE" == "msys" || "$OSTYPE" == "cygwin" ]]; then
MSYS_NO_PATHCONV=1 nmake /f makefile.msc sqlite3.c
else
sh configure
make sqlite3.c
fi
cd ~-
mv sqlite/sqlite3.c build/
mv sqlite/sqlite3.h build/
mv sqlite/sqlite3ext.h build/
mv sqlite/ext/misc/anycollseq.c build/ext/
mv sqlite/ext/misc/base64.c build/ext/
mv sqlite/ext/misc/decimal.c build/ext/
mv sqlite/ext/misc/ieee754.c build/ext/
mv sqlite/ext/misc/regexp.c build/ext/
mv sqlite/ext/misc/series.c build/ext/
mv sqlite/ext/misc/spellfix.c build/ext/
mv sqlite/ext/misc/uint.c build/ext/
cd build
cat *.patch | patch --no-backup-if-mismatch
cd ~-
"$WASI_SDK/clang" --target=wasm32-wasi -std=c23 -g0 -O2 \
-Wall -Wextra -Wno-unused-parameter -Wno-unused-function \
-o bcw2.wasm "build/main.c" \
-I"build" \
-mexec-model=reactor \
-matomics -msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
-Wl,--initial-memory=327680 \
-D_HAVE_SQLITE_CONFIG_H \
-DSQLITE_CUSTOM_INCLUDE=sqlite_opt.h \
$(awk '{print "-Wl,--export="$0}' ../exports.txt)
"$BINARYEN/wasm-ctor-eval" -g -c _initialize bcw2.wasm -o bcw2.tmp
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
bcw2.tmp -o bcw2.wasm \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext

13
embed/bcw2/go.mod Normal file
View File

@@ -0,0 +1,13 @@
module github.com/ncruces/go-sqlite3/embed/bcw2
go 1.21
toolchain go1.23.0
require github.com/ncruces/go-sqlite3 v0.21.0
require (
github.com/ncruces/julianday v1.0.0 // indirect
github.com/tetratelabs/wazero v1.8.2 // indirect
golang.org/x/sys v0.28.0 // indirect
)

10
embed/bcw2/go.sum Normal file
View File

@@ -0,0 +1,10 @@
github.com/ncruces/go-sqlite3 v0.21.0 h1:EwKFoy1hHEopN4sFZarmi+McXdbCcbTuLixhEayXVbQ=
github.com/ncruces/go-sqlite3 v0.21.0/go.mod h1:zxMOaSG5kFYVFK4xQa0pdwIszqxqJ0W0BxBgwdrNjuA=
github.com/ncruces/julianday v1.0.0 h1:fH0OKwa7NWvniGQtxdJRxAgkBMolni2BjDHaWTxqt7M=
github.com/ncruces/julianday v1.0.0/go.mod h1:Dusn2KvZrrovOMJuOt0TNXL6tB7U2E8kvza5fFc9G7g=
github.com/tetratelabs/wazero v1.8.2 h1:yIgLR/b2bN31bjxwXHD8a3d+BogigR952csSDdLYEv4=
github.com/tetratelabs/wazero v1.8.2/go.mod h1:yAI0XTsMBhREkM/YDAK/zNou3GoiAce1P6+rp/wQhjs=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=

23
embed/bcw2/init.go Normal file
View File

@@ -0,0 +1,23 @@
// Package bcw2 embeds SQLite into your application.
//
// Importing package bcw2 initializes the [sqlite3.Binary] variable
// with a build of SQLite that includes the [BEGIN CONCURRENT] and [Wal2] patches:
//
// import _ "github.com/ncruces/go-sqlite3/embed/bcw2"
//
// [BEGIN CONCURRENT]: https://sqlite.org/src/doc/begin-concurrent/doc/begin_concurrent.md
// [Wal2]: https://sqlite.org/cgi/src/doc/wal2/doc/wal2.md
package bcw2
import (
_ "embed"
"github.com/ncruces/go-sqlite3"
)
//go:embed bcw2.wasm
var binary []byte
func init() {
sqlite3.Binary = binary
}

View File

@@ -4,26 +4,27 @@ set -euo pipefail
cd -P -- "$(dirname -- "$0")"
ROOT=../
BINARYEN="$ROOT/tools/binaryen-version_117/bin"
WASI_SDK="$ROOT/tools/wasi-sdk-22.0/bin"
BINARYEN="$ROOT/tools/binaryen/bin"
WASI_SDK="$ROOT/tools/wasi-sdk/bin"
"$WASI_SDK/clang" --target=wasm32-wasi -std=c17 -flto -g0 -O2 \
trap 'rm -f sqlite3.tmp' EXIT
"$WASI_SDK/clang" --target=wasm32-wasi -std=c23 -g0 -O2 \
-Wall -Wextra -Wno-unused-parameter -Wno-unused-function \
-o sqlite3.wasm "$ROOT/sqlite3/main.c" \
-I"$ROOT/sqlite3" \
-mexec-model=reactor \
-msimd128 -mmutable-globals \
-matomics -msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--initial-memory=327680 \
-Wl,--stack-first \
-Wl,--import-undefined \
-Wl,--initial-memory=327680 \
-D_HAVE_SQLITE_CONFIG_H \
-DSQLITE_CUSTOM_INCLUDE=sqlite_opt.h \
$(awk '{print "-Wl,--export="$0}' exports.txt)
trap 'rm -f sqlite3.tmp' EXIT
"$BINARYEN/wasm-ctor-eval" -g -c _initialize sqlite3.wasm -o sqlite3.tmp
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
sqlite3.tmp -o sqlite3.wasm \

View File

@@ -1,7 +1,4 @@
aligned_alloc
free
malloc
malloc_destructor
sqlite3_anycollseq_init
sqlite3_autovacuum_pages_go
sqlite3_backup_finish
@@ -9,7 +6,7 @@ sqlite3_backup_init
sqlite3_backup_pagecount
sqlite3_backup_remaining
sqlite3_backup_step
sqlite3_bind_blob64
sqlite3_bind_blob_go
sqlite3_bind_double
sqlite3_bind_int64
sqlite3_bind_null
@@ -17,7 +14,7 @@ sqlite3_bind_parameter_count
sqlite3_bind_parameter_index
sqlite3_bind_parameter_name
sqlite3_bind_pointer_go
sqlite3_bind_text64
sqlite3_bind_text_go
sqlite3_bind_value
sqlite3_bind_zeroblob64
sqlite3_blob_bytes
@@ -54,33 +51,42 @@ sqlite3_create_collation_go
sqlite3_create_function_go
sqlite3_create_module_go
sqlite3_create_window_function_go
sqlite3_data_count
sqlite3_database_file_object
sqlite3_db_cacheflush
sqlite3_db_config
sqlite3_db_filename
sqlite3_db_name
sqlite3_db_readonly
sqlite3_db_release_memory
sqlite3_db_status
sqlite3_declare_vtab
sqlite3_errcode
sqlite3_errmsg
sqlite3_error_offset
sqlite3_errstr
sqlite3_exec
sqlite3_expanded_sql
sqlite3_file_control
sqlite3_filename_database
sqlite3_filename_journal
sqlite3_filename_wal
sqlite3_finalize
sqlite3_free
sqlite3_get_autocommit
sqlite3_get_auxdata
sqlite3_hard_heap_limit64
sqlite3_interrupt
sqlite3_last_insert_rowid
sqlite3_limit
sqlite3_malloc64
sqlite3_open_v2
sqlite3_overload_function
sqlite3_prepare_v3
sqlite3_progress_handler_go
sqlite3_realloc64
sqlite3_reset
sqlite3_result_blob64
sqlite3_result_blob_go
sqlite3_result_double
sqlite3_result_error
sqlite3_result_error_code
@@ -89,27 +95,30 @@ sqlite3_result_error_toobig
sqlite3_result_int64
sqlite3_result_null
sqlite3_result_pointer_go
sqlite3_result_text64
sqlite3_result_text_go
sqlite3_result_value
sqlite3_result_zeroblob64
sqlite3_rollback_hook_go
sqlite3_set_authorizer_go
sqlite3_set_auxdata_go
sqlite3_set_last_insert_rowid
sqlite3_soft_heap_limit64
sqlite3_step
sqlite3_stmt_busy
sqlite3_stmt_readonly
sqlite3_stmt_status
sqlite3_table_column_metadata
sqlite3_total_changes64
sqlite3_trace_go
sqlite3_txn_state
sqlite3_update_hook_go
sqlite3_uri_key
sqlite3_uri_parameter
sqlite3_value_blob
sqlite3_value_bytes
sqlite3_value_double
sqlite3_value_dup
sqlite3_value_free
sqlite3_value_frombind
sqlite3_value_int64
sqlite3_value_nochange
sqlite3_value_numeric_type

25
embed/init_test.go Normal file
View File

@@ -0,0 +1,25 @@
package embed
import (
"testing"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Test_init(t *testing.T) {
db, err := driver.Open("file:/test.db?vfs=memdb")
if err != nil {
t.Fatal(err)
}
defer db.Close()
var version string
err = db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
if err != nil {
t.Fatal(err)
}
if version != "3.47.2" {
t.Error(version)
}
}

Binary file not shown.

View File

@@ -106,6 +106,11 @@ func (e ErrorCode) Temporary() bool {
return e == BUSY
}
// ExtendedCode returns the extended error code for this error.
func (e ErrorCode) ExtendedCode() ExtendedErrorCode {
return ExtendedErrorCode(e)
}
// Error implements the error interface.
func (e ExtendedErrorCode) Error() string {
return util.ErrorCodeString(uint32(e))
@@ -136,6 +141,11 @@ func (e ExtendedErrorCode) Timeout() bool {
return e == BUSY_TIMEOUT
}
// Code returns the primary error code for this error.
func (e ExtendedErrorCode) Code() ErrorCode {
return ErrorCode(e)
}
func errorCode(err error, def ErrorCode) (msg string, code uint32) {
switch code := err.(type) {
case nil:

View File

@@ -2,6 +2,7 @@ package sqlite3
import (
"errors"
"fmt"
"strings"
"testing"
@@ -10,7 +11,7 @@ import (
func Test_assertErr(t *testing.T) {
err := util.AssertErr()
if s := err.Error(); !strings.HasPrefix(s, "sqlite3: assertion failed") || !strings.HasSuffix(s, "error_test.go:12)") {
if s := err.Error(); !strings.HasPrefix(s, "sqlite3: assertion failed") || !strings.HasSuffix(s, "error_test.go:13)") {
t.Errorf("got %q", s)
}
}
@@ -166,3 +167,32 @@ func Test_ExtendedErrorCode_Error(t *testing.T) {
}
}
}
func Test_errorCode(t *testing.T) {
tests := []struct {
arg error
wantMsg string
wantCode uint32
}{
{nil, "", _OK},
{ERROR, "", util.ERROR},
{IOERR, "", util.IOERR},
{IOERR_READ, "", util.IOERR_READ},
{&Error{code: util.ERROR}, "", util.ERROR},
{fmt.Errorf("%w", ERROR), ERROR.Error(), util.ERROR},
{fmt.Errorf("%w", IOERR), IOERR.Error(), util.IOERR},
{fmt.Errorf("%w", IOERR_READ), IOERR_READ.Error(), util.IOERR_READ},
{fmt.Errorf("error"), "error", util.ERROR},
}
for _, tt := range tests {
t.Run("", func(t *testing.T) {
gotMsg, gotCode := errorCode(tt.arg, ERROR)
if gotMsg != tt.wantMsg {
t.Errorf("errorCode() gotMsg = %q, want %q", gotMsg, tt.wantMsg)
}
if gotCode != uint32(tt.wantCode) {
t.Errorf("errorCode() gotCode = %d, want %d", gotCode, tt.wantCode)
}
})
}
}

37
ext/README.md Normal file
View File

@@ -0,0 +1,37 @@
# Go SQLite Extensions
This folder collects optional SQLite extensions
you can load into your database connections.
### Extensions
- [`github.com/ncruces/go-sqlite3/ext/array`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/array)
provides the [`array`](https://sqlite.org/carray.html) table-valued function.
- [`github.com/ncruces/go-sqlite3/ext/blobio`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/blobio)
simplifies [incremental BLOB I/O](https://sqlite.org/c3ref/blob_open.html).
- [`github.com/ncruces/go-sqlite3/ext/bloom`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/bloom)
provides a [Bloom filter](https://github.com/nalgeon/sqlean/issues/27#issuecomment-1002267134) virtual table.
- [`github.com/ncruces/go-sqlite3/ext/closure`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/closure)
provides a transitive closure virtual table.
- [`github.com/ncruces/go-sqlite3/ext/csv`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/csv)
reads [comma-separated values](https://sqlite.org/csv.html).
- [`github.com/ncruces/go-sqlite3/ext/fileio`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/fileio)
reads, writes and lists files.
- [`github.com/ncruces/go-sqlite3/ext/hash`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/hash)
provides cryptographic hash functions.
- [`github.com/ncruces/go-sqlite3/ext/lines`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/lines)
reads data [line-by-line](https://github.com/asg017/sqlite-lines).
- [`github.com/ncruces/go-sqlite3/ext/pivot`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/pivot)
creates [pivot tables](https://github.com/jakethaw/pivot_vtab).
- [`github.com/ncruces/go-sqlite3/ext/regexp`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/regexp)
provides regular expression functions.
- [`github.com/ncruces/go-sqlite3/ext/statement`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/statement)
creates [parameterized views](https://github.com/0x09/sqlite-statement-vtab).
- [`github.com/ncruces/go-sqlite3/ext/stats`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/stats)
provides [statistics](https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html) functions.
- [`github.com/ncruces/go-sqlite3/ext/unicode`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/unicode)
provides [Unicode aware](https://sqlite.org/src/dir/ext/icu) functions.
- [`github.com/ncruces/go-sqlite3/ext/uuid`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/uuid)
generates [UUIDs](https://en.wikipedia.org/wiki/Universally_unique_identifier).
- [`github.com/ncruces/go-sqlite3/ext/zorder`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/zorder)
maps multidimensional data to one dimension.

View File

@@ -15,8 +15,8 @@ import (
// The argument must be bound to a Go slice or array of
// ints, floats, bools, strings or byte slices,
// using [sqlite3.BindPointer] or [sqlite3.Pointer].
func Register(db *sqlite3.Conn) {
sqlite3.CreateModule(db, "array", nil,
func Register(db *sqlite3.Conn) error {
return sqlite3.CreateModule(db, "array", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (array, error) {
err := db.DeclareVTab(`CREATE TABLE x(value, array HIDDEN)`)
return array{}, err
@@ -62,7 +62,7 @@ func (c *cursor) RowID() (int64, error) {
return int64(c.rowID), nil
}
func (c *cursor) Column(ctx *sqlite3.Context, n int) error {
func (c *cursor) Column(ctx sqlite3.Context, n int) error {
if n != 0 {
return nil
}

View File

@@ -12,13 +12,11 @@ import (
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/array"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Example_driver() {
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
array.Register(c)
return nil
})
db, err := driver.Open("file:/test.db?vfs=memdb", array.Register)
if err != nil {
log.Fatal(err)
}
@@ -53,14 +51,14 @@ func Example_driver() {
}
func Example() {
sqlite3.AutoExtension(array.Register)
db, err := sqlite3.Open(":memory:")
if err != nil {
log.Fatal(err)
}
defer db.Close()
array.Register(db)
stmt, _, err := db.Prepare(`
SELECT name
FROM pragma_function_list
@@ -90,11 +88,9 @@ func Example() {
func Test_cursor_Column(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
array.Register(c)
return nil
})
db, err := driver.Open(tmp, array.Register)
if err != nil {
t.Fatal(err)
}
@@ -126,7 +122,7 @@ func Test_cursor_Column(t *testing.T) {
want = want[1:]
}
if err := rows.Err(); err != nil {
log.Fatal(err)
t.Fatal(err)
}
}
@@ -139,7 +135,10 @@ func Test_array_errors(t *testing.T) {
}
defer db.Close()
array.Register(db)
err = array.Register(db)
if err != nil {
t.Fatal(err)
}
err = db.Exec(`SELECT * FROM array()`)
if err == nil {

View File

@@ -11,11 +11,11 @@ import (
// Register registers the SQL functions:
//
// readblob(schema, table, column, rowid, offset, n)
// readblob(schema, table, column, rowid, offset, n/writer)
//
// Reads n bytes of a blob, starting at offset.
//
// writeblob(schema, table, column, rowid, offset, data)
// writeblob(schema, table, column, rowid, offset, data/reader)
//
// Writes data into a blob, at the given offset.
//
@@ -27,12 +27,17 @@ import (
// using [sqlite3.BindPointer] or [sqlite3.Pointer].
// The optional args will be passed to the callback,
// along with the [sqlite3.Blob] handle.
// The [sqlite3.Blob] handle is only valid during
// the execution of the callback. Callers cannot
// read or write to the handle after the callback
// exits.
//
// https://sqlite.org/c3ref/blob.html
func Register(db *sqlite3.Conn) {
db.CreateFunction("readblob", 6, 0, readblob)
db.CreateFunction("writeblob", 6, 0, writeblob)
db.CreateFunction("openblob", -1, 0, openblob)
func Register(db *sqlite3.Conn) error {
return errors.Join(
db.CreateFunction("readblob", 6, 0, readblob),
db.CreateFunction("writeblob", 6, 0, writeblob),
db.CreateFunction("openblob", -1, 0, openblob))
}
// OpenCallback is the type for the openblob callback.
@@ -42,28 +47,33 @@ func readblob(ctx sqlite3.Context, arg ...sqlite3.Value) {
blob, err := getAuxBlob(ctx, arg, false)
if err != nil {
ctx.ResultError(err)
return
return // notest
}
_, err = blob.Seek(arg[4].Int64(), io.SeekStart)
if err != nil {
ctx.ResultError(err)
return
return // notest
}
n := arg[5].Int64()
if n <= 0 {
return
if p, ok := arg[5].Pointer().(io.Writer); ok {
var n int64
n, err = blob.WriteTo(p)
ctx.ResultInt64(n)
} else {
n := arg[5].Int64()
if n <= 0 {
return
}
buf := make([]byte, n)
_, err = io.ReadFull(blob, buf)
ctx.ResultBlob(buf)
}
buf := make([]byte, n)
_, err = io.ReadFull(blob, buf)
if err != nil {
ctx.ResultError(err)
return
return // notest
}
ctx.ResultBlob(buf)
setAuxBlob(ctx, blob, false)
}
@@ -71,19 +81,25 @@ func writeblob(ctx sqlite3.Context, arg ...sqlite3.Value) {
blob, err := getAuxBlob(ctx, arg, true)
if err != nil {
ctx.ResultError(err)
return
return // notest
}
_, err = blob.Seek(arg[4].Int64(), io.SeekStart)
if err != nil {
ctx.ResultError(err)
return
return // notest
}
_, err = blob.Write(arg[5].RawBlob())
if p, ok := arg[5].Pointer().(io.Reader); ok {
var n int64
n, err = blob.ReadFrom(p)
ctx.ResultInt64(n)
} else {
_, err = blob.Write(arg[5].RawBlob())
}
if err != nil {
ctx.ResultError(err)
return
return // notest
}
setAuxBlob(ctx, blob, false)
@@ -98,14 +114,14 @@ func openblob(ctx sqlite3.Context, arg ...sqlite3.Value) {
blob, err := getAuxBlob(ctx, arg, arg[4].Bool())
if err != nil {
ctx.ResultError(err)
return
return // notest
}
fn := arg[5].Pointer().(OpenCallback)
err = fn(blob, arg[6:]...)
if err != nil {
ctx.ResultError(err)
return
return // notest
}
setAuxBlob(ctx, blob, true)

View File

@@ -5,6 +5,7 @@ import (
"log"
"os"
"reflect"
"strings"
"testing"
"github.com/ncruces/go-sqlite3"
@@ -18,10 +19,7 @@ import (
func Example() {
// Open the database, registering the extension.
db, err := driver.Open("file:/test.db?vfs=memdb", func(conn *sqlite3.Conn) error {
blobio.Register(conn)
return nil
})
db, err := driver.Open("file:/test.db?vfs=memdb", blobio.Register)
if err != nil {
log.Fatal(err)
@@ -36,23 +34,26 @@ func Example() {
const message = "Hello BLOB!"
// Create the BLOB.
_, err = db.Exec(`INSERT INTO test VALUES (?)`, sqlite3.ZeroBlob(len(message)))
r, err := db.Exec(`INSERT INTO test VALUES (?)`, sqlite3.ZeroBlob(len(message)))
if err != nil {
log.Fatal(err)
}
id, err := r.LastInsertId()
if err != nil {
log.Fatal(err)
}
// Write the BLOB.
_, err = db.Exec(`SELECT writeblob('main', 'test', 'col', last_insert_rowid(), 0, ?)`, message)
_, err = db.Exec(`SELECT writeblob('main', 'test', 'col', ?, 0, ?)`,
id, message)
if err != nil {
log.Fatal(err)
}
// Read the BLOB.
_, err = db.Exec(`SELECT openblob('main', 'test', 'col', rowid, false, ?) FROM test`,
sqlite3.Pointer[blobio.OpenCallback](func(blob *sqlite3.Blob, _ ...sqlite3.Value) error {
_, err = io.Copy(os.Stdout, blob)
return err
}))
_, err = db.Exec(`SELECT readblob('main', 'test', 'col', ?, 0, ?)`,
id, sqlite3.Pointer(os.Stdout))
if err != nil {
log.Fatal(err)
}
@@ -60,6 +61,12 @@ func Example() {
// Hello BLOB!
}
func TestMain(m *testing.M) {
sqlite3.AutoExtension(blobio.Register)
sqlite3.AutoExtension(array.Register)
m.Run()
}
func Test_readblob(t *testing.T) {
t.Parallel()
@@ -69,9 +76,6 @@ func Test_readblob(t *testing.T) {
}
defer db.Close()
blobio.Register(db)
array.Register(db)
err = db.Exec(`SELECT readblob()`)
if err == nil {
t.Fatal("want error")
@@ -79,44 +83,135 @@ func Test_readblob(t *testing.T) {
t.Log(err)
}
err = db.Exec(`SELECT readblob('main', 'test1', 'col', 1, 1, 1)`)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
err = db.Exec(`
CREATE TABLE test1 (col);
CREATE TABLE test2 (col);
INSERT INTO test1 VALUES (x'cafe');
INSERT INTO test1 VALUES (x'dead');
INSERT INTO test2 VALUES (x'babe');
`)
if err != nil {
t.Fatal(err)
}
stmt, _, err := db.Prepare(`SELECT readblob('main', value, 'col', 1, 1, 1) FROM array(?)`)
err = db.Exec(`SELECT readblob('main', 'test1', 'col', 1, -1, 1)`)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
err = db.Exec(`SELECT readblob('main', 'test1', 'col', 1, 1, 0)`)
if err != nil {
t.Log(err)
}
tests := []struct {
name string
sql string
want1 string
want2 string
}{
{"rows", `SELECT readblob('main', 'test1', 'col', rowid, 1, 1) FROM test1`, "\xfe", "\xad"},
{"tables", `SELECT readblob('main', value, 'col', 1, 1, 1) FROM array(?)`, "\xfe", "\xbe"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
stmt, _, err := db.Prepare(tt.sql)
if err != nil {
t.Fatal(err)
}
defer stmt.Close()
if stmt.BindCount() == 1 {
err = stmt.BindPointer(1, []string{"test1", "test2"})
if err != nil {
t.Fatal(err)
}
}
if stmt.Step() {
got := stmt.ColumnText(0)
if got != tt.want1 {
t.Errorf("got %q", got)
}
}
if stmt.Step() {
got := stmt.ColumnText(0)
if got != tt.want2 {
t.Errorf("got %q", got)
}
}
err = stmt.Err()
if err != nil {
t.Fatal(err)
}
})
}
}
func Test_writeblob(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
err = db.Exec(`SELECT writeblob()`)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
err = db.Exec(`SELECT writeblob('main', 'test', 'col', 1, 1, x'')`)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
err = db.Exec(`
CREATE TABLE test (col);
INSERT INTO test VALUES (x'cafe');
`)
if err != nil {
t.Fatal(err)
}
err = db.Exec(`SELECT writeblob('main', 'test', 'col', 1, -1, x'')`)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
stmt, _, err := db.Prepare(`SELECT writeblob('main', 'test', 'col', 1, 0, ?)`)
if err != nil {
t.Log(err)
}
defer stmt.Close()
err = stmt.BindPointer(1, []string{"test1", "test2"})
err = stmt.BindPointer(1, strings.NewReader("\xba\xbe"))
if err != nil {
t.Fatal(err)
t.Log(err)
}
if stmt.Step() {
got := stmt.ColumnText(0)
if got != "\xfe" {
t.Errorf("got %q", got)
}
}
if stmt.Step() {
got := stmt.ColumnText(0)
if got != "\xbe" {
t.Errorf("got %q", got)
}
}
err = stmt.Err()
err = stmt.Exec()
if err != nil {
t.Fatal(err)
t.Log(err)
}
}
@@ -129,9 +224,6 @@ func Test_openblob(t *testing.T) {
}
defer db.Close()
blobio.Register(db)
array.Register(db)
err = db.Exec(`SELECT openblob()`)
if err == nil {
t.Fatal("want error")
@@ -139,6 +231,13 @@ func Test_openblob(t *testing.T) {
t.Log(err)
}
err = db.Exec(`SELECT openblob('main', 'test1', 'col', 1, false, NULL)`)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
err = db.Exec(`
CREATE TABLE test1 (col);
CREATE TABLE test2 (col);

View File

@@ -7,21 +7,22 @@
package bloom
import (
"errors"
"fmt"
"io"
"math"
"strconv"
"github.com/dchest/siphash"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers the bloom_filter virtual table:
//
// CREATE VIRTUAL TABLE foo USING bloom_filter(nElements, falseProb, kHashes)
func Register(db *sqlite3.Conn) {
sqlite3.CreateModule(db, "bloom_filter", create, connect)
func Register(db *sqlite3.Conn) error {
return sqlite3.CreateModule(db, "bloom_filter", create, connect)
}
type bloom struct {
@@ -34,7 +35,7 @@ type bloom struct {
}
func create(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom, err error) {
t := bloom{
b := bloom{
db: db,
schema: schema,
storage: table + "_storage",
@@ -47,41 +48,47 @@ func create(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom,
return nil, err
}
if nelem <= 0 {
return nil, errors.New("bloom: number of elements in filter must be positive")
return nil, util.ErrorString("bloom: number of elements in filter must be positive")
}
} else {
nelem = 100
}
if len(arg) > 1 {
t.prob, err = strconv.ParseFloat(arg[1], 64)
b.prob, err = strconv.ParseFloat(arg[1], 64)
if err != nil {
return nil, err
}
if t.prob <= 0 || t.prob >= 1 {
return nil, errors.New("bloom: probability must be in the range (0,1)")
if b.prob <= 0 || b.prob >= 1 {
return nil, util.ErrorString("bloom: probability must be in the range (0,1)")
}
} else {
t.prob = 0.01
b.prob = 0.01
}
if len(arg) > 2 {
t.hashes, err = strconv.Atoi(arg[2])
b.hashes, err = strconv.Atoi(arg[2])
if err != nil {
return nil, err
}
if t.hashes <= 0 {
return nil, errors.New("bloom: number of hash functions must be positive")
if b.hashes <= 0 {
return nil, util.ErrorString("bloom: number of hash functions must be positive")
}
} else {
t.hashes = max(1, numHashes(t.prob))
b.hashes = max(1, numHashes(b.prob))
}
t.bytes = numBytes(nelem, t.prob)
b.bytes = numBytes(nelem, b.prob)
err = db.DeclareVTab(
`CREATE TABLE x(present, word HIDDEN NOT NULL PRIMARY KEY) WITHOUT ROWID`)
if err != nil {
return nil, err
}
err = db.Exec(fmt.Sprintf(
`CREATE TABLE %s.%s (data BLOB, p REAL, n INTEGER, m INTEGER, k INTEGER)`,
sqlite3.QuoteIdentifier(t.schema), sqlite3.QuoteIdentifier(t.storage)))
sqlite3.QuoteIdentifier(b.schema), sqlite3.QuoteIdentifier(b.storage)))
if err != nil {
return nil, err
}
@@ -92,23 +99,17 @@ func create(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom,
err = db.Exec(fmt.Sprintf(
`INSERT INTO %s.%s (rowid, data, p, n, m, k)
VALUES (1, zeroblob(%d), %f, %d, %d, %d)`,
sqlite3.QuoteIdentifier(t.schema), sqlite3.QuoteIdentifier(t.storage),
t.bytes, t.prob, nelem, 8*t.bytes, t.hashes))
sqlite3.QuoteIdentifier(b.schema), sqlite3.QuoteIdentifier(b.storage),
b.bytes, b.prob, nelem, 8*b.bytes, b.hashes))
if err != nil {
b.Destroy()
return nil, err
}
err = db.DeclareVTab(
`CREATE TABLE x(present, word HIDDEN NOT NULL PRIMARY KEY) WITHOUT ROWID`)
if err != nil {
t.Destroy()
return nil, err
}
return &t, nil
return &b, nil
}
func connect(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom, err error) {
t := bloom{
b := bloom{
db: db,
schema: schema,
storage: table + "_storage",
@@ -122,23 +123,23 @@ func connect(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom
load, _, err := db.Prepare(fmt.Sprintf(
`SELECT m/8, p, k FROM %s.%s WHERE rowid = 1`,
sqlite3.QuoteIdentifier(t.schema), sqlite3.QuoteIdentifier(t.storage)))
sqlite3.QuoteIdentifier(b.schema), sqlite3.QuoteIdentifier(b.storage)))
if err != nil {
return nil, err
}
defer load.Close()
if !load.Step() {
if err = load.Err(); err == nil {
err = sqlite3.CORRUPT_VTAB
if err := load.Err(); err != nil {
return nil, err
}
return nil, err
return nil, sqlite3.CORRUPT_VTAB
}
t.bytes = load.ColumnInt64(0)
t.prob = load.ColumnFloat(1)
t.hashes = load.ColumnInt(2)
return &t, nil
b.bytes = load.ColumnInt64(0)
b.prob = load.ColumnFloat(1)
b.hashes = load.ColumnInt(2)
return &b, nil
}
func (b *bloom) Destroy() error {
@@ -160,7 +161,9 @@ func (b *bloom) Rename(new string) error {
return err
}
func (t *bloom) ShadowTables() {}
func (t *bloom) ShadowTables() {
// notest // not meant to be called
}
func (t *bloom) Integrity(schema, table string, flags int) error {
load, _, err := t.db.Prepare(fmt.Sprintf(
@@ -171,7 +174,7 @@ func (t *bloom) Integrity(schema, table string, flags int) error {
}
defer load.Close()
err = errors.New("bloom: invalid parameters")
err = util.ErrorString("bloom: invalid parameters")
if !load.Step() {
return err
}
@@ -196,10 +199,10 @@ func (t *bloom) Integrity(schema, table string, flags int) error {
}
func (b *bloom) BestIndex(idx *sqlite3.IndexInfo) error {
for n, cst := range idx.Constraint {
for i, cst := range idx.Constraint {
if cst.Usable && cst.Column == 1 &&
cst.Op == sqlite3.INDEX_CONSTRAINT_EQ {
idx.ConstraintUsage[n].ArgvIndex = 1
idx.ConstraintUsage[i].ArgvIndex = 1
idx.OrderByConsumed = true
idx.EstimatedRows = 1
idx.EstimatedCost = float64(b.hashes)
@@ -213,11 +216,14 @@ func (b *bloom) BestIndex(idx *sqlite3.IndexInfo) error {
func (b *bloom) Update(arg ...sqlite3.Value) (rowid int64, err error) {
if arg[0].Type() != sqlite3.NULL {
if len(arg) == 1 {
return 0, errors.New("bloom: elements cannot be deleted")
return 0, util.ErrorString("bloom: elements cannot be deleted")
}
return 0, errors.New("bloom: elements cannot be updated")
return 0, util.ErrorString("bloom: elements cannot be updated")
}
if arg[2].NoChange() {
return 0, nil
}
blob := arg[2].RawBlob()
f, err := b.db.OpenBlob(b.schema, b.storage, "data", 1, true)
@@ -262,15 +268,11 @@ func (b *bloom) Open() (sqlite3.VTabCursor, error) {
type cursor struct {
*bloom
eof bool
arg *sqlite3.Value
eof bool
}
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
if len(arg) != 1 {
return nil
}
c.eof = false
c.arg = &arg[0]
blob := arg[0].RawBlob()
@@ -302,7 +304,10 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
return nil
}
func (c *cursor) Column(ctx *sqlite3.Context, n int) error {
func (c *cursor) Column(ctx sqlite3.Context, n int) error {
if ctx.VTabNoChange() {
return nil
}
switch n {
case 0:
ctx.ResultBool(true)
@@ -322,6 +327,7 @@ func (c *cursor) EOF() bool {
}
func (c *cursor) RowID() (int64, error) {
// notest // WITHOUT ROWID
return 0, nil
}

View File

@@ -12,6 +12,11 @@ import (
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
)
func TestMain(m *testing.M) {
sqlite3.AutoExtension(bloom.Register)
m.Run()
}
func TestRegister(t *testing.T) {
t.Parallel()
@@ -21,10 +26,8 @@ func TestRegister(t *testing.T) {
}
defer db.Close()
bloom.Register(db)
err = db.Exec(`
CREATE VIRTUAL TABLE sports_cars USING bloom_filter(20);
CREATE VIRTUAL TABLE sports_cars USING bloom_filter();
INSERT INTO sports_cars VALUES ('ferrari'), ('lamborghini'), ('alfa romeo')
`)
if err != nil {
@@ -66,7 +69,22 @@ func TestRegister(t *testing.T) {
t.Fatal(err)
}
err = db.Exec(`DROP TABLE sports_cars`)
err = db.Exec(`DELETE FROM sports_cars WHERE word = 'lamborghini'`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`UPDATE sports_cars SET word = 'ferrari' WHERE word = 'lamborghini'`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`ALTER TABLE sports_cars RENAME TO fast_cars`)
if err != nil {
t.Fatal(err)
}
err = db.Exec(`DROP TABLE fast_cars`)
if err != nil {
t.Fatal(err)
}
@@ -90,8 +108,6 @@ func Test_compatible(t *testing.T) {
}
defer db.Close()
bloom.Register(db)
query, _, err := db.Prepare(`SELECT COUNT(*) FROM plants(?)`)
if err != nil {
t.Fatal(err)
@@ -138,3 +154,42 @@ func Test_compatible(t *testing.T) {
t.Error(err)
}
}
func Test_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
bloom.Register(db)
err = db.Exec(`CREATE VIRTUAL TABLE sports_cars USING bloom_filter(0)`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`CREATE VIRTUAL TABLE sports_cars USING bloom_filter('a')`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`CREATE VIRTUAL TABLE sports_cars USING bloom_filter(20, 2)`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`CREATE VIRTUAL TABLE sports_cars USING bloom_filter(20, 'a')`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`CREATE VIRTUAL TABLE sports_cars USING bloom_filter(20, 0.9, 0)`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`CREATE VIRTUAL TABLE sports_cars USING bloom_filter(20, 0.9, 'a')`)
if err == nil {
t.Error("want error")
}
}

Binary file not shown.

264
ext/closure/closure.go Normal file
View File

@@ -0,0 +1,264 @@
// Package closure provides a transitive closure virtual table.
//
// The transitive_closure virtual table finds the transitive closure of
// a parent/child relationship in a real table.
//
// https://sqlite.org/src/doc/tip/ext/misc/closure.c
package closure
import (
"fmt"
"math"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/sql3util"
)
const (
_COL_ID = 0
_COL_DEPTH = 1
_COL_ROOT = 2
_COL_TABLENAME = 3
_COL_IDCOLUMN = 4
_COL_PARENTCOLUMN = 5
)
// Register registers the transitive_closure virtual table:
//
// CREATE VIRTUAL TABLE temp.closure USING transitive_closure;
func Register(db *sqlite3.Conn) error {
return sqlite3.CreateModule(db, "transitive_closure", nil,
func(db *sqlite3.Conn, _, _, _ string, arg ...string) (*closure, error) {
var (
table string
column string
parent string
done = util.Set[string]{}
)
for _, arg := range arg {
key, val := sql3util.NamedArg(arg)
if done.Contains(key) {
return nil, fmt.Errorf("transitive_closure: more than one %q parameter", key)
}
switch key {
case "tablename":
table = sql3util.Unquote(val)
case "idcolumn":
column = sql3util.Unquote(val)
case "parentcolumn":
parent = sql3util.Unquote(val)
default:
return nil, fmt.Errorf("transitive_closure: unknown %q parameter", key)
}
done.Add(key)
}
err := db.DeclareVTab(`CREATE TABLE x(id,depth,root HIDDEN,tablename HIDDEN,idcolumn HIDDEN,parentcolumn HIDDEN)`)
if err != nil {
return nil, err
}
return &closure{
db: db,
table: table,
column: column,
parent: parent,
}, nil
})
}
type closure struct {
db *sqlite3.Conn
table string
column string
parent string
}
func (c *closure) Destroy() error { return nil }
func (c *closure) BestIndex(idx *sqlite3.IndexInfo) error {
plan := 0
posi := 1
cost := 1e7
for i, cst := range idx.Constraint {
if !cst.Usable {
continue
}
if plan&1 == 0 && cst.Column == _COL_ROOT {
switch cst.Op {
case sqlite3.INDEX_CONSTRAINT_EQ:
plan |= 1
cost /= 100
idx.ConstraintUsage[i] = sqlite3.IndexConstraintUsage{
ArgvIndex: 1,
Omit: true,
}
}
continue
}
if plan&0xf0 == 0 && cst.Column == _COL_DEPTH {
switch cst.Op {
case sqlite3.INDEX_CONSTRAINT_LT, sqlite3.INDEX_CONSTRAINT_LE, sqlite3.INDEX_CONSTRAINT_EQ:
plan |= posi << 4
cost /= 5
posi += 1
idx.ConstraintUsage[i].ArgvIndex = posi
if cst.Op == sqlite3.INDEX_CONSTRAINT_LT {
plan |= 2
}
}
continue
}
if plan&0xf00 == 0 && cst.Column == _COL_TABLENAME {
switch cst.Op {
case sqlite3.INDEX_CONSTRAINT_EQ:
plan |= posi << 8
cost /= 5
posi += 1
idx.ConstraintUsage[i] = sqlite3.IndexConstraintUsage{
ArgvIndex: posi,
Omit: true,
}
}
continue
}
if plan&0xf000 == 0 && cst.Column == _COL_IDCOLUMN {
switch cst.Op {
case sqlite3.INDEX_CONSTRAINT_EQ:
plan |= posi << 12
posi += 1
idx.ConstraintUsage[i] = sqlite3.IndexConstraintUsage{
ArgvIndex: posi,
Omit: true,
}
}
continue
}
if plan&0xf0000 == 0 && cst.Column == _COL_PARENTCOLUMN {
switch cst.Op {
case sqlite3.INDEX_CONSTRAINT_EQ:
plan |= posi << 16
posi += 1
idx.ConstraintUsage[i] = sqlite3.IndexConstraintUsage{
ArgvIndex: posi,
Omit: true,
}
}
continue
}
}
if plan&1 == 0 ||
c.table == "" && plan&0xf00 == 0 ||
c.column == "" && plan&0xf000 == 0 ||
c.parent == "" && plan&0xf0000 == 0 {
return sqlite3.CONSTRAINT
}
idx.EstimatedCost = cost
idx.IdxNum = plan
return nil
}
func (c *closure) Open() (sqlite3.VTabCursor, error) {
return &cursor{closure: c}, nil
}
type cursor struct {
*closure
nodes []node
}
type node struct {
id int64
depth int
}
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
root := arg[0].Int64()
maxDepth := math.MaxInt
if idxNum&0xf0 != 0 {
maxDepth = arg[(idxNum>>4)&0xf].Int()
if idxNum&2 != 0 {
maxDepth -= 1
}
}
table := c.table
if idxNum&0xf00 != 0 {
table = arg[(idxNum>>8)&0xf].Text()
}
column := c.column
if idxNum&0xf000 != 0 {
column = arg[(idxNum>>12)&0xf].Text()
}
parent := c.parent
if idxNum&0xf0000 != 0 {
parent = arg[(idxNum>>16)&0xf].Text()
}
sql := fmt.Sprintf(
`SELECT %[1]s.%[2]s FROM %[1]s WHERE %[1]s.%[3]s=?`,
sqlite3.QuoteIdentifier(table),
sqlite3.QuoteIdentifier(column),
sqlite3.QuoteIdentifier(parent),
)
stmt, _, err := c.db.Prepare(sql)
if err != nil {
return err
}
defer stmt.Close()
c.nodes = []node{{root, 0}}
set := util.Set[int64]{}
set.Add(root)
for i := 0; i < len(c.nodes); i++ {
curr := c.nodes[i]
if curr.depth >= maxDepth {
continue
}
stmt.BindInt64(1, curr.id)
for stmt.Step() {
if stmt.ColumnType(0) == sqlite3.INTEGER {
next := stmt.ColumnInt64(0)
if !set.Contains(next) {
set.Add(next)
c.nodes = append(c.nodes, node{next, curr.depth + 1})
}
}
}
stmt.Reset()
}
return nil
}
func (c *cursor) Column(ctx sqlite3.Context, n int) error {
switch n {
case _COL_ID:
ctx.ResultInt64(c.nodes[0].id)
case _COL_DEPTH:
ctx.ResultInt(c.nodes[0].depth)
case _COL_TABLENAME:
ctx.ResultText(c.table)
case _COL_IDCOLUMN:
ctx.ResultText(c.column)
case _COL_PARENTCOLUMN:
ctx.ResultText(c.parent)
}
return nil
}
func (c *cursor) Next() error {
c.nodes = c.nodes[1:]
return nil
}
func (c *cursor) EOF() bool {
return len(c.nodes) == 0
}
func (c *cursor) RowID() (int64, error) {
return c.nodes[0].id, nil
}

184
ext/closure/closure_test.go Normal file
View File

@@ -0,0 +1,184 @@
package closure_test
import (
_ "embed"
"fmt"
"log"
"testing"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/closure"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
)
func TestMain(m *testing.M) {
sqlite3.AutoExtension(closure.Register)
m.Run()
}
func Example() {
db, err := sqlite3.Open(":memory:")
if err != nil {
log.Fatal(err)
}
defer db.Close()
closure.Register(db)
err = db.Exec(`
CREATE TABLE employees (
id INTEGER PRIMARY KEY,
parent_id INTEGER,
name TEXT
);
CREATE INDEX employees_parent_idx ON employees(parent_id);
INSERT INTO employees (id, parent_id, name) VALUES
(11, NULL, 'Diane'),
(12, 11, 'Bob'),
(21, 11, 'Emma'),
(22, 21, 'Grace'),
(23, 21, 'Henry'),
(24, 21, 'Irene'),
(25, 21, 'Frank'),
(31, 11, 'Cindy'),
(32, 31, 'Dave'),
(33, 31, 'Alice');
CREATE VIRTUAL TABLE hierarchy USING transitive_closure(
tablename = "employees",
idcolumn = "id",
parentcolumn = "parent_id"
);
`)
if err != nil {
log.Fatal(err)
}
stmt, _, err := db.Prepare(`
SELECT employees.id, name FROM employees, hierarchy
WHERE employees.id = hierarchy.id AND hierarchy.root = 31
`)
if err != nil {
log.Fatal(err)
}
defer stmt.Close()
for stmt.Step() {
fmt.Println(stmt.ColumnInt(0), stmt.ColumnText(1))
}
if err := stmt.Err(); err != nil {
log.Fatal(err)
}
err = stmt.Close()
if err != nil {
log.Fatal(err)
}
err = db.Close()
if err != nil {
log.Fatal(err)
}
// Output:
// 31 Cindy
// 32 Dave
// 33 Alice
}
func TestRegister(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
err = db.Exec(`
CREATE TABLE employees (
id INTEGER PRIMARY KEY,
parent_id INTEGER,
name TEXT
);
CREATE INDEX employees_parent_idx ON employees(parent_id);
INSERT INTO employees (id, parent_id, name) VALUES
(11, NULL, 'Diane'),
(12, 11, 'Bob'),
(21, 11, 'Emma'),
(22, 21, 'Grace'),
(23, 21, 'Henry'),
(24, 21, 'Irene'),
(25, 21, 'Frank'),
(31, 11, 'Cindy'),
(32, 31, 'Dave'),
(33, 31, 'Alice');
CREATE VIRTUAL TABLE temp.closure USING transitive_closure;
`)
if err != nil {
t.Fatal(err)
}
stmt, _, err := db.Prepare(`
SELECT employees.id, name FROM employees, closure
WHERE employees.id = closure.id
AND closure.root = 31
AND closure.depth < 1
AND closure.tablename='employees'
AND closure.idcolumn='id'
AND closure.parentcolumn='parent_id'
`)
if err != nil {
t.Fatal(err)
}
defer stmt.Close()
if !stmt.Step() {
t.Error("want row")
}
if stmt.Step() {
t.Error("don't want row")
}
if err := stmt.Err(); err != nil {
t.Fatal(err)
}
err = stmt.Close()
if err != nil {
t.Fatal(err)
}
err = db.Close()
if err != nil {
t.Fatal(err)
}
}
func Test_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
err = db.Exec(`CREATE VIRTUAL TABLE hierarchy USING transitive_closure(table='employees')`)
if err == nil {
t.Error("want error")
}
err = db.Exec(`CREATE VIRTUAL TABLE hierarchy USING transitive_closure(tablename='employees', tablename="employees")`)
if err == nil {
t.Error("want error")
}
err = db.Exec("CREATE VIRTUAL TABLE hierarchy USING transitive_closure(tablename=`employees`)")
if err != nil {
t.Error(err)
}
err = db.Exec(`SELECT * FROM hierarchy`)
if err == nil {
t.Error("want error")
}
}

View File

@@ -4,8 +4,7 @@ import (
"fmt"
"strconv"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/vtabutil"
"github.com/ncruces/go-sqlite3/util/sql3util"
)
func uintArg(key, val string) (int, error) {
@@ -20,7 +19,7 @@ func boolArg(key, val string) (bool, error) {
if val == "" {
return true, nil
}
b, ok := util.ParseBool(val)
b, ok := sql3util.ParseBool(val)
if ok {
return b, nil
}
@@ -28,7 +27,7 @@ func boolArg(key, val string) (bool, error) {
}
func runeArg(key, val string) (rune, error) {
r, _, tail, err := strconv.UnquoteChar(vtabutil.Unquote(val), 0)
r, _, tail, err := strconv.UnquoteChar(sql3util.Unquote(val), 0)
if tail != "" || err != nil {
return 0, fmt.Errorf("csv: invalid %q parameter: %s", key, val)
}

View File

@@ -3,7 +3,7 @@ package csv
import (
"testing"
"github.com/ncruces/go-sqlite3/util/vtabutil"
"github.com/ncruces/go-sqlite3/util/sql3util"
)
func Test_uintArg(t *testing.T) {
@@ -24,7 +24,7 @@ func Test_uintArg(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.arg, func(t *testing.T) {
key, val := vtabutil.NamedArg(tt.arg)
key, val := sql3util.NamedArg(tt.arg)
if key != tt.key {
t.Errorf("NamedArg() %v, want err %v", key, tt.key)
}
@@ -62,7 +62,7 @@ func Test_boolArg(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.arg, func(t *testing.T) {
key, val := vtabutil.NamedArg(tt.arg)
key, val := sql3util.NamedArg(tt.arg)
if key != tt.key {
t.Errorf("NamedArg() %v, want err %v", key, tt.key)
}
@@ -96,7 +96,7 @@ func Test_runeArg(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.arg, func(t *testing.T) {
key, val := vtabutil.NamedArg(tt.arg)
key, val := sql3util.NamedArg(tt.arg)
if key != tt.key {
t.Errorf("NamedArg() %v, want err %v", key, tt.key)
}

View File

@@ -9,7 +9,6 @@ package csv
import (
"bufio"
"encoding/csv"
"errors"
"fmt"
"io"
"io/fs"
@@ -17,20 +16,21 @@ import (
"strings"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/osutil"
"github.com/ncruces/go-sqlite3/util/vtabutil"
"github.com/ncruces/go-sqlite3/util/sql3util"
)
// Register registers the CSV virtual table.
// If a filename is specified, [os.Open] is used to open the file.
func Register(db *sqlite3.Conn) {
RegisterFS(db, osutil.FS{})
func Register(db *sqlite3.Conn) error {
return RegisterFS(db, osutil.FS{})
}
// RegisterFS registers the CSV virtual table.
// If a filename is specified, fsys is used to open the file.
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
declare := func(db *sqlite3.Conn, _, _, _ string, arg ...string) (_ *table, err error) {
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
declare := func(db *sqlite3.Conn, _, _, _ string, arg ...string) (res *table, err error) {
var (
filename string
data string
@@ -40,21 +40,21 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
comma rune = ','
comment rune
done = map[string]struct{}{}
done = util.Set[string]{}
)
for _, arg := range arg {
key, val := vtabutil.NamedArg(arg)
if _, ok := done[key]; ok {
key, val := sql3util.NamedArg(arg)
if done.Contains(key) {
return nil, fmt.Errorf("csv: more than one %q parameter", key)
}
switch key {
case "filename":
filename = vtabutil.Unquote(val)
filename = sql3util.Unquote(val)
case "data":
data = vtabutil.Unquote(val)
data = sql3util.Unquote(val)
case "schema":
schema = vtabutil.Unquote(val)
schema = sql3util.Unquote(val)
case "header":
header, err = boolArg(key, val)
case "columns":
@@ -69,14 +69,14 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
if err != nil {
return nil, err
}
done[key] = struct{}{}
done.Add(key)
}
if (filename == "") == (data == "") {
return nil, errors.New(`csv: must specify either "filename" or "data" but not both`)
return nil, util.ErrorString(`csv: must specify either "filename" or "data" but not both`)
}
table := &table{
t := &table{
fsys: fsys,
name: filename,
data: data,
@@ -88,7 +88,7 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
if schema == "" {
var row []string
if header || columns < 0 {
csv, c, err := table.newReader()
csv, c, err := t.newReader()
defer c.Close()
if err != nil {
return nil, err
@@ -100,25 +100,23 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
}
schema = getSchema(header, columns, row)
} else {
defer func() {
if err == nil {
table.typs, err = getColumnAffinities(schema)
}
}()
t.typs, err = getColumnAffinities(schema)
if err != nil {
return nil, err
}
}
err = db.DeclareVTab(schema)
if err == nil {
err = db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
}
if err != nil {
return nil, err
}
err = db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
if err != nil {
return nil, err
}
return table, nil
return t, nil
}
sqlite3.CreateModule(db, "csv", declare, declare)
return sqlite3.CreateModule(db, "csv", declare, declare)
}
type table struct {
@@ -239,7 +237,7 @@ func (c *cursor) RowID() (int64, error) {
return c.rowID, nil
}
func (c *cursor) Column(ctx *sqlite3.Context, col int) error {
func (c *cursor) Column(ctx sqlite3.Context, col int) error {
if col < len(c.row) {
typ := text
if col < len(c.table.typs) {

View File

@@ -18,7 +18,10 @@ func Example() {
}
defer db.Close()
csv.Register(db)
err = csv.Register(db)
if err != nil {
log.Fatal(err)
}
err = db.Exec(`
CREATE VIRTUAL TABLE eurofxref USING csv(
@@ -51,6 +54,11 @@ func Example() {
// On Twosday, 1€ = $1.1342
}
func TestMain(m *testing.M) {
sqlite3.AutoExtension(csv.Register)
m.Run()
}
func TestRegister(t *testing.T) {
t.Parallel()
@@ -60,8 +68,6 @@ func TestRegister(t *testing.T) {
}
defer db.Close()
csv.Register(db)
const data = `
# Comment
"Rob" "Pike" rob
@@ -124,8 +130,6 @@ func TestAffinity(t *testing.T) {
}
defer db.Close()
csv.Register(db)
const data = "01\n0.10\ne"
err = db.Exec(`
CREATE VIRTUAL TABLE temp.nums USING csv(
@@ -168,8 +172,6 @@ func TestRegister_errors(t *testing.T) {
}
defer db.Close()
csv.Register(db)
err = db.Exec(`CREATE VIRTUAL TABLE temp.users USING csv()`)
if err == nil {
t.Fatal("want error")

View File

@@ -1,10 +1,9 @@
package csv
import (
_ "embed"
"strings"
"github.com/ncruces/go-sqlite3/util/vtabutil"
"github.com/ncruces/go-sqlite3/util/sql3util"
)
type affinity byte
@@ -18,16 +17,15 @@ const (
)
func getColumnAffinities(schema string) ([]affinity, error) {
tab, err := vtabutil.Parse(schema)
tab, err := sql3util.ParseTable(schema)
if err != nil {
return nil, err
}
defer tab.Close()
types := make([]affinity, tab.NumColumns())
for i := range types {
col := tab.Column(i)
types[i] = getAffinity(col.Type())
columns := tab.Columns
types := make([]affinity, len(columns))
for i, col := range columns {
types[i] = getAffinity(col.Type)
}
return types, nil
}

View File

@@ -1,9 +1,6 @@
package csv
import (
_ "embed"
"testing"
)
import "testing"
func Test_getAffinity(t *testing.T) {
tests := []struct {

View File

@@ -1,3 +1,5 @@
//go:build !go1.23
package fileio
import (

View File

@@ -14,24 +14,28 @@ import (
// Register registers SQL functions readfile, writefile, lsmode,
// and the table-valued function fsdir.
func Register(db *sqlite3.Conn) {
RegisterFS(db, nil)
func Register(db *sqlite3.Conn) error {
return RegisterFS(db, nil)
}
// Register registers SQL functions readfile, lsmode,
// and the table-valued function fsdir;
// fsys will be used to read files and list directories.
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
db.CreateFunction("lsmode", 1, sqlite3.DETERMINISTIC, lsmode)
db.CreateFunction("readfile", 1, sqlite3.DIRECTONLY, readfile(fsys))
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
var err error
if fsys == nil {
db.CreateFunction("writefile", -1, sqlite3.DIRECTONLY, writefile)
err = db.CreateFunction("writefile", -1, sqlite3.DIRECTONLY, writefile)
}
sqlite3.CreateModule(db, "fsdir", nil, func(db *sqlite3.Conn, _, _, _ string, _ ...string) (fsdir, error) {
err := db.DeclareVTab(`CREATE TABLE x(name,mode,mtime TIMESTAMP,data,path HIDDEN,dir HIDDEN)`)
db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
return fsdir{fsys}, err
})
return errors.Join(err,
db.CreateFunction("readfile", 1, sqlite3.DIRECTONLY, readfile(fsys)),
db.CreateFunction("lsmode", 1, sqlite3.DETERMINISTIC, lsmode),
sqlite3.CreateModule(db, "fsdir", nil, func(db *sqlite3.Conn, _, _, _ string, _ ...string) (fsdir, error) {
err := db.DeclareVTab(`CREATE TABLE x(name,mode,mtime TIMESTAMP,data,path HIDDEN,dir HIDDEN)`)
if err == nil {
err = db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
}
return fsdir{fsys}, err
}))
}
func lsmode(ctx sqlite3.Context, arg ...sqlite3.Value) {
@@ -53,7 +57,7 @@ func readfile(fsys fs.FS) func(ctx sqlite3.Context, arg ...sqlite3.Value) {
case err == nil:
ctx.ResultBlob(data)
case !errors.Is(err, fs.ErrNotExist):
ctx.ResultError(fmt.Errorf("readfile: %w", err))
ctx.ResultError(fmt.Errorf("readfile: %w", err)) // notest
}
}
}

View File

@@ -12,15 +12,14 @@ import (
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/fileio"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Test_lsmode(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
fileio.Register(c)
return nil
})
db, err := driver.Open(tmp, fileio.Register)
if err != nil {
t.Fatal(err)
}
@@ -54,7 +53,9 @@ func Test_readfile(t *testing.T) {
for _, fsys := range []fs.FS{nil, os.DirFS(".")} {
t.Run("", func(t *testing.T) {
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
tmp := memdb.TestDB(t)
db, err := driver.Open(tmp, func(c *sqlite3.Conn) error {
fileio.RegisterFS(c, fsys)
return nil
})

View File

@@ -10,13 +10,22 @@ import (
"github.com/ncruces/go-sqlite3"
)
const (
_COL_NAME = 0
_COL_MODE = 1
_COL_TIME = 2
_COL_DATA = 3
_COL_ROOT = 4
_COL_BASE = 5
)
type fsdir struct{ fsys fs.FS }
func (d fsdir) BestIndex(idx *sqlite3.IndexInfo) error {
var root, base bool
for i, cst := range idx.Constraint {
switch cst.Column {
case 4: // root
case _COL_ROOT:
if !cst.Usable || cst.Op != sqlite3.INDEX_CONSTRAINT_EQ {
return sqlite3.CONSTRAINT
}
@@ -25,7 +34,7 @@ func (d fsdir) BestIndex(idx *sqlite3.IndexInfo) error {
ArgvIndex: 1,
}
root = true
case 5: // base
case _COL_BASE:
if !cst.Usable || cst.Op != sqlite3.INDEX_CONSTRAINT_EQ {
return sqlite3.CONSTRAINT
}
@@ -54,7 +63,7 @@ func (d fsdir) Open() (sqlite3.VTabCursor, error) {
type cursor struct {
fsdir
base string
resume func(struct{}) (entry, bool)
resume resume
cancel func()
curr entry
eof bool
@@ -92,25 +101,14 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
c.base = base
}
c.resume, c.cancel = coroNew(func(_ struct{}, yield func(entry) struct{}) entry {
walkDir := func(path string, d fs.DirEntry, err error) error {
yield(entry{d, err, path})
return nil
}
if c.fsys != nil {
fs.WalkDir(c.fsys, root, walkDir)
} else {
filepath.WalkDir(root, walkDir)
}
return entry{}
})
c.resume, c.cancel = pull(c, root)
c.eof = false
c.rowID = 0
return c.Next()
}
func (c *cursor) Next() error {
curr, ok := c.resume(struct{}{})
curr, ok := next(c)
c.curr = curr
c.eof = !ok
c.rowID++
@@ -125,27 +123,27 @@ func (c *cursor) RowID() (int64, error) {
return c.rowID, nil
}
func (c *cursor) Column(ctx *sqlite3.Context, n int) error {
func (c *cursor) Column(ctx sqlite3.Context, n int) error {
switch n {
case 0: // name
case _COL_NAME:
name := strings.TrimPrefix(c.curr.path, c.base)
ctx.ResultText(name)
case 1: // mode
case _COL_MODE:
i, err := c.curr.Info()
if err != nil {
return err
}
ctx.ResultInt64(int64(i.Mode()))
case 2: // mtime
case _COL_TIME:
i, err := c.curr.Info()
if err != nil {
return err
}
ctx.ResultTime(i.ModTime(), sqlite3.TimeFormatUnixFrac)
case 3: // data
case _COL_DATA:
switch typ := c.curr.Type(); {
case typ.IsRegular():
var data []byte

29
ext/fileio/fsdir_coro.go Normal file
View File

@@ -0,0 +1,29 @@
//go:build !go1.23
package fileio
import (
"io/fs"
"path/filepath"
)
type resume = func(struct{}) (entry, bool)
func next(c *cursor) (entry, bool) {
return c.resume(struct{}{})
}
func pull(c *cursor, root string) (resume, func()) {
return coroNew(func(_ struct{}, yield func(entry) struct{}) entry {
walkDir := func(path string, d fs.DirEntry, err error) error {
yield(entry{d, err, path})
return nil
}
if c.fsys != nil {
fs.WalkDir(c.fsys, root, walkDir)
} else {
filepath.WalkDir(root, walkDir)
}
return entry{}
})
}

31
ext/fileio/fsdir_iter.go Normal file
View File

@@ -0,0 +1,31 @@
//go:build go1.23
package fileio
import (
"io/fs"
"iter"
"path/filepath"
)
type resume = func() (entry, bool)
func next(c *cursor) (entry, bool) {
return c.resume()
}
func pull(c *cursor, root string) (resume, func()) {
return iter.Pull(func(yield func(entry) bool) {
walkDir := func(path string, d fs.DirEntry, err error) error {
if yield(entry{d, err, path}) {
return nil
}
return fs.SkipAll
}
if c.fsys != nil {
fs.WalkDir(c.fsys, root, walkDir)
} else {
filepath.WalkDir(root, walkDir)
}
})
}

View File

@@ -13,6 +13,7 @@ import (
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/fileio"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Test_fsdir(t *testing.T) {
@@ -20,7 +21,9 @@ func Test_fsdir(t *testing.T) {
for _, fsys := range []fs.FS{nil, os.DirFS(".")} {
t.Run("", func(t *testing.T) {
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
tmp := memdb.TestDB(t)
db, err := driver.Open(tmp, func(c *sqlite3.Conn) error {
fileio.RegisterFS(c, fsys)
return nil
})
@@ -68,7 +71,10 @@ func Test_fsdir_errors(t *testing.T) {
}
defer db.Close()
fileio.Register(db)
err = fileio.Register(db)
if err != nil {
t.Fatal(err)
}
err = db.Exec(`SELECT name FROM fsdir()`)
if err == nil {

View File

@@ -29,7 +29,7 @@ func writefile(ctx sqlite3.Context, arg ...sqlite3.Value) {
n, err := createFileAndDir(file, mode, arg[1])
if err != nil {
if len(arg) > 2 {
ctx.ResultError(fmt.Errorf("writefile: %w", err))
ctx.ResultError(fmt.Errorf("writefile: %w", err)) // notest
}
return
}
@@ -39,7 +39,7 @@ func writefile(ctx sqlite3.Context, arg ...sqlite3.Value) {
err := os.Chmod(file, mode.Perm())
if err != nil {
ctx.ResultError(fmt.Errorf("writefile: %w", err))
return
return // notest
}
}
@@ -48,7 +48,7 @@ func writefile(ctx sqlite3.Context, arg ...sqlite3.Value) {
err := os.Chtimes(file, time.Time{}, mtime)
if err != nil {
ctx.ResultError(fmt.Errorf("writefile: %w", err))
return
return // notest
}
}
}

View File

@@ -7,19 +7,17 @@ import (
"testing"
"time"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Test_writefile(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
Register(c)
return nil
})
db, err := driver.Open(tmp, Register)
if err != nil {
t.Fatal(err)
}

View File

@@ -21,47 +21,60 @@ package hash
import (
"crypto"
"errors"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers cryptographic hash functions for a database connection.
func Register(db *sqlite3.Conn) {
flags := sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
var errs util.ErrorJoiner
if crypto.MD4.Available() {
db.CreateFunction("md4", 1, flags, md4Func)
errs.Join(
db.CreateFunction("md4", 1, flags, md4Func))
}
if crypto.MD5.Available() {
db.CreateFunction("md5", 1, flags, md5Func)
errs.Join(
db.CreateFunction("md5", 1, flags, md5Func))
}
if crypto.SHA1.Available() {
db.CreateFunction("sha1", 1, flags, sha1Func)
errs.Join(
db.CreateFunction("sha1", 1, flags, sha1Func))
}
if crypto.SHA3_512.Available() {
db.CreateFunction("sha3", 1, flags, sha3Func)
db.CreateFunction("sha3", 2, flags, sha3Func)
errs.Join(
db.CreateFunction("sha3", 1, flags, sha3Func),
db.CreateFunction("sha3", 2, flags, sha3Func))
}
if crypto.SHA256.Available() {
db.CreateFunction("sha224", 1, flags, sha224Func)
db.CreateFunction("sha256", 1, flags, sha256Func)
db.CreateFunction("sha256", 2, flags, sha256Func)
errs.Join(
db.CreateFunction("sha224", 1, flags, sha224Func),
db.CreateFunction("sha256", 1, flags, sha256Func),
db.CreateFunction("sha256", 2, flags, sha256Func))
}
if crypto.SHA512.Available() {
db.CreateFunction("sha384", 1, flags, sha384Func)
db.CreateFunction("sha512", 1, flags, sha512Func)
db.CreateFunction("sha512", 2, flags, sha512Func)
errs.Join(
db.CreateFunction("sha384", 1, flags, sha384Func),
db.CreateFunction("sha512", 1, flags, sha512Func),
db.CreateFunction("sha512", 2, flags, sha512Func))
}
if crypto.BLAKE2s_256.Available() {
db.CreateFunction("blake2s", 1, flags, blake2sFunc)
errs.Join(
db.CreateFunction("blake2s", 1, flags, blake2sFunc))
}
if crypto.BLAKE2b_512.Available() {
db.CreateFunction("blake2b", 1, flags, blake2bFunc)
db.CreateFunction("blake2b", 2, flags, blake2bFunc)
errs.Join(
db.CreateFunction("blake2b", 1, flags, blake2bFunc),
db.CreateFunction("blake2b", 2, flags, blake2bFunc))
}
if crypto.RIPEMD160.Available() {
db.CreateFunction("ripemd160", 1, flags, ripemd160Func)
errs.Join(
db.CreateFunction("ripemd160", 1, flags, ripemd160Func))
}
return errors.Join(errs...)
}
func md4Func(ctx sqlite3.Context, arg ...sqlite3.Value) {

View File

@@ -7,19 +7,21 @@ import (
_ "crypto/sha512"
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
_ "golang.org/x/crypto/blake2b"
_ "golang.org/x/crypto/blake2s"
_ "golang.org/x/crypto/md4"
_ "golang.org/x/crypto/ripemd160"
_ "golang.org/x/crypto/sha3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func TestRegister(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
tests := []struct {
name string
@@ -53,10 +55,7 @@ func TestRegister(t *testing.T) {
{"blake2b('', 256)", "0E5751C026E543B2E8AB2EB06099DAA1D1E5DF47778F7787FAAB45CDF12FE3A8"},
}
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
Register(c)
return nil
})
db, err := driver.Open(tmp, Register)
if err != nil {
t.Fatal(err)
}

View File

@@ -13,6 +13,7 @@ package lines
import (
"bufio"
"bytes"
"errors"
"fmt"
"io"
"io/fs"
@@ -25,27 +26,32 @@ import (
// The lines function reads from a database blob or text.
// The lines_read function reads from a file or an [io.Reader].
// If a filename is specified, [os.Open] is used to open the file.
func Register(db *sqlite3.Conn) {
RegisterFS(db, osutil.FS{})
func Register(db *sqlite3.Conn) error {
return RegisterFS(db, osutil.FS{})
}
// RegisterFS registers the lines and lines_read table-valued functions.
// The lines function reads from a database blob or text.
// The lines_read function reads from a file or an [io.Reader].
// If a filename is specified, fsys is used to open the file.
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
sqlite3.CreateModule(db, "lines", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (lines, error) {
err := db.DeclareVTab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
db.VTabConfig(sqlite3.VTAB_INNOCUOUS)
return lines{}, err
})
sqlite3.CreateModule(db, "lines_read", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (lines, error) {
err := db.DeclareVTab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
return lines{fsys}, err
})
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
return errors.Join(
sqlite3.CreateModule(db, "lines", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (lines, error) {
err := db.DeclareVTab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
if err == nil {
err = db.VTabConfig(sqlite3.VTAB_INNOCUOUS)
}
return lines{}, err
}),
sqlite3.CreateModule(db, "lines_read", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (lines, error) {
err := db.DeclareVTab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
if err == nil {
err = db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
}
return lines{fsys}, err
}))
}
type lines struct {
@@ -89,7 +95,7 @@ func (c *cursor) RowID() (int64, error) {
return c.rowID, nil
}
func (c *cursor) Column(ctx *sqlite3.Context, n int) error {
func (c *cursor) Column(ctx sqlite3.Context, n int) error {
if n == 0 {
ctx.ResultRawText(c.line)
}

View File

@@ -15,13 +15,11 @@ import (
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/lines"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Example() {
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
lines.Register(c)
return nil
})
db, err := driver.Open("file:/test.db?vfs=memdb", lines.Register)
if err != nil {
log.Fatal(err)
}
@@ -69,11 +67,9 @@ func Example() {
func Test_lines(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
lines.Register(c)
return nil
})
db, err := driver.Open(tmp, lines.Register)
if err != nil {
log.Fatal(err)
}
@@ -102,11 +98,9 @@ func Test_lines(t *testing.T) {
func Test_lines_error(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
lines.Register(c)
return nil
})
db, err := driver.Open(tmp, lines.Register)
if err != nil {
log.Fatal(err)
}
@@ -129,11 +123,9 @@ func Test_lines_error(t *testing.T) {
func Test_lines_read(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
lines.Register(c)
return nil
})
db, err := driver.Open(tmp, lines.Register)
if err != nil {
log.Fatal(err)
}
@@ -163,11 +155,9 @@ func Test_lines_read(t *testing.T) {
func Test_lines_test(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
lines.Register(c)
return nil
})
db, err := driver.Open(tmp, lines.Register)
if err != nil {
log.Fatal(err)
}

View File

@@ -9,11 +9,12 @@ import (
"strings"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers the pivot virtual table.
func Register(db *sqlite3.Conn) {
sqlite3.CreateModule(db, "pivot", declare, declare)
func Register(db *sqlite3.Conn) error {
return sqlite3.CreateModule(db, "pivot", declare, declare)
}
type table struct {
@@ -24,15 +25,15 @@ type table struct {
cols []*sqlite3.Value
}
func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (_ *table, err error) {
func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (res *table, err error) {
if len(arg) != 3 {
return nil, fmt.Errorf("pivot: wrong number of arguments")
}
table := &table{db: db}
t := &table{db: db}
defer func() {
if err != nil {
table.Close()
if res == nil {
t.Close()
}
}()
@@ -41,17 +42,17 @@ func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (_ *table, err err
create.WriteString("CREATE TABLE x(")
// Row key query.
table.scan = "SELECT * FROM\n" + arg[0]
stmt, _, err := db.Prepare(table.scan)
t.scan = "SELECT * FROM\n" + arg[0]
stmt, _, err := db.Prepare(t.scan)
if err != nil {
return nil, err
}
defer stmt.Close()
table.keys = make([]string, stmt.ColumnCount())
for i := range table.keys {
t.keys = make([]string, stmt.ColumnCount())
for i := range t.keys {
name := sqlite3.QuoteIdentifier(stmt.ColumnName(i))
table.keys[i] = name
t.keys[i] = name
create.WriteString(sep)
create.WriteString(name)
sep = ","
@@ -65,28 +66,28 @@ func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (_ *table, err err
}
if stmt.ColumnCount() != 2 {
return nil, errors.New("pivot: column definition query expects 2 result columns")
return nil, util.ErrorString("pivot: column definition query expects 2 result columns")
}
for stmt.Step() {
name := sqlite3.QuoteIdentifier(stmt.ColumnText(1))
table.cols = append(table.cols, stmt.ColumnValue(0).Dup())
t.cols = append(t.cols, stmt.ColumnValue(0).Dup())
create.WriteString(",")
create.WriteString(name)
}
stmt.Close()
// Pivot cell query.
table.cell = "SELECT * FROM\n" + arg[2]
stmt, _, err = db.Prepare(table.cell)
t.cell = "SELECT * FROM\n" + arg[2]
stmt, _, err = db.Prepare(t.cell)
if err != nil {
return nil, err
}
if stmt.ColumnCount() != 1 {
return nil, errors.New("pivot: cell query expects 1 result columns")
return nil, util.ErrorString("pivot: cell query expects 1 result columns")
}
if stmt.BindCount() != len(table.keys)+1 {
return nil, fmt.Errorf("pivot: cell query expects %d bound parameters", len(table.keys)+1)
if stmt.BindCount() != len(t.keys)+1 {
return nil, fmt.Errorf("pivot: cell query expects %d bound parameters", len(t.keys)+1)
}
create.WriteByte(')')
@@ -94,12 +95,12 @@ func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (_ *table, err err
if err != nil {
return nil, err
}
return table, nil
return t, nil
}
func (t *table) Close() error {
for i := range t.cols {
t.cols[i].Close()
for _, c := range t.cols {
c.Close()
}
return nil
}
@@ -224,7 +225,7 @@ func (c *cursor) RowID() (int64, error) {
return c.rowID, nil
}
func (c *cursor) Column(ctx *sqlite3.Context, col int) error {
func (c *cursor) Column(ctx sqlite3.Context, col int) error {
count := c.scan.ColumnCount()
if col < count {
ctx.ResultValue(c.scan.ColumnValue(col))

View File

@@ -14,14 +14,14 @@ import (
// https://antonz.org/sqlite-pivot-table/
func Example() {
sqlite3.AutoExtension(pivot.Register)
db, err := sqlite3.Open(":memory:")
if err != nil {
log.Fatal(err)
}
defer db.Close()
pivot.Register(db)
err = db.Exec(`
CREATE TABLE sales(product TEXT, year INT, income DECIMAL);
INSERT INTO sales(product, year, income) VALUES
@@ -83,6 +83,11 @@ func Example() {
// gamma 80 75 78 80
}
func TestMain(m *testing.M) {
sqlite3.AutoExtension(pivot.Register)
m.Run()
}
func TestRegister(t *testing.T) {
t.Parallel()
@@ -92,8 +97,6 @@ func TestRegister(t *testing.T) {
}
defer db.Close()
pivot.Register(db)
err = db.Exec(`
CREATE TABLE r AS
SELECT 1 id UNION SELECT 2 UNION SELECT 3;
@@ -142,6 +145,11 @@ func TestRegister(t *testing.T) {
t.Errorf("got %d, want 3", got)
}
}
err = db.Exec(`ALTER TABLE v_x RENAME TO v_y`)
if err != nil {
t.Fatal(err)
}
}
func TestRegister_errors(t *testing.T) {
@@ -153,8 +161,6 @@ func TestRegister_errors(t *testing.T) {
}
defer db.Close()
pivot.Register(db)
err = db.Exec(`CREATE VIRTUAL TABLE pivot USING pivot()`)
if err == nil {
t.Fatal("want error")

240
ext/regexp/regexp.go Normal file
View File

@@ -0,0 +1,240 @@
// Package regexp provides additional regular expression functions.
//
// It provides the following Unicode aware functions:
// - regexp_like(text, pattern),
// - regexp_count(text, pattern [, start]),
// - regexp_instr(text, pattern [, start [, N [, endoption [, subexpr ]]]]),
// - regexp_substr(text, pattern [, start [, N [, subexpr ]]]),
// - regexp_replace(text, pattern, replacement [, start [, N ]]),
// - and a REGEXP operator.
//
// The implementation uses Go [regexp/syntax] for regular expressions.
//
// https://github.com/nalgeon/sqlean/blob/main/docs/regexp.md
package regexp
import (
"errors"
"regexp"
"strings"
"github.com/ncruces/go-sqlite3"
)
// Register registers Unicode aware functions for a database connection.
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
return errors.Join(
db.CreateFunction("regexp", 2, flags, regex),
db.CreateFunction("regexp_like", 2, flags, regexLike),
db.CreateFunction("regexp_count", 2, flags, regexCount),
db.CreateFunction("regexp_count", 3, flags, regexCount),
db.CreateFunction("regexp_instr", 2, flags, regexInstr),
db.CreateFunction("regexp_instr", 3, flags, regexInstr),
db.CreateFunction("regexp_instr", 4, flags, regexInstr),
db.CreateFunction("regexp_instr", 5, flags, regexInstr),
db.CreateFunction("regexp_instr", 6, flags, regexInstr),
db.CreateFunction("regexp_substr", 2, flags, regexSubstr),
db.CreateFunction("regexp_substr", 3, flags, regexSubstr),
db.CreateFunction("regexp_substr", 4, flags, regexSubstr),
db.CreateFunction("regexp_substr", 5, flags, regexSubstr),
db.CreateFunction("regexp_replace", 3, flags, regexReplace),
db.CreateFunction("regexp_replace", 4, flags, regexReplace),
db.CreateFunction("regexp_replace", 5, flags, regexReplace))
}
// GlobPrefix returns a GLOB for a regular expression
// appropriate to take advantage of the [LIKE optimization]
// in a query such as:
//
// SELECT column WHERE column GLOB :glob_prefix AND column REGEXP :regexp
//
// [LIKE optimization]: https://sqlite.org/optoverview.html#the_like_optimization
func GlobPrefix(re *regexp.Regexp) string {
prefix, complete := re.LiteralPrefix()
i := strings.IndexAny(prefix, "*?[")
if i < 0 {
if complete {
return prefix
}
i = len(prefix)
}
return prefix[:i] + "*"
}
func load(ctx sqlite3.Context, i int, expr string) (*regexp.Regexp, error) {
re, ok := ctx.GetAuxData(i).(*regexp.Regexp)
if !ok {
r, err := regexp.Compile(expr)
if err != nil {
return nil, err
}
re = r
ctx.SetAuxData(0, r)
}
return re, nil
}
func regex(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 0, arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[1].RawText()
ctx.ResultBool(re.Match(text))
}
func regexLike(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[0].RawText()
ctx.ResultBool(re.Match(text))
}
func regexCount(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[0].RawText()
if len(arg) > 2 {
pos := arg[2].Int()
text = text[skip(text, pos):]
}
ctx.ResultInt(len(re.FindAll(text, -1)))
}
func regexSubstr(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[0].RawText()
var pos, n, subexpr int
if len(arg) > 2 {
pos = arg[2].Int()
}
if len(arg) > 3 {
n = arg[3].Int()
}
if len(arg) > 4 {
subexpr = arg[4].Int()
}
loc := regexFind(re, text, pos, n, subexpr)
if loc != nil {
ctx.ResultRawText(text[loc[0]:loc[1]])
}
}
func regexInstr(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[0].RawText()
var pos, n, end, subexpr int
if len(arg) > 2 {
pos = arg[2].Int()
}
if len(arg) > 3 {
n = arg[3].Int()
}
if len(arg) > 4 && arg[4].Bool() {
end = 1
}
if len(arg) > 5 {
subexpr = arg[5].Int()
}
loc := regexFind(re, text, pos, n, subexpr)
if loc != nil {
ctx.ResultInt(loc[end] + 1)
}
}
func regexReplace(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[0].RawText()
repl := arg[2].RawText()
var pos, n int
if len(arg) > 3 {
pos = arg[3].Int()
}
if len(arg) > 4 {
n = arg[4].Int()
}
res := text
pos = skip(text, pos)
if n > 0 {
all := re.FindAllSubmatchIndex(text[pos:], n)
if n <= len(all) {
loc := all[n-1]
res = text[:pos+loc[0]]
res = re.Expand(res, repl, text[pos:], loc)
res = append(res, text[pos+loc[1]:]...)
}
} else {
res = append(text[:pos], re.ReplaceAll(text[pos:], repl)...)
}
ctx.ResultRawText(res)
}
func regexFind(re *regexp.Regexp, text []byte, pos, n, subexpr int) (loc []int) {
pos = skip(text, pos)
text = text[pos:]
if n <= 1 {
if subexpr == 0 {
loc = re.FindIndex(text)
} else {
loc = re.FindSubmatchIndex(text)
}
} else {
if subexpr == 0 {
all := re.FindAllIndex(text, n)
if n <= len(all) {
loc = all[n-1]
}
} else {
all := re.FindAllSubmatchIndex(text, n)
if n <= len(all) {
loc = all[n-1]
}
}
}
if 2+2*subexpr <= len(loc) {
loc = loc[2*subexpr : 2+2*subexpr]
loc[0] += pos
loc[1] += pos
return loc
}
return nil
}
func skip(text []byte, start int) int {
for pos := range string(text) {
if start--; start <= 0 {
return pos
}
}
return len(text)
}

126
ext/regexp/regexp_test.go Normal file
View File

@@ -0,0 +1,126 @@
package regexp
import (
"database/sql"
"regexp"
"testing"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func TestRegister(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(tmp, Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
tests := []struct {
test string
want string
}{
{`'Hello' REGEXP 'elo'`, "0"},
{`'Hello' REGEXP 'ell'`, "1"},
{`'Hello' REGEXP 'el.'`, "1"},
{`regexp_like('Hello', 'elo')`, "0"},
{`regexp_like('Hello', 'ell')`, "1"},
{`regexp_like('Hello', 'el.')`, "1"},
{`regexp_count('Hello', 'l')`, "2"},
{`regexp_instr('Hello', 'el.')`, "2"},
{`regexp_instr('Hello', '.', 6)`, ""},
{`regexp_substr('Hello', 'el.')`, "ell"},
{`regexp_replace('Hello', 'llo', 'll')`, "Hell"},
// https://www.postgresql.org/docs/current/functions-matching.html
{`regexp_count('ABCABCAXYaxy', 'A.')`, "3"},
{`regexp_count('ABCABCAXYaxy', '(?i)A.', 1)`, "4"},
{`regexp_instr('number of your street, town zip, FR', '[^,]+', 1, 2)`, "23"},
{`regexp_instr('ABCDEFGHI', '(?i)(c..)(...)', 1, 1, 0, 2)`, "6"},
{`regexp_substr('number of your street, town zip, FR', '[^,]+', 1, 2)`, " town zip"},
{`regexp_substr('ABCDEFGHI', '(?i)(c..)(...)', 1, 1, 2)`, "FGH"},
{`regexp_replace('foobarbaz', 'b..', 'X', 1, 1)`, "fooXbaz"},
{`regexp_replace('foobarbaz', 'b..', 'X')`, "fooXX"},
{`regexp_replace('foobarbaz', 'b(..)', 'X${1}Y')`, "fooXarYXazY"},
{`regexp_replace('A PostgreSQL function', '(?i)a|e|i|o|u', 'X', 1, 0)`, "X PXstgrXSQL fXnctXXn"},
{`regexp_replace('A PostgreSQL function', '(?i)a|e|i|o|u', 'X', 1, 3)`, "A PostgrXSQL function"},
// https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/REGEXP_COUNT.html
{`regexp_count('123123123123123', '(12)3', 1)`, "5"},
{`regexp_count('123123123123', '123', 3)`, "3"},
{`regexp_instr('500 Oracle Parkway, Redwood Shores, CA', '[^ ]+', 1, 6)`, "37"},
{`regexp_instr('500 Oracle Parkway, Redwood Shores, CA', '(?i)[s|r|p][[:alpha:]]{6}', 3, 2, 1)`, "28"},
{`regexp_instr('1234567890', '(123)(4(56)(78))', 1, 1, 0, 1)`, "1"},
{`regexp_instr('1234567890', '(123)(4(56)(78))', 1, 1, 0, 2)`, "4"},
{`regexp_instr('1234567890', '(123)(4(56)(78))', 1, 1, 0, 4)`, "7"},
{`regexp_substr('500 Oracle Parkway, Redwood Shores, CA', ',[^,]+,')`, ", Redwood Shores,"},
{`regexp_substr('http://www.example.com/products', 'http://([[:alnum:]]+\.?){3,4}/?')`, "http://www.example.com/"},
{`regexp_substr('1234567890', '(123)(4(56)(78))', 1, 1, 1)`, "123"},
{`regexp_substr('1234567890', '(123)(4(56)(78))', 1, 1, 4)`, "78"},
{`regexp_substr('123123123123', '1(.)3', 3, 2, 1)`, "2"},
{`regexp_replace('500 Oracle Parkway, Redwood Shores, CA', '( ){2,}', ' ')`, "500 Oracle Parkway, Redwood Shores, CA"},
}
for _, tt := range tests {
var got sql.NullString
err := db.QueryRow(`SELECT ` + tt.test).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got.String != tt.want {
t.Errorf("got %q, want %q", got.String, tt.want)
}
}
}
func TestRegister_errors(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(tmp, Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
tests := []string{
`'' REGEXP ?`,
`regexp_like('', ?)`,
`regexp_count('', ?)`,
`regexp_instr('', ?)`,
`regexp_substr('', ?)`,
`regexp_replace('', ?, '')`,
}
for _, tt := range tests {
err := db.QueryRow(`SELECT `+tt, `\`).Scan(nil)
if err == nil {
t.Fatal("want error")
}
}
}
func TestGlobPrefix(t *testing.T) {
tests := []struct {
re string
want string
}{
{``, ""},
{`a`, "a"},
{`a*`, "*"},
{`a+`, "a*"},
{`ab*`, "a*"},
{`ab+`, "ab*"},
{`a\?b`, "a*"},
}
for _, tt := range tests {
t.Run(tt.re, func(t *testing.T) {
if got := GlobPrefix(regexp.MustCompile(tt.re)); got != tt.want {
t.Errorf("GlobPrefix() = %v, want %v", got, tt.want)
}
})
}
}

View File

@@ -8,17 +8,17 @@ package statement
import (
"encoding/json"
"errors"
"strconv"
"strings"
"unsafe"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers the statement virtual table.
func Register(db *sqlite3.Conn) {
sqlite3.CreateModule(db, "statement", declare, declare)
func Register(db *sqlite3.Conn) error {
return sqlite3.CreateModule(db, "statement", declare, declare)
}
type table struct {
@@ -29,12 +29,12 @@ type table struct {
func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (*table, error) {
if len(arg) != 1 {
return nil, errors.New("statement: wrong number of arguments")
return nil, util.ErrorString("statement: wrong number of arguments")
}
sql := "SELECT * FROM\n" + arg[0]
stmt, _, err := db.Prepare(sql)
stmt, _, err := db.PrepareFlags(sql, sqlite3.PREPARE_PERSISTENT)
if err != nil {
return nil, err
}
@@ -123,12 +123,11 @@ func (t *table) BestIndex(idx *sqlite3.IndexInfo) error {
return nil
}
func (t *table) Open() (sqlite3.VTabCursor, error) {
func (t *table) Open() (_ sqlite3.VTabCursor, err error) {
stmt := t.stmt
if !t.inuse {
t.inuse = true
} else {
var err error
stmt, _, err = t.stmt.Conn().Prepare(t.sql)
if err != nil {
return nil, err
@@ -202,7 +201,7 @@ func (c *cursor) RowID() (int64, error) {
return c.rowID, nil
}
func (c *cursor) Column(ctx *sqlite3.Context, col int) error {
func (c *cursor) Column(ctx sqlite3.Context, col int) error {
switch outputs := c.stmt.ColumnCount(); {
case col < outputs:
ctx.ResultValue(c.stmt.ColumnValue(col))

View File

@@ -12,14 +12,14 @@ import (
)
func Example() {
sqlite3.AutoExtension(statement.Register)
db, err := sqlite3.Open(":memory:")
if err != nil {
log.Fatal(err)
}
defer db.Close()
statement.Register(db)
err = db.Exec(`
CREATE VIRTUAL TABLE split_date USING statement((
SELECT
@@ -48,6 +48,11 @@ func Example() {
// Twosday was 2022-2-22
}
func TestMain(m *testing.M) {
sqlite3.AutoExtension(statement.Register)
m.Run()
}
func TestRegister(t *testing.T) {
t.Parallel()
@@ -57,8 +62,6 @@ func TestRegister(t *testing.T) {
}
defer db.Close()
statement.Register(db)
err = db.Exec(`
CREATE VIRTUAL TABLE arguments USING statement((SELECT ? AS a, ? AS b, ? AS c))
`)
@@ -107,8 +110,6 @@ func TestRegister_errors(t *testing.T) {
}
defer db.Close()
statement.Register(db)
err = db.Exec(`CREATE VIRTUAL TABLE split_date USING statement()`)
if err == nil {
t.Fatal("want error")

View File

@@ -26,21 +26,21 @@ func (b *boolean) Value(ctx sqlite3.Context) {
}
func (b *boolean) Step(ctx sqlite3.Context, arg ...sqlite3.Value) {
if arg[0].Type() == sqlite3.NULL {
return
}
if arg[0].Bool() {
a := arg[0]
if a.Bool() {
b.count++
}
b.total++
if a.Type() != sqlite3.NULL {
b.total++
}
}
func (b *boolean) Inverse(ctx sqlite3.Context, arg ...sqlite3.Value) {
if arg[0].Type() == sqlite3.NULL {
return
}
if arg[0].Bool() {
a := arg[0]
if a.Bool() {
b.count--
}
b.total--
if a.Type() != sqlite3.NULL {
b.total--
}
}

View File

@@ -5,7 +5,6 @@ import (
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/stats"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
)
@@ -18,8 +17,6 @@ func TestRegister_boolean(t *testing.T) {
}
defer db.Close()
stats.Register(db)
err = db.Exec(`CREATE TABLE data (x)`)
if err != nil {
t.Fatal(err)

View File

@@ -13,6 +13,7 @@ import (
const (
median = iota
percentile_100
percentile_cont
percentile_disc
)
@@ -28,17 +29,25 @@ type percentile struct {
}
func (q *percentile) Step(ctx sqlite3.Context, arg ...sqlite3.Value) {
if a := arg[0]; a.NumericType() != sqlite3.NULL {
q.nums = append(q.nums, a.Float())
a := arg[0]
f := a.Float()
if f != 0.0 || a.NumericType() != sqlite3.NULL {
q.nums = append(q.nums, f)
}
if q.kind != median {
q.arg1 = arg[1].Blob(q.arg1[:0])
if q.kind != median && q.arg1 == nil {
q.arg1 = append(q.arg1, arg[1].RawText()...)
}
}
func (q *percentile) Inverse(ctx sqlite3.Context, arg ...sqlite3.Value) {
// Implementing inverse allows certain queries that don't really need it to succeed.
ctx.ResultError(util.ErrorString("percentile: may not be used as a window function"))
a := arg[0]
f := a.Float()
if f != 0.0 || a.NumericType() != sqlite3.NULL {
i := slices.Index(q.nums, f)
l := len(q.nums) - 1
q.nums[i] = q.nums[l]
q.nums = q.nums[:l]
}
}
func (q *percentile) Value(ctx sqlite3.Context) {
@@ -52,39 +61,42 @@ func (q *percentile) Value(ctx sqlite3.Context) {
floats []float64
)
if q.kind == median {
float, err = getPercentile(q.nums, 0.5, false)
float, err = q.at(0.5)
ctx.ResultFloat(float)
} else if err = json.Unmarshal(q.arg1, &float); err == nil {
float, err = getPercentile(q.nums, float, q.kind == percentile_disc)
float, err = q.at(float)
ctx.ResultFloat(float)
} else if err = json.Unmarshal(q.arg1, &floats); err == nil {
err = getPercentiles(q.nums, floats, q.kind == percentile_disc)
err = q.atMore(floats)
ctx.ResultJSON(floats)
}
if err != nil {
ctx.ResultError(fmt.Errorf("percentile: %w", err))
ctx.ResultError(fmt.Errorf("percentile: %w", err)) // notest
}
}
func getPercentile(nums []float64, pos float64, disc bool) (float64, error) {
func (q *percentile) at(pos float64) (float64, error) {
if q.kind == percentile_100 {
pos = pos / 100
}
if pos < 0 || pos > 1 {
return 0, util.ErrorString("invalid pos")
}
i, f := math.Modf(pos * float64(len(nums)-1))
m0 := quick.Select(nums, int(i))
i, f := math.Modf(pos * float64(len(q.nums)-1))
m0 := quick.Select(q.nums, int(i))
if f == 0 || disc {
if f == 0 || q.kind == percentile_disc {
return m0, nil
}
m1 := slices.Min(nums[int(i)+1:])
return math.FMA(f, m1, -math.FMA(f, m0, -m0)), nil
m1 := slices.Min(q.nums[int(i)+1:])
return util.Lerp(m0, m1, f), nil
}
func getPercentiles(nums []float64, pos []float64, disc bool) error {
func (q *percentile) atMore(pos []float64) error {
for i := range pos {
v, err := getPercentile(nums, pos[i], disc)
v, err := q.at(pos[i])
if err != nil {
return err
}

View File

@@ -6,7 +6,6 @@ import (
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/stats"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
)
@@ -19,8 +18,6 @@ func TestRegister_percentile(t *testing.T) {
}
defer db.Close()
stats.Register(db)
err = db.Exec(`CREATE TABLE data (x)`)
if err != nil {
t.Fatal(err)
@@ -34,6 +31,7 @@ func TestRegister_percentile(t *testing.T) {
stmt, _, err := db.Prepare(`
SELECT
median(x),
percentile(x, 50),
percentile_disc(x, 0.5),
percentile_cont(x, '[0.25, 0.5, 0.75]')
FROM data`)
@@ -44,11 +42,14 @@ func TestRegister_percentile(t *testing.T) {
if got := stmt.ColumnFloat(0); got != 10 {
t.Errorf("got %v, want 10", got)
}
if got := stmt.ColumnFloat(1); got != 7 {
if got := stmt.ColumnFloat(1); got != 10 {
t.Errorf("got %v, want 10", got)
}
if got := stmt.ColumnFloat(2); got != 7 {
t.Errorf("got %v, want 7", got)
}
var got []float64
if err := stmt.ColumnJSON(2, &got); err != nil {
if err := stmt.ColumnJSON(3, &got); err != nil {
t.Error(err)
}
if !slices.Equal(got, []float64{6.25, 10, 13.75}) {
@@ -57,9 +58,44 @@ func TestRegister_percentile(t *testing.T) {
}
stmt.Close()
stmt, _, err = db.Prepare(`
SELECT
median(x) OVER (ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)
FROM data`)
if err != nil {
t.Fatal(err)
}
if stmt.Step() {
if got := stmt.ColumnFloat(0); got != 5.5 {
t.Errorf("got %v, want 5.5", got)
}
}
if stmt.Step() {
if got := stmt.ColumnFloat(0); got != 7 {
t.Errorf("got %v, want 7", got)
}
}
if stmt.Step() {
if got := stmt.ColumnFloat(0); got != 10 {
t.Errorf("got %v, want 10", got)
}
}
if stmt.Step() {
if got := stmt.ColumnFloat(0); got != 14.5 {
t.Errorf("got %v, want 14.5", got)
}
}
if stmt.Step() {
if got := stmt.ColumnFloat(0); got != 16 {
t.Errorf("got %v, want 16", got)
}
}
stmt.Close()
stmt, _, err = db.Prepare(`
SELECT
median(x),
percentile(x, 50),
percentile_disc(x, 0.5),
percentile_cont(x, '[0.25, 0.5, 0.75]')
FROM data
@@ -74,8 +110,11 @@ func TestRegister_percentile(t *testing.T) {
if got := stmt.ColumnFloat(1); got != 4 {
t.Errorf("got %v, want 4", got)
}
if got := stmt.ColumnFloat(2); got != 4 {
t.Errorf("got %v, want 4", got)
}
var got []float64
if err := stmt.ColumnJSON(2, &got); err != nil {
if err := stmt.ColumnJSON(3, &got); err != nil {
t.Error(err)
}
if !slices.Equal(got, []float64{4, 4, 4}) {
@@ -87,6 +126,7 @@ func TestRegister_percentile(t *testing.T) {
stmt, _, err = db.Prepare(`
SELECT
median(x),
percentile(x, 50),
percentile_disc(x, 0.5),
percentile_cont(x, '[0.25, 0.5, 0.75]')
FROM data
@@ -104,6 +144,9 @@ func TestRegister_percentile(t *testing.T) {
if got := stmt.ColumnType(2); got != sqlite3.NULL {
t.Error("want NULL")
}
if got := stmt.ColumnType(3); got != sqlite3.NULL {
t.Error("want NULL")
}
}
stmt.Close()

View File

@@ -44,33 +44,40 @@
// [ANSI SQL Aggregate Functions]: https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html
package stats
import "github.com/ncruces/go-sqlite3"
import (
"errors"
"github.com/ncruces/go-sqlite3"
)
// Register registers statistics functions.
func Register(db *sqlite3.Conn) {
flags := sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
db.CreateWindowFunction("var_pop", 1, flags, newVariance(var_pop))
db.CreateWindowFunction("var_samp", 1, flags, newVariance(var_samp))
db.CreateWindowFunction("stddev_pop", 1, flags, newVariance(stddev_pop))
db.CreateWindowFunction("stddev_samp", 1, flags, newVariance(stddev_samp))
db.CreateWindowFunction("covar_pop", 2, flags, newCovariance(var_pop))
db.CreateWindowFunction("covar_samp", 2, flags, newCovariance(var_samp))
db.CreateWindowFunction("corr", 2, flags, newCovariance(corr))
db.CreateWindowFunction("regr_r2", 2, flags, newCovariance(regr_r2))
db.CreateWindowFunction("regr_sxx", 2, flags, newCovariance(regr_sxx))
db.CreateWindowFunction("regr_syy", 2, flags, newCovariance(regr_syy))
db.CreateWindowFunction("regr_sxy", 2, flags, newCovariance(regr_sxy))
db.CreateWindowFunction("regr_avgx", 2, flags, newCovariance(regr_avgx))
db.CreateWindowFunction("regr_avgy", 2, flags, newCovariance(regr_avgy))
db.CreateWindowFunction("regr_slope", 2, flags, newCovariance(regr_slope))
db.CreateWindowFunction("regr_intercept", 2, flags, newCovariance(regr_intercept))
db.CreateWindowFunction("regr_count", 2, flags, newCovariance(regr_count))
db.CreateWindowFunction("regr_json", 2, flags, newCovariance(regr_json))
db.CreateWindowFunction("median", 1, flags, newPercentile(median))
db.CreateWindowFunction("percentile_cont", 2, flags, newPercentile(percentile_cont))
db.CreateWindowFunction("percentile_disc", 2, flags, newPercentile(percentile_disc))
db.CreateWindowFunction("every", 1, flags, newBoolean(every))
db.CreateWindowFunction("some", 1, flags, newBoolean(some))
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
const order = sqlite3.SELFORDER1 | flags
return errors.Join(
db.CreateWindowFunction("var_pop", 1, flags, newVariance(var_pop)),
db.CreateWindowFunction("var_samp", 1, flags, newVariance(var_samp)),
db.CreateWindowFunction("stddev_pop", 1, flags, newVariance(stddev_pop)),
db.CreateWindowFunction("stddev_samp", 1, flags, newVariance(stddev_samp)),
db.CreateWindowFunction("covar_pop", 2, flags, newCovariance(var_pop)),
db.CreateWindowFunction("covar_samp", 2, flags, newCovariance(var_samp)),
db.CreateWindowFunction("corr", 2, flags, newCovariance(corr)),
db.CreateWindowFunction("regr_r2", 2, flags, newCovariance(regr_r2)),
db.CreateWindowFunction("regr_sxx", 2, flags, newCovariance(regr_sxx)),
db.CreateWindowFunction("regr_syy", 2, flags, newCovariance(regr_syy)),
db.CreateWindowFunction("regr_sxy", 2, flags, newCovariance(regr_sxy)),
db.CreateWindowFunction("regr_avgx", 2, flags, newCovariance(regr_avgx)),
db.CreateWindowFunction("regr_avgy", 2, flags, newCovariance(regr_avgy)),
db.CreateWindowFunction("regr_slope", 2, flags, newCovariance(regr_slope)),
db.CreateWindowFunction("regr_intercept", 2, flags, newCovariance(regr_intercept)),
db.CreateWindowFunction("regr_count", 2, flags, newCovariance(regr_count)),
db.CreateWindowFunction("regr_json", 2, flags, newCovariance(regr_json)),
db.CreateWindowFunction("median", 1, order, newPercentile(median)),
db.CreateWindowFunction("percentile", 2, order, newPercentile(percentile_100)),
db.CreateWindowFunction("percentile_cont", 2, order, newPercentile(percentile_cont)),
db.CreateWindowFunction("percentile_disc", 2, order, newPercentile(percentile_disc)),
db.CreateWindowFunction("every", 1, flags, newBoolean(every)),
db.CreateWindowFunction("some", 1, flags, newBoolean(some)))
}
const (
@@ -116,14 +123,18 @@ func (fn *variance) Value(ctx sqlite3.Context) {
}
func (fn *variance) Step(ctx sqlite3.Context, arg ...sqlite3.Value) {
if a := arg[0]; a.NumericType() != sqlite3.NULL {
fn.enqueue(a.Float())
a := arg[0]
f := a.Float()
if f != 0.0 || a.NumericType() != sqlite3.NULL {
fn.enqueue(f)
}
}
func (fn *variance) Inverse(ctx sqlite3.Context, arg ...sqlite3.Value) {
if a := arg[0]; a.NumericType() != sqlite3.NULL {
fn.dequeue(a.Float())
a := arg[0]
f := a.Float()
if f != 0.0 || a.NumericType() != sqlite3.NULL {
fn.dequeue(f)
}
}
@@ -172,15 +183,23 @@ func (fn *covariance) Value(ctx sqlite3.Context) {
}
func (fn *covariance) Step(ctx sqlite3.Context, arg ...sqlite3.Value) {
a, b := arg[0], arg[1]
if a.NumericType() != sqlite3.NULL && b.NumericType() != sqlite3.NULL {
fn.enqueue(a.Float(), b.Float())
b, a := arg[1], arg[0] // avoid a bounds check
fa := a.Float()
fb := b.Float()
if true &&
(fa != 0.0 || a.NumericType() != sqlite3.NULL) &&
(fb != 0.0 || b.NumericType() != sqlite3.NULL) {
fn.enqueue(fa, fb)
}
}
func (fn *covariance) Inverse(ctx sqlite3.Context, arg ...sqlite3.Value) {
a, b := arg[0], arg[1]
if a.NumericType() != sqlite3.NULL && b.NumericType() != sqlite3.NULL {
fn.dequeue(a.Float(), b.Float())
b, a := arg[1], arg[0] // avoid a bounds check
fa := a.Float()
fb := b.Float()
if true &&
(fa != 0.0 || a.NumericType() != sqlite3.NULL) &&
(fb != 0.0 || b.NumericType() != sqlite3.NULL) {
fn.dequeue(fa, fb)
}
}

View File

@@ -10,6 +10,11 @@ import (
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
)
func TestMain(m *testing.M) {
sqlite3.AutoExtension(stats.Register)
m.Run()
}
func TestRegister_variance(t *testing.T) {
t.Parallel()
@@ -19,8 +24,6 @@ func TestRegister_variance(t *testing.T) {
}
defer db.Close()
stats.Register(db)
err = db.Exec(`CREATE TABLE data (x)`)
if err != nil {
t.Fatal(err)
@@ -88,8 +91,6 @@ func TestRegister_covariance(t *testing.T) {
}
defer db.Close()
stats.Register(db)
err = db.Exec(`CREATE TABLE data (y, x)`)
if err != nil {
t.Fatal(err)
@@ -217,8 +218,6 @@ func Benchmark_variance(b *testing.B) {
}
defer db.Close()
stats.Register(db)
stmt, _, err := db.Prepare(`SELECT var_pop(value) FROM generate_series(0, ?)`)
if err != nil {
b.Fatal(err)

View File

@@ -5,6 +5,10 @@
// - LIKE and REGEXP operators,
// - collation sequences.
//
// It also provides, from PostgreSQL:
// - unaccent(),
// - initcap().
//
// The implementation is not 100% compatible with the [ICU extension]:
// - upper() and lower() use [strings.ToUpper], [strings.ToLower] and [cases];
// - the LIKE operator follows [strings.EqualFold] rules;
@@ -18,41 +22,61 @@ package unicode
import (
"bytes"
"errors"
"regexp"
"strings"
"unicode"
"unicode/utf8"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"golang.org/x/text/cases"
"golang.org/x/text/collate"
"golang.org/x/text/language"
"golang.org/x/text/runes"
"golang.org/x/text/transform"
"golang.org/x/text/unicode/norm"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Set RegisterLike to false to not register a Unicode aware LIKE operator.
// Overriding the built-in LIKE operator disables the [LIKE optimization].
//
// [LIKE optimization]: https://sqlite.org/optoverview.html#the_like_optimization
var RegisterLike = true
// Register registers Unicode aware functions for a database connection.
func Register(db *sqlite3.Conn) {
flags := sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
var errs util.ErrorJoiner
if RegisterLike {
errs.Join(
db.CreateFunction("like", 2, flags, like),
db.CreateFunction("like", 3, flags, like))
}
errs.Join(
db.CreateFunction("upper", 1, flags, upper),
db.CreateFunction("upper", 2, flags, upper),
db.CreateFunction("lower", 1, flags, lower),
db.CreateFunction("lower", 2, flags, lower),
db.CreateFunction("regexp", 2, flags, regex),
db.CreateFunction("initcap", 1, flags, initcap),
db.CreateFunction("initcap", 2, flags, initcap),
db.CreateFunction("unaccent", 1, flags, unaccent),
db.CreateFunction("icu_load_collation", 2, sqlite3.DIRECTONLY,
func(ctx sqlite3.Context, arg ...sqlite3.Value) {
name := arg[1].Text()
if name == "" {
return
}
db.CreateFunction("like", 2, flags, like)
db.CreateFunction("like", 3, flags, like)
db.CreateFunction("upper", 1, flags, upper)
db.CreateFunction("upper", 2, flags, upper)
db.CreateFunction("lower", 1, flags, lower)
db.CreateFunction("lower", 2, flags, lower)
db.CreateFunction("regexp", 2, flags, regex)
db.CreateFunction("icu_load_collation", 2, sqlite3.DIRECTONLY,
func(ctx sqlite3.Context, arg ...sqlite3.Value) {
name := arg[1].Text()
if name == "" {
return
}
err := RegisterCollation(db, arg[0].Text(), name)
if err != nil {
ctx.ResultError(err)
return
}
})
err := RegisterCollation(ctx.Conn(), arg[0].Text(), name)
if err != nil {
ctx.ResultError(err)
return // notest
}
}))
return errors.Join(errs...)
}
// RegisterCollation registers a Unicode collation sequence for a database connection.
@@ -64,6 +88,15 @@ func RegisterCollation(db *sqlite3.Conn, locale, name string) error {
return db.CreateCollation(name, collate.New(tag).Compare)
}
// RegisterCollationsNeeded registers Unicode collation sequences on demand for a database connection.
func RegisterCollationsNeeded(db *sqlite3.Conn) error {
return db.CollationNeeded(func(db *sqlite3.Conn, name string) {
if tag, err := language.Parse(name); err == nil {
db.CreateCollation(name, collate.New(tag).Compare)
}
})
}
func upper(ctx sqlite3.Context, arg ...sqlite3.Value) {
if len(arg) == 1 {
ctx.ResultRawText(bytes.ToUpper(arg[0].RawText()))
@@ -74,7 +107,7 @@ func upper(ctx sqlite3.Context, arg ...sqlite3.Value) {
t, err := language.Parse(arg[1].Text())
if err != nil {
ctx.ResultError(err)
return
return // notest
}
c := cases.Upper(t)
ctx.SetAuxData(1, c)
@@ -93,7 +126,7 @@ func lower(ctx sqlite3.Context, arg ...sqlite3.Value) {
t, err := language.Parse(arg[1].Text())
if err != nil {
ctx.ResultError(err)
return
return // notest
}
c := cases.Lower(t)
ctx.SetAuxData(1, c)
@@ -102,16 +135,45 @@ func lower(ctx sqlite3.Context, arg ...sqlite3.Value) {
ctx.ResultRawText(cs.Bytes(arg[0].RawText()))
}
func initcap(ctx sqlite3.Context, arg ...sqlite3.Value) {
if len(arg) == 1 {
ctx.ResultRawText(bytes.Title(arg[0].RawText()))
return
}
cs, ok := ctx.GetAuxData(1).(cases.Caser)
if !ok {
t, err := language.Parse(arg[1].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
c := cases.Title(t)
ctx.SetAuxData(1, c)
cs = c
}
ctx.ResultRawText(cs.Bytes(arg[0].RawText()))
}
func unaccent(ctx sqlite3.Context, arg ...sqlite3.Value) {
unaccent := transform.Chain(norm.NFD, runes.Remove(runes.In(unicode.Mn)), norm.NFC)
res, _, err := transform.Bytes(unaccent, arg[0].RawText())
if err != nil {
ctx.ResultError(err) // notest
} else {
ctx.ResultRawText(res)
}
}
func regex(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, ok := ctx.GetAuxData(0).(*regexp.Regexp)
if !ok {
r, err := regexp.Compile(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return
return // notest
}
re = r
ctx.SetAuxData(0, re)
ctx.SetAuxData(0, r)
}
ctx.ResultBool(re.Match(arg[1].RawText()))
}

View File

@@ -47,6 +47,9 @@ func TestRegister(t *testing.T) {
{`upper('istanbul', 'tr-TR')`, "İSTANBUL"},
{`lower('Dünyanın İlk Borsası', 'tr-TR')`, "dünyanın ilk borsası"},
{`upper('Dünyanın İlk Borsası', 'tr-TR')`, "DÜNYANIN İLK BORSASI"},
{`initcap('Kad je hladno Marko nosi džemper')`, "Kad Je Hladno Marko Nosi Džemper"},
{`initcap('Kad je hladno Marko nosi džemper', 'hr-HR')`, "Kad Je Hladno Marko Nosi Džemper"},
{`unaccent('Hôtel')`, "Hotel"},
{`'Hello' REGEXP 'ell'`, "1"},
{`'Hello' REGEXP 'el.'`, "1"},
{`'Hello' LIKE 'hel_'`, "0"},
@@ -92,7 +95,7 @@ func TestRegister_collation(t *testing.T) {
t.Fatal(err)
}
err = db.Exec(`SELECT icu_load_collation('fr_FR', 'french')`)
err = db.Exec(`SELECT icu_load_collation('fr-FR', 'french')`)
if err != nil {
t.Fatal(err)
}
@@ -127,6 +130,57 @@ func TestRegister_collation(t *testing.T) {
}
}
func TestRegisterCollationsNeeded(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
RegisterCollationsNeeded(db)
err = db.Exec(`CREATE TABLE words (word VARCHAR(10))`)
if err != nil {
t.Fatal(err)
}
err = db.Exec(`INSERT INTO words (word) VALUES ('côte'), ('cote'), ('coter'), ('coté'), ('cotée'), ('côté')`)
if err != nil {
t.Fatal(err)
}
stmt, _, err := db.Prepare(`SELECT word FROM words ORDER BY word COLLATE fr_FR`)
if err != nil {
t.Fatal(err)
}
defer stmt.Close()
got, want := []string{}, []string{"cote", "coté", "côte", "côté", "cotée", "coter"}
for stmt.Step() {
got = append(got, stmt.ColumnText(0))
}
if err := stmt.Err(); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(got, want) {
t.Error("not equal")
}
err = stmt.Close()
if err != nil {
t.Fatal(err)
}
err = db.Close()
if err != nil {
t.Fatal(err)
}
}
func TestRegister_error(t *testing.T) {
t.Parallel()

169
ext/uuid/uuid.go Normal file
View File

@@ -0,0 +1,169 @@
// Package uuid provides functions to generate RFC 4122 UUIDs.
//
// https://sqlite.org/src/file/ext/misc/uuid.c
package uuid
import (
"bytes"
"errors"
"fmt"
"github.com/google/uuid"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers the SQL functions:
//
// uuid([version], [domain/namespace], [id/data])
//
// Generates a UUID as a string.
//
// uuid_str(u)
//
// Converts a UUID into a well-formed UUID string.
//
// uuid_blob(u)
//
// Converts a UUID into a 16-byte blob.
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
return errors.Join(
db.CreateFunction("uuid", 0, sqlite3.INNOCUOUS, generate),
db.CreateFunction("uuid", 1, sqlite3.INNOCUOUS, generate),
db.CreateFunction("uuid", 2, sqlite3.INNOCUOUS, generate),
db.CreateFunction("uuid", 3, sqlite3.INNOCUOUS, generate),
db.CreateFunction("uuid_str", 1, flags, toString),
db.CreateFunction("uuid_blob", 1, flags, toBlob))
}
func generate(ctx sqlite3.Context, arg ...sqlite3.Value) {
var (
ver int
err error
u uuid.UUID
)
if len(arg) > 0 {
ver = arg[0].Int()
} else {
ver = 4
}
switch ver {
case 1:
u, err = uuid.NewUUID()
case 4:
u, err = uuid.NewRandom()
case 6:
u, err = uuid.NewV6()
case 7:
u, err = uuid.NewV7()
case 2:
var domain uuid.Domain
if len(arg) > 1 {
domain = uuid.Domain(arg[1].Int64())
if domain == 0 {
if txt := arg[1].RawText(); len(txt) > 0 {
switch txt[0] | 0x20 { // to lower
case 'g': // group
domain = uuid.Group
case 'o': // org
domain = uuid.Org
}
}
}
}
switch {
case len(arg) > 2:
u, err = uuid.NewDCESecurity(domain, uint32(arg[2].Int64()))
case domain == uuid.Person:
u, err = uuid.NewDCEPerson()
case domain == uuid.Group:
u, err = uuid.NewDCEGroup()
default:
err = util.ErrorString("missing id")
}
case 3, 5:
if len(arg) < 2 {
err = util.ErrorString("missing data")
break
}
ns, err := fromValue(arg[1])
if err != nil {
space := arg[1].RawText()
switch {
case bytes.EqualFold(space, []byte("url")):
ns = uuid.NameSpaceURL
case bytes.EqualFold(space, []byte("oid")):
ns = uuid.NameSpaceOID
case bytes.EqualFold(space, []byte("dns")):
ns = uuid.NameSpaceDNS
case bytes.EqualFold(space, []byte("fqdn")):
ns = uuid.NameSpaceDNS
case bytes.EqualFold(space, []byte("x500")):
ns = uuid.NameSpaceX500
default:
ctx.ResultError(err)
return // notest
}
}
if ver == 3 {
u = uuid.NewMD5(ns, arg[2].RawBlob())
} else {
u = uuid.NewSHA1(ns, arg[2].RawBlob())
}
default:
err = fmt.Errorf("invalid version: %d", ver)
}
if err != nil {
ctx.ResultError(fmt.Errorf("uuid: %w", err)) // notest
} else {
ctx.ResultText(u.String())
}
}
func fromValue(arg sqlite3.Value) (u uuid.UUID, err error) {
switch t := arg.Type(); t {
case sqlite3.TEXT:
u, err = uuid.ParseBytes(arg.RawText())
if err != nil {
err = fmt.Errorf("uuid: %w", err)
}
case sqlite3.BLOB:
blob := arg.RawBlob()
if len := len(blob); len != 16 {
err = fmt.Errorf("uuid: invalid BLOB length: %d", len)
} else {
copy(u[:], blob)
}
default:
err = fmt.Errorf("uuid: invalid type: %v", t)
}
return u, err
}
func toBlob(ctx sqlite3.Context, arg ...sqlite3.Value) {
u, err := fromValue(arg[0])
if err != nil {
ctx.ResultError(err) // notest
} else {
ctx.ResultBlob(u[:])
}
}
func toString(ctx sqlite3.Context, arg ...sqlite3.Value) {
u, err := fromValue(arg[0])
if err != nil {
ctx.ResultError(err) // notest
} else {
ctx.ResultText(u.String())
}
}

181
ext/uuid/uuid_test.go Normal file
View File

@@ -0,0 +1,181 @@
package uuid
import (
"testing"
"github.com/google/uuid"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Test_generate(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(tmp, Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var u uuid.UUID
// Version 4, SQLite compatible
err = db.QueryRow(`SELECT uuid()`).Scan(&u)
if err != nil {
t.Fatal(err)
}
if got := u.Version(); got != 4 {
t.Errorf("got %d, want 4", got)
}
// Invalid version
err = db.QueryRow(`SELECT uuid(8)`).Scan(&u)
if err == nil {
t.Error("want error")
}
// Custom version, no arguments
for _, want := range []uuid.Version{1, 2, 4, 6, 7} {
err = db.QueryRow(`SELECT uuid(?)`, want).Scan(&u)
if err != nil {
t.Fatal(err)
}
if got := u.Version(); got != want {
t.Errorf("got %d, want %d", got, want)
}
}
// Version 2, custom arguments
err = db.QueryRow(`SELECT uuid(2, 4)`).Scan(&u)
if err == nil {
t.Error("want error")
}
err = db.QueryRow(`SELECT uuid(2, 'group')`).Scan(&u)
if err != nil {
t.Fatal(err)
}
if got := u.Version(); got != 2 {
t.Errorf("got %d, want 2", got)
}
if got := u.Domain(); got != uuid.Group {
t.Errorf("got %d, want 1", got)
}
dce := []struct {
out uuid.Domain
in any
id uint32
}{
{uuid.Person, "user", 42},
{uuid.Group, "group", 42},
{uuid.Org, "org", 42},
{uuid.Person, 0, 42},
{uuid.Group, 1, 42},
{uuid.Org, 2, 42},
{3, 3, 42},
}
for _, tt := range dce {
err = db.QueryRow(`SELECT uuid(2, ?, ?)`, tt.in, tt.id).Scan(&u)
if err != nil {
t.Fatal(err)
}
if got := u.Version(); got != 2 {
t.Errorf("got %d, want 2", got)
}
if got := u.Domain(); got != tt.out {
t.Errorf("got %d, want %d", got, tt.out)
}
if got := u.ID(); got != tt.id {
t.Errorf("got %d, want %d", got, tt.id)
}
}
// Versions 3 and 5
err = db.QueryRow(`SELECT uuid(3)`).Scan(&u)
if err == nil {
t.Error("want error")
}
err = db.QueryRow(`SELECT uuid(3, 0, '')`).Scan(&u)
if err == nil {
t.Error("want error")
}
hash := []struct {
ver uuid.Version
ns any
data string
u uuid.UUID
}{
{3, "oid", "2.999", uuid.MustParse("31cb1efa-18c4-3d19-89ba-df6a74ddbd1d")},
{3, "dns", "www.example.com", uuid.MustParse("5df41881-3aed-3515-88a7-2f4a814cf09e")},
{3, "fqdn", "www.example.com", uuid.MustParse("5df41881-3aed-3515-88a7-2f4a814cf09e")},
{3, "url", "https://www.example.com/", uuid.MustParse("7fed185f-0864-319f-875b-a3d5458e30ac")},
{3, "x500", "CN=Test User 1, O=Example Organization, ST=California, C=US", uuid.MustParse("addf5e97-9287-3834-abfd-7edcbe7db56f")},
{3, "url", "https://www.php.net", uuid.MustParse("3f703955-aaba-3e70-a3cb-baff6aa3b28f")},
{5, "url", "https://www.php.net", uuid.MustParse("a8f6ae40-d8a7-58f0-be05-a22f94eca9ec")},
}
for _, tt := range hash {
err = db.QueryRow(`SELECT uuid(?, ?, ?)`, tt.ver, tt.ns, tt.data).Scan(&u)
if err != nil {
t.Fatal(err)
}
if u != tt.u {
t.Errorf("got %v, want %v", u, tt.u)
}
}
}
func Test_convert(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(tmp, Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var u uuid.UUID
lits := []string{
"'6ba7b8119dad11d180b400c04fd430c8'",
"'6ba7b811-9dad-11d1-80b4-00c04fd430c8'",
"'{6ba7b811-9dad-11d1-80b4-00c04fd430c8}'",
"X'6ba7b8119dad11d180b400c04fd430c8'",
}
for _, tt := range lits {
err = db.QueryRow(`SELECT uuid_str(` + tt + `)`).Scan(&u)
if err != nil {
t.Fatal(err)
}
if u != uuid.NameSpaceURL {
t.Errorf("got %v, want %v", u, uuid.NameSpaceURL)
}
}
for _, tt := range lits {
err = db.QueryRow(`SELECT uuid_blob(` + tt + `)`).Scan(&u)
if err != nil {
t.Fatal(err)
}
if u != uuid.NameSpaceURL {
t.Errorf("got %v, want %v", u, uuid.NameSpaceURL)
}
}
err = db.QueryRow(`SELECT uuid_str(X'cafe')`).Scan(&u)
if err == nil {
t.Fatal("want error")
}
err = db.QueryRow(`SELECT uuid_blob(X'cafe')`).Scan(&u)
if err == nil {
t.Fatal("want error")
}
}

View File

@@ -4,30 +4,33 @@
package zorder
import (
"errors"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers the zorder and unzorder SQL functions.
func Register(db *sqlite3.Conn) {
flags := sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
db.CreateFunction("zorder", -1, flags, zorder)
db.CreateFunction("unzorder", 3, flags, unzorder)
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
return errors.Join(
db.CreateFunction("zorder", -1, flags, zorder),
db.CreateFunction("unzorder", 3, flags, unzorder))
}
func zorder(ctx sqlite3.Context, arg ...sqlite3.Value) {
var x [63]int64
for i := range arg {
x[i] = arg[i].Int64()
}
if len(arg) > len(x) {
ctx.ResultError(util.ErrorString("zorder: too many parameters"))
return
}
for i := range arg {
x[i] = arg[i].Int64()
}
var z int64
if len(arg) > 0 {
for i := 0; i < 63; i++ {
for i := range x {
j := i % len(arg)
z |= (x[j] & 1) << i
x[j] >>= 1
@@ -44,9 +47,9 @@ func zorder(ctx sqlite3.Context, arg ...sqlite3.Value) {
}
func unzorder(ctx sqlite3.Context, arg ...sqlite3.Value) {
z := arg[0].Int64()
n := arg[1].Int64()
i := arg[2].Int64()
n := arg[1].Int64()
z := arg[0].Int64()
var k int
var x int64

View File

@@ -1,22 +1,22 @@
package zorder_test
import (
"strconv"
"strings"
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/zorder"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func TestRegister_zorder(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
zorder.Register(c)
return nil
})
db, err := driver.Open(tmp, zorder.Register)
if err != nil {
t.Fatal(err)
}
@@ -59,11 +59,9 @@ func TestRegister_zorder(t *testing.T) {
func TestRegister_unzorder(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
zorder.Register(c)
return nil
})
db, err := driver.Open(tmp, zorder.Register)
if err != nil {
t.Fatal(err)
}
@@ -89,11 +87,9 @@ func TestRegister_unzorder(t *testing.T) {
func TestRegister_error(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
zorder.Register(c)
return nil
})
db, err := driver.Open(tmp, zorder.Register)
if err != nil {
t.Fatal(err)
}
@@ -104,4 +100,16 @@ func TestRegister_error(t *testing.T) {
if err == nil {
t.Error("want error")
}
var buf strings.Builder
buf.WriteString("SELECT zorder(0")
for i := 1; i < 80; i++ {
buf.WriteByte(',')
buf.WriteString(strconv.Itoa(0))
}
buf.WriteByte(')')
err = db.QueryRow(buf.String()).Scan(&got)
if err == nil {
t.Error("want error")
}
}

34
func.go
View File

@@ -4,8 +4,9 @@ import (
"context"
"sync"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/tetratelabs/wazero/api"
"github.com/ncruces/go-sqlite3/internal/util"
)
// CollationNeeded registers a callback to be invoked
@@ -31,17 +32,25 @@ func (c *Conn) CollationNeeded(cb func(db *Conn, name string)) error {
//
// This can be used to load schemas that contain
// one or more unknown collating sequences.
func (c *Conn) AnyCollationNeeded() {
c.call("sqlite3_anycollseq_init", uint64(c.handle), 0, 0)
func (c Conn) AnyCollationNeeded() error {
r := c.call("sqlite3_anycollseq_init", uint64(c.handle), 0, 0)
if err := c.error(r); err != nil {
return err
}
c.collation = nil
return nil
}
// CreateCollation defines a new collating sequence.
//
// https://sqlite.org/c3ref/create_collation.html
func (c *Conn) CreateCollation(name string, fn func(a, b []byte) int) error {
var funcPtr uint32
defer c.arena.mark()()
namePtr := c.arena.string(name)
funcPtr := util.AddHandle(c.ctx, fn)
if fn != nil {
funcPtr = util.AddHandle(c.ctx, fn)
}
r := c.call("sqlite3_create_collation_go",
uint64(c.handle), uint64(namePtr), uint64(funcPtr))
return c.error(r)
@@ -51,9 +60,12 @@ func (c *Conn) CreateCollation(name string, fn func(a, b []byte) int) error {
//
// https://sqlite.org/c3ref/create_function.html
func (c *Conn) CreateFunction(name string, nArg int, flag FunctionFlag, fn ScalarFunction) error {
var funcPtr uint32
defer c.arena.mark()()
namePtr := c.arena.string(name)
funcPtr := util.AddHandle(c.ctx, fn)
if fn != nil {
funcPtr = util.AddHandle(c.ctx, fn)
}
r := c.call("sqlite3_create_function_go",
uint64(c.handle), uint64(namePtr), uint64(nArg),
uint64(flag), uint64(funcPtr))
@@ -70,10 +82,13 @@ type ScalarFunction func(ctx Context, arg ...Value)
//
// https://sqlite.org/c3ref/create_function.html
func (c *Conn) CreateWindowFunction(name string, nArg int, flag FunctionFlag, fn func() AggregateFunction) error {
var funcPtr uint32
defer c.arena.mark()()
call := "sqlite3_create_aggregate_function_go"
namePtr := c.arena.string(name)
funcPtr := util.AddHandle(c.ctx, fn)
if fn != nil {
funcPtr = util.AddHandle(c.ctx, fn)
}
call := "sqlite3_create_aggregate_function_go"
if _, ok := fn().(WindowFunction); ok {
call = "sqlite3_create_window_function_go"
}
@@ -183,11 +198,12 @@ func callbackAggregate(db *Conn, pAgg, pApp uint32) (AggregateFunction, uint32)
// We need to create the aggregate.
fn := util.GetHandle(db.ctx, pApp).(func() AggregateFunction)()
handle := util.AddHandle(db.ctx, fn)
if pAgg != 0 {
handle := util.AddHandle(db.ctx, fn)
util.WriteUint32(db.mod, pAgg, handle)
return fn, handle
}
return fn, handle
return fn, 0
}
func callbackArgs(db *Conn, arg []Value, pArg uint32) {

View File

@@ -130,8 +130,8 @@ func ExampleContext_SetAuxData() {
ctx.ResultError(err)
return
}
ctx.SetAuxData(0, r)
re = r
ctx.SetAuxData(0, r)
}
ctx.ResultBool(re.Match(arg[1].RawText()))
})

View File

@@ -78,7 +78,7 @@ func (f *countASCII) isASCII(arg sqlite3.Value) bool {
if arg.Type() != sqlite3.TEXT {
return false
}
for _, c := range arg.RawBlob() {
for _, c := range arg.RawText() {
if c > unicode.MaxASCII {
return false
}

22
go.mod
View File

@@ -2,17 +2,23 @@ module github.com/ncruces/go-sqlite3
go 1.21
toolchain go1.23.0
require (
github.com/dchest/siphash v1.2.3
github.com/ncruces/julianday v1.0.0
github.com/ncruces/sort v0.1.2
github.com/psanford/httpreadat v0.1.0
github.com/tetratelabs/wazero v1.7.3
golang.org/x/crypto v0.24.0
golang.org/x/sync v0.7.0
golang.org/x/sys v0.21.0
golang.org/x/text v0.16.0
lukechampine.com/adiantum v1.1.1
github.com/tetratelabs/wazero v1.8.2
golang.org/x/crypto v0.31.0
golang.org/x/sys v0.28.0
)
require (
github.com/dchest/siphash v1.2.3 // ext/bloom
github.com/google/uuid v1.6.0 // ext/uuid
github.com/psanford/httpreadat v0.1.0 // example
golang.org/x/sync v0.10.0 // test
golang.org/x/text v0.21.0 // ext/unicode
lukechampine.com/adiantum v1.1.1 // vfs/adiantum
)
retract v0.4.0 // tagged from the wrong branch

22
go.sum
View File

@@ -1,20 +1,22 @@
github.com/dchest/siphash v1.2.3 h1:QXwFc8cFOR2dSa/gE6o/HokBMWtLUaNDVd+22aKHeEA=
github.com/dchest/siphash v1.2.3/go.mod h1:0NvQU092bT0ipiFN++/rXm69QG9tVxLAlQHIXMPAkHc=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/ncruces/julianday v1.0.0 h1:fH0OKwa7NWvniGQtxdJRxAgkBMolni2BjDHaWTxqt7M=
github.com/ncruces/julianday v1.0.0/go.mod h1:Dusn2KvZrrovOMJuOt0TNXL6tB7U2E8kvza5fFc9G7g=
github.com/ncruces/sort v0.1.2 h1:zKQ9CA4fpHPF6xsUhRTfi5EEryspuBpe/QA4VWQOV1U=
github.com/ncruces/sort v0.1.2/go.mod h1:vEJUTBJtebIuCMmXD18GKo5GJGhsay+xZFOoBEIXFmE=
github.com/psanford/httpreadat v0.1.0 h1:VleW1HS2zO7/4c7c7zNl33fO6oYACSagjJIyMIwZLUE=
github.com/psanford/httpreadat v0.1.0/go.mod h1:Zg7P+TlBm3bYbyHTKv/EdtSJZn3qwbPwpfZ/I9GKCRE=
github.com/tetratelabs/wazero v1.7.3 h1:PBH5KVahrt3S2AHgEjKu4u+LlDbbk+nsGE3KLucy6Rw=
github.com/tetratelabs/wazero v1.7.3/go.mod h1:ytl6Zuh20R/eROuyDaGPkp82O9C/DJfXAwJfQ3X6/7Y=
golang.org/x/crypto v0.24.0 h1:mnl8DM0o513X8fdIkmyFE/5hTYxbwYOjDS/+rK6qpRI=
golang.org/x/crypto v0.24.0/go.mod h1:Z1PMYSOR5nyMcyAVAIQSKCDwalqy85Aqn1x3Ws4L5DM=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.21.0 h1:rF+pYz3DAGSQAxAu1CbC7catZg4ebC4UIeIhKxBZvws=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
github.com/tetratelabs/wazero v1.8.2 h1:yIgLR/b2bN31bjxwXHD8a3d+BogigR952csSDdLYEv4=
github.com/tetratelabs/wazero v1.8.2/go.mod h1:yAI0XTsMBhREkM/YDAK/zNou3GoiAce1P6+rp/wQhjs=
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
lukechampine.com/adiantum v1.1.1 h1:4fp6gTxWCqpEbLy40ExiYDDED3oUNWx5cTqBCtPdZqA=
lukechampine.com/adiantum v1.1.1/go.mod h1:LrAYVnTYLnUtE/yMp5bQr0HstAf060YUF8nM0B6+rUw=

View File

@@ -3,4 +3,5 @@ go 1.21
use (
.
./gormlite
./embed/bcw2
)

View File

@@ -1,7 +1,16 @@
github.com/ncruces/go-sqlite3 v0.21.0/go.mod h1:zxMOaSG5kFYVFK4xQa0pdwIszqxqJ0W0BxBgwdrNjuA=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.21.0/go.mod h1:ooXLefLobQVslOqselCNF4SxFAaoS6KujMbsGzSDmX0=
golang.org/x/term v0.22.0/go.mod h1:F3qCibpT5AMpCRfhfT53vVJwhLtIVHhB9XDjfFvnMI4=
golang.org/x/term v0.23.0/go.mod h1:DgV24QBUrK6jhZXl+20l6UWznPlwAHm1Q1mGHtydmSk=
golang.org/x/term v0.24.0/go.mod h1:lOBK/LVxemqiMij05LGJ0tzNr8xlmwBRJ81PX6wVLH8=
golang.org/x/term v0.25.0/go.mod h1:RPyXicDX+6vLxogjjRxjgD2TKtmAO6NZBsBRfrOLu7M=
golang.org/x/term v0.26.0/go.mod h1:Si5m1o57C5nBNQo5z1iq+XDijt21BDBDp2bK0QI8e3E=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=

View File

@@ -3,9 +3,11 @@ set -euo pipefail
cd -P -- "$(dirname -- "$0")"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/ddlmod.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/ddlmod_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/error_translator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/migrator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/sqlite.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/sqlite_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.6/ddlmod.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.6/ddlmod_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.6/error_translator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.6/migrator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.6/sqlite.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.6/sqlite_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.6/sqlite_test.go"
curl -#L "https://github.com/glebarez/sqlite/raw/v1.11.0/sqlite_error_translator_test.go" > error_translator_test.go

View File

@@ -3,8 +3,9 @@ package gormlite
import (
"errors"
"github.com/ncruces/go-sqlite3"
"gorm.io/gorm"
"github.com/ncruces/go-sqlite3"
)
func (_Dialector) Translate(err error) error {

View File

@@ -0,0 +1,47 @@
package gormlite
import (
"testing"
"gorm.io/gorm"
"gorm.io/gorm/logger"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func TestErrorTranslator(t *testing.T) {
// This is the example object for testing the unique constraint error
type Article struct {
ArticleNumber string `gorm:"unique"`
}
db, err := gorm.Open(Open(memdb.TestDB(t)), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
TranslateError: true})
if err != nil {
t.Errorf("Expected Open to succeed; got error: %v", err)
}
if db == nil {
t.Errorf("Expected db to be non-nil.")
}
err = db.AutoMigrate(&Article{})
if err != nil {
t.Errorf("Expected to migrate database models to succeed: %v", err)
}
err = db.Create(&Article{ArticleNumber: "A00000XX"}).Error
if err != nil {
t.Errorf("Expected first create to succeed: %v", err)
}
err = db.Create(&Article{ArticleNumber: "A00000XX"}).Error
if err == nil {
t.Errorf("Expected second create to fail.")
}
if err != gorm.ErrDuplicatedKey {
t.Errorf("Expected error from second create to be gorm.ErrDuplicatedKey: %v", err)
}
}

View File

@@ -2,15 +2,18 @@ module github.com/ncruces/go-sqlite3/gormlite
go 1.21
toolchain go1.23.0
require (
github.com/ncruces/go-sqlite3 v0.16.1
gorm.io/gorm v1.25.10
github.com/ncruces/go-sqlite3 v0.21.0
gorm.io/gorm v1.25.12
)
require (
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/ncruces/julianday v1.0.0 // indirect
github.com/tetratelabs/wazero v1.7.3 // indirect
golang.org/x/sys v0.21.0 // indirect
github.com/tetratelabs/wazero v1.8.2 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/text v0.21.0 // indirect
)

View File

@@ -2,15 +2,15 @@ github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/ncruces/go-sqlite3 v0.16.1 h1:1wHv7s8y+fWK44UIliotJ42ZV41A5T0sjIAqGmnMrkc=
github.com/ncruces/go-sqlite3 v0.16.1/go.mod h1:feFXbBcbLtxNk6XWG1ROt8MS9+E45yCW3G8o4ixIqZ8=
github.com/ncruces/go-sqlite3 v0.21.0 h1:EwKFoy1hHEopN4sFZarmi+McXdbCcbTuLixhEayXVbQ=
github.com/ncruces/go-sqlite3 v0.21.0/go.mod h1:zxMOaSG5kFYVFK4xQa0pdwIszqxqJ0W0BxBgwdrNjuA=
github.com/ncruces/julianday v1.0.0 h1:fH0OKwa7NWvniGQtxdJRxAgkBMolni2BjDHaWTxqt7M=
github.com/ncruces/julianday v1.0.0/go.mod h1:Dusn2KvZrrovOMJuOt0TNXL6tB7U2E8kvza5fFc9G7g=
github.com/tetratelabs/wazero v1.7.3 h1:PBH5KVahrt3S2AHgEjKu4u+LlDbbk+nsGE3KLucy6Rw=
github.com/tetratelabs/wazero v1.7.3/go.mod h1:ytl6Zuh20R/eROuyDaGPkp82O9C/DJfXAwJfQ3X6/7Y=
golang.org/x/sys v0.21.0 h1:rF+pYz3DAGSQAxAu1CbC7catZg4ebC4UIeIhKxBZvws=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
gorm.io/gorm v1.25.10 h1:dQpO+33KalOA+aFYGlK+EfxcI5MbO7EP2yYygwh9h+s=
gorm.io/gorm v1.25.10/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
github.com/tetratelabs/wazero v1.8.2 h1:yIgLR/b2bN31bjxwXHD8a3d+BogigR952csSDdLYEv4=
github.com/tetratelabs/wazero v1.8.2/go.mod h1:yAI0XTsMBhREkM/YDAK/zNou3GoiAce1P6+rp/wQhjs=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
gorm.io/gorm v1.25.12 h1:I0u8i2hWQItBq1WfE0o2+WuL9+8L21K9e2HHSTE/0f8=
gorm.io/gorm v1.25.12/go.mod h1:xh7N7RHfYlNc5EmcI/El95gXusucDrQnHXe0+CgWcLQ=

View File

@@ -2,7 +2,6 @@
package gormlite
import (
"database/sql"
"strconv"
"gorm.io/gorm"
@@ -21,7 +20,7 @@ func Open(dsn string) gorm.Dialector {
}
// Open opens a GORM dialector from a database handle.
func OpenDB(db *sql.DB) gorm.Dialector {
func OpenDB(db gorm.ConnPool) gorm.Dialector {
return &_Dialector{Conn: db}
}
@@ -38,7 +37,7 @@ func (dialector _Dialector) Initialize(db *gorm.DB) (err error) {
if dialector.Conn != nil {
db.ConnPool = dialector.Conn
} else {
conn, err := driver.Open(dialector.DSN, nil)
conn, err := driver.Open(dialector.DSN)
if err != nil {
return err
}

View File

@@ -10,14 +10,14 @@ import (
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
)
func TestDialector(t *testing.T) {
// This is the DSN of the in-memory SQLite database for these tests.
const InMemoryDSN = "file:testdatabase?mode=memory&cache=shared"
tmp := memdb.TestDB(t)
// Custom connection with a custom function called "my_custom_function".
db, err := driver.Open(InMemoryDSN, func(conn *sqlite3.Conn) error {
db, err := driver.Open(tmp, func(conn *sqlite3.Conn) error {
return conn.CreateFunction("my_custom_function", 0, sqlite3.DETERMINISTIC,
func(ctx sqlite3.Context, arg ...sqlite3.Value) {
ctx.ResultText("my-result")
@@ -36,14 +36,14 @@ func TestDialector(t *testing.T) {
}{
{
description: "Default driver",
dialector: Open(InMemoryDSN),
dialector: Open(tmp),
openSuccess: true,
query: "SELECT 1",
querySuccess: true,
},
{
description: "Custom function",
dialector: Open(InMemoryDSN),
dialector: Open(tmp),
openSuccess: true,
query: "SELECT my_custom_function()",
querySuccess: false,

View File

@@ -7,7 +7,7 @@ rm -rf gorm/ tests/
go work use -r .
go test
git clone --branch v1.25.10 --filter=blob:none https://github.com/go-gorm/gorm.git
git clone --branch v1.25.12 --filter=blob:none https://github.com/go-gorm/gorm.git
mv gorm/tests tests
rm -rf gorm/

Some files were not shown because too many files have changed in this diff Show More