Compare commits

...

100 Commits

Author SHA1 Message Date
Nuno Cruces
ec609ea131 F2FS. 2024-04-16 02:52:37 +01:00
Nuno Cruces
7bab8bb949 wazero 1.7.1. 2024-04-16 01:59:29 +01:00
Nuno Cruces
ce97e820d5 wasi-sdk-22. 2024-04-16 01:58:49 +01:00
Nuno Cruces
7d8249efa5 SQLite 3.45.3. 2024-04-16 01:58:48 +01:00
Nuno Cruces
d2362b0311 Use coroutines. 2024-04-16 01:05:24 +01:00
Nuno Cruces
17f1681477 Remove test. 2024-04-15 13:55:12 +01:00
Nuno Cruces
cc0b011e8d Readonly WAL. 2024-04-13 17:11:13 +01:00
dependabot[bot]
46086916d4 Bump cross-platform-actions/action from 0.23.0 to 0.24.0 (#76)
Bumps [cross-platform-actions/action](https://github.com/cross-platform-actions/action) from 0.23.0 to 0.24.0.
- [Release notes](https://github.com/cross-platform-actions/action/releases)
- [Changelog](https://github.com/cross-platform-actions/action/blob/master/changelog.md)
- [Commits](https://github.com/cross-platform-actions/action/compare/v0.23.0...v0.24.0)

---
updated-dependencies:
- dependency-name: cross-platform-actions/action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-13 00:45:41 +01:00
Nuno Cruces
4322c71a09 Fix race. 2024-04-12 16:28:53 +01:00
Nuno Cruces
da9077cbea Fix repeat runs. 2024-04-12 15:54:48 +01:00
Nuno Cruces
1c3ad12434 WAL and vacuum hooks. 2024-04-12 15:02:01 +01:00
Nuno Cruces
7260962aba Update README.md 2024-04-11 15:18:49 +01:00
Nuno Cruces
e503be641a Refactors. 2024-04-11 12:00:17 +01:00
Nuno Cruces
11c03a16f9 Implement shared memory WAL. (#71)
- enabled by default on 64-bit macOS and Linux (`amd64`/`arm64`)
- depends on merged but unreleased wazero
- may cause small performance regression
- users may need WithMemoryLimitPages if not enough address space available
- needs docs
2024-04-10 13:15:36 +01:00
dependabot[bot]
f1c376cb49 Bump golang.org/x/sync from 0.6.0 to 0.7.0 (#72)
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.6.0 to 0.7.0.
- [Commits](https://github.com/golang/sync/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-04 23:52:16 +01:00
dependabot[bot]
91fd1457aa Bump golang.org/x/crypto from 0.21.0 to 0.22.0 (#74)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.21.0 to 0.22.0.
- [Commits](https://github.com/golang/crypto/compare/v0.21.0...v0.22.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-04 23:50:02 +01:00
Nuno Cruces
63938d5705 Simplify tests. 2024-04-04 01:25:52 +01:00
Nuno Cruces
10daa594f5 Gorm v1.25.8. 2024-03-28 00:10:07 +00:00
Nuno Cruces
2c2b6835b4 Tweaks, docs. 2024-03-27 07:54:15 +00:00
Nuno Cruces
af7fc3dcb7 Remove deprecations. 2024-03-27 07:54:08 +00:00
Nuno Cruces
0f9ce387b9 Documentation. 2024-03-22 00:21:00 +00:00
Nuno Cruces
b7d22e8fbf Fdatasync. 2024-03-21 15:04:59 +00:00
Nuno Cruces
617982f947 F2FS atomic writes. (#66)
https://sqlite.org/cgi/src/technote/714f6cbbf7
2024-03-21 13:59:47 +00:00
Nuno Cruces
36583542e1 Updated dependencies. 2024-03-17 16:34:02 +00:00
Nuno Cruces
fd3a3a3499 Locking improvements. 2024-03-17 16:17:48 +00:00
dependabot[bot]
ec96c77715 Bump github.com/tetratelabs/wazero from 1.7.0-pre.1 to 1.7.0 (#65)
Bumps [github.com/tetratelabs/wazero](https://github.com/tetratelabs/wazero) from 1.7.0-pre.1 to 1.7.0.
- [Release notes](https://github.com/tetratelabs/wazero/releases)
- [Commits](https://github.com/tetratelabs/wazero/compare/v1.7.0-pre.1...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/tetratelabs/wazero
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 23:23:02 +00:00
Nuno Cruces
c61f7b90f6 Locking improvements. 2024-03-15 16:14:49 +00:00
Nuno Cruces
7bd31c3443 Use strchrnul. 2024-03-15 14:15:12 +00:00
Nuno Cruces
f2f698b78a Remove clear. 2024-03-15 14:13:00 +00:00
Nuno Cruces
846b95d2d4 Persistent WAL. 2024-03-14 14:31:16 +00:00
Nuno Cruces
b9453aefb6 SQLite 3.45.2.
Also, remove FTS3/4.
2024-03-12 14:24:11 +00:00
Nuno Cruces
28b6fedef0 wazero v1.7.0-pre.1. (#60)
This enables the wazevo next gen compiler.
2024-03-09 10:11:50 +00:00
Nuno Cruces
fed9ce6e1c Backport date from 3.46. 2024-03-07 03:04:28 -08:00
dependabot[bot]
0ec08c2e74 Bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#63)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.20.0 to 0.21.0.
- [Commits](https://github.com/golang/crypto/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-04 23:10:29 +00:00
Nuno Cruces
4439cd302c binaryen-version_117. 2024-02-28 12:48:05 +00:00
dependabot[bot]
705eab456a Bump golang.org/x/crypto from 0.19.0 to 0.20.0 (#62)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.19.0 to 0.20.0.
- [Commits](https://github.com/golang/crypto/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-26 23:04:54 +00:00
Nuno Cruces
d1d5e355c4 CI testing. 2024-02-24 00:22:40 +00:00
Nuno Cruces
d3da8cc4f3 BSD. 2024-02-23 23:56:54 +00:00
Nuno Cruces
6def6f735c Warn about Git LFS. 2024-02-20 10:08:42 +00:00
Nuno Cruces
e02c5b5db0 BSD. 2024-02-19 00:19:36 +00:00
Nuno Cruces
52d42e4b21 Naming. 2024-02-10 10:09:53 +00:00
Nuno Cruces
396e6537b4 GORM v1.25.7. (#59) 2024-02-10 10:03:12 +00:00
Nuno Cruces
78cb9abefd Lebesgue/Morton order. 2024-02-08 00:39:39 +00:00
dependabot[bot]
b76cb33e62 Bump golang.org/x/crypto from 0.18.0 to 0.19.0 (#57)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.18.0 to 0.19.0.
- [Commits](https://github.com/golang/crypto/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-07 23:21:55 +00:00
Nuno Cruces
c7eea620a3 Tweaks. 2024-02-05 12:47:55 +00:00
Nuno Cruces
76e2733fef Updated dependencies. 2024-02-03 02:12:35 +00:00
Nuno Cruces
58df08329d SQLite 3.45.1. 2024-02-03 02:00:45 +00:00
Nuno Cruces
cdab468a92 Flaky tests. 2024-02-03 02:00:40 +00:00
Nuno Cruces
7438fdb664 Busy handlers. 2024-02-03 00:34:15 +00:00
Nuno Cruces
da0e98f17e BSD tests. 2024-02-02 19:10:56 +00:00
Nuno Cruces
bb0c77c6fa Test on M1. 2024-02-02 17:49:46 +00:00
Nuno Cruces
ea8894162b Update dependencies. 2024-02-02 17:43:24 +00:00
Nuno Cruces
9898fbfffa Rename. 2024-02-02 17:40:55 +00:00
Nuno Cruces
031087327d Update, authorizer callbacks. 2024-01-27 10:57:46 +00:00
Nuno Cruces
c9cc893ed7 Commit callback. 2024-01-27 10:05:31 +00:00
Nuno Cruces
99ad7ff766 Collation callback. 2024-01-26 23:52:45 +00:00
Nuno Cruces
019c71fb55 Towards callbacks. 2024-01-26 15:41:36 +00:00
Nuno Cruces
88cf845651 JSON stats. 2024-01-25 11:03:33 +00:00
Nuno Cruces
354242e528 Locking tweaks. 2024-01-24 09:58:05 +00:00
Nuno Cruces
3d906d47dd Avoid allocations. 2024-01-23 17:50:11 +00:00
Nuno Cruces
9df3488964 Backport ISO week from 3.46. 2024-01-22 11:21:30 +00:00
Nuno Cruces
d998b5f36c Limits, and tweaks. 2024-01-18 15:53:00 +00:00
Nuno Cruces
9f58a5d669 Rename. 2024-01-18 15:11:04 +00:00
Nuno Cruces
7d52cb259b Updated dependencies. 2024-01-17 16:36:26 +00:00
Nuno Cruces
35bbd8a0b0 New APIs. 2024-01-17 15:39:13 +00:00
Nuno Cruces
bce66299ab Remove flag. 2024-01-17 10:52:47 +00:00
Nuno Cruces
bc840dcefb SQLite 3.45.0. 2024-01-16 15:53:47 +00:00
Nuno Cruces
c822fa95c7 Batch column scans. (#52) 2024-01-16 15:18:14 +00:00
Nuno Cruces
1b2c267b2b Optimize interrupts. 2024-01-16 15:08:26 +00:00
Nuno Cruces
3d99af86bf Ensure arena alignment. 2024-01-15 10:43:36 +00:00
Nuno Cruces
145bc228af Avoid allocation. 2024-01-12 13:35:21 +00:00
Nuno Cruces
6b0c2c0554 Optimize. (#51) 2024-01-11 02:18:12 +00:00
Nuno Cruces
97f2b73701 Optimize. 2024-01-10 16:53:18 +00:00
Nuno Cruces
cb1e33a32d Benchmarks. 2024-01-10 12:27:19 +00:00
Nuno Cruces
ee48dd5c96 More stats. 2024-01-10 11:39:26 +00:00
Nuno Cruces
af42af2978 More stats. 2024-01-09 03:20:59 +00:00
dependabot[bot]
d48a92fcdf Bump golang.org/x/crypto from 0.17.0 to 0.18.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.17.0 to 0.18.0.
- [Commits](https://github.com/golang/crypto/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-08 23:05:11 +00:00
Nuno Cruces
69937fbee5 More vtab API. 2024-01-08 19:23:56 +00:00
dependabot[bot]
2fb325b223 Bump golang.org/x/sync from 0.5.0 to 0.6.0
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.5.0 to 0.6.0.
- [Commits](https://github.com/golang/sync/compare/v0.5.0...v0.6.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-05 02:16:24 +00:00
dependabot[bot]
f0c583a581 Bump golang.org/x/sys from 0.15.0 to 0.16.0
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.15.0 to 0.16.0.
- [Commits](https://github.com/golang/sys/compare/v0.15.0...v0.16.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-05 02:00:28 +00:00
Nuno Cruces
17ce949c55 Create osutil. 2024-01-03 12:54:26 +00:00
Nuno Cruces
ae850191c8 Refactor extensions. 2024-01-03 12:43:03 +00:00
Nuno Cruces
fab70ddbec IEEE754 extension. 2023-12-30 10:50:35 +00:00
Nuno Cruces
a3c5f47d79 Update README.md 2023-12-30 00:47:16 +00:00
Nuno Cruces
16b5d80ef7 Internal JSON and pointer wrappers. 2023-12-29 23:42:37 +00:00
Nuno Cruces
7e5a143214 Hash functions. 2023-12-29 23:42:30 +00:00
dependabot[bot]
92d75f7446 Bump cross-platform-actions/action from 0.21.1 to 0.22.0
Bumps [cross-platform-actions/action](https://github.com/cross-platform-actions/action) from 0.21.1 to 0.22.0.
- [Release notes](https://github.com/cross-platform-actions/action/releases)
- [Changelog](https://github.com/cross-platform-actions/action/blob/master/changelog.md)
- [Commits](https://github.com/cross-platform-actions/action/compare/v0.21.1...v0.22.0)

---
updated-dependencies:
- dependency-name: cross-platform-actions/action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-28 00:16:27 +00:00
Nuno Cruces
d56ee4ac2c Error logging. 2023-12-27 14:16:00 +00:00
Nuno Cruces
e944d5d8e7 Config. 2023-12-23 14:53:15 +00:00
Nuno Cruces
fde2277b4a wazero v1.6.0. 2023-12-23 13:19:33 +00:00
Nuno Cruces
1ebdeed565 Documentation, issue #45. 2023-12-22 02:45:26 +00:00
Nuno Cruces
89202629ec Increase various limits, fix #45. 2023-12-21 15:08:19 +00:00
Nuno Cruces
cb62771a45 Examples. 2023-12-20 16:59:16 +00:00
Nuno Cruces
0bb1cd5e2e Rework error messages, see #45. 2023-12-20 16:10:50 +00:00
Danlock
7bbd4f1e3c Fix regex link typo 2023-12-19 16:01:58 +00:00
Nuno Cruces
ed4a3a894b Extension API tweaks. 2023-12-19 15:24:54 +00:00
Nuno Cruces
f1b00a9944 wasi-sdk-21. 2023-12-19 00:33:04 +00:00
Nuno Cruces
9281948f57 Extension API tweaks. 2023-12-19 00:13:51 +00:00
Nuno Cruces
b0b27439b5 Fix macOS osAllocate.
Mozilla is just wrong.
https://searchfox.org/mozilla-central/source/xpcom/glue/FileUtils.cpp
2023-12-17 05:19:27 +00:00
Nuno Cruces
c938577763 Update README.md 2023-12-15 11:05:53 +00:00
172 changed files with 5426 additions and 1793 deletions

11
.github/workflows/bsd.sh vendored Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
echo 'set -euo pipefail' > test.sh
for p in $(go list ./...); do
dir=".${p#github.com/ncruces/go-sqlite3}"
name="$(basename "$p").test"
(cd ${dir}; GOOS=freebsd go test -c)
[ -f "${dir}/${name}" ] && echo "(cd ${dir}; ./${name} -test.v)" >> test.sh
done

View File

@@ -1,24 +0,0 @@
name: BSD
on:
workflow_dispatch:
jobs:
test:
runs-on: macos-12
steps:
- uses: actions/checkout@v4
with:
lfs: 'true'
- name: Test
uses: cross-platform-actions/action@v0.21.1
with:
operating_system: freebsd
version: '13.2'
memory: 8G
sync_files: runner-to-vm
run: |
sudo pkg install -y go121
go121 test -v ./...

View File

@@ -1,40 +0,0 @@
name: CPUs
on:
workflow_dispatch:
jobs:
test-386:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
lfs: 'true'
- name: Set up
uses: actions/setup-go@v5
with:
go-version: stable
- name: Test
run: GOARCH=386 go test -v ./...
test-arm:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
lfs: 'true'
- name: Set up
uses: actions/setup-go@v5
with:
go-version: stable
- name: Install QEMU
uses: docker/setup-qemu-action@v3
- name: Test
run: GOARCH=arm64 go test -v ./...

View File

@@ -1,4 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
echo android ; GOOS=android GOARCH=amd64 go build .
echo darwin ; GOOS=darwin GOARCH=amd64 go build .
@@ -12,11 +13,13 @@ echo openbsd ; GOOS=openbsd GOARCH=amd64 go build .
echo plan9 ; GOOS=plan9 GOARCH=amd64 go build .
echo solaris ; GOOS=solaris GOARCH=amd64 go build .
echo windows ; GOOS=windows GOARCH=amd64 go build .
# echo aix ; GOOS=aix GOARCH=ppc64 go build .
echo aix ; GOOS=aix GOARCH=ppc64 go build .
echo js ; GOOS=js GOARCH=wasm go build .
echo wasip1 ; GOOS=wasip1 GOARCH=wasm go build .
echo darwin-flock ; GOOS=darwin GOARCH=amd64 go build -tags sqlite3_flock .
echo darwin-noshm ; GOOS=darwin GOARCH=amd64 go build -tags sqlite3_noshm .
echo darwin-nosys ; GOOS=darwin GOARCH=amd64 go build -tags sqlite3_nosys .
echo linux-noshm ; GOOS=linux GOARCH=amd64 go build -tags sqlite3_noshm .
echo linux-nosys ; GOOS=linux GOARCH=amd64 go build -tags sqlite3_nosys .
echo windows-nosys ; GOOS=windows GOARCH=amd64 go build -tags sqlite3_nosys .
echo freebsd-nosys ; GOOS=freebsd GOARCH=amd64 go build -tags sqlite3_nosys .

View File

@@ -10,8 +10,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Set up
uses: actions/setup-go@v5
- uses: actions/setup-go@v5
with:
go-version: stable

View File

@@ -1,63 +0,0 @@
name: Go
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
test:
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
with:
lfs: 'true'
- name: Set up
uses: actions/setup-go@v5
with:
go-version: stable
- name: Format
run: gofmt -s -w . && git diff --exit-code
if: matrix.os != 'windows-latest'
- name: Tidy
run: go mod tidy && git diff --exit-code
- name: Download
run: go mod download
- name: Verify
run: go mod verify
- name: Vet
run: go vet ./...
- name: Build
run: go build -v ./...
- name: Test
run: go test -v ./...
- name: Test no locks
run: go test -v -tags sqlite3_nosys ./tests -run TestDB_nolock
- name: Test BSD locks
run: go test -v -tags sqlite3_flock ./...
if: matrix.os == 'macos-latest'
- name: Coverage report
uses: ncruces/go-coverage-report@v0
with:
chart: true
amend: true
if: |
github.event_name == 'push' &&
matrix.os == 'ubuntu-latest'

View File

@@ -2,22 +2,22 @@
set -euo pipefail
if [[ "$OSTYPE" == "linux"* ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-linux.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_116/binaryen-version_116-x86_64-linux.tar.gz"
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-22/wasi-sdk-22.0-linux.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_117/binaryen-version_117-x86_64-linux.tar.gz"
elif [[ "$OSTYPE" == "darwin"* ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-macos.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_116/binaryen-version_116-x86_64-macos.tar.gz"
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-22/wasi-sdk-22.0-macos.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_117/binaryen-version_117-x86_64-macos.tar.gz"
elif [[ "$OSTYPE" == "msys" || "$OSTYPE" == "cygwin" ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0.m-mingw.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_116/binaryen-version_116-x86_64-windows.tar.gz"
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-22/wasi-sdk-22.0.m-mingw.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_117/binaryen-version_117-x86_64-windows.tar.gz"
fi
# Download tools
mkdir -p tools
mkdir -p tools/
[ -d "tools/wasi-sdk"* ] || curl -#L "$WASI_SDK" | tar xzC tools &
[ -d "tools/binaryen-version"* ] || curl -#L "$BINARYEN" | tar xzC tools &
wait
sqlite3/download.sh # Download SQLite
embed/build.sh # Build WASM
embed/build.sh # Build Wasm
git diff --exit-code # Check diffs

View File

@@ -15,8 +15,7 @@ jobs:
with:
lfs: 'true'
- name: Set up
uses: actions/setup-go@v5
- uses: actions/setup-go@v5
with:
go-version: stable

133
.github/workflows/test.yml vendored Normal file
View File

@@ -0,0 +1,133 @@
name: Test
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
test:
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
with: { lfs: 'true' }
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Format
run: gofmt -s -w . && git diff --exit-code
if: matrix.os != 'windows-latest'
- name: Tidy
run: go mod tidy && git diff --exit-code
- name: Download
run: go mod download
- name: Verify
run: go mod verify
- name: Vet
run: go vet ./...
- name: Build
run: go build -v ./...
- name: Test
run: go test -v ./...
- name: Test BSD locks
run: go test -v -tags sqlite3_flock ./...
if: matrix.os == 'macos-latest'
- name: Test no shared memory
run: go test -v -tags sqlite3_noshm ./...
if: matrix.os == 'ubuntu-latest'
- name: Test no locks
run: go test -v -tags sqlite3_nosys ./tests -run TestDB_nolock
- name: Test GORM
run: gormlite/test.sh
- uses: ncruces/go-coverage-report@v0
with:
chart: true
amend: true
if: |
github.event_name == 'push' &&
matrix.os == 'ubuntu-latest'
test-bsd:
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
with: { lfs: 'true' }
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Build
run: .github/workflows/bsd.sh
- name: Test
uses: cross-platform-actions/action@v0.24.0
with:
operating_system: freebsd
version: '14.0'
shell: bash
run: source test.sh
sync_files: runner-to-vm
test-m1:
runs-on: macos-14
needs: test
steps:
- uses: actions/checkout@v4
with: { lfs: 'true' }
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Test
run: go test -v ./...
test-386:
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
with: { lfs: 'true' }
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Test
run: GOARCH=386 go test -v -short ./...
test-arm:
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
with: { lfs: 'true' }
- uses: actions/setup-go@v5
with: { go-version: stable }
- uses: docker/setup-qemu-action@v3
- name: Test
run: GOARCH=arm64 go test -v -short ./...

View File

@@ -4,8 +4,14 @@
[![Go Report](https://goreportcard.com/badge/github.com/ncruces/go-sqlite3)](https://goreportcard.com/report/github.com/ncruces/go-sqlite3)
[![Go Coverage](https://github.com/ncruces/go-sqlite3/wiki/coverage.svg)](https://github.com/ncruces/go-sqlite3/wiki/Test-coverage-report)
Go module `github.com/ncruces/go-sqlite3` wraps a [WASM](https://webassembly.org/) build of [SQLite](https://sqlite.org/),
and uses [wazero](https://wazero.io/) to provide `cgo`-free SQLite bindings.
Go module `github.com/ncruces/go-sqlite3` is `cgo`-free [SQLite](https://sqlite.org/) wrapper.\
It provides a [`database/sql`](https://pkg.go.dev/database/sql) compatible driver,
as well as direct access to most of the [C SQLite API](https://sqlite.org/cintro.html).
It wraps a [Wasm](https://webassembly.org/) build of SQLite, and uses [wazero](https://wazero.io/) as the runtime.\
Go, wazero and [`x/sys`](https://pkg.go.dev/golang.org/x/sys) are the _only_ runtime dependencies.
### Packages
- [`github.com/ncruces/go-sqlite3`](https://pkg.go.dev/github.com/ncruces/go-sqlite3)
wraps the [C SQLite API](https://sqlite.org/cintro.html)
@@ -20,7 +26,7 @@ and uses [wazero](https://wazero.io/) to provide `cgo`-free SQLite bindings.
- [`github.com/ncruces/go-sqlite3/gormlite`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/gormlite)
provides a [GORM](https://gorm.io) driver.
### Loadable extensions
### Extensions
- [`github.com/ncruces/go-sqlite3/ext/array`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/array)
provides the [`array`](https://sqlite.org/carray.html) table-valued function.
@@ -30,6 +36,8 @@ and uses [wazero](https://wazero.io/) to provide `cgo`-free SQLite bindings.
reads [comma-separated values](https://sqlite.org/csv.html).
- [`github.com/ncruces/go-sqlite3/ext/fileio`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/fileio)
reads, writes and lists files.
- [`github.com/ncruces/go-sqlite3/ext/hash`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/hash)
provides cryptographic hash functions.
- [`github.com/ncruces/go-sqlite3/ext/lines`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/lines)
reads data [line-by-line](https://github.com/asg017/sqlite-lines).
- [`github.com/ncruces/go-sqlite3/ext/pivot`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/pivot)
@@ -40,6 +48,8 @@ and uses [wazero](https://wazero.io/) to provide `cgo`-free SQLite bindings.
provides [statistics](https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html) functions.
- [`github.com/ncruces/go-sqlite3/ext/unicode`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/unicode)
provides [Unicode aware](https://sqlite.org/src/dir/ext/icu) functions.
- [`github.com/ncruces/go-sqlite3/ext/zorder`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/zorder)
maps multidimensional data to one dimension.
- [`github.com/ncruces/go-sqlite3/vfs/memdb`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/memdb)
implements an in-memory VFS.
- [`github.com/ncruces/go-sqlite3/vfs/readervfs`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readervfs)
@@ -47,64 +57,42 @@ and uses [wazero](https://wazero.io/) to provide `cgo`-free SQLite bindings.
### Advanced features
- [x] [incremental BLOB I/O](https://sqlite.org/c3ref/blob_open.html)
- [x] [nested transactions](https://sqlite.org/lang_savepoint.html)
- [x] [custom functions](https://sqlite.org/c3ref/create_function.html)
- [x] [virtual tables](https://sqlite.org/vtab.html)
- [x] [custom VFSes](https://sqlite.org/vfs.html)
- [x] [online backup](https://sqlite.org/backup.html)
- [x] [JSON support](https://sqlite.org/json1.html)
- [x] [Unicode support](https://sqlite.org/src/dir/ext/icu)
- [incremental BLOB I/O](https://sqlite.org/c3ref/blob_open.html)
- [nested transactions](https://sqlite.org/lang_savepoint.html)
- [custom functions](https://sqlite.org/c3ref/create_function.html)
- [virtual tables](https://sqlite.org/vtab.html)
- [custom VFSes](https://sqlite.org/vfs.html)
- [online backup](https://sqlite.org/backup.html)
- [JSON support](https://sqlite.org/json1.html)
- [math functions](https://sqlite.org/lang_mathfunc.html)
- [full-text search](https://sqlite.org/fts5.html)
- [geospatial search](https://sqlite.org/geopoly.html)
- [and more…](embed/README.md)
### Caveats
This module replaces the SQLite [OS Interface](https://sqlite.org/vfs.html)
(aka VFS) with a [pure Go](vfs/) implementation.
This has benefits, but also comes with some drawbacks.
(aka VFS) with a [pure Go](vfs/) implementation,
which has advantages and disadvantages.
#### Write-Ahead Logging
Read more about the Go VFS design [here](vfs/README.md).
Because WASM does not support shared memory,
[WAL](https://sqlite.org/wal.html) support is [limited](https://sqlite.org/wal.html#noshm).
### Testing
To work around this limitation, SQLite is [patched](sqlite3/locking_mode.patch)
to always use `EXCLUSIVE` locking mode for WAL databases.
This project aims for [high test coverage](https://github.com/ncruces/go-sqlite3/wiki/Test-coverage-report).
It also benefits greatly from [SQLite's](https://sqlite.org/testing.html) and
[wazero's](https://tetrate.io/blog/introducing-wazero-from-tetrate/#:~:text=Rock%2Dsolid%20test%20approach) thorough testing.
Because connection pooling is incompatible with `EXCLUSIVE` locking mode,
to use the [`database/sql`](https://pkg.go.dev/database/sql) driver
with WAL mode databases you should disable connection pooling by calling
[`db.SetMaxOpenConns(1)`](https://pkg.go.dev/database/sql#DB.SetMaxOpenConns).
#### File Locking
POSIX advisory locks, which SQLite uses on Unix, are
[broken by design](https://sqlite.org/src/artifact/2e8b12?ln=1073-1161).
On Linux, macOS and illumos, this module uses
[OFD locks](https://www.gnu.org/software/libc/manual/html_node/Open-File-Description-Locks.html)
to synchronize access to database files.
OFD locks are fully compatible with POSIX advisory locks.
On BSD Unixes, this module uses
[BSD locks](https://man.freebsd.org/cgi/man.cgi?query=flock&sektion=2).
On BSD Unixes, BSD locks are fully compatible with POSIX advisory locks.
On Windows, this module uses `LockFile`, `LockFileEx`, and `UnlockFile`,
like SQLite.
On all other platforms, file locking is not supported, and you must use
[`nolock=1`](https://sqlite.org/uri.html#urinolock)
to open database files.
To use the [`database/sql`](https://pkg.go.dev/database/sql) driver
with `nolock=1` you must disable connection pooling by calling
[`db.SetMaxOpenConns(1)`](https://pkg.go.dev/database/sql#DB.SetMaxOpenConns).
#### Testing
The pure Go VFS is tested by running SQLite's
The Go VFS is tested by running SQLite's
[mptest](https://github.com/sqlite/sqlite/blob/master/mptest/mptest.c)
on Linux, macOS, Windows and FreeBSD.
Performance is tested by running
### Performance
Perfomance of the [`database/sql`](https://pkg.go.dev/database/sql) driver is
[competitive](https://github.com/cvilsmeier/go-sqlite-bench) with alternatives.
The Wasm and VFS layers are also tested by running SQLite's
[speedtest1](https://github.com/sqlite/sqlite/blob/master/test/speedtest1.c).
### Alternatives

View File

@@ -121,7 +121,7 @@ func (b *Backup) Step(nPage int) (done bool, err error) {
// https://sqlite.org/c3ref/backup_finish.html#sqlite3backupremaining
func (b *Backup) Remaining() int {
r := b.c.call("sqlite3_backup_remaining", uint64(b.handle))
return int(r)
return int(int32(r))
}
// PageCount returns the total number of pages in the source database
@@ -130,5 +130,5 @@ func (b *Backup) Remaining() int {
// https://sqlite.org/c3ref/backup_finish.html#sqlite3backuppagecount
func (b *Backup) PageCount() int {
r := b.c.call("sqlite3_backup_pagecount", uint64(b.handle))
return int(r)
return int(int32(r))
}

164
config.go Normal file
View File

@@ -0,0 +1,164 @@
package sqlite3
import (
"context"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/tetratelabs/wazero/api"
)
// Config makes configuration changes to a database connection.
// Only boolean configuration options are supported.
// Called with no arg reads the current configuration value,
// called with one arg sets and returns the new value.
//
// https://sqlite.org/c3ref/db_config.html
func (c *Conn) Config(op DBConfig, arg ...bool) (bool, error) {
defer c.arena.mark()()
argsPtr := c.arena.new(2 * ptrlen)
var flag int
switch {
case len(arg) == 0:
flag = -1
case arg[0]:
flag = 1
}
util.WriteUint32(c.mod, argsPtr+0*ptrlen, uint32(flag))
util.WriteUint32(c.mod, argsPtr+1*ptrlen, argsPtr)
r := c.call("sqlite3_db_config", uint64(c.handle),
uint64(op), uint64(argsPtr))
return util.ReadUint32(c.mod, argsPtr) != 0, c.error(r)
}
// ConfigLog sets up the error logging callback for the connection.
//
// https://sqlite.org/errlog.html
func (c *Conn) ConfigLog(cb func(code ExtendedErrorCode, msg string)) error {
var enable uint64
if cb != nil {
enable = 1
}
r := c.call("sqlite3_config_log_go", enable)
if err := c.error(r); err != nil {
return err
}
c.log = cb
return nil
}
func logCallback(ctx context.Context, mod api.Module, _, iCode, zMsg uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.log != nil {
msg := util.ReadString(mod, zMsg, _MAX_LENGTH)
c.log(xErrorCode(iCode), msg)
}
}
// Limit allows the size of various constructs to be
// limited on a connection by connection basis.
//
// https://sqlite.org/c3ref/limit.html
func (c *Conn) Limit(id LimitCategory, value int) int {
r := c.call("sqlite3_limit", uint64(c.handle), uint64(id), uint64(value))
return int(int32(r))
}
// SetAuthorizer registers an authorizer callback with the database connection.
//
// https://sqlite.org/c3ref/set_authorizer.html
func (c *Conn) SetAuthorizer(cb func(action AuthorizerActionCode, name3rd, name4th, schema, nameInner string) AuthorizerReturnCode) error {
var enable uint64
if cb != nil {
enable = 1
}
r := c.call("sqlite3_set_authorizer_go", uint64(c.handle), enable)
if err := c.error(r); err != nil {
return err
}
c.authorizer = cb
return nil
}
func authorizerCallback(ctx context.Context, mod api.Module, pDB uint32, action AuthorizerActionCode, zName3rd, zName4th, zSchema, zNameInner uint32) (rc AuthorizerReturnCode) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.authorizer != nil {
var name3rd, name4th, schema, nameInner string
if zName3rd != 0 {
name3rd = util.ReadString(mod, zName3rd, _MAX_NAME)
}
if zName4th != 0 {
name4th = util.ReadString(mod, zName4th, _MAX_NAME)
}
if zSchema != 0 {
schema = util.ReadString(mod, zSchema, _MAX_NAME)
}
if zNameInner != 0 {
nameInner = util.ReadString(mod, zNameInner, _MAX_NAME)
}
rc = c.authorizer(action, name3rd, name4th, schema, nameInner)
}
return rc
}
// WalCheckpoint checkpoints a WAL database.
//
// https://sqlite.org/c3ref/wal_checkpoint_v2.html
func (c *Conn) WalCheckpoint(schema string, mode CheckpointMode) (nLog, nCkpt int, err error) {
defer c.arena.mark()()
nLogPtr := c.arena.new(ptrlen)
nCkptPtr := c.arena.new(ptrlen)
schemaPtr := c.arena.string(schema)
r := c.call("sqlite3_wal_checkpoint_v2",
uint64(c.handle), uint64(schemaPtr), uint64(mode),
uint64(nLogPtr), uint64(nCkptPtr))
nLog = int(int32(util.ReadUint32(c.mod, nLogPtr)))
nCkpt = int(int32(util.ReadUint32(c.mod, nCkptPtr)))
return nLog, nCkpt, c.error(r)
}
// WalAutoCheckpoint configures WAL auto-checkpoints.
//
// https://sqlite.org/c3ref/wal_autocheckpoint.html
func (c *Conn) WalAutoCheckpoint(pages int) error {
r := c.call("sqlite3_wal_autocheckpoint", uint64(c.handle), uint64(pages))
return c.error(r)
}
// WalHook registers a callback function to be invoked
// each time data is committed to a database in WAL mode.
//
// https://sqlite.org/c3ref/wal_hook.html
func (c *Conn) WalHook(cb func(db *Conn, schema string, pages int) error) {
var enable uint64
if cb != nil {
enable = 1
}
c.call("sqlite3_wal_hook_go", uint64(c.handle), enable)
c.wal = cb
}
func walCallback(ctx context.Context, mod api.Module, _, pDB, zSchema uint32, pages int32) (rc uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.wal != nil {
schema := util.ReadString(mod, zSchema, _MAX_NAME)
err := c.wal(c, schema, int(pages))
_, rc = errorCode(err, ERROR)
}
return rc
}
// AutoVacuumPages registers a autovacuum compaction amount callback.
//
// https://sqlite.org/c3ref/autovacuum_pages.html
func (c *Conn) AutoVacuumPages(cb func(schema string, dbPages, freePages, bytesPerPage uint) uint) error {
funcPtr := util.AddHandle(c.ctx, cb)
r := c.call("sqlite3_autovacuum_pages_go", uint64(c.handle), uint64(funcPtr))
return c.error(r)
}
func autoVacuumCallback(ctx context.Context, mod api.Module, pApp, zSchema, nDbPage, nFreePage, nBytePerPage uint32) uint32 {
fn := util.GetHandle(ctx, pApp).(func(schema string, dbPages, freePages, bytesPerPage uint) uint)
schema := util.ReadString(mod, zSchema, _MAX_NAME)
return uint32(fn(schema, uint(nDbPage), uint(nFreePage), uint(nBytePerPage)))
}

147
conn.go
View File

@@ -4,8 +4,10 @@ import (
"context"
"errors"
"fmt"
"math"
"net/url"
"strings"
"time"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/tetratelabs/wazero/api"
@@ -18,9 +20,17 @@ import (
type Conn struct {
*sqlite
interrupt context.Context
pending *Stmt
arena arena
interrupt context.Context
pending *Stmt
busy func(int) bool
log func(xErrorCode, string)
collation func(*Conn, string)
authorizer func(AuthorizerActionCode, string, string, string, string) AuthorizerReturnCode
update func(AuthorizerActionCode, string, string, int64)
commit func() bool
rollback func()
wal func(*Conn, string, int) error
arena arena
handle uint32
}
@@ -193,6 +203,32 @@ func (c *Conn) PrepareFlags(sql string, flags PrepareFlag) (stmt *Stmt, tail str
return stmt, tail, nil
}
// DBName returns the schema name for n-th database on the database connection.
//
// https://sqlite.org/c3ref/db_name.html
func (c *Conn) DBName(n int) string {
r := c.call("sqlite3_db_name", uint64(c.handle), uint64(n))
ptr := uint32(r)
if ptr == 0 {
return ""
}
return util.ReadString(c.mod, ptr, _MAX_NAME)
}
// ReadOnly determines if a database is read-only.
//
// https://sqlite.org/c3ref/db_readonly.html
func (c *Conn) ReadOnly(schema string) (ro bool, ok bool) {
var ptr uint32
if schema != "" {
defer c.arena.mark()()
ptr = c.arena.string(schema)
}
r := c.call("sqlite3_db_readonly", uint64(c.handle), uint64(ptr))
return int32(r) > 0, int32(r) < 0
}
// GetAutocommit tests the connection for auto-commit mode.
//
// https://sqlite.org/c3ref/get_autocommit.html
@@ -210,6 +246,14 @@ func (c *Conn) LastInsertRowID() int64 {
return int64(r)
}
// SetLastInsertRowID allows the application to set the value returned by
// [Conn.LastInsertRowID].
//
// https://sqlite.org/c3ref/set_last_insert_rowid.html
func (c *Conn) SetLastInsertRowID(id int64) {
c.call("sqlite3_set_last_insert_rowid", uint64(c.handle), uint64(id))
}
// Changes returns the number of rows modified, inserted or deleted
// by the most recently completed INSERT, UPDATE or DELETE statement
// on the database connection.
@@ -220,6 +264,30 @@ func (c *Conn) Changes() int64 {
return int64(r)
}
// TotalChanges returns the number of rows modified, inserted or deleted
// by all INSERT, UPDATE or DELETE statements completed
// since the database connection was opened.
//
// https://sqlite.org/c3ref/total_changes.html
func (c *Conn) TotalChanges() int64 {
r := c.call("sqlite3_total_changes64", uint64(c.handle))
return int64(r)
}
// ReleaseMemory frees memory used by a database connection.
//
// https://sqlite.org/c3ref/db_release_memory.html
func (c *Conn) ReleaseMemory() error {
r := c.call("sqlite3_db_release_memory", uint64(c.handle))
return c.error(r)
}
// GetInterrupt gets the context set with [Conn.SetInterrupt],
// or nil if none was set.
func (c *Conn) GetInterrupt() context.Context {
return c.interrupt
}
// SetInterrupt interrupts a long-running query when a context is done.
//
// Subsequent uses of the connection will return [INTERRUPT]
@@ -243,51 +311,68 @@ func (c *Conn) SetInterrupt(ctx context.Context) (old context.Context) {
// A busy SQL statement prevents SQLite from ignoring an interrupt
// that comes before any other statements are started.
if c.pending == nil {
c.pending, _, _ = c.Prepare(`SELECT 1 UNION ALL SELECT 2`)
} else {
c.pending.Reset()
c.pending, _, _ = c.Prepare(`WITH RECURSIVE c(x) AS (VALUES(0) UNION ALL SELECT x FROM c) SELECT x FROM c`)
}
old = c.interrupt
c.interrupt = ctx
if ctx == nil || ctx.Done() == nil {
return old
}
c.pending.Step()
if old != nil && old.Done() != nil && (ctx == nil || ctx.Err() == nil) {
c.pending.Reset()
}
if ctx != nil && ctx.Done() != nil {
c.pending.Step()
}
return old
}
func progressCallback(ctx context.Context, mod api.Module, _ uint32) uint32 {
if c, ok := ctx.Value(connKey{}).(*Conn); ok {
if c.interrupt != nil && c.interrupt.Err() != nil {
return 1
}
}
return 0
}
func (c *Conn) checkInterrupt() {
if c.interrupt != nil && c.interrupt.Err() != nil {
c.call("sqlite3_interrupt", uint64(c.handle))
}
}
// Pragma executes a PRAGMA statement and returns any results.
//
// https://sqlite.org/pragma.html
func (c *Conn) Pragma(str string) ([]string, error) {
stmt, _, err := c.Prepare(`PRAGMA ` + str)
if err != nil {
return nil, err
func progressCallback(ctx context.Context, mod api.Module, pDB uint32) (interrupt uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.commit != nil {
if c.interrupt != nil && c.interrupt.Err() != nil {
interrupt = 1
}
}
defer stmt.Close()
return interrupt
}
var pragmas []string
for stmt.Step() {
pragmas = append(pragmas, stmt.ColumnText(0))
// BusyTimeout sets a busy timeout.
//
// https://sqlite.org/c3ref/busy_timeout.html
func (c *Conn) BusyTimeout(timeout time.Duration) error {
ms := min((timeout+time.Millisecond-1)/time.Millisecond, math.MaxInt32)
r := c.call("sqlite3_busy_timeout", uint64(c.handle), uint64(ms))
return c.error(r)
}
// BusyHandler registers a callback to handle [BUSY] errors.
//
// https://sqlite.org/c3ref/busy_handler.html
func (c *Conn) BusyHandler(cb func(count int) (retry bool)) error {
var enable uint64
if cb != nil {
enable = 1
}
return pragmas, stmt.Close()
r := c.call("sqlite3_busy_handler_go", uint64(c.handle), enable)
if err := c.error(r); err != nil {
return err
}
c.busy = cb
return nil
}
func busyCallback(ctx context.Context, mod api.Module, pDB uint32, count int32) (retry uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.busy != nil {
if c.busy(int(count)) {
retry = 1
}
}
return retry
}
func (c *Conn) error(rc uint64, sql ...string) error {

130
const.go
View File

@@ -9,10 +9,11 @@ const (
_UTF8 = 1
_MAX_NAME = 512 // Used for short strings: names, error messages…
_MAX_NAME = 1e6 // Self-imposed limit for most NUL terminated strings.
_MAX_LENGTH = 1e9
_MAX_SQL_LENGTH = 1e9
_MAX_ALLOCATION_SIZE = 0x7ffffeff
_MAX_FUNCTION_ARG = 100
ptrlen = 4
)
@@ -200,6 +201,133 @@ const (
STMTSTATUS_MEMUSED StmtStatus = 99
)
// DBConfig are the available database connection configuration options.
//
// https://sqlite.org/c3ref/c_dbconfig_defensive.html
type DBConfig uint32
const (
// DBCONFIG_MAINDBNAME DBConfig = 1000
// DBCONFIG_LOOKASIDE DBConfig = 1001
DBCONFIG_ENABLE_FKEY DBConfig = 1002
DBCONFIG_ENABLE_TRIGGER DBConfig = 1003
DBCONFIG_ENABLE_FTS3_TOKENIZER DBConfig = 1004
DBCONFIG_ENABLE_LOAD_EXTENSION DBConfig = 1005
DBCONFIG_NO_CKPT_ON_CLOSE DBConfig = 1006
DBCONFIG_ENABLE_QPSG DBConfig = 1007
DBCONFIG_TRIGGER_EQP DBConfig = 1008
DBCONFIG_RESET_DATABASE DBConfig = 1009
DBCONFIG_DEFENSIVE DBConfig = 1010
DBCONFIG_WRITABLE_SCHEMA DBConfig = 1011
DBCONFIG_LEGACY_ALTER_TABLE DBConfig = 1012
DBCONFIG_DQS_DML DBConfig = 1013
DBCONFIG_DQS_DDL DBConfig = 1014
DBCONFIG_ENABLE_VIEW DBConfig = 1015
DBCONFIG_LEGACY_FILE_FORMAT DBConfig = 1016
DBCONFIG_TRUSTED_SCHEMA DBConfig = 1017
DBCONFIG_STMT_SCANSTATUS DBConfig = 1018
DBCONFIG_REVERSE_SCANORDER DBConfig = 1019
)
// LimitCategory are the available run-time limit categories.
//
// https://sqlite.org/c3ref/c_limit_attached.html
type LimitCategory uint32
const (
LIMIT_LENGTH LimitCategory = 0
LIMIT_SQL_LENGTH LimitCategory = 1
LIMIT_COLUMN LimitCategory = 2
LIMIT_EXPR_DEPTH LimitCategory = 3
LIMIT_COMPOUND_SELECT LimitCategory = 4
LIMIT_VDBE_OP LimitCategory = 5
LIMIT_FUNCTION_ARG LimitCategory = 6
LIMIT_ATTACHED LimitCategory = 7
LIMIT_LIKE_PATTERN_LENGTH LimitCategory = 8
LIMIT_VARIABLE_NUMBER LimitCategory = 9
LIMIT_TRIGGER_DEPTH LimitCategory = 10
LIMIT_WORKER_THREADS LimitCategory = 11
)
// AuthorizerActionCode are the integer action codes
// that the authorizer callback may be passed.
//
// https://sqlite.org/c3ref/c_alter_table.html
type AuthorizerActionCode uint32
const (
/***************************************************** 3rd ************ 4th ***********/
AUTH_CREATE_INDEX AuthorizerActionCode = 1 /* Index Name Table Name */
AUTH_CREATE_TABLE AuthorizerActionCode = 2 /* Table Name NULL */
AUTH_CREATE_TEMP_INDEX AuthorizerActionCode = 3 /* Index Name Table Name */
AUTH_CREATE_TEMP_TABLE AuthorizerActionCode = 4 /* Table Name NULL */
AUTH_CREATE_TEMP_TRIGGER AuthorizerActionCode = 5 /* Trigger Name Table Name */
AUTH_CREATE_TEMP_VIEW AuthorizerActionCode = 6 /* View Name NULL */
AUTH_CREATE_TRIGGER AuthorizerActionCode = 7 /* Trigger Name Table Name */
AUTH_CREATE_VIEW AuthorizerActionCode = 8 /* View Name NULL */
AUTH_DELETE AuthorizerActionCode = 9 /* Table Name NULL */
AUTH_DROP_INDEX AuthorizerActionCode = 10 /* Index Name Table Name */
AUTH_DROP_TABLE AuthorizerActionCode = 11 /* Table Name NULL */
AUTH_DROP_TEMP_INDEX AuthorizerActionCode = 12 /* Index Name Table Name */
AUTH_DROP_TEMP_TABLE AuthorizerActionCode = 13 /* Table Name NULL */
AUTH_DROP_TEMP_TRIGGER AuthorizerActionCode = 14 /* Trigger Name Table Name */
AUTH_DROP_TEMP_VIEW AuthorizerActionCode = 15 /* View Name NULL */
AUTH_DROP_TRIGGER AuthorizerActionCode = 16 /* Trigger Name Table Name */
AUTH_DROP_VIEW AuthorizerActionCode = 17 /* View Name NULL */
AUTH_INSERT AuthorizerActionCode = 18 /* Table Name NULL */
AUTH_PRAGMA AuthorizerActionCode = 19 /* Pragma Name 1st arg or NULL */
AUTH_READ AuthorizerActionCode = 20 /* Table Name Column Name */
AUTH_SELECT AuthorizerActionCode = 21 /* NULL NULL */
AUTH_TRANSACTION AuthorizerActionCode = 22 /* Operation NULL */
AUTH_UPDATE AuthorizerActionCode = 23 /* Table Name Column Name */
AUTH_ATTACH AuthorizerActionCode = 24 /* Filename NULL */
AUTH_DETACH AuthorizerActionCode = 25 /* Database Name NULL */
AUTH_ALTER_TABLE AuthorizerActionCode = 26 /* Database Name Table Name */
AUTH_REINDEX AuthorizerActionCode = 27 /* Index Name NULL */
AUTH_ANALYZE AuthorizerActionCode = 28 /* Table Name NULL */
AUTH_CREATE_VTABLE AuthorizerActionCode = 29 /* Table Name Module Name */
AUTH_DROP_VTABLE AuthorizerActionCode = 30 /* Table Name Module Name */
AUTH_FUNCTION AuthorizerActionCode = 31 /* NULL Function Name */
AUTH_SAVEPOINT AuthorizerActionCode = 32 /* Operation Savepoint Name */
AUTH_COPY AuthorizerActionCode = 0 /* No longer used */
AUTH_RECURSIVE AuthorizerActionCode = 33 /* NULL NULL */
)
// AuthorizerReturnCode are the integer codes
// that the authorizer callback may return.
//
// https://sqlite.org/c3ref/c_deny.html
type AuthorizerReturnCode uint32
const (
AUTH_OK AuthorizerReturnCode = 0
AUTH_DENY AuthorizerReturnCode = 1 /* Abort the SQL statement with an error */
AUTH_IGNORE AuthorizerReturnCode = 2 /* Don't allow access, but don't generate an error */
)
// CheckpointMode are all the checkpoint mode values.
//
// https://sqlite.org/c3ref/c_checkpoint_full.html
type CheckpointMode uint32
const (
CHECKPOINT_PASSIVE CheckpointMode = 0 /* Do as much as possible w/o blocking */
CHECKPOINT_FULL CheckpointMode = 1 /* Wait for writers, then checkpoint */
CHECKPOINT_RESTART CheckpointMode = 2 /* Like FULL but wait for readers */
CHECKPOINT_TRUNCATE CheckpointMode = 3 /* Like RESTART but also truncate WAL */
)
// TxnState are the allowed return values from [Conn.TxnState].
//
// https://sqlite.org/c3ref/c_txn_none.html
type TxnState uint32
const (
TXN_NONE TxnState = 0
TXN_READ TxnState = 1
TXN_WRITE TxnState = 2
)
// Datatype is a fundamental datatype of SQLite.
//
// https://sqlite.org/c3ref/c_blob.html

View File

@@ -184,7 +184,7 @@ func (ctx Context) ResultJSON(value any) {
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultValue(value Value) {
if value.sqlite != ctx.c.sqlite {
if value.c != ctx.c {
ctx.ResultError(MISUSE)
return
}
@@ -218,3 +218,12 @@ func (ctx Context) ResultError(err error) {
uint64(ctx.handle), uint64(code))
}
}
// VTabNoChange may return true if a column is being fetched as part
// of an update during which the column value will not change.
//
// https://sqlite.org/c3ref/vtab_nochange.html
func (ctx Context) VTabNoChange() bool {
r := ctx.c.call("sqlite3_vtab_nochange", uint64(ctx.handle))
return r != 0
}

View File

@@ -50,6 +50,7 @@ import (
"net/url"
"strings"
"time"
"unsafe"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
@@ -169,7 +170,7 @@ func (n *connector) Connect(ctx context.Context) (_ driver.Conn, err error) {
defer c.Conn.SetInterrupt(old)
if !n.pragmas {
err = c.Conn.Exec(`PRAGMA busy_timeout=60000`)
err = c.Conn.BusyTimeout(60 * time.Second)
if err != nil {
return nil, err
}
@@ -417,10 +418,10 @@ func (s *stmt) setupBindings(args []driver.NamedValue) error {
err = s.Stmt.BindZeroBlob(id, int64(a))
case time.Time:
err = s.Stmt.BindTime(id, a, s.tmWrite)
case interface{ Pointer() any }:
err = s.Stmt.BindPointer(id, a.Pointer())
case interface{ JSON() any }:
err = s.Stmt.BindJSON(id, a.JSON())
case util.JSON:
err = s.Stmt.BindJSON(id, a.Value)
case util.PointerUnwrap:
err = s.Stmt.BindPointer(id, util.UnwrapPointer(a))
case nil:
err = s.Stmt.BindNull(id)
default:
@@ -437,9 +438,8 @@ func (s *stmt) setupBindings(args []driver.NamedValue) error {
func (s *stmt) CheckNamedValue(arg *driver.NamedValue) error {
switch arg.Value.(type) {
case bool, int, int64, float64, string, []byte,
sqlite3.ZeroBlob, time.Time,
interface{ Pointer() any },
interface{ JSON() any },
time.Time, sqlite3.ZeroBlob,
util.JSON, util.PointerUnwrap,
nil:
return nil
default:
@@ -533,46 +533,39 @@ func (r *rows) Next(dest []driver.Value) error {
return io.EOF
}
data := unsafe.Slice((*any)(unsafe.SliceData(dest)), len(dest))
err := r.Stmt.Columns(data)
for i := range dest {
t := r.Stmt.ColumnType(i)
if tm, ok := r.decodeTime(i, t); ok {
dest[i] = tm
if t, ok := r.decodeTime(i, dest[i]); ok {
dest[i] = t
continue
}
switch t {
case sqlite3.INTEGER:
dest[i] = r.Stmt.ColumnInt64(i)
case sqlite3.FLOAT:
dest[i] = r.Stmt.ColumnFloat(i)
case sqlite3.BLOB:
dest[i] = r.Stmt.ColumnRawBlob(i)
case sqlite3.TEXT:
dest[i] = stringOrTime(r.Stmt.ColumnText(i))
case sqlite3.NULL:
dest[i] = nil
default:
panic(util.AssertErr())
if s, ok := dest[i].(string); ok {
t, ok := maybeTime(s)
if ok {
dest[i] = t
}
}
}
return r.Stmt.Err()
return err
}
func (r *rows) decodeTime(i int, typ sqlite3.Datatype) (_ time.Time, _ bool) {
func (r *rows) decodeTime(i int, v any) (_ time.Time, _ bool) {
if r.tmRead == sqlite3.TimeFormatDefault {
return
}
switch typ {
case sqlite3.INTEGER, sqlite3.FLOAT, sqlite3.TEXT:
// maybe
default:
return
}
switch r.declType(i) {
case "DATE", "TIME", "DATETIME", "TIMESTAMP":
// maybe
default:
return
}
return r.Stmt.ColumnTime(i, r.tmRead), r.Stmt.Err() == nil
switch v.(type) {
case int64, float64, string:
// maybe
default:
return
}
t, err := r.tmRead.Decode(v)
return t, err == nil
}

View File

@@ -153,7 +153,7 @@ func Test_BeginTx(t *testing.T) {
t.Fatal(err)
}
_, err = tx1.Exec(`CREATE TABLE IF NOT EXISTS test (col)`)
_, err = tx1.Exec(`CREATE TABLE test (col)`)
if err == nil {
t.Error("want error")
}
@@ -310,7 +310,7 @@ func Test_time(t *testing.T) {
twosday := time.Date(2022, 2, 22, 22, 22, 22, 0, time.UTC)
_, err = db.Exec(`CREATE TABLE IF NOT EXISTS test (at DATETIME)`)
_, err = db.Exec(`CREATE TABLE test (at DATETIME)`)
if err != nil {
t.Fatal(err)
}

View File

@@ -18,7 +18,7 @@ func Example_json() {
defer db.Close()
_, err = db.Exec(`
CREATE TABLE IF NOT EXISTS orders (
CREATE TABLE orders (
cart_id INTEGER PRIMARY KEY,
user_id INTEGER NOT NULL,
cart TEXT

View File

@@ -16,7 +16,7 @@ func ExampleSavepoint() {
}
defer db.Close()
_, err = db.Exec(`CREATE TABLE IF NOT EXISTS users (id INT, name VARCHAR(10))`)
_, err = db.Exec(`CREATE TABLE users (id INT, name VARCHAR(10))`)
if err != nil {
log.Fatal(err)
}

View File

@@ -1,7 +1,6 @@
package driver
import (
"database/sql/driver"
"time"
)
@@ -9,23 +8,24 @@ import (
// if it roundtrips back to the same string.
// This way times can be persisted to, and recovered from, the database,
// but if a string is needed, [database/sql] will recover the same string.
func stringOrTime(text string) driver.Value {
func maybeTime(text string) (_ time.Time, _ bool) {
// Weed out (some) values that can't possibly be
// [time.RFC3339Nano] timestamps.
if len(text) < len("2006-01-02T15:04:05Z") {
return text
return
}
if len(text) > len(time.RFC3339Nano) {
return text
return
}
if text[4] != '-' || text[10] != 'T' || text[16] != ':' {
return text
return
}
// Slow path.
var buf [len(time.RFC3339Nano)]byte
date, err := time.Parse(time.RFC3339Nano, text)
if err == nil && date.Format(time.RFC3339Nano) == text {
return date
if err == nil && text == string(date.AppendFormat(buf[:0], time.RFC3339Nano)) {
return date, true
}
return text
return
}

View File

@@ -22,26 +22,18 @@ func Fuzz_stringOrTime_1(f *testing.F) {
f.Add("Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.")
f.Fuzz(func(t *testing.T, str string) {
value := stringOrTime(str)
switch v := value.(type) {
case time.Time:
v, ok := maybeTime(str)
if ok {
// Make sure times round-trip to the same string:
// https://pkg.go.dev/database/sql#Rows.Scan
if v.Format(time.RFC3339Nano) != str {
t.Fatalf("did not round-trip: %q", str)
}
case string:
if v != str {
t.Fatalf("did not round-trip: %q", str)
}
} else {
date, err := time.Parse(time.RFC3339Nano, str)
if err == nil && date.Format(time.RFC3339Nano) == str {
t.Fatalf("would round-trip: %q", str)
}
default:
t.Fatalf("invalid type %T: %q", v, str)
}
})
}
@@ -59,24 +51,20 @@ func Fuzz_stringOrTime_2(f *testing.F) {
f.Add(int64(-763421161058), int64(222_222_222)) // twosday, year 22222BC
checkTime := func(t testing.TB, date time.Time) {
value := stringOrTime(date.Format(time.RFC3339Nano))
switch v := value.(type) {
case time.Time:
v, ok := maybeTime(date.Format(time.RFC3339Nano))
if ok {
// Make sure times round-trip to the same time:
if !v.Equal(date) {
t.Fatalf("did not round-trip: %v", date)
}
// Make with the same zone offset:
// With the same zone offset:
_, off1 := v.Zone()
_, off2 := date.Zone()
if off1 != off2 {
t.Fatalf("did not round-trip: %v", date)
}
case string:
} else {
t.Fatalf("was not recovered: %v", date)
default:
t.Fatalf("invalid type %T: %v", v, date)
}
}

View File

@@ -1,17 +1,18 @@
# Embeddable WASM build of SQLite
# Embeddable Wasm build of SQLite
This folder includes an embeddable WASM build of SQLite 3.44.2 for use with
This folder includes an embeddable Wasm build of SQLite 3.45.3 for use with
[`github.com/ncruces/go-sqlite3`](https://pkg.go.dev/github.com/ncruces/go-sqlite3).
The following optional features are compiled in:
- [math functions](https://sqlite.org/lang_mathfunc.html)
- [FTS3/4](https://sqlite.org/fts3.html)/[5](https://sqlite.org/fts5.html)
- [FTS5](https://sqlite.org/fts5.html)
- [JSON](https://sqlite.org/json1.html)
- [R*Tree](https://sqlite.org/rtree.html)
- [GeoPoly](https://sqlite.org/geopoly.html)
- [soundex](https://sqlite.org/lang_corefunc.html#soundex)
- [base64](https://github.com/sqlite/sqlite/blob/master/ext/misc/base64.c)
- [decimal](https://github.com/sqlite/sqlite/blob/master/ext/misc/decimal.c)
- [ieee754](https://github.com/sqlite/sqlite/blob/master/ext/misc/ieee754.c)
- [regexp](https://github.com/sqlite/sqlite/blob/master/ext/misc/regexp.c)
- [series](https://github.com/sqlite/sqlite/blob/master/ext/misc/series.c)
- [uint](https://github.com/sqlite/sqlite/blob/master/ext/misc/uint.c)

View File

@@ -4,10 +4,11 @@ set -euo pipefail
cd -P -- "$(dirname -- "$0")"
ROOT=../
BINARYEN="$ROOT/tools/binaryen-version_116/bin"
WASI_SDK="$ROOT/tools/wasi-sdk-20.0/bin"
BINARYEN="$ROOT/tools/binaryen-version_117/bin"
WASI_SDK="$ROOT/tools/wasi-sdk-22.0/bin"
"$WASI_SDK/clang" --target=wasm32-wasi -flto -g0 -O2 \
"$WASI_SDK/clang" --target=wasm32-wasi -std=c17 -flto -g0 -O2 \
-Wall -Wextra -Wno-unused-parameter \
-o sqlite3.wasm "$ROOT/sqlite3/main.c" \
-I"$ROOT/sqlite3" \
-mexec-model=reactor \

View File

@@ -1,8 +1,9 @@
aligned_alloc
free
malloc
malloc_destructor
sqlite3_aggregate_context
sqlite3_anycollseq_init
sqlite3_autovacuum_pages_go
sqlite3_backup_finish
sqlite3_backup_init
sqlite3_backup_pagecount
@@ -25,10 +26,13 @@ sqlite3_blob_open
sqlite3_blob_read
sqlite3_blob_reopen
sqlite3_blob_write
sqlite3_busy_handler_go
sqlite3_busy_timeout
sqlite3_changes64
sqlite3_clear_bindings
sqlite3_close
sqlite3_close_v2
sqlite3_collation_needed_go
sqlite3_column_blob
sqlite3_column_bytes
sqlite3_column_count
@@ -39,11 +43,18 @@ sqlite3_column_name
sqlite3_column_text
sqlite3_column_type
sqlite3_column_value
sqlite3_columns_go
sqlite3_commit_hook_go
sqlite3_config_log_go
sqlite3_create_aggregate_function_go
sqlite3_create_collation_go
sqlite3_create_function_go
sqlite3_create_module_go
sqlite3_create_window_function_go
sqlite3_db_config
sqlite3_db_name
sqlite3_db_readonly
sqlite3_db_release_memory
sqlite3_declare_vtab
sqlite3_errcode
sqlite3_errmsg
@@ -55,6 +66,7 @@ sqlite3_get_autocommit
sqlite3_get_auxdata
sqlite3_interrupt
sqlite3_last_insert_rowid
sqlite3_limit
sqlite3_open_v2
sqlite3_overload_function
sqlite3_prepare_v3
@@ -72,14 +84,19 @@ sqlite3_result_pointer_go
sqlite3_result_text64
sqlite3_result_value
sqlite3_result_zeroblob64
sqlite3_rollback_hook_go
sqlite3_set_authorizer_go
sqlite3_set_auxdata_go
sqlite3_set_last_insert_rowid
sqlite3_step
sqlite3_stmt_busy
sqlite3_stmt_readonly
sqlite3_stmt_status
sqlite3_total_changes64
sqlite3_txn_state
sqlite3_update_hook_go
sqlite3_uri_key
sqlite3_uri_parameter
sqlite3_user_data
sqlite3_value_blob
sqlite3_value_bytes
sqlite3_value_double
@@ -87,6 +104,7 @@ sqlite3_value_dup
sqlite3_value_free
sqlite3_value_int64
sqlite3_value_nochange
sqlite3_value_numeric_type
sqlite3_value_pointer_go
sqlite3_value_text
sqlite3_value_type
@@ -98,4 +116,7 @@ sqlite3_vtab_in_first
sqlite3_vtab_in_next
sqlite3_vtab_nochange
sqlite3_vtab_on_conflict
sqlite3_vtab_rhs_value
sqlite3_vtab_rhs_value
sqlite3_wal_autocheckpoint
sqlite3_wal_checkpoint_v2
sqlite3_wal_hook_go

Binary file not shown.

View File

@@ -138,14 +138,14 @@ func (e ExtendedErrorCode) Timeout() bool {
func errorCode(err error, def ErrorCode) (msg string, code uint32) {
switch code := err.(type) {
case nil:
return "", _OK
case ErrorCode:
return "", uint32(code)
case ExtendedErrorCode:
case xErrorCode:
return "", uint32(code)
case *Error:
return code.msg, uint32(code.code)
case nil:
return "", _OK
}
var ecode ErrorCode

View File

@@ -16,7 +16,7 @@ func Example() {
log.Fatal(err)
}
err = db.Exec(`CREATE TABLE IF NOT EXISTS users (id INT, name VARCHAR(10))`)
err = db.Exec(`CREATE TABLE users (id INT, name VARCHAR(10))`)
if err != nil {
log.Fatal(err)
}

View File

@@ -8,15 +8,17 @@ import (
"reflect"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers the array single-argument, table-valued SQL function.
// The argument must be an [sqlite3.Pointer] to a Go slice or array
// of ints, floats, bools, strings or blobs.
// The argument must be bound to a Go slice or array of
// ints, floats, bools, strings or byte slices,
// using [sqlite3.BindPointer] or [sqlite3.Pointer].
func Register(db *sqlite3.Conn) {
sqlite3.CreateModule[array](db, "array", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (array, error) {
err := db.DeclareVtab(`CREATE TABLE x(value, array HIDDEN)`)
err := db.DeclareVTab(`CREATE TABLE x(value, array HIDDEN)`)
return array{}, err
})
}
@@ -102,7 +104,7 @@ func (c *cursor) Column(ctx *sqlite3.Context, n int) error {
ctx.ResultBlob(v.Bytes())
default:
return fmt.Errorf("array: unsupported element:%.0w %v", sqlite3.MISMATCH, v.Type())
return fmt.Errorf("array: unsupported element:%.0w %v", sqlite3.MISMATCH, util.ReflectType(v))
}
return nil
}
@@ -120,16 +122,15 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
}
func indexable(v reflect.Value) (reflect.Value, error) {
if v.Kind() == reflect.Slice {
switch v.Kind() {
case reflect.Slice:
return v, nil
}
if v.Kind() == reflect.Array {
case reflect.Array:
return v, nil
}
if v.Kind() == reflect.Pointer {
case reflect.Pointer:
if v := v.Elem(); v.Kind() == reflect.Array {
return v, nil
}
}
return v, fmt.Errorf("array: unsupported argument:%.0w %v", sqlite3.MISMATCH, v)
return v, fmt.Errorf("array: unsupported argument:%.0w %v", sqlite3.MISMATCH, util.ReflectType(v))
}

View File

@@ -13,7 +13,7 @@ import (
"github.com/ncruces/go-sqlite3/ext/array"
)
func Example() {
func Example_driver() {
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
array.Register(c)
return nil
@@ -51,6 +51,42 @@ func Example() {
// geopoly_within
}
func Example() {
db, err := sqlite3.Open(":memory:")
if err != nil {
log.Fatal(err)
}
defer db.Close()
array.Register(db)
stmt, _, err := db.Prepare(`
SELECT name
FROM pragma_function_list
WHERE name like 'geopoly%' AND narg IN array(?)`)
if err != nil {
log.Fatal(err)
}
defer stmt.Close()
err = stmt.BindPointer(1, [...]int{2, 3, 4})
if err != nil {
log.Fatal(err)
}
for stmt.Step() {
fmt.Printf("%s\n", stmt.ColumnText(0))
}
if err := stmt.Err(); err != nil {
log.Fatal(err)
}
// Unordered output:
// geopoly_regular
// geopoly_overlap
// geopoly_contains_point
// geopoly_within
}
func Test_cursor_Column(t *testing.T) {
t.Parallel()

View File

@@ -23,15 +23,16 @@ import (
//
// Opens blobs for reading or writing.
// The callback is invoked for each open blob,
// and must be an [sqlite3.Pointer] to an [OpenCallback].
// and must be bound to an [OpenCallback],
// using [sqlite3.BindPointer] or [sqlite3.Pointer].
// The optional args will be passed to the callback,
// along with the [sqlite3.Blob] handle.
//
// https://sqlite.org/c3ref/blob.html
func Register(db *sqlite3.Conn) {
db.CreateFunction("readblob", 6, sqlite3.DIRECTONLY, readblob)
db.CreateFunction("writeblob", 6, sqlite3.DIRECTONLY, writeblob)
db.CreateFunction("openblob", -1, sqlite3.DIRECTONLY, openblob)
db.CreateFunction("readblob", 6, 0, readblob)
db.CreateFunction("writeblob", 6, 0, writeblob)
db.CreateFunction("openblob", -1, 0, openblob)
}
// OpenCallback is the type for the openblob callback.
@@ -127,12 +128,12 @@ func getAuxBlob(ctx sqlite3.Context, arg []sqlite3.Value, write bool) (*sqlite3.
return ctx.Conn().OpenBlob(db, table, column, row, write)
}
func setAuxBlob(ctx sqlite3.Context, blob *sqlite3.Blob, writer bool) {
func setAuxBlob(ctx sqlite3.Context, blob *sqlite3.Blob, open bool) {
// This ensures the blob is closed if db, table, column or write change.
ctx.SetAuxData(0, blob) // db
ctx.SetAuxData(1, blob) // table
ctx.SetAuxData(2, blob) // column
if writer {
if open {
ctx.SetAuxData(4, blob) // write
}
}

View File

@@ -27,7 +27,7 @@ func Example() {
}
defer db.Close()
_, err = db.Exec(`CREATE TABLE IF NOT EXISTS test (col)`)
_, err = db.Exec(`CREATE TABLE test (col)`)
if err != nil {
log.Fatal(err)
}
@@ -79,8 +79,8 @@ func Test_readblob(t *testing.T) {
}
err = db.Exec(`
CREATE TABLE IF NOT EXISTS test1 (col);
CREATE TABLE IF NOT EXISTS test2 (col);
CREATE TABLE test1 (col);
CREATE TABLE test2 (col);
INSERT INTO test1 VALUES (x'cafe');
INSERT INTO test2 VALUES (x'babe');
`)
@@ -139,8 +139,8 @@ func Test_openblob(t *testing.T) {
}
err = db.Exec(`
CREATE TABLE IF NOT EXISTS test1 (col);
CREATE TABLE IF NOT EXISTS test2 (col);
CREATE TABLE test1 (col);
CREATE TABLE test2 (col);
INSERT INTO test1 VALUES (x'cafe');
INSERT INTO test2 VALUES (x'babe');
`)

36
ext/csv/arg.go Normal file
View File

@@ -0,0 +1,36 @@
package csv
import (
"fmt"
"strconv"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/vtabutil"
)
func uintArg(key, val string) (int, error) {
i, err := strconv.ParseUint(val, 10, 15)
if err != nil {
return 0, fmt.Errorf("csv: invalid %q parameter: %s", key, val)
}
return int(i), nil
}
func boolArg(key, val string) (bool, error) {
if val == "" {
return true, nil
}
b, ok := util.ParseBool(val)
if ok {
return b, nil
}
return false, fmt.Errorf("csv: invalid %q parameter: %s", key, val)
}
func runeArg(key, val string) (rune, error) {
r, _, tail, err := strconv.UnquoteChar(vtabutil.Unquote(val), 0)
if tail != "" || err != nil {
return 0, fmt.Errorf("csv: invalid %q parameter: %s", key, val)
}
return r, nil
}

View File

@@ -1,8 +1,12 @@
package csv
import "testing"
import (
"testing"
func Test_uintParam(t *testing.T) {
"github.com/ncruces/go-sqlite3/util/vtabutil"
)
func Test_uintArg(t *testing.T) {
t.Parallel()
tests := []struct {
@@ -20,22 +24,22 @@ func Test_uintParam(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.arg, func(t *testing.T) {
key, val := getParam(tt.arg)
key, val := vtabutil.NamedArg(tt.arg)
if key != tt.key {
t.Errorf("getParam() %v, want err %v", key, tt.key)
t.Errorf("NamedArg() %v, want err %v", key, tt.key)
}
got, err := uintParam(key, val)
got, err := uintArg(key, val)
if (err != nil) != tt.err {
t.Fatalf("uintParam() error = %v, want err %v", err, tt.err)
t.Fatalf("uintArg() error = %v, want err %v", err, tt.err)
}
if got != tt.val {
t.Errorf("uintParam() = %v, want %v", got, tt.val)
t.Errorf("uintArg() = %v, want %v", got, tt.val)
}
})
}
}
func Test_boolParam(t *testing.T) {
func Test_boolArg(t *testing.T) {
tests := []struct {
arg string
key string
@@ -56,22 +60,22 @@ func Test_boolParam(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.arg, func(t *testing.T) {
key, val := getParam(tt.arg)
key, val := vtabutil.NamedArg(tt.arg)
if key != tt.key {
t.Errorf("getParam() %v, want err %v", key, tt.key)
t.Errorf("NamedArg() %v, want err %v", key, tt.key)
}
got, err := boolParam(key, val)
got, err := boolArg(key, val)
if (err != nil) != tt.err {
t.Fatalf("boolParam() error = %v, want err %v", err, tt.err)
t.Fatalf("boolArg() error = %v, want err %v", err, tt.err)
}
if got != tt.val {
t.Errorf("boolParam() = %v, want %v", got, tt.val)
t.Errorf("boolArg() = %v, want %v", got, tt.val)
}
})
}
}
func Test_runeParam(t *testing.T) {
func Test_runeArg(t *testing.T) {
tests := []struct {
arg string
key string
@@ -88,16 +92,16 @@ func Test_runeParam(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.arg, func(t *testing.T) {
key, val := getParam(tt.arg)
key, val := vtabutil.NamedArg(tt.arg)
if key != tt.key {
t.Errorf("getParam() %v, want err %v", key, tt.key)
t.Errorf("NamedArg() %v, want err %v", key, tt.key)
}
got, err := runeParam(key, val)
got, err := runeArg(key, val)
if (err != nil) != tt.err {
t.Fatalf("runeParam() error = %v, want err %v", err, tt.err)
t.Fatalf("runeArg() error = %v, want err %v", err, tt.err)
}
if got != tt.val {
t.Errorf("runeParam() = %v, want %v", got, tt.val)
t.Errorf("runeArg() = %v, want %v", got, tt.val)
}
})
}

View File

@@ -7,27 +7,27 @@
package csv
import (
"bufio"
"encoding/csv"
"fmt"
"io"
"math"
"os"
"io/fs"
"strings"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/util/osutil"
"github.com/ncruces/go-sqlite3/util/vtabutil"
)
// Register registers the CSV virtual table.
// If a filename is specified, `os.Open` is used to read it from disk.
// If a filename is specified, [os.Open] is used to open the file.
func Register(db *sqlite3.Conn) {
RegisterOpen(db, func(name string) (io.ReaderAt, error) {
return os.Open(name)
})
RegisterFS(db, osutil.FS{})
}
// RegisterOpen registers the CSV virtual table.
// If a filename is specified, open is used to open the file.
func RegisterOpen(db *sqlite3.Conn, open func(name string) (io.ReaderAt, error)) {
// RegisterFS registers the CSV virtual table.
// If a filename is specified, fsys is used to open the file.
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
declare := func(db *sqlite3.Conn, _, _, _ string, arg ...string) (_ *table, err error) {
var (
filename string
@@ -41,23 +41,23 @@ func RegisterOpen(db *sqlite3.Conn, open func(name string) (io.ReaderAt, error))
)
for _, arg := range arg {
key, val := getParam(arg)
key, val := vtabutil.NamedArg(arg)
if _, ok := done[key]; ok {
return nil, fmt.Errorf("csv: more than one %q parameter", key)
}
switch key {
case "filename":
filename = unquoteParam(val)
filename = vtabutil.Unquote(val)
case "data":
data = unquoteParam(val)
data = vtabutil.Unquote(val)
case "schema":
schema = unquoteParam(val)
schema = vtabutil.Unquote(val)
case "header":
header, err = boolParam(key, val)
header, err = boolArg(key, val)
case "columns":
columns, err = uintParam(key, val)
columns, err = uintArg(key, val)
case "comma":
comma, err = runeParam(key, val)
comma, err = runeArg(key, val)
default:
return nil, fmt.Errorf("csv: unknown %q parameter", key)
}
@@ -71,32 +71,23 @@ func RegisterOpen(db *sqlite3.Conn, open func(name string) (io.ReaderAt, error))
return nil, fmt.Errorf(`csv: must specify either "filename" or "data" but not both`)
}
var r io.ReaderAt
if filename != "" {
r, err = open(filename)
} else {
r = strings.NewReader(data)
}
if err != nil {
return nil, err
}
table := &table{
r: r,
fsys: fsys,
name: filename,
data: data,
comma: comma,
header: header,
bom: -1,
}
defer func() {
if err != nil {
table.Close()
}
}()
if schema == "" {
var row []string
if header || columns < 0 {
row, err = table.newReader().Read()
csv, c, err := table.newReader()
defer c.Close()
if err != nil {
return nil, err
}
row, err = csv.Read()
if err != nil {
return nil, err
}
@@ -104,11 +95,11 @@ func RegisterOpen(db *sqlite3.Conn, open func(name string) (io.ReaderAt, error))
schema = getSchema(header, columns, row)
}
err = db.DeclareVtab(schema)
err = db.DeclareVTab(schema)
if err != nil {
return nil, err
}
err = db.VtabConfig(sqlite3.VTAB_DIRECTONLY)
err = db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
if err != nil {
return nil, err
}
@@ -119,19 +110,11 @@ func RegisterOpen(db *sqlite3.Conn, open func(name string) (io.ReaderAt, error))
}
type table struct {
r io.ReaderAt
fsys fs.FS
name string
data string
comma rune
header bool
bom int8
}
func (t *table) Close() error {
if c, ok := t.r.(io.Closer); ok {
err := c.Close()
t.r = nil
return err
}
return nil
}
func (t *table) BestIndex(idx *sqlite3.IndexInfo) error {
@@ -147,38 +130,76 @@ func (t *table) Rename(new string) error {
return nil
}
func (t *table) Integrity(schema, table string, flags int) (err error) {
if flags&1 == 0 {
_, err = t.newReader().ReadAll()
func (t *table) Integrity(schema, table string, flags int) error {
if flags&1 != 0 {
return nil
}
csv, c, err := t.newReader()
if err != nil {
return err
}
defer c.Close()
_, err = csv.ReadAll()
return err
}
func (t *table) newReader() (*csv.Reader, io.Closer, error) {
var r io.Reader
var c io.Closer
if t.name != "" {
f, err := t.fsys.Open(t.name)
if err != nil {
return nil, f, err
}
buf := bufio.NewReader(f)
bom, err := buf.Peek(3)
if err != nil {
return nil, f, err
}
if string(bom) == "\xEF\xBB\xBF" {
buf.Discard(3)
}
r = buf
c = f
} else {
r = strings.NewReader(t.data)
c = io.NopCloser(r)
}
csv := csv.NewReader(r)
csv.ReuseRecord = true
csv.Comma = t.comma
return csv, c, nil
}
type cursor struct {
table *table
closer io.Closer
csv *csv.Reader
row []string
rowID int64
}
func (c *cursor) Close() (err error) {
if c.closer != nil {
err = c.closer.Close()
c.closer = nil
}
return err
}
func (t *table) newReader() *csv.Reader {
if t.bom < 0 {
var bom [3]byte
t.r.ReadAt(bom[:], 0)
if string(bom[:]) == "\xEF\xBB\xBF" {
t.bom = 3
} else {
t.bom = 0
}
}
csv := csv.NewReader(io.NewSectionReader(t.r, int64(t.bom), math.MaxInt64))
csv.ReuseRecord = true
csv.Comma = t.comma
return csv
}
type cursor struct {
table *table
csv *csv.Reader
row []string
rowID int64
}
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
c.csv = c.table.newReader()
err := c.Close()
if err != nil {
return err
}
c.csv, c.closer, err = c.table.newReader()
if err != nil {
return err
}
if c.table.header {
c.Next() // skip header
}

View File

@@ -20,7 +20,7 @@ func Example() {
csv.Register(db)
err = db.Exec(`
CREATE VIRTUAL TABLE IF NOT EXISTS eurofxref USING csv(
CREATE VIRTUAL TABLE eurofxref USING csv(
filename = 'testdata/eurofxref.csv',
header = YES,
columns = 42,
@@ -61,7 +61,7 @@ func TestRegister(t *testing.T) {
csv.Register(db)
const data = "\xEF\xBB\xBF" + `
const data = `
"Rob" "Pike" rob
"Ken" Thompson ken
Robert "Griesemer" "gri"`
@@ -84,8 +84,8 @@ Robert "Griesemer" "gri"`
if !stmt.Step() {
t.Fatal("no rows")
}
if got := stmt.ColumnText(1); got != "Pike" {
t.Errorf("got %q want Pike", got)
if got := stmt.ColumnText(0); got != "Rob" {
t.Errorf("got %q want Rob", got)
}
if stmt.Step() {
t.Fatal("more rows")
@@ -98,12 +98,17 @@ Robert "Griesemer" "gri"`
err = db.Exec(`PRAGMA integrity_check`)
if err != nil {
t.Fatal(err)
t.Error(err)
}
err = db.Exec(`PRAGMA quick_check`)
if err != nil {
t.Error(err)
}
err = db.Exec(`DROP TABLE temp.csv`)
if err != nil {
log.Fatal(err)
t.Error(err)
}
}

View File

@@ -1,65 +0,0 @@
package csv
import (
"fmt"
"strconv"
"strings"
)
func getParam(arg string) (key, val string) {
key, val, _ = strings.Cut(arg, "=")
key = strings.TrimSpace(key)
val = strings.TrimSpace(val)
return
}
func uintParam(key, val string) (int, error) {
i, err := strconv.ParseUint(val, 10, 15)
if err != nil {
return 0, fmt.Errorf("csv: invalid %q parameter: %s", key, val)
}
return int(i), nil
}
func boolParam(key, val string) (bool, error) {
if val == "" || val == "1" ||
strings.EqualFold(val, "true") ||
strings.EqualFold(val, "yes") ||
strings.EqualFold(val, "on") {
return true, nil
}
if val == "0" ||
strings.EqualFold(val, "false") ||
strings.EqualFold(val, "no") ||
strings.EqualFold(val, "off") {
return false, nil
}
return false, fmt.Errorf("csv: invalid %q parameter: %s", key, val)
}
func runeParam(key, val string) (rune, error) {
r, _, tail, err := strconv.UnquoteChar(unquoteParam(val), 0)
if tail != "" || err != nil {
return 0, fmt.Errorf("csv: invalid %q parameter: %s", key, val)
}
return r, nil
}
func unquoteParam(val string) string {
if len(val) < 2 {
return val
}
if val[0] != val[len(val)-1] {
return val
}
var old, new string
switch val[0] {
default:
return val
case '"':
old, new = `""`, `"`
case '\'':
old, new = `''`, `'`
}
return strings.ReplaceAll(val[1:len(val)-1], old, new)
}

View File

@@ -1,4 +1,4 @@
Date,USD,JPY,BGN,CYP,CZK,DKK,EEK,GBP,HUF,LTL,LVL,MTL,PLN,ROL,RON,SEK,SIT,SKK,CHF,ISK,NOK,HRK,RUB,TRL,TRY,AUD,BRL,CAD,CNY,HKD,IDR,ILS,INR,KRW,MXN,MYR,NZD,PHP,SGD,THB,ZAR,
Date,USD,JPY,BGN,CYP,CZK,DKK,EEK,GBP,HUF,LTL,LVL,MTL,PLN,ROL,RON,SEK,SIT,SKK,CHF,ISK,NOK,HRK,RUB,TRL,TRY,AUD,BRL,CAD,CNY,HKD,IDR,ILS,INR,KRW,MXN,MYR,NZD,PHP,SGD,THB,ZAR,
2022-12-30,1.0666,140.66,1.9558,N/A,24.116,7.4365,N/A,0.88693,400.87,N/A,N/A,N/A,4.6808,N/A,4.9495,11.1218,N/A,N/A,0.9847,151.5,10.5138,7.5365,N/A,N/A,19.9649,1.5693,5.6386,1.444,7.3582,8.3163,16519.82,3.7554,88.171,1344.09,20.856,4.6984,1.6798,59.32,1.43,36.835,18.0986,
2022-12-29,1.0649,142.24,1.9558,N/A,24.191,7.4365,N/A,0.88549,399.6,N/A,N/A,N/A,4.6855,N/A,4.9493,11.158,N/A,N/A,0.984,152.5,10.55,7.5365,N/A,N/A,19.934,1.5859,5.5351,1.4475,7.4151,8.2994,16680.38,3.7575,88.2295,1350.18,20.651,4.7106,1.6887,59.367,1.436,36.877,18.1967,
2022-12-28,1.064,142.21,1.9558,N/A,24.252,7.4365,N/A,0.88058,403.3,N/A,N/A,N/A,4.7008,N/A,4.946,11.1038,N/A,N/A,0.9863,151.9,10.4495,7.5365,N/A,N/A,19.9144,1.566,5.6109,1.4361,7.4224,8.2931,16765.93,3.7526,88.0943,1348.59,20.6856,4.7055,1.6772,59.613,1.4323,36.953,18.289,
1 Date USD JPY BGN CYP CZK DKK EEK GBP HUF LTL LVL MTL PLN ROL RON SEK SIT SKK CHF ISK NOK HRK RUB TRL TRY AUD BRL CAD CNY HKD IDR ILS INR KRW MXN MYR NZD PHP SGD THB ZAR
2 2022-12-30 1.0666 140.66 1.9558 N/A 24.116 7.4365 N/A 0.88693 400.87 N/A N/A N/A 4.6808 N/A 4.9495 11.1218 N/A N/A 0.9847 151.5 10.5138 7.5365 N/A N/A 19.9649 1.5693 5.6386 1.444 7.3582 8.3163 16519.82 3.7554 88.171 1344.09 20.856 4.6984 1.6798 59.32 1.43 36.835 18.0986
3 2022-12-29 1.0649 142.24 1.9558 N/A 24.191 7.4365 N/A 0.88549 399.6 N/A N/A N/A 4.6855 N/A 4.9493 11.158 N/A N/A 0.984 152.5 10.55 7.5365 N/A N/A 19.934 1.5859 5.5351 1.4475 7.4151 8.2994 16680.38 3.7575 88.2295 1350.18 20.651 4.7106 1.6887 59.367 1.436 36.877 18.1967
4 2022-12-28 1.064 142.21 1.9558 N/A 24.252 7.4365 N/A 0.88058 403.3 N/A N/A N/A 4.7008 N/A 4.946 11.1038 N/A N/A 0.9863 151.9 10.4495 7.5365 N/A N/A 19.9144 1.566 5.6109 1.4361 7.4224 8.2931 16765.93 3.7526 88.0943 1348.59 20.6856 4.7055 1.6772 59.613 1.4323 36.953 18.289

68
ext/fileio/coro.go Normal file
View File

@@ -0,0 +1,68 @@
package fileio
import (
"fmt"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Adapted from: https://research.swtch.com/coro
const errCoroCanceled = util.ErrorString("coroutine canceled")
func coroNew[In, Out any](f func(In, func(Out) In) Out) (resume func(In) (Out, bool), cancel func()) {
type msg[T any] struct {
panic any
val T
}
cin := make(chan msg[In])
cout := make(chan msg[Out])
running := true
resume = func(in In) (out Out, ok bool) {
if !running {
return
}
cin <- msg[In]{val: in}
m := <-cout
if m.panic != nil {
panic(m.panic)
}
return m.val, running
}
cancel = func() {
if !running {
return
}
e := fmt.Errorf("%w", errCoroCanceled)
cin <- msg[In]{panic: e}
m := <-cout
if m.panic != nil && m.panic != e {
panic(m.panic)
}
}
yield := func(out Out) In {
cout <- msg[Out]{val: out}
m := <-cin
if m.panic != nil {
panic(m.panic)
}
return m.val
}
go func() {
defer func() {
if running {
running = false
cout <- msg[Out]{panic: recover()}
}
}()
var out Out
m := <-cin
if m.panic == nil {
out = f(m.val, yield)
}
running = false
cout <- msg[Out]{val: out}
}()
return resume, cancel
}

View File

@@ -13,23 +13,23 @@ import (
)
// Register registers SQL functions readfile, writefile, lsmode,
// and the eponymous virtual table fsdir.
// and the table-valued function fsdir.
func Register(db *sqlite3.Conn) {
RegisterFS(db, nil)
}
// Register registers SQL functions readfile, lsmode,
// and the eponymous virtual table fsdir;
// fs will be used to read files and list directories.
// and the table-valued function fsdir;
// fsys will be used to read files and list directories.
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
db.CreateFunction("lsmode", 1, 0, lsmode)
db.CreateFunction("lsmode", 1, sqlite3.DETERMINISTIC, lsmode)
db.CreateFunction("readfile", 1, sqlite3.DIRECTONLY, readfile(fsys))
if fsys == nil {
db.CreateFunction("writefile", -1, sqlite3.DIRECTONLY, writefile)
}
sqlite3.CreateModule(db, "fsdir", nil, func(db *sqlite3.Conn, module, schema, table string, arg ...string) (fsdir, error) {
err := db.DeclareVtab(`CREATE TABLE x(name,mode,mtime,data,path HIDDEN,dir HIDDEN)`)
db.VtabConfig(sqlite3.VTAB_DIRECTONLY)
sqlite3.CreateModule(db, "fsdir", nil, func(db *sqlite3.Conn, _, _, _ string, _ ...string) (fsdir, error) {
err := db.DeclareVTab(`CREATE TABLE x(name,mode,mtime TIMESTAMP,data,path HIDDEN,dir HIDDEN)`)
db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
return fsdir{fsys}, err
})
}

View File

@@ -48,32 +48,28 @@ func (d fsdir) BestIndex(idx *sqlite3.IndexInfo) error {
}
func (d fsdir) Open() (sqlite3.VTabCursor, error) {
return &cursor{fsys: d.fsys}, nil
return &cursor{fsdir: d}, nil
}
type cursor struct {
fsys fs.FS
base string
rowID int64
eof bool
curr entry
next chan entry
done chan struct{}
fsdir
base string
resume func(struct{}) (entry, bool)
cancel func()
curr entry
eof bool
rowID int64
}
type entry struct {
path string
fs.DirEntry
err error
err error
path string
}
func (c *cursor) Close() error {
if c.done != nil {
close(c.done)
s := <-c.next
c.done = nil
c.next = nil
return s.err
if c.cancel != nil {
c.cancel()
}
return nil
}
@@ -96,16 +92,25 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
c.base = base
}
c.rowID = 0
c.resume, c.cancel = coroNew(func(_ struct{}, yield func(entry) struct{}) entry {
walkDir := func(path string, d fs.DirEntry, err error) error {
yield(entry{d, err, path})
return nil
}
if c.fsys != nil {
fs.WalkDir(c.fsys, root, walkDir)
} else {
filepath.WalkDir(root, walkDir)
}
return entry{}
})
c.eof = false
c.next = make(chan entry)
c.done = make(chan struct{})
go c.WalkDir(root)
c.rowID = 0
return c.Next()
}
func (c *cursor) Next() error {
curr, ok := <-c.next
curr, ok := c.resume(struct{}{})
c.curr = curr
c.eof = !ok
c.rowID++
@@ -165,22 +170,3 @@ func (c *cursor) Column(ctx *sqlite3.Context, n int) error {
}
return nil
}
func (c *cursor) WalkDir(path string) {
defer close(c.next)
if c.fsys != nil {
fs.WalkDir(c.fsys, path, c.WalkDirFunc)
} else {
filepath.WalkDir(path, c.WalkDirFunc)
}
}
func (c *cursor) WalkDirFunc(path string, d fs.DirEntry, err error) error {
select {
case <-c.done:
return fs.SkipAll
case c.next <- entry{path, d, err}:
return nil
}
}

View File

@@ -28,7 +28,7 @@ func Test_fsdir(t *testing.T) {
}
defer db.Close()
rows, err := db.Query(`SELECT * FROM fsdir('.', '.') LIMIT 4`)
rows, err := db.Query(`SELECT * FROM fsdir('.', '.')`)
if err != nil {
t.Fatal(err)
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/fsutil"
)
func writefile(ctx sqlite3.Context, arg ...sqlite3.Value) {
@@ -22,7 +23,7 @@ func writefile(ctx sqlite3.Context, arg ...sqlite3.Value) {
var mode fs.FileMode
if len(arg) > 2 {
mode = fixMode(fs.FileMode(arg[2].Int()))
mode = fsutil.FileModeFromValue(arg[2])
}
n, err := createFileAndDir(file, mode, arg[1])
@@ -88,40 +89,6 @@ func createFile(path string, mode fs.FileMode, data sqlite3.Value) (int, error)
return 0, fmt.Errorf("invalid mode: %v", mode)
}
func fixMode(mode fs.FileMode) fs.FileMode {
const (
S_IFMT fs.FileMode = 0170000
S_IFIFO fs.FileMode = 0010000
S_IFCHR fs.FileMode = 0020000
S_IFDIR fs.FileMode = 0040000
S_IFBLK fs.FileMode = 0060000
S_IFREG fs.FileMode = 0100000
S_IFLNK fs.FileMode = 0120000
S_IFSOCK fs.FileMode = 0140000
)
switch mode & S_IFMT {
case S_IFDIR:
mode |= fs.ModeDir
case S_IFLNK:
mode |= fs.ModeSymlink
case S_IFBLK:
mode |= fs.ModeDevice
case S_IFCHR:
mode |= fs.ModeCharDevice | fs.ModeDevice
case S_IFIFO:
mode |= fs.ModeNamedPipe
case S_IFSOCK:
mode |= fs.ModeSocket
case S_IFREG, 0:
//
default:
mode |= fs.ModeIrregular
}
return mode &^ S_IFMT
}
func fixPerm(mode fs.FileMode, def fs.FileMode) fs.FileMode {
if mode.Perm() == 0 {
return def

View File

@@ -56,7 +56,7 @@ func Test_writefile(t *testing.T) {
var mode fs.FileMode
var mtime time.Time
var data sql.NullString
err := rows.Scan(&name, &mode, sqlite3.TimeFormatUnixFrac.Scanner(&mtime), &data)
err := rows.Scan(&name, &mode, &mtime, &data)
if err != nil {
t.Fatal(err)
}
@@ -90,26 +90,3 @@ func Test_writefile(t *testing.T) {
t.Log(err)
}
}
func Test_fixMode(t *testing.T) {
tests := []struct {
mode fs.FileMode
want fs.FileMode
}{
{0010754, 0754 | fs.ModeNamedPipe},
{0020754, 0754 | fs.ModeCharDevice | fs.ModeDevice},
{0040754, 0754 | fs.ModeDir},
{0060754, 0754 | fs.ModeDevice},
{0100754, 0754},
{0120754, 0754 | fs.ModeSymlink},
{0140754, 0754 | fs.ModeSocket},
{0170754, 0754 | fs.ModeIrregular},
}
for _, tt := range tests {
t.Run(tt.mode.String(), func(t *testing.T) {
if got := fixMode(tt.mode); got != tt.want {
t.Errorf("fixMode() = %o, want %o", got, tt.want)
}
})
}
}

30
ext/hash/blake2.go Normal file
View File

@@ -0,0 +1,30 @@
package hash
import (
"crypto"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
func blake2sFunc(ctx sqlite3.Context, arg ...sqlite3.Value) {
hashFunc(ctx, arg[0], crypto.BLAKE2s_256)
}
func blake2bFunc(ctx sqlite3.Context, arg ...sqlite3.Value) {
size := 512
if len(arg) > 1 {
size = arg[1].Int()
}
switch size {
case 256:
hashFunc(ctx, arg[0], crypto.BLAKE2b_256)
case 384:
hashFunc(ctx, arg[0], crypto.BLAKE2b_384)
case 512:
hashFunc(ctx, arg[0], crypto.BLAKE2b_512)
default:
ctx.ResultError(util.ErrorString("blake2b: size must be 256, 384, 512"))
}
}

97
ext/hash/hash.go Normal file
View File

@@ -0,0 +1,97 @@
// Package hash provides cryptographic hash functions.
//
// Provided functions:
// - md4(data)
// - md5(data)
// - sha1(data)
// - sha3(data, size) (default size 256)
// - sha224(data)
// - sha256(data, size) (default size 256)
// - sha384(data)
// - sha512(data, size) (default size 512)
// - blake2s(data)
// - blake2b(data, size) (default size 512)
// - ripemd160(data)
//
// Each SQL function will only be registered if the corresponding
// [crypto.Hash] function is available.
// To ensure a specific hash function is available,
// import the implementing package.
package hash
import (
"crypto"
"github.com/ncruces/go-sqlite3"
)
// Register registers cryptographic hash functions for a database connection.
func Register(db *sqlite3.Conn) {
flags := sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
if crypto.MD4.Available() {
db.CreateFunction("md4", 1, flags, md4Func)
}
if crypto.MD5.Available() {
db.CreateFunction("md5", 1, flags, md5Func)
}
if crypto.SHA1.Available() {
db.CreateFunction("sha1", 1, flags, sha1Func)
}
if crypto.SHA3_512.Available() {
db.CreateFunction("sha3", 1, flags, sha3Func)
db.CreateFunction("sha3", 2, flags, sha3Func)
}
if crypto.SHA256.Available() {
db.CreateFunction("sha224", 1, flags, sha224Func)
db.CreateFunction("sha256", 1, flags, sha256Func)
db.CreateFunction("sha256", 2, flags, sha256Func)
}
if crypto.SHA512.Available() {
db.CreateFunction("sha384", 1, flags, sha384Func)
db.CreateFunction("sha512", 1, flags, sha512Func)
db.CreateFunction("sha512", 2, flags, sha512Func)
}
if crypto.BLAKE2s_256.Available() {
db.CreateFunction("blake2s", 1, flags, blake2sFunc)
}
if crypto.BLAKE2b_512.Available() {
db.CreateFunction("blake2b", 1, flags, blake2bFunc)
db.CreateFunction("blake2b", 2, flags, blake2bFunc)
}
if crypto.RIPEMD160.Available() {
db.CreateFunction("ripemd160", 1, flags, ripemd160Func)
}
}
func md4Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
hashFunc(ctx, arg[0], crypto.MD4)
}
func md5Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
hashFunc(ctx, arg[0], crypto.MD5)
}
func sha1Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
hashFunc(ctx, arg[0], crypto.SHA1)
}
func ripemd160Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
hashFunc(ctx, arg[0], crypto.RIPEMD160)
}
func hashFunc(ctx sqlite3.Context, arg sqlite3.Value, fn crypto.Hash) {
var data []byte
switch arg.Type() {
case sqlite3.NULL:
return
case sqlite3.BLOB:
data = arg.RawBlob()
default:
data = arg.RawText()
}
h := fn.New()
h.Write(data)
ctx.ResultBlob(h.Sum(nil))
}

98
ext/hash/hash_test.go Normal file
View File

@@ -0,0 +1,98 @@
package hash
import (
_ "crypto/md5"
_ "crypto/sha1"
_ "crypto/sha256"
_ "crypto/sha512"
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "golang.org/x/crypto/blake2b"
_ "golang.org/x/crypto/blake2s"
_ "golang.org/x/crypto/md4"
_ "golang.org/x/crypto/ripemd160"
_ "golang.org/x/crypto/sha3"
)
func TestRegister(t *testing.T) {
t.Parallel()
tests := []struct {
name string
hash string
}{
{"md4(NULL)", ""},
{"md4(X'')", "31D6CFE0D16AE931B73C59D7E0C089C0"},
{"md4('The quick brown fox jumps over the lazy dog')", "1BEE69A46BA811185C194762ABAEAE90"},
{"md5('')", "D41D8CD98F00B204E9800998ECF8427E"},
{"sha1('')", "DA39A3EE5E6B4B0D3255BFEF95601890AFD80709"},
{"ripemd160('')", "9C1185A5C5E9FC54612808977EE8F548B2258D31"},
{"sha224('')", "D14A028C2A3A2BC9476102BB288234C415A2B01F828EA62AC5B3E42F"},
{"sha256('')", "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855"},
{"sha256('', 224)", "D14A028C2A3A2BC9476102BB288234C415A2B01F828EA62AC5B3E42F"},
{"sha384('')", "38B060A751AC96384CD9327EB1B1E36A21FDB71114BE07434C0CC7BF63F6E1DA274EDEBFE76F65FBD51AD2F14898B95B"},
{"sha512('')", "CF83E1357EEFB8BDF1542850D66D8007D620E4050B5715DC83F4A921D36CE9CE47D0D13C5D85F2B0FF8318D2877EEC2F63B931BD47417A81A538327AF927DA3E"},
{"sha512('', 224)", "6ED0DD02806FA89E25DE060C19D3AC86CABB87D6A0DDD05C333B84F4"},
{"sha512('', 256)", "C672B8D1EF56ED28AB87C3622C5114069BDD3AD7B8F9737498D0C01ECEF0967A"},
{"sha512('', 384)", "38B060A751AC96384CD9327EB1B1E36A21FDB71114BE07434C0CC7BF63F6E1DA274EDEBFE76F65FBD51AD2F14898B95B"},
{"sha3('')", "A7FFC6F8BF1ED76651C14756A061D662F580FF4DE43B49FA82D80A4B80F8434A"},
{"sha3('', 224)", "6B4E03423667DBB73B6E15454F0EB1ABD4597F9A1B078E3F5B5A6BC7"},
{"sha3('', 384)", "0C63A75B845E4F7D01107D852E4C2485C51A50AAAA94FC61995E71BBEE983A2AC3713831264ADB47FB6BD1E058D5F004"},
{"sha3('', 512)", "A69F73CCA23A9AC5C8B567DC185A756E97C982164FE25859E0D1DCC1475C80A615B2123AF1F5F94C11E3E9402C3AC558F500199D95B6D3E301758586281DCD26"},
{"blake2s('')", "69217A3079908094E11121D042354A7C1F55B6482CA1A51E1B250DFD1ED0EEF9"},
{"blake2b('')", "786A02F742015903C6C6FD852552D272912F4740E15847618A86E217F71F5419D25E1031AFEE585313896444934EB04B903A685B1448B755D56F701AFE9BE2CE"},
{"blake2b('', 384)", "B32811423377F52D7862286EE1A72EE540524380FDA1724A6F25D7978C6FD3244A6CAF0498812673C5E05EF583825100"},
{"blake2b('', 256)", "0E5751C026E543B2E8AB2EB06099DAA1D1E5DF47778F7787FAAB45CDF12FE3A8"},
}
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
Register(c)
return nil
})
if err != nil {
t.Fatal(err)
}
defer db.Close()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var hash string
err = db.QueryRow(`SELECT hex(` + tt.name + `)`).Scan(&hash)
if err != nil {
t.Fatal(err)
}
if hash != tt.hash {
t.Errorf("got %s, want %s", hash, tt.hash)
}
})
}
_, err = db.Exec(`SELECT sha256('', 255)`)
if err == nil {
t.Error("want error")
}
_, err = db.Exec(`SELECT sha512('', 255)`)
if err == nil {
t.Error("want error")
}
_, err = db.Exec(`SELECT sha3('', 255)`)
if err == nil {
t.Error("want error")
}
_, err = db.Exec(`SELECT blake2b('', 255)`)
if err == nil {
t.Error("want error")
}
}

53
ext/hash/sha2.go Normal file
View File

@@ -0,0 +1,53 @@
package hash
import (
"crypto"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
func sha224Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
hashFunc(ctx, arg[0], crypto.SHA224)
}
func sha384Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
hashFunc(ctx, arg[0], crypto.SHA384)
}
func sha256Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
size := 256
if len(arg) > 1 {
size = arg[1].Int()
}
switch size {
case 224:
hashFunc(ctx, arg[0], crypto.SHA224)
case 256:
hashFunc(ctx, arg[0], crypto.SHA256)
default:
ctx.ResultError(util.ErrorString("sha256: size must be 224, 256"))
}
}
func sha512Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
size := 512
if len(arg) > 1 {
size = arg[1].Int()
}
switch size {
case 224:
hashFunc(ctx, arg[0], crypto.SHA512_224)
case 256:
hashFunc(ctx, arg[0], crypto.SHA512_256)
case 384:
hashFunc(ctx, arg[0], crypto.SHA384)
case 512:
hashFunc(ctx, arg[0], crypto.SHA512)
default:
ctx.ResultError(util.ErrorString("sha512: size must be 224, 256, 384, 512"))
}
}

28
ext/hash/sha3.go Normal file
View File

@@ -0,0 +1,28 @@
package hash
import (
"crypto"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
func sha3Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
size := 256
if len(arg) > 1 {
size = arg[1].Int()
}
switch size {
case 224:
hashFunc(ctx, arg[0], crypto.SHA3_224)
case 256:
hashFunc(ctx, arg[0], crypto.SHA3_256)
case 384:
hashFunc(ctx, arg[0], crypto.SHA3_384)
case 512:
hashFunc(ctx, arg[0], crypto.SHA3_512)
default:
ctx.ResultError(util.ErrorString("sha3: size must be 224, 256, 384, 512"))
}
}

View File

@@ -15,31 +15,42 @@ import (
"bytes"
"fmt"
"io"
"math"
"os"
"io/fs"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/util/osutil"
)
// Register registers the lines and lines_read virtual tables.
// The lines virtual table reads from a database blob or text.
// The lines_read virtual table reads from a file or an [io.ReaderAt].
// Register registers the lines and lines_read table-valued functions.
// The lines function reads from a database blob or text.
// The lines_read function reads from a file or an [io.Reader].
// If a filename is specified, [os.Open] is used to open the file.
func Register(db *sqlite3.Conn) {
RegisterFS(db, osutil.FS{})
}
// RegisterFS registers the lines and lines_read table-valued functions.
// The lines function reads from a database blob or text.
// The lines_read function reads from a file or an [io.Reader].
// If a filename is specified, fsys is used to open the file.
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) {
sqlite3.CreateModule[lines](db, "lines", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (lines, error) {
err := db.DeclareVtab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
db.VtabConfig(sqlite3.VTAB_INNOCUOUS)
return false, err
err := db.DeclareVTab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
db.VTabConfig(sqlite3.VTAB_INNOCUOUS)
return lines{}, err
})
sqlite3.CreateModule[lines](db, "lines_read", nil,
func(db *sqlite3.Conn, _, _, _ string, _ ...string) (lines, error) {
err := db.DeclareVtab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
db.VtabConfig(sqlite3.VTAB_DIRECTONLY)
return true, err
err := db.DeclareVTab(`CREATE TABLE x(line TEXT, data HIDDEN)`)
db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
return lines{fsys}, err
})
}
type lines bool
type lines struct {
fsys fs.FS
}
func (l lines) BestIndex(idx *sqlite3.IndexInfo) error {
for i, cst := range idx.Constraint {
@@ -57,81 +68,126 @@ func (l lines) BestIndex(idx *sqlite3.IndexInfo) error {
}
func (l lines) Open() (sqlite3.VTabCursor, error) {
return &cursor{reader: bool(l)}, nil
if l.fsys != nil {
return &reader{fsys: l.fsys}, nil
} else {
return &buffer{}, nil
}
}
type cursor struct {
scanner *bufio.Scanner
closer io.Closer
rowID int64
eof bool
reader bool
}
func (c *cursor) Close() (err error) {
if c.closer != nil {
err = c.closer.Close()
c.closer = nil
}
return err
line []byte
rowID int64
eof bool
}
func (c *cursor) EOF() bool {
return c.eof
}
func (c *cursor) Next() error {
c.rowID++
c.eof = !c.scanner.Scan()
return c.scanner.Err()
}
func (c *cursor) RowID() (int64, error) {
return c.rowID, nil
}
func (c *cursor) Column(ctx *sqlite3.Context, n int) error {
if n == 0 {
ctx.ResultRawText(c.scanner.Bytes())
ctx.ResultRawText(c.line)
}
return nil
}
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
type reader struct {
fsys fs.FS
reader *bufio.Reader
closer io.Closer
cursor
}
func (c *reader) Close() (err error) {
if c.closer != nil {
err = c.closer.Close()
c.closer = nil
}
return err
}
func (c *reader) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
if err := c.Close(); err != nil {
return err
}
var r io.Reader
data := arg[0]
typ := data.Type()
if c.reader {
switch typ {
case sqlite3.NULL:
if p, ok := data.Pointer().(io.ReaderAt); ok {
r = io.NewSectionReader(p, 0, math.MaxInt64)
}
case sqlite3.TEXT:
f, err := os.Open(data.Text())
if err != nil {
return err
}
c.closer = f
r = f
typ := arg[0].Type()
switch typ {
case sqlite3.NULL:
if p, ok := arg[0].Pointer().(io.Reader); ok {
r = p
}
} else {
switch typ {
case sqlite3.TEXT:
r = bytes.NewReader(data.RawText())
case sqlite3.BLOB:
r = bytes.NewReader(data.RawBlob())
case sqlite3.TEXT:
f, err := c.fsys.Open(arg[0].Text())
if err != nil {
return err
}
r = f
}
if r == nil {
return fmt.Errorf("lines: unsupported argument:%.0w %v", sqlite3.MISMATCH, typ)
}
c.scanner = bufio.NewScanner(r)
c.reader = bufio.NewReader(r)
c.closer, _ = r.(io.Closer)
c.rowID = 0
return c.Next()
}
func (c *reader) Next() (err error) {
c.line = c.line[:0]
for more := true; more; {
var line []byte
line, more, err = c.reader.ReadLine()
c.line = append(c.line, line...)
}
if err == io.EOF {
c.eof = true
err = nil
}
c.rowID++
return err
}
type buffer struct {
data []byte
cursor
}
func (c *buffer) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
typ := arg[0].Type()
switch typ {
case sqlite3.TEXT:
c.data = arg[0].RawText()
case sqlite3.BLOB:
c.data = arg[0].RawBlob()
default:
return fmt.Errorf("lines: unsupported argument:%.0w %v", sqlite3.MISMATCH, typ)
}
c.rowID = 0
return c.Next()
}
func (c *buffer) Next() error {
i := bytes.IndexByte(c.data, '\n')
j := i + 1
switch {
case i < 0:
i = len(c.data)
j = i
case i > 0 && c.data[i-1] == '\r':
i--
}
c.eof = len(c.data) == 0
c.line = c.data[:i]
c.data = c.data[j:]
c.rowID++
return nil
}

View File

@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"log"
"net/http"
"os"
"strings"
"testing"
@@ -25,12 +26,11 @@ func Example() {
}
defer db.Close()
// https://storage.googleapis.com/quickdraw_dataset/full/simplified/calendar.ndjson
f, err := os.Open("calendar.ndjson")
res, err := http.Get("https://storage.googleapis.com/quickdraw_dataset/full/simplified/calendar.ndjson")
if err != nil {
log.Fatal(err)
}
defer f.Close()
defer res.Body.Close()
rows, err := db.Query(`
SELECT
@@ -40,7 +40,7 @@ func Example() {
GROUP BY 1
ORDER BY 2 DESC
LIMIT 5`,
sqlite3.Pointer(f))
sqlite3.Pointer(res.Body))
if err != nil {
log.Fatal(err)
}
@@ -58,7 +58,7 @@ func Example() {
if err := rows.Err(); err != nil {
log.Fatal(err)
}
// Sample output:
// Output:
// US: 141001
// GB: 22560
// CA: 11759
@@ -78,7 +78,7 @@ func Test_lines(t *testing.T) {
}
defer db.Close()
const data = "line 1\nline 2\nline 3"
const data = "line 1\nline 2\r\nline 3\n"
rows, err := db.Query(`SELECT rowid, line FROM lines(?)`, data)
if err != nil {
@@ -93,6 +93,9 @@ func Test_lines(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if want := fmt.Sprintf("line %d", id); line != want {
t.Errorf("got %q, want %q", line, want)
}
}
}
@@ -135,7 +138,7 @@ func Test_lines_read(t *testing.T) {
}
defer db.Close()
const data = "line 1\nline 2\nline 3"
const data = "line 1\nline 2\r\nline 3\n"
rows, err := db.Query(`SELECT rowid, line FROM lines_read(?)`,
sqlite3.Pointer(strings.NewReader(data)))
@@ -151,6 +154,9 @@ func Test_lines_read(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if want := fmt.Sprintf("line %d", id); line != want {
t.Errorf("got %q, want %q", line, want)
}
}
}

View File

@@ -90,7 +90,7 @@ func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (_ *table, err err
}
create.WriteByte(')')
err = db.DeclareVtab(create.String())
err = db.DeclareVTab(create.String())
if err != nil {
return nil, err
}
@@ -142,6 +142,8 @@ func (t *table) BestIndex(idx *sqlite3.IndexInfo) error {
}
idxStr.WriteString(sep)
idxStr.WriteString(t.keys[ord.Column])
idxStr.WriteString(" COLLATE ")
idxStr.WriteString(idx.Collation(ord.Column))
if ord.Desc {
idxStr.WriteString(" DESC")
}

View File

@@ -67,7 +67,7 @@ func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (*table, error) {
}
str.WriteByte(')')
err = db.DeclareVtab(str.String())
err = db.DeclareVTab(str.String())
if err != nil {
stmt.Close()
return nil, err

47
ext/stats/TODO.md Normal file
View File

@@ -0,0 +1,47 @@
# ANSI SQL Aggregate Functions
https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html
## Built in aggregates
- [x] `COUNT(*)`
- [x] `COUNT(expression)`
- [x] `SUM(expression)`
- [x] `AVG(expression)`
- [x] `MIN(expression)`
- [x] `MAX(expression)`
https://sqlite.org/lang_aggfunc.html
## Statistical aggregates
- [x] `STDDEV_POP(expression)`
- [x] `STDDEV_SAMP(expression)`
- [x] `VAR_POP(expression)`
- [x] `VAR_SAMP(expression)`
- [x] `COVAR_POP(dependent, independent)`
- [x] `COVAR_SAMP(dependent, independent)`
- [x] `CORR(dependent, independent)`
## Linear regression aggregates
- [X] `REGR_AVGX(dependent, independent)`
- [X] `REGR_AVGY(dependent, independent)`
- [X] `REGR_SXX(dependent, independent)`
- [X] `REGR_SYY(dependent, independent)`
- [X] `REGR_SXY(dependent, independent)`
- [X] `REGR_COUNT(dependent, independent)`
- [X] `REGR_SLOPE(dependent, independent)`
- [X] `REGR_INTERCEPT(dependent, independent)`
- [X] `REGR_R2(dependent, independent)`
## Set aggregates
- [X] `CUME_DIST() OVER window`
- [X] `RANK() OVER window`
- [X] `DENSE_RANK() OVER window`
- [X] `PERCENT_RANK() OVER window`
- [ ] `PERCENTILE_CONT(percentile) OVER window`
- [ ] `PERCENTILE_DISC(percentile) OVER window`
https://sqlite.org/windowfunctions.html#builtins

View File

@@ -1,6 +1,6 @@
// Package stats provides aggregate functions for statistics.
//
// Functions:
// Provided functions:
// - stddev_pop: population standard deviation
// - stddev_samp: sample standard deviation
// - var_pop: population variance
@@ -8,9 +8,27 @@
// - covar_pop: population covariance
// - covar_samp: sample covariance
// - corr: correlation coefficient
// - regr_r2: correlation coefficient squared
// - regr_avgx: average of the independent variable
// - regr_avgy: average of the dependent variable
// - regr_sxx: sum of the squares of the independent variable
// - regr_syy: sum of the squares of the dependent variable
// - regr_sxy: sum of the products of each pair of variables
// - regr_count: count non-null pairs of variables
// - regr_slope: slope of the least-squares-fit linear equation
// - regr_intercept: y-intercept of the least-squares-fit linear equation
// - regr_json: all regr stats in a JSON object
//
// These join the [Built-in Aggregate Functions]:
// - count: count rows/values
// - sum: sum values
// - avg: average value
// - min: minimum value
// - max: maximum value
//
// See: [ANSI SQL Aggregate Functions]
//
// [Built-in Aggregate Functions]: https://sqlite.org/lang_aggfunc.html
// [ANSI SQL Aggregate Functions]: https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html
package stats
@@ -26,6 +44,16 @@ func Register(db *sqlite3.Conn) {
db.CreateWindowFunction("covar_pop", 2, flags, newCovariance(var_pop))
db.CreateWindowFunction("covar_samp", 2, flags, newCovariance(var_samp))
db.CreateWindowFunction("corr", 2, flags, newCovariance(corr))
db.CreateWindowFunction("regr_r2", 2, flags, newCovariance(regr_r2))
db.CreateWindowFunction("regr_sxx", 2, flags, newCovariance(regr_sxx))
db.CreateWindowFunction("regr_syy", 2, flags, newCovariance(regr_syy))
db.CreateWindowFunction("regr_sxy", 2, flags, newCovariance(regr_sxy))
db.CreateWindowFunction("regr_avgx", 2, flags, newCovariance(regr_avgx))
db.CreateWindowFunction("regr_avgy", 2, flags, newCovariance(regr_avgy))
db.CreateWindowFunction("regr_slope", 2, flags, newCovariance(regr_slope))
db.CreateWindowFunction("regr_intercept", 2, flags, newCovariance(regr_intercept))
db.CreateWindowFunction("regr_count", 2, flags, newCovariance(regr_count))
db.CreateWindowFunction("regr_json", 2, flags, newCovariance(regr_json))
}
const (
@@ -34,6 +62,16 @@ const (
stddev_pop
stddev_samp
corr
regr_r2
regr_sxx
regr_syy
regr_sxy
regr_avgx
regr_avgy
regr_slope
regr_intercept
regr_count
regr_json
)
func newVariance(kind int) func() sqlite3.AggregateFunction {
@@ -61,13 +99,13 @@ func (fn *variance) Value(ctx sqlite3.Context) {
}
func (fn *variance) Step(ctx sqlite3.Context, arg ...sqlite3.Value) {
if a := arg[0]; a.Type() != sqlite3.NULL {
if a := arg[0]; a.NumericType() != sqlite3.NULL {
fn.enqueue(a.Float())
}
}
func (fn *variance) Inverse(ctx sqlite3.Context, arg ...sqlite3.Value) {
if a := arg[0]; a.Type() != sqlite3.NULL {
if a := arg[0]; a.NumericType() != sqlite3.NULL {
fn.dequeue(a.Float())
}
}
@@ -90,20 +128,42 @@ func (fn *covariance) Value(ctx sqlite3.Context) {
r = fn.covar_samp()
case corr:
r = fn.correlation()
case regr_r2:
r = fn.regr_r2()
case regr_sxx:
r = fn.regr_sxx()
case regr_syy:
r = fn.regr_syy()
case regr_sxy:
r = fn.regr_sxy()
case regr_avgx:
r = fn.regr_avgx()
case regr_avgy:
r = fn.regr_avgy()
case regr_slope:
r = fn.regr_slope()
case regr_intercept:
r = fn.regr_intercept()
case regr_count:
ctx.ResultInt64(fn.regr_count())
return
case regr_json:
ctx.ResultText(fn.regr_json())
return
}
ctx.ResultFloat(r)
}
func (fn *covariance) Step(ctx sqlite3.Context, arg ...sqlite3.Value) {
a, b := arg[0], arg[1]
if a.Type() != sqlite3.NULL && b.Type() != sqlite3.NULL {
if a.NumericType() != sqlite3.NULL && b.NumericType() != sqlite3.NULL {
fn.enqueue(a.Float(), b.Float())
}
}
func (fn *covariance) Inverse(ctx sqlite3.Context, arg ...sqlite3.Value) {
a, b := arg[0], arg[1]
if a.Type() != sqlite3.NULL && b.Type() != sqlite3.NULL {
if a.NumericType() != sqlite3.NULL && b.NumericType() != sqlite3.NULL {
fn.dequeue(a.Float(), b.Float())
}
}

View File

@@ -20,7 +20,7 @@ func TestRegister_variance(t *testing.T) {
stats.Register(db)
err = db.Exec(`CREATE TABLE IF NOT EXISTS data (x)`)
err = db.Exec(`CREATE TABLE data (x)`)
if err != nil {
t.Fatal(err)
}
@@ -92,18 +92,23 @@ func TestRegister_covariance(t *testing.T) {
stats.Register(db)
err = db.Exec(`CREATE TABLE IF NOT EXISTS data (x, y)`)
err = db.Exec(`CREATE TABLE data (y, x)`)
if err != nil {
t.Fatal(err)
}
err = db.Exec(`INSERT INTO data (x, y) VALUES (3, 70), (5, 80), (2, 60), (7, 90), (4, 75)`)
err = db.Exec(`INSERT INTO data (y, x) VALUES (3, 70), (5, 80), (2, 60), (7, 90), (4, 75)`)
if err != nil {
t.Fatal(err)
}
stmt, _, err := db.Prepare(`SELECT
corr(x, y), covar_samp(x, y), covar_pop(x, y) FROM data`)
corr(y, x), covar_samp(y, x), covar_pop(y, x),
regr_avgy(y, x), regr_avgx(y, x),
regr_syy(y, x), regr_sxx(y, x), regr_sxy(y, x),
regr_slope(y, x), regr_intercept(y, x), regr_r2(y, x),
regr_count(y, x), regr_json(y, x)
FROM data`)
if err != nil {
t.Fatal(err)
}
@@ -119,10 +124,43 @@ func TestRegister_covariance(t *testing.T) {
if got := stmt.ColumnFloat(2); got != 17 {
t.Errorf("got %v, want 17", got)
}
if got := stmt.ColumnFloat(3); got != 4.2 {
t.Errorf("got %v, want 4.2", got)
}
if got := stmt.ColumnFloat(4); got != 75 {
t.Errorf("got %v, want 75", got)
}
if got := stmt.ColumnFloat(5); got != 14.8 {
t.Errorf("got %v, want 14.8", got)
}
if got := stmt.ColumnFloat(6); got != 500 {
t.Errorf("got %v, want 500", got)
}
if got := stmt.ColumnFloat(7); got != 85 {
t.Errorf("got %v, want 85", got)
}
if got := stmt.ColumnFloat(8); got != 0.17 {
t.Errorf("got %v, want 0.17", got)
}
if got := stmt.ColumnFloat(9); got != -8.55 {
t.Errorf("got %v, want -8.55", got)
}
if got := stmt.ColumnFloat(10); got != 0.9763513513513513 {
t.Errorf("got %v, want 0.9763513513513513", got)
}
if got := stmt.ColumnInt(11); got != 5 {
t.Errorf("got %v, want 5", got)
}
var a map[string]float64
if err := stmt.ColumnJSON(12, &a); err != nil {
t.Error(err)
} else if got := a["count"]; got != 5 {
t.Errorf("got %v, want 5", got)
}
}
{
stmt, _, err := db.Prepare(`SELECT covar_samp(x, y) OVER (ROWS 1 PRECEDING) FROM data`)
stmt, _, err := db.Prepare(`SELECT covar_samp(y, x) OVER (ROWS 1 PRECEDING) FROM data`)
if err != nil {
t.Fatal(err)
}
@@ -139,3 +177,67 @@ func TestRegister_covariance(t *testing.T) {
}
}
}
func Benchmark_average(b *testing.B) {
db, err := sqlite3.Open(":memory:")
if err != nil {
b.Fatal(err)
}
defer db.Close()
stmt, _, err := db.Prepare(`SELECT avg(value) FROM generate_series(0, ?)`)
if err != nil {
b.Fatal(err)
}
defer stmt.Close()
err = stmt.BindInt(1, b.N)
if err != nil {
b.Fatal(err)
}
if stmt.Step() {
want := float64(b.N) / 2
if got := stmt.ColumnFloat(0); got != want {
b.Errorf("got %v, want %v", got, want)
}
}
err = stmt.Err()
if err != nil {
b.Error(err)
}
}
func Benchmark_variance(b *testing.B) {
db, err := sqlite3.Open(":memory:")
if err != nil {
b.Fatal(err)
}
defer db.Close()
stats.Register(db)
stmt, _, err := db.Prepare(`SELECT var_pop(value) FROM generate_series(0, ?)`)
if err != nil {
b.Fatal(err)
}
defer stmt.Close()
err = stmt.BindInt(1, b.N)
if err != nil {
b.Fatal(err)
}
if stmt.Step() && b.N > 100 {
want := float64(b.N*b.N) / 12
if got := stmt.ColumnFloat(0); want > (got-want)*float64(b.N) {
b.Errorf("got %v, want %v", got, want)
}
}
err = stmt.Err()
if err != nil {
b.Error(err)
}
}

View File

@@ -1,14 +1,21 @@
package stats
import "math"
import (
"math"
"strconv"
"strings"
)
// Welford's algorithm with Kahan summation:
// https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm
// https://en.wikipedia.org/wiki/Kahan_summation_algorithm
// See also:
// https://duckdb.org/docs/sql/aggregates.html#statistical-aggregates
type welford struct {
m1, m2 kahan
n uint64
n int64
}
func (w welford) average() float64 {
@@ -48,10 +55,10 @@ func (w *welford) dequeue(x float64) {
}
type welford2 struct {
m1x, m2x kahan
m1y, m2y kahan
m1x, m2x kahan
cov kahan
n uint64
n int64
}
func (w welford2) covar_pop() float64 {
@@ -63,33 +70,98 @@ func (w welford2) covar_samp() float64 {
}
func (w welford2) correlation() float64 {
return w.cov.hi / math.Sqrt(w.m2x.hi*w.m2y.hi)
return w.cov.hi / math.Sqrt(w.m2y.hi*w.m2x.hi)
}
func (w *welford2) enqueue(x, y float64) {
func (w welford2) regr_avgy() float64 {
return w.m1y.hi
}
func (w welford2) regr_avgx() float64 {
return w.m1x.hi
}
func (w welford2) regr_syy() float64 {
return w.m2y.hi
}
func (w welford2) regr_sxx() float64 {
return w.m2x.hi
}
func (w welford2) regr_sxy() float64 {
return w.cov.hi
}
func (w welford2) regr_count() int64 {
return w.n
}
func (w welford2) regr_slope() float64 {
return w.cov.hi / w.m2x.hi
}
func (w welford2) regr_intercept() float64 {
slope := -w.regr_slope()
hi := math.FMA(slope, w.m1x.hi, w.m1y.hi)
lo := math.FMA(slope, w.m1x.lo, w.m1y.lo)
return hi + lo
}
func (w welford2) regr_r2() float64 {
return w.cov.hi * w.cov.hi / (w.m2y.hi * w.m2x.hi)
}
func (w welford2) regr_json() string {
var json strings.Builder
var num [32]byte
json.Grow(128)
json.WriteString(`{"count":`)
json.Write(strconv.AppendInt(num[:0], w.regr_count(), 10))
json.WriteString(`,"avgy":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_avgy(), 'g', -1, 64))
json.WriteString(`,"avgx":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_avgx(), 'g', -1, 64))
json.WriteString(`,"syy":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_syy(), 'g', -1, 64))
json.WriteString(`,"sxx":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_sxx(), 'g', -1, 64))
json.WriteString(`,"sxy":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_sxy(), 'g', -1, 64))
json.WriteString(`,"slope":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_slope(), 'g', -1, 64))
json.WriteString(`,"intercept":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_intercept(), 'g', -1, 64))
json.WriteString(`,"r2":`)
json.Write(strconv.AppendFloat(num[:0], w.regr_r2(), 'g', -1, 64))
json.WriteByte('}')
return json.String()
}
func (w *welford2) enqueue(y, x float64) {
w.n++
d1x := x - w.m1x.hi - w.m1x.lo
d1y := y - w.m1y.hi - w.m1y.lo
w.m1x.add(d1x / float64(w.n))
d1x := x - w.m1x.hi - w.m1x.lo
w.m1y.add(d1y / float64(w.n))
d2x := x - w.m1x.hi - w.m1x.lo
w.m1x.add(d1x / float64(w.n))
d2y := y - w.m1y.hi - w.m1y.lo
w.m2x.add(d1x * d2x)
d2x := x - w.m1x.hi - w.m1x.lo
w.m2y.add(d1y * d2y)
w.cov.add(d1x * d2y)
w.m2x.add(d1x * d2x)
w.cov.add(d1y * d2x)
}
func (w *welford2) dequeue(x, y float64) {
func (w *welford2) dequeue(y, x float64) {
w.n--
d1x := x - w.m1x.hi - w.m1x.lo
d1y := y - w.m1y.hi - w.m1y.lo
w.m1x.sub(d1x / float64(w.n))
d1x := x - w.m1x.hi - w.m1x.lo
w.m1y.sub(d1y / float64(w.n))
d2x := x - w.m1x.hi - w.m1x.lo
w.m1x.sub(d1x / float64(w.n))
d2y := y - w.m1y.hi - w.m1y.lo
w.m2x.sub(d1x * d2x)
d2x := x - w.m1x.hi - w.m1x.lo
w.m2y.sub(d1y * d2y)
w.cov.sub(d1x * d2y)
w.m2x.sub(d1x * d2x)
w.cov.sub(d1y * d2x)
}
type kahan struct{ hi, lo float64 }

View File

@@ -1,16 +0,0 @@
sha3
sha3_query
ieee754
ieee754_exponent
ieee754_from_blob
ieee754_inc
ieee754_mantissa
ieee754_to_blob
zipfile
zipfile_cds
sqlar_compress
sqlar_uncompress
remember

View File

@@ -8,7 +8,7 @@
// The implementation is not 100% compatible with the [ICU extension]:
// - upper() and lower() use [strings.ToUpper], [strings.ToLower] and [cases];
// - the LIKE operator follows [strings.EqualFold] rules;
// - the REGEXP operator uses Go [regex/syntax];
// - the REGEXP operator uses Go [regexp/syntax];
// - collation sequences use [collate].
//
// Expect subtle differences (e.g.) in the handling of Turkish case folding.

View File

@@ -81,7 +81,7 @@ func TestRegister_collation(t *testing.T) {
Register(db)
err = db.Exec(`CREATE TABLE IF NOT EXISTS words (word VARCHAR(10))`)
err = db.Exec(`CREATE TABLE words (word VARCHAR(10))`)
if err != nil {
t.Fatal(err)
}

58
ext/zorder/zorder.go Normal file
View File

@@ -0,0 +1,58 @@
// Package zorder provides functions for z-order transformations.
//
// https://sqlite.org/src/doc/tip/ext/misc/zorder.c
package zorder
import (
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Register registers the zorder and unzorder SQL functions.
func Register(db *sqlite3.Conn) {
flags := sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
db.CreateFunction("zorder", -1, flags, zorder)
db.CreateFunction("unzorder", 3, flags, unzorder)
}
func zorder(ctx sqlite3.Context, arg ...sqlite3.Value) {
var x [63]int64
for i := range arg {
x[i] = arg[i].Int64()
}
if len(arg) > len(x) {
ctx.ResultError(util.ErrorString("zorder: too many parameters"))
return
}
var z int64
if len(arg) > 0 {
for i := 0; i < 63; i++ {
j := i % len(arg)
z |= (x[j] & 1) << i
x[j] >>= 1
}
}
for i := range arg {
if x[i] != 0 {
ctx.ResultError(util.ErrorString("zorder: parameter too large"))
return
}
}
ctx.ResultInt64(z)
}
func unzorder(ctx sqlite3.Context, arg ...sqlite3.Value) {
z := arg[0].Int64()
n := arg[1].Int64()
i := arg[2].Int64()
var k int
var x int64
for j := i; j < 63; j += n {
x |= ((z >> j) & 1) << k
k++
}
ctx.ResultInt64(x)
}

106
ext/zorder/zorder_test.go Normal file
View File

@@ -0,0 +1,106 @@
package zorder_test
import (
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/zorder"
)
func TestRegister_zorder(t *testing.T) {
t.Parallel()
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
zorder.Register(c)
return nil
})
if err != nil {
t.Fatal(err)
}
defer db.Close()
var got int64
err = db.QueryRow(`SELECT zorder(2, 3)`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != 14 {
t.Errorf("got %d, want 14", got)
}
err = db.QueryRow(`SELECT zorder(4, 5)`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != 50 {
t.Errorf("got %d, want 14", got)
}
var check bool
err = db.QueryRow(`SELECT zorder(3, 4) BETWEEN zorder(2, 3) AND zorder(4, 5)`).Scan(&check)
if err != nil {
t.Fatal(err)
}
if !check {
t.Error("want true")
}
err = db.QueryRow(`SELECT zorder(2, 2) NOT BETWEEN zorder(2, 3) AND zorder(4, 5)`).Scan(&check)
if err != nil {
t.Fatal(err)
}
if !check {
t.Error("want true")
}
}
func TestRegister_unzorder(t *testing.T) {
t.Parallel()
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
zorder.Register(c)
return nil
})
if err != nil {
t.Fatal(err)
}
defer db.Close()
var got int64
err = db.QueryRow(`SELECT unzorder(zorder(3, 4), 2, 0)`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != 3 {
t.Errorf("got %d, want 3", got)
}
err = db.QueryRow(`SELECT unzorder(zorder(3, 4), 2, 1)`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != 4 {
t.Errorf("got %d, want 4", got)
}
}
func TestRegister_error(t *testing.T) {
t.Parallel()
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
zorder.Register(c)
return nil
})
if err != nil {
t.Fatal(err)
}
defer db.Close()
var got int64
err = db.QueryRow(`SELECT zorder(1, 2, 3, 100000)`).Scan(&got)
if err == nil {
t.Error("want error")
}
}

153
func.go
View File

@@ -2,13 +2,31 @@ package sqlite3
import (
"context"
"sync"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/tetratelabs/wazero/api"
)
// AnyCollationNeeded registers a fake collating function
// for any unknown collating sequence.
// CollationNeeded registers a callback to be invoked
// whenever an unknown collation sequence is required.
//
// https://sqlite.org/c3ref/collation_needed.html
func (c *Conn) CollationNeeded(cb func(db *Conn, name string)) error {
var enable uint64
if cb != nil {
enable = 1
}
r := c.call("sqlite3_collation_needed_go", uint64(c.handle), enable)
if err := c.error(r); err != nil {
return err
}
c.collation = cb
return nil
}
// AnyCollationNeeded uses [Conn.CollationNeeded] to register
// a fake collating function for any unknown collating sequence.
// The fake collating function works like BINARY.
//
// This can be used to load schemas that contain
@@ -43,6 +61,7 @@ func (c *Conn) CreateFunction(name string, nArg int, flag FunctionFlag, fn Scala
}
// ScalarFunction is the type of a scalar SQL function.
// Implementations must not retain arg.
type ScalarFunction func(ctx Context, arg ...Value)
// CreateWindowFunction defines a new aggregate or aggregate window SQL function.
@@ -69,7 +88,8 @@ func (c *Conn) CreateWindowFunction(name string, nArg int, flag FunctionFlag, fn
// https://sqlite.org/appfunc.html
type AggregateFunction interface {
// Step is invoked to add a row to the current window.
// The function arguments, if any, corresponding to the row being added are passed to Step.
// The function arguments, if any, corresponding to the row being added, are passed to Step.
// Implementations must not retain arg.
Step(ctx Context, arg ...Value)
// Value is invoked to return the current (or final) value of the aggregate.
@@ -84,92 +104,111 @@ type WindowFunction interface {
// Inverse is invoked to remove the oldest presently aggregated result of Step from the current window.
// The function arguments, if any, are those passed to Step for the row being removed.
// Implementations must not retain arg.
Inverse(ctx Context, arg ...Value)
}
// OverloadFunction overloads a function for a virtual table.
//
// https://sqlite.org/c3ref/overload_function.html
func (c *Conn) OverloadFunction(name string, nArg int) error {
defer c.arena.mark()()
namePtr := c.arena.string(name)
r := c.call("sqlite3_overload_function",
uint64(c.handle), uint64(namePtr), uint64(nArg))
return c.error(r)
}
func destroyCallback(ctx context.Context, mod api.Module, pApp uint32) {
util.DelHandle(ctx, pApp)
}
func collationCallback(ctx context.Context, mod api.Module, pArg, pDB, eTextRep, zName uint32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.collation != nil {
name := util.ReadString(mod, zName, _MAX_NAME)
c.collation(c, name)
}
}
func compareCallback(ctx context.Context, mod api.Module, pApp, nKey1, pKey1, nKey2, pKey2 uint32) uint32 {
fn := util.GetHandle(ctx, pApp).(func(a, b []byte) int)
return uint32(fn(util.View(mod, pKey1, uint64(nKey1)), util.View(mod, pKey2, uint64(nKey2))))
}
func funcCallback(ctx context.Context, mod api.Module, pCtx, nArg, pArg uint32) {
func funcCallback(ctx context.Context, mod api.Module, pCtx, pApp, nArg, pArg uint32) {
args := getFuncArgs()
defer putFuncArgs(args)
db := ctx.Value(connKey{}).(*Conn)
fn := userDataHandle(db, pCtx).(ScalarFunction)
fn(Context{db, pCtx}, callbackArgs(db, nArg, pArg)...)
fn := util.GetHandle(db.ctx, pApp).(ScalarFunction)
callbackArgs(db, args[:nArg], pArg)
fn(Context{db, pCtx}, args[:nArg]...)
}
func stepCallback(ctx context.Context, mod api.Module, pCtx, nArg, pArg uint32) {
func stepCallback(ctx context.Context, mod api.Module, pCtx, pAgg, pApp, nArg, pArg uint32) {
args := getFuncArgs()
defer putFuncArgs(args)
db := ctx.Value(connKey{}).(*Conn)
fn := aggregateCtxHandle(db, pCtx, nil)
fn.Step(Context{db, pCtx}, callbackArgs(db, nArg, pArg)...)
callbackArgs(db, args[:nArg], pArg)
fn, _ := callbackAggregate(db, pAgg, pApp)
fn.Step(Context{db, pCtx}, args[:nArg]...)
}
func finalCallback(ctx context.Context, mod api.Module, pCtx uint32) {
var handle uint32
func finalCallback(ctx context.Context, mod api.Module, pCtx, pAgg, pApp uint32) {
db := ctx.Value(connKey{}).(*Conn)
fn := aggregateCtxHandle(db, pCtx, &handle)
fn, handle := callbackAggregate(db, pAgg, pApp)
fn.Value(Context{db, pCtx})
if err := util.DelHandle(ctx, handle); err != nil {
Context{db, pCtx}.ResultError(err)
}
util.DelHandle(ctx, handle)
}
func valueCallback(ctx context.Context, mod api.Module, pCtx uint32) {
func valueCallback(ctx context.Context, mod api.Module, pCtx, pAgg uint32) {
db := ctx.Value(connKey{}).(*Conn)
fn := aggregateCtxHandle(db, pCtx, nil)
fn := util.GetHandle(db.ctx, pAgg).(AggregateFunction)
fn.Value(Context{db, pCtx})
}
func inverseCallback(ctx context.Context, mod api.Module, pCtx, nArg, pArg uint32) {
func inverseCallback(ctx context.Context, mod api.Module, pCtx, pAgg, nArg, pArg uint32) {
args := getFuncArgs()
defer putFuncArgs(args)
db := ctx.Value(connKey{}).(*Conn)
fn := aggregateCtxHandle(db, pCtx, nil).(WindowFunction)
fn.Inverse(Context{db, pCtx}, callbackArgs(db, nArg, pArg)...)
callbackArgs(db, args[:nArg], pArg)
fn := util.GetHandle(db.ctx, pAgg).(WindowFunction)
fn.Inverse(Context{db, pCtx}, args[:nArg]...)
}
func userDataHandle(db *Conn, pCtx uint32) any {
pApp := uint32(db.call("sqlite3_user_data", uint64(pCtx)))
return util.GetHandle(db.ctx, pApp)
func callbackAggregate(db *Conn, pAgg, pApp uint32) (AggregateFunction, uint32) {
if pApp == 0 {
handle := util.ReadUint32(db.mod, pAgg)
return util.GetHandle(db.ctx, handle).(AggregateFunction), handle
}
// We need to create the aggregate.
fn := util.GetHandle(db.ctx, pApp).(func() AggregateFunction)()
handle := util.AddHandle(db.ctx, fn)
if pAgg != 0 {
util.WriteUint32(db.mod, pAgg, handle)
}
return fn, handle
}
func aggregateCtxHandle(db *Conn, pCtx uint32, close *uint32) AggregateFunction {
// On close, we're getting rid of the aggregate.
// Don't allocate space to store it.
var size uint64
if close == nil {
size = ptrlen
}
ptr := uint32(db.call("sqlite3_aggregate_context", uint64(pCtx), size))
// If we already have an aggregate, return it.
if ptr != 0 {
if handle := util.ReadUint32(db.mod, ptr); handle != 0 {
fn := util.GetHandle(db.ctx, handle).(AggregateFunction)
if close != nil {
*close = handle
}
return fn
}
}
// Create a new aggregate, and store it if needed.
fn := userDataHandle(db, pCtx).(func() AggregateFunction)()
if ptr != 0 {
util.WriteUint32(db.mod, ptr, util.AddHandle(db.ctx, fn))
}
return fn
}
func callbackArgs(db *Conn, nArg, pArg uint32) []Value {
args := make([]Value, nArg)
for i := range args {
args[i] = Value{
sqlite: db.sqlite,
func callbackArgs(db *Conn, arg []Value, pArg uint32) {
for i := range arg {
arg[i] = Value{
c: db,
handle: util.ReadUint32(db.mod, pArg+ptrlen*uint32(i)),
}
}
return args
}
var funcArgsPool sync.Pool
func putFuncArgs(p *[_MAX_FUNCTION_ARG]Value) {
funcArgsPool.Put(p)
}
func getFuncArgs() *[_MAX_FUNCTION_ARG]Value {
if p := funcArgsPool.Get(); p == nil {
return new([_MAX_FUNCTION_ARG]Value)
} else {
return p.(*[_MAX_FUNCTION_ARG]Value)
}
}

View File

@@ -6,11 +6,9 @@ import (
"log"
"regexp"
"golang.org/x/text/collate"
"golang.org/x/text/language"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/unicode"
)
func ExampleConn_CreateCollation() {
@@ -20,7 +18,7 @@ func ExampleConn_CreateCollation() {
}
defer db.Close()
err = db.Exec(`CREATE TABLE IF NOT EXISTS words (word VARCHAR(10))`)
err = db.Exec(`CREATE TABLE words (word VARCHAR(10))`)
if err != nil {
log.Fatal(err)
}
@@ -30,12 +28,17 @@ func ExampleConn_CreateCollation() {
log.Fatal(err)
}
err = db.CreateCollation("french", collate.New(language.French).Compare)
err = db.CollationNeeded(func(db *sqlite3.Conn, name string) {
err := unicode.RegisterCollation(db, name, name)
if err != nil {
log.Fatal(err)
}
})
if err != nil {
log.Fatal(err)
}
stmt, _, err := db.Prepare(`SELECT word FROM words ORDER BY word COLLATE french`)
stmt, _, err := db.Prepare(`SELECT word FROM words ORDER BY word COLLATE fr_FR`)
if err != nil {
log.Fatal(err)
}
@@ -63,7 +66,7 @@ func ExampleConn_CreateFunction() {
}
defer db.Close()
err = db.Exec(`CREATE TABLE IF NOT EXISTS words (word VARCHAR(10))`)
err = db.Exec(`CREATE TABLE words (word VARCHAR(10))`)
if err != nil {
log.Fatal(err)
}
@@ -108,7 +111,7 @@ func ExampleContext_SetAuxData() {
}
defer db.Close()
err = db.Exec(`CREATE TABLE IF NOT EXISTS words (word VARCHAR(10))`)
err = db.Exec(`CREATE TABLE words (word VARCHAR(10))`)
if err != nil {
log.Fatal(err)
}

View File

@@ -16,7 +16,7 @@ func ExampleConn_CreateWindowFunction() {
}
defer db.Close()
err = db.Exec(`CREATE TABLE IF NOT EXISTS words (word VARCHAR(10))`)
err = db.Exec(`CREATE TABLE words (word VARCHAR(10))`)
if err != nil {
log.Fatal(err)
}

7
go.mod
View File

@@ -5,9 +5,10 @@ go 1.21
require (
github.com/ncruces/julianday v1.0.0
github.com/psanford/httpreadat v0.1.0
github.com/tetratelabs/wazero v1.5.0
golang.org/x/sync v0.5.0
golang.org/x/sys v0.15.0
github.com/tetratelabs/wazero v1.7.1
golang.org/x/crypto v0.22.0
golang.org/x/sync v0.7.0
golang.org/x/sys v0.19.0
golang.org/x/text v0.14.0
)

14
go.sum
View File

@@ -2,11 +2,13 @@ github.com/ncruces/julianday v1.0.0 h1:fH0OKwa7NWvniGQtxdJRxAgkBMolni2BjDHaWTxqt
github.com/ncruces/julianday v1.0.0/go.mod h1:Dusn2KvZrrovOMJuOt0TNXL6tB7U2E8kvza5fFc9G7g=
github.com/psanford/httpreadat v0.1.0 h1:VleW1HS2zO7/4c7c7zNl33fO6oYACSagjJIyMIwZLUE=
github.com/psanford/httpreadat v0.1.0/go.mod h1:Zg7P+TlBm3bYbyHTKv/EdtSJZn3qwbPwpfZ/I9GKCRE=
github.com/tetratelabs/wazero v1.5.0 h1:Yz3fZHivfDiZFUXnWMPUoiW7s8tC1sjdBtlJn08qYa0=
github.com/tetratelabs/wazero v1.5.0/go.mod h1:0U0G41+ochRKoPKCJlh0jMg1CHkyfK8kDqiirMmKY8A=
golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE=
golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
github.com/tetratelabs/wazero v1.7.1 h1:QtSfd6KLc41DIMpDYlJdoMc6k7QTN246DM2+n2Y/Dx8=
github.com/tetratelabs/wazero v1.7.1/go.mod h1:ytl6Zuh20R/eROuyDaGPkp82O9C/DJfXAwJfQ3X6/7Y=
golang.org/x/crypto v0.22.0 h1:g1v0xeRhjcugydODzvb3mEM9SQ0HGp9s/nh3COQ/C30=
golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.19.0 h1:q5f1RH2jigJ1MoAWp2KTp3gm5zAGFUTarQZ5U386+4o=
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=

View File

@@ -1,4 +1,8 @@
golang.org/x/mod v0.8.0 h1:LUYupSeNrTNCGzR/hVBk2NHZO4hXcVaW1k4Qx7rjPx8=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.21.0 h1:AQyQV4dYCvJ7vGmJyKki9+PBdyvhkSd8EIx/qb0AYv4=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/term v0.19.0 h1:+ThwsDv+tYfnJFhF4L8jITxu1tdTWRTZpdsWgEgjL6Q=
golang.org/x/term v0.19.0/go.mod h1:2CuTdWZ7KHSQwUzKva0cbMg6q2DMI3Mmxp+gKJbskEk=
golang.org/x/tools v0.6.0 h1:BOw41kyTf3PuCW1pVQf8+Cyg8pMlkYB1oo9iJ6D/lKM=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=

View File

@@ -13,6 +13,7 @@ import (
var (
sqliteSeparator = "`|\"|'|\t"
uniqueRegexp = regexp.MustCompile(fmt.Sprintf(`^CONSTRAINT [%v]?[\w-]+[%v]? UNIQUE (.*)$`, sqliteSeparator, sqliteSeparator))
indexRegexp = regexp.MustCompile(fmt.Sprintf(`(?is)CREATE(?: UNIQUE)? INDEX [%v]?[\w\d-]+[%v]?(?s:.*?)ON (.*)$`, sqliteSeparator, sqliteSeparator))
tableRegexp = regexp.MustCompile(fmt.Sprintf(`(?is)(CREATE TABLE [%v]?[\w\d-]+[%v]?)(?:\s*\((.*)\))?`, sqliteSeparator, sqliteSeparator))
separatorRegexp = regexp.MustCompile(fmt.Sprintf("[%v]", sqliteSeparator))
@@ -103,11 +104,24 @@ func parseDDL(strs ...string) (*ddl, error) {
for _, f := range result.fields {
fUpper := strings.ToUpper(f)
if strings.HasPrefix(fUpper, "CHECK") ||
strings.HasPrefix(fUpper, "CONSTRAINT") {
if strings.HasPrefix(fUpper, "CHECK") {
continue
}
if strings.HasPrefix(fUpper, "CONSTRAINT") {
matches := uniqueRegexp.FindStringSubmatch(f)
if len(matches) > 0 {
if columns := getAllColumns(matches[1]); len(columns) == 1 {
for idx, column := range result.columns {
if column.NameValue.String == columns[0] {
column.UniqueValue = sql.NullBool{Bool: true, Valid: true}
result.columns[idx] = column
break
}
}
}
}
continue
}
if strings.HasPrefix(fUpper, "PRIMARY KEY") {
for _, name := range getAllColumns(f) {
for idx, column := range result.columns {
@@ -159,14 +173,7 @@ func parseDDL(strs ...string) (*ddl, error) {
}
}
} else if matches := indexRegexp.FindStringSubmatch(str); len(matches) > 0 {
for _, column := range getAllColumns(matches[1]) {
for idx, c := range result.columns {
if c.NameValue.String == column {
c.UniqueValue = sql.NullBool{Bool: strings.ToUpper(strings.Fields(str)[1]) == "UNIQUE", Valid: true}
result.columns[idx] = c
}
}
}
// don't report Unique by UniqueIndex
} else {
return nil, errors.New("invalid DDL")
}
@@ -269,20 +276,6 @@ func (d *ddl) getColumns() []string {
return res
}
func (d *ddl) alterColumn(name, sql string) bool {
reg := regexp.MustCompile("^(`|'|\"| )" + regexp.QuoteMeta(name) + "(`|'|\"| ) .*?$")
for i := 0; i < len(d.fields); i++ {
if reg.MatchString(d.fields[i]) {
d.fields[i] = sql
return false
}
}
d.fields = append(d.fields, sql)
return true
}
func (d *ddl) removeColumn(name string) bool {
reg := regexp.MustCompile("^(`|'|\"| )" + regexp.QuoteMeta(name) + "(`|'|\"| ) .*?$")

View File

@@ -16,11 +16,12 @@ func TestParseDDL(t *testing.T) {
columns []migrator.ColumnType
}{
{"with_fk", []string{
"CREATE TABLE `notes` (`id` integer NOT NULL,`text` varchar(500) DEFAULT \"hello\",`age` integer DEFAULT 18,`user_id` integer,PRIMARY KEY (`id`),CONSTRAINT `fk_users_notes` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`))",
"CREATE TABLE `notes` (" +
"`id` integer NOT NULL,`text` varchar(500) DEFAULT \"hello\",`age` integer DEFAULT 18,`user_id` integer,PRIMARY KEY (`id`),CONSTRAINT `fk_users_notes` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`))",
"CREATE UNIQUE INDEX `idx_profiles_refer` ON `profiles`(`text`)",
}, 6, []migrator.ColumnType{
{NameValue: sql.NullString{String: "id", Valid: true}, DataTypeValue: sql.NullString{String: "integer", Valid: true}, ColumnTypeValue: sql.NullString{String: "integer", Valid: true}, PrimaryKeyValue: sql.NullBool{Bool: true, Valid: true}, NullableValue: sql.NullBool{Valid: true}, UniqueValue: sql.NullBool{Valid: true}, DefaultValueValue: sql.NullString{Valid: false}},
{NameValue: sql.NullString{String: "text", Valid: true}, DataTypeValue: sql.NullString{String: "varchar", Valid: true}, LengthValue: sql.NullInt64{Int64: 500, Valid: true}, ColumnTypeValue: sql.NullString{String: "varchar(500)", Valid: true}, DefaultValueValue: sql.NullString{String: "hello", Valid: true}, NullableValue: sql.NullBool{Bool: true, Valid: true}, UniqueValue: sql.NullBool{Bool: true, Valid: true}, PrimaryKeyValue: sql.NullBool{Valid: true}},
{NameValue: sql.NullString{String: "text", Valid: true}, DataTypeValue: sql.NullString{String: "varchar", Valid: true}, LengthValue: sql.NullInt64{Int64: 500, Valid: true}, ColumnTypeValue: sql.NullString{String: "varchar(500)", Valid: true}, DefaultValueValue: sql.NullString{String: "hello", Valid: true}, NullableValue: sql.NullBool{Bool: true, Valid: true}, UniqueValue: sql.NullBool{Bool: false, Valid: true}, PrimaryKeyValue: sql.NullBool{Valid: true}},
{NameValue: sql.NullString{String: "age", Valid: true}, DataTypeValue: sql.NullString{String: "integer", Valid: true}, ColumnTypeValue: sql.NullString{String: "integer", Valid: true}, DefaultValueValue: sql.NullString{String: "18", Valid: true}, NullableValue: sql.NullBool{Bool: true, Valid: true}, UniqueValue: sql.NullBool{Valid: true}, PrimaryKeyValue: sql.NullBool{Valid: true}},
{NameValue: sql.NullString{String: "user_id", Valid: true}, DataTypeValue: sql.NullString{String: "integer", Valid: true}, ColumnTypeValue: sql.NullString{String: "integer", Valid: true}, DefaultValueValue: sql.NullString{Valid: false}, NullableValue: sql.NullBool{Bool: true, Valid: true}, UniqueValue: sql.NullBool{Valid: true}, PrimaryKeyValue: sql.NullBool{Valid: true}},
},
@@ -56,28 +57,54 @@ func TestParseDDL(t *testing.T) {
ColumnTypeValue: sql.NullString{String: "int", Valid: true},
NullableValue: sql.NullBool{Bool: false, Valid: true},
DefaultValueValue: sql.NullString{Valid: false},
UniqueValue: sql.NullBool{Bool: true, Valid: true},
UniqueValue: sql.NullBool{Bool: false, Valid: true},
PrimaryKeyValue: sql.NullBool{Valid: true},
},
},
},
{
}, {
"unique index",
[]string{
"CREATE TABLE `test-b` (`field` integer NOT NULL)",
"CREATE UNIQUE INDEX `idx_uq` ON `test-b`(`field`) WHERE field = 0",
},
1,
[]migrator.ColumnType{
{
NameValue: sql.NullString{String: "field", Valid: true},
DataTypeValue: sql.NullString{String: "integer", Valid: true},
ColumnTypeValue: sql.NullString{String: "integer", Valid: true},
PrimaryKeyValue: sql.NullBool{Bool: false, Valid: true},
UniqueValue: sql.NullBool{Bool: true, Valid: true},
NullableValue: sql.NullBool{Bool: false, Valid: true},
},
[]migrator.ColumnType{{
NameValue: sql.NullString{String: "field", Valid: true},
DataTypeValue: sql.NullString{String: "integer", Valid: true},
ColumnTypeValue: sql.NullString{String: "integer", Valid: true},
PrimaryKeyValue: sql.NullBool{Bool: false, Valid: true},
UniqueValue: sql.NullBool{Bool: false, Valid: true},
NullableValue: sql.NullBool{Bool: false, Valid: true},
}},
}, {
"normal index",
[]string{
"CREATE TABLE `test-c` (`field` integer NOT NULL)",
"CREATE INDEX `idx_uq` ON `test-c`(`field`)",
},
1,
[]migrator.ColumnType{{
NameValue: sql.NullString{String: "field", Valid: true},
DataTypeValue: sql.NullString{String: "integer", Valid: true},
ColumnTypeValue: sql.NullString{String: "integer", Valid: true},
PrimaryKeyValue: sql.NullBool{Bool: false, Valid: true},
UniqueValue: sql.NullBool{Bool: false, Valid: true},
NullableValue: sql.NullBool{Bool: false, Valid: true},
}},
}, {
"unique constraint",
[]string{
"CREATE TABLE `unique_struct` (`name` text,CONSTRAINT `uni_unique_struct_name` UNIQUE (`name`))",
},
2,
[]migrator.ColumnType{{
NameValue: sql.NullString{String: "name", Valid: true},
DataTypeValue: sql.NullString{String: "text", Valid: true},
ColumnTypeValue: sql.NullString{String: "text", Valid: true},
PrimaryKeyValue: sql.NullBool{Bool: false, Valid: true},
UniqueValue: sql.NullBool{Bool: true, Valid: true},
NullableValue: sql.NullBool{Bool: true, Valid: true},
}},
},
{
"non-unique index",

View File

@@ -3,9 +3,9 @@ set -euo pipefail
cd -P -- "$(dirname -- "$0")"
curl -#OL "https://github.com/go-gorm/sqlite/raw/master/ddlmod.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/master/ddlmod_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/master/error_translator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/master/migrator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/master/sqlite.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/master/sqlite_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/ddlmod.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/ddlmod_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/error_translator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/migrator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/sqlite.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.5/sqlite_test.go"

View File

@@ -3,14 +3,14 @@ module github.com/ncruces/go-sqlite3/gormlite
go 1.21
require (
github.com/ncruces/go-sqlite3 v0.11.0
gorm.io/gorm v1.25.5
github.com/ncruces/go-sqlite3 v0.13.0
gorm.io/gorm v1.25.9
)
require (
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/ncruces/julianday v1.0.0 // indirect
github.com/tetratelabs/wazero v1.5.0 // indirect
golang.org/x/sys v0.15.0 // indirect
github.com/tetratelabs/wazero v1.7.0 // indirect
golang.org/x/sys v0.19.0 // indirect
)

View File

@@ -2,15 +2,15 @@ github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/ncruces/go-sqlite3 v0.11.0 h1:PDjs8Ve2Z0GWmHyKQHGUyG78grCXKhiHCUZQI8CqXO8=
github.com/ncruces/go-sqlite3 v0.11.0/go.mod h1:zaYJ6xP+EQiWJCa3nd3h28cD8DuSIcIqh+LrJMrBN9k=
github.com/ncruces/go-sqlite3 v0.13.0 h1:+N1VHVLnrCJasyXdAKQL9MNTKC3wHZa8pLMUuP8wb3k=
github.com/ncruces/go-sqlite3 v0.13.0/go.mod h1:y9zPUI+C42V9xuuJeN+tGhpOjr4gUHz2Pi2RLFVEdZg=
github.com/ncruces/julianday v1.0.0 h1:fH0OKwa7NWvniGQtxdJRxAgkBMolni2BjDHaWTxqt7M=
github.com/ncruces/julianday v1.0.0/go.mod h1:Dusn2KvZrrovOMJuOt0TNXL6tB7U2E8kvza5fFc9G7g=
github.com/tetratelabs/wazero v1.5.0 h1:Yz3fZHivfDiZFUXnWMPUoiW7s8tC1sjdBtlJn08qYa0=
github.com/tetratelabs/wazero v1.5.0/go.mod h1:0U0G41+ochRKoPKCJlh0jMg1CHkyfK8kDqiirMmKY8A=
golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
github.com/tetratelabs/wazero v1.7.0 h1:jg5qPydno59wqjpGrHph81lbtHzTrWzwwtD4cD88+hQ=
github.com/tetratelabs/wazero v1.7.0/go.mod h1:ytl6Zuh20R/eROuyDaGPkp82O9C/DJfXAwJfQ3X6/7Y=
golang.org/x/sys v0.19.0 h1:q5f1RH2jigJ1MoAWp2KTp3gm5zAGFUTarQZ5U386+4o=
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
gorm.io/gorm v1.25.5 h1:zR9lOiiYf09VNh5Q1gphfyia1JpiClIWG9hQaxB/mls=
gorm.io/gorm v1.25.5/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
gorm.io/gorm v1.25.9 h1:wct0gxZIELDk8+ZqF/MVnHLkA1rvYlBWUMv2EdsK1g8=
gorm.io/gorm v1.25.9/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=

View File

@@ -79,14 +79,28 @@ func (m _Migrator) AlterColumn(value interface{}, name string) error {
return m.RunWithoutForeignKey(func() error {
return m.recreateTable(value, nil, func(ddl *ddl, stmt *gorm.Statement) (*ddl, []interface{}, error) {
if field := stmt.Schema.LookUpField(name); field != nil {
if ddl.alterColumn(field.DBName, fmt.Sprintf("`%s` ?", field.DBName)) {
return nil, nil, fmt.Errorf("field `%s` not found in origin ddl, ddl= '%s'", name, ddl.compile())
var sqlArgs []interface{}
for i, f := range ddl.fields {
if matches := columnRegexp.FindStringSubmatch(f); len(matches) > 1 && matches[1] == field.DBName {
ddl.fields[i] = fmt.Sprintf("`%v` ?", field.DBName)
sqlArgs = []interface{}{m.FullDataTypeOf(field)}
// table created by old version might look like `CREATE TABLE ? (? varchar(10) UNIQUE)`.
// FullDataTypeOf doesn't contain UNIQUE, so we need to add unique constraint.
if strings.Contains(strings.ToUpper(matches[3]), " UNIQUE") {
uniName := m.DB.NamingStrategy.UniqueName(stmt.Table, field.DBName)
uni, _ := m.GuessConstraintInterfaceAndTable(stmt, uniName)
if uni != nil {
uniSQL, uniArgs := uni.Build()
ddl.addConstraint(uniName, uniSQL)
sqlArgs = append(sqlArgs, uniArgs...)
}
}
break
}
}
return ddl, []interface{}{m.FullDataTypeOf(field)}, nil
return ddl, sqlArgs, nil
}
return nil, nil, fmt.Errorf("failed to alter field with name `%s`", name)
return nil, nil, fmt.Errorf("failed to alter field with name %v", name)
})
})
}
@@ -153,7 +167,7 @@ func (m _Migrator) DropColumn(value interface{}, name string) error {
func (m _Migrator) CreateConstraint(value interface{}, name string) error {
return m.RunWithValue(value, func(stmt *gorm.Statement) error {
constraint, chk, table := m.GuessConstraintAndTable(stmt, name)
constraint, table := m.GuessConstraintInterfaceAndTable(stmt, name)
return m.recreateTable(value, &table,
func(ddl *ddl, stmt *gorm.Statement) (*ddl, []interface{}, error) {
@@ -164,12 +178,8 @@ func (m _Migrator) CreateConstraint(value interface{}, name string) error {
)
if constraint != nil {
constraintName = constraint.Name
constraintSql, constraintValues = buildConstraint(constraint)
} else if chk != nil {
constraintName = chk.Name
constraintSql = "CONSTRAINT ? CHECK (?)"
constraintValues = []interface{}{clause.Column{Name: chk.Name}, clause.Expr{SQL: chk.Constraint}}
constraintName = constraint.GetName()
constraintSql, constraintValues = constraint.Build()
} else {
return nil, nil, nil
}
@@ -182,11 +192,9 @@ func (m _Migrator) CreateConstraint(value interface{}, name string) error {
func (m _Migrator) DropConstraint(value interface{}, name string) error {
return m.RunWithValue(value, func(stmt *gorm.Statement) error {
constraint, chk, table := m.GuessConstraintAndTable(stmt, name)
constraint, table := m.GuessConstraintInterfaceAndTable(stmt, name)
if constraint != nil {
name = constraint.Name
} else if chk != nil {
name = chk.Name
name = constraint.GetName()
}
return m.recreateTable(value, &table,
@@ -200,11 +208,9 @@ func (m _Migrator) DropConstraint(value interface{}, name string) error {
func (m _Migrator) HasConstraint(value interface{}, name string) bool {
var count int64
m.RunWithValue(value, func(stmt *gorm.Statement) error {
constraint, chk, table := m.GuessConstraintAndTable(stmt, name)
constraint, table := m.GuessConstraintInterfaceAndTable(stmt, name)
if constraint != nil {
name = constraint.Name
} else if chk != nil {
name = chk.Name
name = constraint.GetName()
}
m.DB.Raw(
@@ -317,26 +323,44 @@ func (m _Migrator) DropIndex(value interface{}, name string) error {
})
}
func buildConstraint(constraint *schema.Constraint) (sql string, results []interface{}) {
sql = "CONSTRAINT ? FOREIGN KEY ? REFERENCES ??"
if constraint.OnDelete != "" {
sql += " ON DELETE " + constraint.OnDelete
}
type _Index struct {
Seq int
Name string
Unique bool
Origin string
Partial bool
}
if constraint.OnUpdate != "" {
sql += " ON UPDATE " + constraint.OnUpdate
}
var foreignKeys, references []interface{}
for _, field := range constraint.ForeignKeys {
foreignKeys = append(foreignKeys, clause.Column{Name: field.DBName})
}
for _, field := range constraint.References {
references = append(references, clause.Column{Name: field.DBName})
}
results = append(results, clause.Table{Name: constraint.Name}, foreignKeys, clause.Table{Name: constraint.ReferenceSchema.Table}, references)
return
// GetIndexes return Indexes []gorm.Index and execErr error,
// See the [doc]
//
// [doc]: https://sqlite.org/pragma.html#pragma_index_list
func (m _Migrator) GetIndexes(value interface{}) ([]gorm.Index, error) {
indexes := make([]gorm.Index, 0)
err := m.RunWithValue(value, func(stmt *gorm.Statement) error {
rst := make([]*_Index, 0)
if err := m.DB.Debug().Raw("SELECT * FROM PRAGMA_index_list(?)", stmt.Table).Scan(&rst).Error; err != nil { // alias `PRAGMA index_list(?)`
return err
}
for _, index := range rst {
if index.Origin == "u" { // skip the index was created by a UNIQUE constraint
continue
}
var columns []string
if err := m.DB.Raw("SELECT name FROM PRAGMA_index_info(?)", index.Name).Scan(&columns).Error; err != nil { // alias `PRAGMA index_info(?)`
return err
}
indexes = append(indexes, &migrator.Index{
TableName: stmt.Table,
NameValue: index.Name,
ColumnList: columns,
PrimaryKeyValue: sql.NullBool{Bool: index.Origin == "pk", Valid: true}, // The exceptions are INTEGER PRIMARY KEY
UniqueValue: sql.NullBool{Bool: index.Unique, Valid: true},
})
}
return nil
})
return indexes, err
}
func (m _Migrator) getRawDDL(table string) (string, error) {

View File

@@ -2,7 +2,6 @@
package gormlite
import (
"context"
"database/sql"
"strconv"
@@ -46,23 +45,12 @@ func (dialector _Dialector) Initialize(db *gorm.DB) (err error) {
db.ConnPool = conn
}
var version string
if err := db.ConnPool.QueryRowContext(context.Background(), "select sqlite_version()").Scan(&version); err != nil {
return err
}
// https://sqlite.org/releaselog/3_35_0.html
if compareVersion(version, "3.35.0") >= 0 {
callbacks.RegisterDefaultCallbacks(db, &callbacks.Config{
CreateClauses: []string{"INSERT", "VALUES", "ON CONFLICT", "RETURNING"},
UpdateClauses: []string{"UPDATE", "SET", "FROM", "WHERE", "RETURNING"},
DeleteClauses: []string{"DELETE", "FROM", "WHERE", "RETURNING"},
LastInsertIDReversed: true,
})
} else {
callbacks.RegisterDefaultCallbacks(db, &callbacks.Config{
LastInsertIDReversed: true,
})
}
callbacks.RegisterDefaultCallbacks(db, &callbacks.Config{
CreateClauses: []string{"INSERT", "VALUES", "ON CONFLICT", "RETURNING"},
UpdateClauses: []string{"UPDATE", "SET", "FROM", "WHERE", "RETURNING"},
DeleteClauses: []string{"DELETE", "FROM", "WHERE", "RETURNING"},
LastInsertIDReversed: true,
})
for k, v := range dialector.ClauseBuilders() {
db.ClauseBuilders[k] = v
@@ -231,27 +219,3 @@ func (dialectopr _Dialector) RollbackTo(tx *gorm.DB, name string) error {
tx.Exec("ROLLBACK TO SAVEPOINT " + name)
return nil
}
func compareVersion(version1, version2 string) int {
n, m := len(version1), len(version2)
i, j := 0, 0
for i < n || j < m {
x := 0
for ; i < n && version1[i] != '.'; i++ {
x = x*10 + int(version1[i]-'0')
}
i++
y := 0
for ; j < m && version2[j] != '.'; j++ {
y = y*10 + int(version2[j]-'0')
}
j++
if x > y {
return 1
}
if x < y {
return -1
}
}
return 0
}

View File

@@ -3,10 +3,11 @@ set -euo pipefail
cd -P -- "$(dirname -- "$0")"
rm -rf gorm/ tests/
go work use -r .
go test
rm -rf gorm/ tests/
git clone --filter=blob:none https://github.com/go-gorm/gorm.git
git clone --branch v1.25.9 --filter=blob:none https://github.com/go-gorm/gorm.git
mv gorm/tests tests
rm -rf gorm/

71
internal/util/alloc.go Normal file
View File

@@ -0,0 +1,71 @@
//go:build unix
package util
import (
"math"
"github.com/tetratelabs/wazero/experimental"
"golang.org/x/sys/unix"
)
func mmappedAllocator(cap, max uint64) experimental.LinearMemory {
// Round up to the page size.
rnd := uint64(unix.Getpagesize() - 1)
max = (max + rnd) &^ rnd
cap = (cap + rnd) &^ rnd
if max > math.MaxInt {
// This ensures int(max) overflows to a negative value,
// and unix.Mmap returns EINVAL.
max = math.MaxUint64
}
// Reserve max bytes of address space, to ensure we won't need to move it.
// A protected, private, anonymous mapping should not commit memory.
b, err := unix.Mmap(-1, 0, int(max), unix.PROT_NONE, unix.MAP_PRIVATE|unix.MAP_ANON)
if err != nil {
panic(err)
}
// Commit the initial cap bytes of memory.
err = unix.Mprotect(b[:cap], unix.PROT_READ|unix.PROT_WRITE)
if err != nil {
unix.Munmap(b)
panic(err)
}
return &mmappedMemory{buf: b[:cap]}
}
// The slice covers the entire mmapped memory:
// - len(buf) is the already committed memory,
// - cap(buf) is the reserved address space.
type mmappedMemory struct {
buf []byte
}
func (m *mmappedMemory) Reallocate(size uint64) []byte {
if com := uint64(len(m.buf)); com < size {
// Round up to the page size.
rnd := uint64(unix.Getpagesize() - 1)
new := (size + rnd) &^ rnd
// Commit additional memory up to new bytes.
err := unix.Mprotect(m.buf[com:new], unix.PROT_READ|unix.PROT_WRITE)
if err != nil {
panic(err)
}
// Update committed memory.
m.buf = m.buf[:new]
}
// Limit returned capacity because bytes beyond
// len(m.buf) have not yet been committed.
return m.buf[:size:len(m.buf)]
}
func (m *mmappedMemory) Free() {
err := unix.Munmap(m.buf[:cap(m.buf)])
if err != nil {
panic(err)
}
m.buf = nil
}

View File

@@ -23,6 +23,19 @@ func ExportFuncVI[T0 i32](mod wazero.HostModuleBuilder, name string, fn func(con
Export(name)
}
type funcVII[T0, T1 i32] func(context.Context, api.Module, T0, T1)
func (fn funcVII[T0, T1]) Call(ctx context.Context, mod api.Module, stack []uint64) {
fn(ctx, mod, T0(stack[0]), T1(stack[1]))
}
func ExportFuncVII[T0, T1 i32](mod wazero.HostModuleBuilder, name string, fn func(context.Context, api.Module, T0, T1)) {
mod.NewFunctionBuilder().
WithGoModuleFunction(funcVII[T0, T1](fn),
[]api.ValueType{api.ValueTypeI32, api.ValueTypeI32}, nil).
Export(name)
}
type funcVIII[T0, T1, T2 i32] func(context.Context, api.Module, T0, T1, T2)
func (fn funcVIII[T0, T1, T2]) Call(ctx context.Context, mod api.Module, stack []uint64) {
@@ -36,6 +49,45 @@ func ExportFuncVIII[T0, T1, T2 i32](mod wazero.HostModuleBuilder, name string, f
Export(name)
}
type funcVIIII[T0, T1, T2, T3 i32] func(context.Context, api.Module, T0, T1, T2, T3)
func (fn funcVIIII[T0, T1, T2, T3]) Call(ctx context.Context, mod api.Module, stack []uint64) {
fn(ctx, mod, T0(stack[0]), T1(stack[1]), T2(stack[2]), T3(stack[3]))
}
func ExportFuncVIIII[T0, T1, T2, T3 i32](mod wazero.HostModuleBuilder, name string, fn func(context.Context, api.Module, T0, T1, T2, T3)) {
mod.NewFunctionBuilder().
WithGoModuleFunction(funcVIIII[T0, T1, T2, T3](fn),
[]api.ValueType{api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32}, nil).
Export(name)
}
type funcVIIIII[T0, T1, T2, T3, T4 i32] func(context.Context, api.Module, T0, T1, T2, T3, T4)
func (fn funcVIIIII[T0, T1, T2, T3, T4]) Call(ctx context.Context, mod api.Module, stack []uint64) {
fn(ctx, mod, T0(stack[0]), T1(stack[1]), T2(stack[2]), T3(stack[3]), T4(stack[4]))
}
func ExportFuncVIIIII[T0, T1, T2, T3, T4 i32](mod wazero.HostModuleBuilder, name string, fn func(context.Context, api.Module, T0, T1, T2, T3, T4)) {
mod.NewFunctionBuilder().
WithGoModuleFunction(funcVIIIII[T0, T1, T2, T3, T4](fn),
[]api.ValueType{api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32}, nil).
Export(name)
}
type funcVIIIIJ[T0, T1, T2, T3 i32, T4 i64] func(context.Context, api.Module, T0, T1, T2, T3, T4)
func (fn funcVIIIIJ[T0, T1, T2, T3, T4]) Call(ctx context.Context, mod api.Module, stack []uint64) {
fn(ctx, mod, T0(stack[0]), T1(stack[1]), T2(stack[2]), T3(stack[3]), T4(stack[4]))
}
func ExportFuncVIIIIJ[T0, T1, T2, T3 i32, T4 i64](mod wazero.HostModuleBuilder, name string, fn func(context.Context, api.Module, T0, T1, T2, T3, T4)) {
mod.NewFunctionBuilder().
WithGoModuleFunction(funcVIIIIJ[T0, T1, T2, T3, T4](fn),
[]api.ValueType{api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI64}, nil).
Export(name)
}
type funcII[TR, T0 i32] func(context.Context, api.Module, T0) TR
func (fn funcII[TR, T0]) Call(ctx context.Context, mod api.Module, stack []uint64) {
@@ -101,6 +153,19 @@ func ExportFuncIIIIII[TR, T0, T1, T2, T3, T4 i32](mod wazero.HostModuleBuilder,
Export(name)
}
type funcIIIIIII[TR, T0, T1, T2, T3, T4, T5 i32] func(context.Context, api.Module, T0, T1, T2, T3, T4, T5) TR
func (fn funcIIIIIII[TR, T0, T1, T2, T3, T4, T5]) Call(ctx context.Context, mod api.Module, stack []uint64) {
stack[0] = uint64(fn(ctx, mod, T0(stack[0]), T1(stack[1]), T2(stack[2]), T3(stack[3]), T4(stack[4]), T5(stack[5])))
}
func ExportFuncIIIIIII[TR, T0, T1, T2, T3, T4, T5 i32](mod wazero.HostModuleBuilder, name string, fn func(context.Context, api.Module, T0, T1, T2, T3, T4, T5) TR) {
mod.NewFunctionBuilder().
WithGoModuleFunction(funcIIIIIII[TR, T0, T1, T2, T3, T4, T5](fn),
[]api.ValueType{api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32, api.ValueTypeI32}, []api.ValueType{api.ValueTypeI32}).
Export(name)
}
type funcIIIIJ[TR, T0, T1, T2 i32, T3 i64] func(context.Context, api.Module, T0, T1, T2, T3) TR
func (fn funcIIIIJ[TR, T0, T1, T2, T3]) Call(ctx context.Context, mod api.Module, stack []uint64) {

View File

@@ -3,21 +3,11 @@ package util
import (
"context"
"io"
"github.com/tetratelabs/wazero/experimental"
)
type handleKey struct{}
type handleState struct {
handles []any
empty int
}
func NewContext(ctx context.Context) context.Context {
state := new(handleState)
ctx = experimental.WithCloseNotifier(ctx, state)
ctx = context.WithValue(ctx, handleKey{}, state)
return ctx
holes int
}
func (s *handleState) CloseNotify(ctx context.Context, exitCode uint32) {
@@ -27,14 +17,14 @@ func (s *handleState) CloseNotify(ctx context.Context, exitCode uint32) {
}
}
s.handles = nil
s.empty = 0
s.holes = 0
}
func GetHandle(ctx context.Context, id uint32) any {
if id == 0 {
return nil
}
s := ctx.Value(handleKey{}).(*handleState)
s := ctx.Value(moduleKey{}).(*moduleState)
return s.handles[^id]
}
@@ -42,10 +32,10 @@ func DelHandle(ctx context.Context, id uint32) error {
if id == 0 {
return nil
}
s := ctx.Value(handleKey{}).(*handleState)
s := ctx.Value(moduleKey{}).(*moduleState)
a := s.handles[^id]
s.handles[^id] = nil
s.empty++
s.holes++
if c, ok := a.(io.Closer); ok {
return c.Close()
}
@@ -56,13 +46,13 @@ func AddHandle(ctx context.Context, a any) (id uint32) {
if a == nil {
panic(NilErr)
}
s := ctx.Value(handleKey{}).(*handleState)
s := ctx.Value(moduleKey{}).(*moduleState)
// Find an empty slot.
if s.empty > cap(s.handles)-len(s.handles) {
if s.holes > cap(s.handles)-len(s.handles) {
for id, h := range s.handles {
if h == nil {
s.empty--
s.holes--
s.handles[id] = a
return ^uint32(id)
}

35
internal/util/json.go Normal file
View File

@@ -0,0 +1,35 @@
package util
import (
"encoding/json"
"strconv"
"time"
"unsafe"
)
type JSON struct{ Value any }
func (j JSON) Scan(value any) error {
var buf []byte
switch v := value.(type) {
case []byte:
buf = v
case string:
buf = unsafe.Slice(unsafe.StringData(v), len(v))
case int64:
buf = strconv.AppendInt(nil, v, 10)
case float64:
buf = strconv.AppendFloat(nil, v, 'g', -1, 64)
case time.Time:
buf = append(buf, '"')
buf = v.AppendFormat(buf, time.RFC3339Nano)
buf = append(buf, '"')
case nil:
buf = append(buf, "null"...)
default:
panic(AssertErr())
}
return json.Unmarshal(buf, j.Value)
}

106
internal/util/mmap.go Normal file
View File

@@ -0,0 +1,106 @@
//go:build (darwin || linux || illumos) && (amd64 || arm64 || riscv64) && !sqlite3_flock && !sqlite3_noshm && !sqlite3_nosys
package util
import (
"context"
"os"
"unsafe"
"github.com/tetratelabs/wazero/api"
"github.com/tetratelabs/wazero/experimental"
"golang.org/x/sys/unix"
)
type mmapState struct {
regions []*MappedRegion
enabled bool
}
func (s *mmapState) init(ctx context.Context, enabled bool) context.Context {
if s.enabled = enabled; enabled {
return experimental.WithMemoryAllocator(ctx,
experimental.MemoryAllocatorFunc(mmappedAllocator))
}
return ctx
}
func CanMap(ctx context.Context) bool {
s := ctx.Value(moduleKey{}).(*moduleState)
return s.mmapState.enabled
}
func (s *mmapState) new(ctx context.Context, mod api.Module, size int32) *MappedRegion {
// Find unused region.
for _, r := range s.regions {
if !r.used && r.size == size {
return r
}
}
// Allocate page aligned memmory.
alloc := mod.ExportedFunction("aligned_alloc")
stack := [2]uint64{
uint64(unix.Getpagesize()),
uint64(size),
}
if err := alloc.CallWithStack(ctx, stack[:]); err != nil {
panic(err)
}
if stack[0] == 0 {
panic(OOMErr)
}
// Save the newly allocated region.
ptr := uint32(stack[0])
buf := View(mod, ptr, uint64(size))
addr := uintptr(unsafe.Pointer(&buf[0]))
s.regions = append(s.regions, &MappedRegion{
Ptr: ptr,
addr: addr,
size: size,
})
return s.regions[len(s.regions)-1]
}
type MappedRegion struct {
addr uintptr
Ptr uint32
size int32
used bool
}
func MapRegion(ctx context.Context, mod api.Module, f *os.File, offset int64, size int32, prot int) (*MappedRegion, error) {
s := ctx.Value(moduleKey{}).(*moduleState)
r := s.new(ctx, mod, size)
err := r.mmap(f, offset, prot)
if err != nil {
return nil, err
}
return r, nil
}
func (r *MappedRegion) Unmap() error {
// We can't munmap the region, otherwise it could be remaped.
// Instead, convert it to a protected, private, anonymous mapping.
// If successful, it can be reused for a subsequent mmap.
_, err := mmap(r.addr, uintptr(r.size),
unix.PROT_NONE, unix.MAP_PRIVATE|unix.MAP_ANON|unix.MAP_FIXED,
-1, 0)
r.used = err != nil
return err
}
func (r *MappedRegion) mmap(f *os.File, offset int64, prot int) error {
_, err := mmap(r.addr, uintptr(r.size),
prot, unix.MAP_SHARED|unix.MAP_FIXED,
int(f.Fd()), offset)
r.used = err == nil
return err
}
// We need the low level mmap for MAP_FIXED to work.
// Bind the syscall version hoping that it is more stable.
//go:linkname mmap syscall.mmap
func mmap(addr, length uintptr, prot, flag, fd int, pos int64) (*byte, error)

View File

@@ -0,0 +1,15 @@
//go:build !(darwin || linux || illumos) || !(amd64 || arm64 || riscv64) || sqlite3_flock || sqlite3_noshm || sqlite3_nosys
package util
import "context"
type mmapState struct{}
func (s *mmapState) init(ctx context.Context, _ bool) context.Context {
return ctx
}
func CanMap(ctx context.Context) bool {
return false
}

21
internal/util/module.go Normal file
View File

@@ -0,0 +1,21 @@
package util
import (
"context"
"github.com/tetratelabs/wazero/experimental"
)
type moduleKey struct{}
type moduleState struct {
handleState
mmapState
}
func NewContext(ctx context.Context, mappableMemory bool) context.Context {
state := new(moduleState)
ctx = context.WithValue(ctx, moduleKey{}, state)
ctx = experimental.WithCloseNotifier(ctx, state)
ctx = state.mmapState.init(ctx, mappableMemory)
return ctx
}

11
internal/util/pointer.go Normal file
View File

@@ -0,0 +1,11 @@
package util
type Pointer[T any] struct{ Value T }
func (p Pointer[T]) unwrap() any { return p.Value }
type PointerUnwrap interface{ unwrap() any }
func UnwrapPointer(p PointerUnwrap) any {
return p.unwrap()
}

View File

@@ -0,0 +1,15 @@
package util_test
import (
"math"
"testing"
"github.com/ncruces/go-sqlite3/internal/util"
)
func TestUnwrapPointer(t *testing.T) {
p := util.Pointer[float64]{Value: math.Pi}
if got := util.UnwrapPointer(p); got != math.Pi {
t.Errorf("want π, got %v", got)
}
}

10
internal/util/reflect.go Normal file
View File

@@ -0,0 +1,10 @@
package util
import "reflect"
func ReflectType(v reflect.Value) reflect.Type {
if v.Kind() != reflect.Invalid {
return v.Type()
}
return nil
}

View File

@@ -0,0 +1,21 @@
package util
import (
"fmt"
"math"
"reflect"
"testing"
)
func TestReflectType(t *testing.T) {
tests := []any{nil, 1, math.Pi, "abc"}
for _, tt := range tests {
t.Run(fmt.Sprint(tt), func(t *testing.T) {
want := fmt.Sprintf("%T", tt)
got := fmt.Sprintf("%v", ReflectType(reflect.ValueOf(tt)))
if got != want {
t.Errorf("ReflectType() = %v, want %v", got, want)
}
})
}
}

41
json.go
View File

@@ -1,46 +1,11 @@
package sqlite3
import (
"encoding/json"
"strconv"
"time"
"unsafe"
"github.com/ncruces/go-sqlite3/internal/util"
)
import "github.com/ncruces/go-sqlite3/internal/util"
// JSON returns a value that can be used as an argument to
// [database/sql.DB.Exec], [database/sql.Row.Scan] and similar methods to
// store value as JSON, or decode JSON into value.
// JSON should NOT be used with [BindJSON] or [ResultJSON].
func JSON(value any) any {
return jsonValue{value}
}
type jsonValue struct{ any }
func (j jsonValue) JSON() any { return j.any }
func (j jsonValue) Scan(value any) error {
var buf []byte
switch v := value.(type) {
case []byte:
buf = v
case string:
buf = unsafe.Slice(unsafe.StringData(v), len(v))
case int64:
buf = strconv.AppendInt(nil, v, 10)
case float64:
buf = strconv.AppendFloat(nil, v, 'g', -1, 64)
case time.Time:
buf = append(buf, '"')
buf = v.AppendFormat(buf, time.RFC3339Nano)
buf = append(buf, '"')
case nil:
buf = append(buf, "null"...)
default:
panic(util.AssertErr())
}
return json.Unmarshal(buf, j.any)
return util.JSON{Value: value}
}

View File

@@ -1,14 +1,12 @@
package sqlite3
// Pointer returns a pointer to a value
// that can be used as an argument to
import "github.com/ncruces/go-sqlite3/internal/util"
// Pointer returns a pointer to a value that can be used as an argument to
// [database/sql.DB.Exec] and similar methods.
// Pointer should NOT be used with [BindPointer] or [ResultPointer].
//
// https://sqlite.org/bindptr.html
func Pointer[T any](val T) any {
return pointer[T]{val}
func Pointer[T any](value T) any {
return util.Pointer[T]{Value: value}
}
type pointer[T any] struct{ val T }
func (p pointer[T]) Pointer() any { return p.val }

View File

@@ -15,14 +15,14 @@ import (
"github.com/tetratelabs/wazero/api"
)
// Configure SQLite WASM.
// Configure SQLite Wasm.
//
// Importing package embed initializes these
// Importing package embed initializes [Binary]
// with an appropriate build of SQLite:
//
// import _ "github.com/ncruces/go-sqlite3/embed"
var (
Binary []byte // WASM binary to load.
Binary []byte // Wasm binary to load.
Path string // Path to load the binary from.
RuntimeConfig wazero.RuntimeConfig
@@ -85,10 +85,10 @@ func instantiateSQLite() (sqlt *sqlite, err error) {
}
sqlt = new(sqlite)
sqlt.ctx = util.NewContext(context.Background())
sqlt.ctx = util.NewContext(context.Background(), vfs.SupportsSharedMemory)
sqlt.mod, err = instance.runtime.InstantiateModule(sqlt.ctx,
instance.compiled, wazero.NewModuleConfig())
instance.compiled, wazero.NewModuleConfig().WithName(""))
if err != nil {
return nil, err
}
@@ -99,8 +99,8 @@ func instantiateSQLite() (sqlt *sqlite, err error) {
}
sqlt.freer = util.ReadUint32(sqlt.mod, uint32(global.Get()))
if err != nil {
return nil, err
if sqlt.freer == 0 {
return nil, util.BadBinaryErr
}
return sqlt, nil
}
@@ -126,7 +126,7 @@ func (sqlt *sqlite) error(rc uint64, handle uint32, sql ...string) error {
if handle != 0 {
if r := sqlt.call("sqlite3_errmsg", uint64(handle)); r != 0 {
err.msg = util.ReadString(sqlt.mod, uint32(r), _MAX_NAME)
err.msg = util.ReadString(sqlt.mod, uint32(r), _MAX_LENGTH)
}
if sql != nil {
@@ -216,6 +216,8 @@ func (sqlt *sqlite) newString(s string) uint32 {
}
func (sqlt *sqlite) newArena(size uint64) arena {
// Ensure the arena's size is a multiple of 8.
size = (size + 7) &^ 7
return arena{
sqlt: sqlt,
size: uint32(size),
@@ -255,6 +257,12 @@ func (a *arena) mark() (reset func()) {
}
func (a *arena) new(size uint64) uint32 {
// Align the next address, to 4 or 8 bytes.
if size&7 != 0 {
a.next = (a.next + 3) &^ 3
} else {
a.next = (a.next + 7) &^ 7
}
if size <= uint64(a.size-a.next) {
ptr := a.base + a.next
a.next += uint32(size)
@@ -266,7 +274,7 @@ func (a *arena) new(size uint64) uint32 {
}
func (a *arena) bytes(b []byte) uint32 {
if b == nil {
if (*[0]byte)(b) == nil {
return 0
}
ptr := a.new(uint64(len(b)))
@@ -281,16 +289,25 @@ func (a *arena) string(s string) uint32 {
}
func exportCallbacks(env wazero.HostModuleBuilder) wazero.HostModuleBuilder {
util.ExportFuncII(env, "go_progress", progressCallback)
util.ExportFuncIII(env, "go_busy_handler", busyCallback)
util.ExportFuncII(env, "go_progress_handler", progressCallback)
util.ExportFuncII(env, "go_commit_hook", commitCallback)
util.ExportFuncVI(env, "go_rollback_hook", rollbackCallback)
util.ExportFuncVIIIIJ(env, "go_update_hook", updateCallback)
util.ExportFuncIIIII(env, "go_wal_hook", walCallback)
util.ExportFuncIIIIII(env, "go_autovacuum_pages", autoVacuumCallback)
util.ExportFuncIIIIIII(env, "go_authorizer", authorizerCallback)
util.ExportFuncVIII(env, "go_log", logCallback)
util.ExportFuncVI(env, "go_destroy", destroyCallback)
util.ExportFuncVIII(env, "go_func", funcCallback)
util.ExportFuncVIII(env, "go_step", stepCallback)
util.ExportFuncVI(env, "go_final", finalCallback)
util.ExportFuncVI(env, "go_value", valueCallback)
util.ExportFuncVIII(env, "go_inverse", inverseCallback)
util.ExportFuncVIIII(env, "go_func", funcCallback)
util.ExportFuncVIIIII(env, "go_step", stepCallback)
util.ExportFuncVIII(env, "go_final", finalCallback)
util.ExportFuncVII(env, "go_value", valueCallback)
util.ExportFuncVIIII(env, "go_inverse", inverseCallback)
util.ExportFuncVIIII(env, "go_collation_needed", collationCallback)
util.ExportFuncIIIIII(env, "go_compare", compareCallback)
util.ExportFuncIIIIII(env, "go_vtab_create", vtabModuleCallback(0))
util.ExportFuncIIIIII(env, "go_vtab_connect", vtabModuleCallback(1))
util.ExportFuncIIIIII(env, "go_vtab_create", vtabModuleCallback(xCreate))
util.ExportFuncIIIIII(env, "go_vtab_connect", vtabModuleCallback(xConnect))
util.ExportFuncII(env, "go_vtab_disconnect", vtabDisconnectCallback)
util.ExportFuncII(env, "go_vtab_destroy", vtabDestroyCallback)
util.ExportFuncIII(env, "go_vtab_best_index", vtabBestIndexCallback)

51
sqlite3/column.c Normal file
View File

@@ -0,0 +1,51 @@
#include <stddef.h>
#include "sqlite3.h"
union sqlite3_data {
sqlite3_int64 i;
double d;
struct {
const void *ptr;
int len;
};
};
int sqlite3_columns_go(sqlite3_stmt *stmt, int nCol, char *aType,
union sqlite3_data *aData) {
if (nCol != sqlite3_column_count(stmt)) {
return SQLITE_MISUSE;
}
int rc = SQLITE_OK;
for (int i = 0; i < nCol; ++i) {
const void *ptr = NULL;
switch (aType[i] = sqlite3_column_type(stmt, i)) {
default: // SQLITE_NULL
aData[i] = (union sqlite3_data){};
case SQLITE_INTEGER:
aData[i].i = sqlite3_column_int64(stmt, i);
continue;
case SQLITE_FLOAT:
aData[i].d = sqlite3_column_double(stmt, i);
continue;
case SQLITE_TEXT:
ptr = sqlite3_column_text(stmt, i);
break;
case SQLITE_BLOB:
ptr = sqlite3_column_blob(stmt, i);
break;
}
if (ptr == NULL && rc == SQLITE_OK) {
rc = sqlite3_errcode(sqlite3_db_handle(stmt));
}
aData[i].ptr = ptr;
aData[i].len = sqlite3_column_bytes(stmt, i);
}
return rc;
}
static_assert(offsetof(union sqlite3_data, i) == 0, "Unexpected offset");
static_assert(offsetof(union sqlite3_data, d) == 0, "Unexpected offset");
static_assert(offsetof(union sqlite3_data, ptr) == 0, "Unexpected offset");
static_assert(offsetof(union sqlite3_data, len) == 4, "Unexpected offset");
static_assert(sizeof(union sqlite3_data) == 8, "Unexpected size");

556
sqlite3/date.patch Normal file
View File

@@ -0,0 +1,556 @@
# Backport from 3.46.
# https://sqlite.org/draft/releaselog/current.html
--- sqlite3.c.orig
+++ sqlite3.c
@@ -71,13 +71,14 @@ struct DateTime {
int tz; /* Timezone offset in minutes */
double s; /* Seconds */
char validJD; /* True (1) if iJD is valid */
- char rawS; /* Raw numeric value stored in s */
char validYMD; /* True (1) if Y,M,D are valid */
char validHMS; /* True (1) if h,m,s are valid */
- char validTZ; /* True (1) if tz is valid */
- char tzSet; /* Timezone was set explicitly */
- char isError; /* An overflow has occurred */
- char useSubsec; /* Display subsecond precision */
+ char nFloor; /* Days to implement "floor" */
+ unsigned rawS : 1; /* Raw numeric value stored in s */
+ unsigned isError : 1; /* An overflow has occurred */
+ unsigned useSubsec : 1; /* Display subsecond precision */
+ unsigned isUtc : 1; /* Time is known to be UTC */
+ unsigned isLocal : 1; /* Time is known to be localtime */
};
@@ -175,6 +176,8 @@ static int parseTimezone(const char *zDate, DateTime *p){
sgn = +1;
}else if( c=='Z' || c=='z' ){
zDate++;
+ p->isLocal = 0;
+ p->isUtc = 1;
goto zulu_time;
}else{
return c!=0;
@@ -187,7 +190,6 @@ static int parseTimezone(const char *zDate, DateTime *p){
p->tz = sgn*(nMn + nHr*60);
zulu_time:
while( sqlite3Isspace(*zDate) ){ zDate++; }
- p->tzSet = 1;
return *zDate!=0;
}
@@ -231,7 +233,6 @@ static int parseHhMmSs(const char *zDate, DateTime *p){
p->m = m;
p->s = s + ms;
if( parseTimezone(zDate, p) ) return 1;
- p->validTZ = (p->tz!=0)?1:0;
return 0;
}
@@ -278,15 +279,40 @@ static void computeJD(DateTime *p){
p->validJD = 1;
if( p->validHMS ){
p->iJD += p->h*3600000 + p->m*60000 + (sqlite3_int64)(p->s*1000 + 0.5);
- if( p->validTZ ){
+ if( p->tz ){
p->iJD -= p->tz*60000;
p->validYMD = 0;
p->validHMS = 0;
- p->validTZ = 0;
+ p->tz = 0;
+ p->isUtc = 1;
+ p->isLocal = 0;
}
}
}
+/*
+** Given the YYYY-MM-DD information current in p, determine if there
+** is day-of-month overflow and set nFloor to the number of days that
+** would need to be subtracted from the date in order to bring the
+** date back to the end of the month.
+*/
+static void computeFloor(DateTime *p){
+ assert( p->validYMD || p->isError );
+ assert( p->D>=0 && p->D<=31 );
+ assert( p->M>=0 && p->M<=12 );
+ if( p->D<=28 ){
+ p->nFloor = 0;
+ }else if( (1<<p->M) & 0x15aa ){
+ p->nFloor = 0;
+ }else if( p->M!=2 ){
+ p->nFloor = (p->D==31);
+ }else if( p->Y%4!=0 || (p->Y%100==0 && p->Y%400!=0) ){
+ p->nFloor = p->D - 28;
+ }else{
+ p->nFloor = p->D - 29;
+ }
+}
+
/*
** Parse dates of the form
**
@@ -325,12 +351,16 @@ static int parseYyyyMmDd(const char *zDate, DateTime *p){
p->Y = neg ? -Y : Y;
p->M = M;
p->D = D;
- if( p->validTZ ){
+ computeFloor(p);
+ if( p->tz ){
computeJD(p);
}
return 0;
}
+
+static void clearYMD_HMS_TZ(DateTime *p); /* Forward declaration */
+
/*
** Set the time to the current time reported by the VFS.
**
@@ -340,6 +370,9 @@ static int setDateTimeToCurrent(sqlite3_context *context, DateTime *p){
p->iJD = sqlite3StmtCurrentTime(context);
if( p->iJD>0 ){
p->validJD = 1;
+ p->isUtc = 1;
+ p->isLocal = 0;
+ clearYMD_HMS_TZ(p);
return 0;
}else{
return 1;
@@ -478,7 +511,7 @@ static void computeYMD_HMS(DateTime *p){
static void clearYMD_HMS_TZ(DateTime *p){
p->validYMD = 0;
p->validHMS = 0;
- p->validTZ = 0;
+ p->tz = 0;
}
#ifndef SQLITE_OMIT_LOCALTIME
@@ -610,7 +643,7 @@ static int toLocaltime(
p->validHMS = 1;
p->validJD = 0;
p->rawS = 0;
- p->validTZ = 0;
+ p->tz = 0;
p->isError = 0;
return SQLITE_OK;
}
@@ -630,12 +663,12 @@ static const struct {
float rLimit; /* Maximum NNN value for this transform */
float rXform; /* Constant used for this transform */
} aXformType[] = {
- { 6, "second", 4.6427e+14, 1.0 },
- { 6, "minute", 7.7379e+12, 60.0 },
- { 4, "hour", 1.2897e+11, 3600.0 },
- { 3, "day", 5373485.0, 86400.0 },
- { 5, "month", 176546.0, 2592000.0 },
- { 4, "year", 14713.0, 31536000.0 },
+ /* 0 */ { 6, "second", 4.6427e+14, 1.0 },
+ /* 1 */ { 6, "minute", 7.7379e+12, 60.0 },
+ /* 2 */ { 4, "hour", 1.2897e+11, 3600.0 },
+ /* 3 */ { 3, "day", 5373485.0, 86400.0 },
+ /* 4 */ { 5, "month", 176546.0, 30.0*86400.0 },
+ /* 5 */ { 4, "year", 14713.0, 365.0*86400.0 },
};
/*
@@ -667,14 +700,20 @@ static void autoAdjustDate(DateTime *p){
** NNN.NNNN seconds
** NNN months
** NNN years
+** +/-YYYY-MM-DD HH:MM:SS.SSS
+** ceiling
+** floor
** start of month
** start of year
** start of week
** start of day
** weekday N
** unixepoch
+** auto
** localtime
** utc
+** subsec
+** subsecond
**
** Return 0 on success and 1 if there is any kind of error. If the error
** is in a system call (i.e. localtime()), then an error message is written
@@ -705,6 +744,37 @@ static int parseModifier(
}
break;
}
+ case 'c': {
+ /*
+ ** ceiling
+ **
+ ** Resolve day-of-month overflow by rolling forward into the next
+ ** month. As this is the default action, this modifier is really
+ ** a no-op that is only included for symmetry. See "floor".
+ */
+ if( sqlite3_stricmp(z, "ceiling")==0 ){
+ computeJD(p);
+ clearYMD_HMS_TZ(p);
+ rc = 0;
+ p->nFloor = 0;
+ }
+ break;
+ }
+ case 'f': {
+ /*
+ ** floor
+ **
+ ** Resolve day-of-month overflow by rolling back to the end of the
+ ** previous month.
+ */
+ if( sqlite3_stricmp(z, "floor")==0 ){
+ computeJD(p);
+ p->iJD -= p->nFloor*86400000;
+ clearYMD_HMS_TZ(p);
+ rc = 0;
+ }
+ break;
+ }
case 'j': {
/*
** julianday
@@ -731,7 +801,9 @@ static int parseModifier(
** show local time.
*/
if( sqlite3_stricmp(z, "localtime")==0 && sqlite3NotPureFunc(pCtx) ){
- rc = toLocaltime(p, pCtx);
+ rc = p->isLocal ? SQLITE_OK : toLocaltime(p, pCtx);
+ p->isUtc = 0;
+ p->isLocal = 1;
}
break;
}
@@ -756,7 +828,7 @@ static int parseModifier(
}
#ifndef SQLITE_OMIT_LOCALTIME
else if( sqlite3_stricmp(z, "utc")==0 && sqlite3NotPureFunc(pCtx) ){
- if( p->tzSet==0 ){
+ if( p->isUtc==0 ){
i64 iOrigJD; /* Original localtime */
i64 iGuess; /* Guess at the corresponding utc time */
int cnt = 0; /* Safety to prevent infinite loop */
@@ -779,7 +851,8 @@ static int parseModifier(
memset(p, 0, sizeof(*p));
p->iJD = iGuess;
p->validJD = 1;
- p->tzSet = 1;
+ p->isUtc = 1;
+ p->isLocal = 0;
}
rc = SQLITE_OK;
}
@@ -799,7 +872,7 @@ static int parseModifier(
&& r>=0.0 && r<7.0 && (n=(int)r)==r ){
sqlite3_int64 Z;
computeYMD_HMS(p);
- p->validTZ = 0;
+ p->tz = 0;
p->validJD = 0;
computeJD(p);
Z = ((p->iJD + 129600000)/86400000) % 7;
@@ -839,7 +912,7 @@ static int parseModifier(
p->h = p->m = 0;
p->s = 0.0;
p->rawS = 0;
- p->validTZ = 0;
+ p->tz = 0;
p->validJD = 0;
if( sqlite3_stricmp(z,"month")==0 ){
p->D = 1;
@@ -910,6 +983,7 @@ static int parseModifier(
x = p->M>0 ? (p->M-1)/12 : (p->M-12)/12;
p->Y += x;
p->M -= x*12;
+ computeFloor(p);
computeJD(p);
p->validHMS = 0;
p->validYMD = 0;
@@ -956,11 +1030,12 @@ static int parseModifier(
z += n;
while( sqlite3Isspace(*z) ) z++;
n = sqlite3Strlen30(z);
- if( n>10 || n<3 ) break;
+ if( n<3 || n>10 ) break;
if( sqlite3UpperToLower[(u8)z[n-1]]=='s' ) n--;
computeJD(p);
assert( rc==1 );
rRounder = r<0 ? -0.5 : +0.5;
+ p->nFloor = 0;
for(i=0; i<ArraySize(aXformType); i++){
if( aXformType[i].nName==n
&& sqlite3_strnicmp(aXformType[i].zName, z, n)==0
@@ -968,21 +1043,24 @@ static int parseModifier(
){
switch( i ){
case 4: { /* Special processing to add months */
- assert( strcmp(aXformType[i].zName,"month")==0 );
+ assert( strcmp(aXformType[4].zName,"month")==0 );
computeYMD_HMS(p);
p->M += (int)r;
x = p->M>0 ? (p->M-1)/12 : (p->M-12)/12;
p->Y += x;
p->M -= x*12;
+ computeFloor(p);
p->validJD = 0;
r -= (int)r;
break;
}
case 5: { /* Special processing to add years */
int y = (int)r;
- assert( strcmp(aXformType[i].zName,"year")==0 );
+ assert( strcmp(aXformType[5].zName,"year")==0 );
computeYMD_HMS(p);
+ assert( p->M>=0 && p->M<=12 );
p->Y += y;
+ computeFloor(p);
p->validJD = 0;
r -= (int)r;
break;
@@ -1236,22 +1314,83 @@ static void dateFunc(
}
}
+/*
+** Compute the number of days after the most recent January 1.
+**
+** In other words, compute the zero-based day number for the
+** current year:
+**
+** Jan01 = 0, Jan02 = 1, ..., Jan31 = 30, Feb01 = 31, ...
+** Dec31 = 364 or 365.
+*/
+static int daysAfterJan01(DateTime *pDate){
+ DateTime jan01 = *pDate;
+ assert( jan01.validYMD );
+ assert( jan01.validHMS );
+ assert( pDate->validJD );
+ jan01.validJD = 0;
+ jan01.M = 1;
+ jan01.D = 1;
+ computeJD(&jan01);
+ return (int)((pDate->iJD-jan01.iJD+43200000)/86400000);
+}
+
+/*
+** Return the number of days after the most recent Monday.
+**
+** In other words, return the day of the week according
+** to this code:
+**
+** 0=Monday, 1=Tuesday, 2=Wednesday, ..., 6=Sunday.
+*/
+static int daysAfterMonday(DateTime *pDate){
+ assert( pDate->validJD );
+ return (int)((pDate->iJD+43200000)/86400000) % 7;
+}
+
+/*
+** Return the number of days after the most recent Sunday.
+**
+** In other words, return the day of the week according
+** to this code:
+**
+** 0=Sunday, 1=Monday, 2=Tues, ..., 6=Saturday
+*/
+static int daysAfterSunday(DateTime *pDate){
+ assert( pDate->validJD );
+ return (int)((pDate->iJD+129600000)/86400000) % 7;
+}
+
/*
** strftime( FORMAT, TIMESTRING, MOD, MOD, ...)
**
** Return a string described by FORMAT. Conversions as follows:
**
-** %d day of month
+** %d day of month 01-31
+** %e day of month 1-31
** %f ** fractional seconds SS.SSS
+** %F ISO date. YYYY-MM-DD
+** %G ISO year corresponding to %V 0000-9999.
+** %g 2-digit ISO year corresponding to %V 00-99
** %H hour 00-24
-** %j day of year 000-366
+** %k hour 0-24 (leading zero converted to space)
+** %I hour 01-12
+** %j day of year 001-366
** %J ** julian day number
+** %l hour 1-12 (leading zero converted to space)
** %m month 01-12
** %M minute 00-59
+** %p "am" or "pm"
+** %P "AM" or "PM"
+** %R time as HH:MM
** %s seconds since 1970-01-01
** %S seconds 00-59
-** %w day of week 0-6 Sunday==0
-** %W week of year 00-53
+** %T time as HH:MM:SS
+** %u day of week 1-7 Monday==1, Sunday==7
+** %w day of week 0-6 Sunday==0, Monday==1
+** %U week of year 00-53 (First Sunday is start of week 01)
+** %V week of year 01-53 (First week containing Thursday is week 01)
+** %W week of year 00-53 (First Monday is start of week 01)
** %Y year 0000-9999
** %% %
*/
@@ -1288,7 +1427,7 @@ static void strftimeFunc(
sqlite3_str_appendf(&sRes, cf=='d' ? "%02d" : "%2d", x.D);
break;
}
- case 'f': {
+ case 'f': { /* Fractional seconds. (Non-standard) */
double s = x.s;
if( s>59.999 ) s = 59.999;
sqlite3_str_appendf(&sRes, "%06.3f", s);
@@ -1298,6 +1437,21 @@ static void strftimeFunc(
sqlite3_str_appendf(&sRes, "%04d-%02d-%02d", x.Y, x.M, x.D);
break;
}
+ case 'G': /* Fall thru */
+ case 'g': {
+ DateTime y = x;
+ assert( y.validJD );
+ /* Move y so that it is the Thursday in the same week as x */
+ y.iJD += (3 - daysAfterMonday(&x))*86400000;
+ y.validYMD = 0;
+ computeYMD(&y);
+ if( cf=='g' ){
+ sqlite3_str_appendf(&sRes, "%02d", y.Y%100);
+ }else{
+ sqlite3_str_appendf(&sRes, "%04d", y.Y);
+ }
+ break;
+ }
case 'H':
case 'k': {
sqlite3_str_appendf(&sRes, cf=='H' ? "%02d" : "%2d", x.h);
@@ -1311,25 +1465,11 @@ static void strftimeFunc(
sqlite3_str_appendf(&sRes, cf=='I' ? "%02d" : "%2d", h);
break;
}
- case 'W': /* Fall thru */
- case 'j': {
- int nDay; /* Number of days since 1st day of year */
- DateTime y = x;
- y.validJD = 0;
- y.M = 1;
- y.D = 1;
- computeJD(&y);
- nDay = (int)((x.iJD-y.iJD+43200000)/86400000);
- if( cf=='W' ){
- int wd; /* 0=Monday, 1=Tuesday, ... 6=Sunday */
- wd = (int)(((x.iJD+43200000)/86400000)%7);
- sqlite3_str_appendf(&sRes,"%02d",(nDay+7-wd)/7);
- }else{
- sqlite3_str_appendf(&sRes,"%03d",nDay+1);
- }
+ case 'j': { /* Day of year. Jan01==1, Jan02==2, and so forth */
+ sqlite3_str_appendf(&sRes,"%03d",daysAfterJan01(&x)+1);
break;
}
- case 'J': {
+ case 'J': { /* Julian day number. (Non-standard) */
sqlite3_str_appendf(&sRes,"%.16g",x.iJD/86400000.0);
break;
}
@@ -1372,13 +1512,33 @@ static void strftimeFunc(
sqlite3_str_appendf(&sRes,"%02d:%02d:%02d", x.h, x.m, (int)x.s);
break;
}
- case 'u': /* Fall thru */
- case 'w': {
- char c = (char)(((x.iJD+129600000)/86400000) % 7) + '0';
+ case 'u': /* Day of week. 1 to 7. Monday==1, Sunday==7 */
+ case 'w': { /* Day of week. 0 to 6. Sunday==0, Monday==1 */
+ char c = (char)daysAfterSunday(&x) + '0';
if( c=='0' && cf=='u' ) c = '7';
sqlite3_str_appendchar(&sRes, 1, c);
break;
}
+ case 'U': { /* Week num. 00-53. First Sun of the year is week 01 */
+ sqlite3_str_appendf(&sRes,"%02d",
+ (daysAfterJan01(&x)-daysAfterSunday(&x)+7)/7);
+ break;
+ }
+ case 'V': { /* Week num. 01-53. First week with a Thur is week 01 */
+ DateTime y = x;
+ /* Adjust y so that is the Thursday in the same week as x */
+ assert( y.validJD );
+ y.iJD += (3 - daysAfterMonday(&x))*86400000;
+ y.validYMD = 0;
+ computeYMD(&y);
+ sqlite3_str_appendf(&sRes,"%02d", daysAfterJan01(&y)/7+1);
+ break;
+ }
+ case 'W': { /* Week num. 00-53. First Mon of the year is week 01 */
+ sqlite3_str_appendf(&sRes,"%02d",
+ (daysAfterJan01(&x)-daysAfterMonday(&x)+7)/7);
+ break;
+ }
case 'Y': {
sqlite3_str_appendf(&sRes,"%04d",x.Y);
break;
@@ -1525,9 +1685,7 @@ static void timediffFunc(
d1.iJD = d2.iJD - d1.iJD;
d1.iJD += (u64)1486995408 * (u64)100000;
}
- d1.validYMD = 0;
- d1.validHMS = 0;
- d1.validTZ = 0;
+ clearYMD_HMS_TZ(&d1);
computeYMD_HMS(&d1);
sqlite3StrAccumInit(&sRes, 0, 0, 0, 100);
sqlite3_str_appendf(&sRes, "%c%04d-%02d-%02d %02d:%02d:%06.3f",
@@ -1596,6 +1754,36 @@ static void currentTimeFunc(
}
#endif
+#if !defined(SQLITE_OMIT_DATETIME_FUNCS) && defined(SQLITE_DEBUG)
+/*
+** datedebug(...)
+**
+** This routine returns JSON that describes the internal DateTime object.
+** Used for debugging and testing only. Subject to change.
+*/
+static void datedebugFunc(
+ sqlite3_context *context,
+ int argc,
+ sqlite3_value **argv
+){
+ DateTime x;
+ if( isDate(context, argc, argv, &x)==0 ){
+ char *zJson;
+ zJson = sqlite3_mprintf(
+ "{iJD:%lld,Y:%d,M:%d,D:%d,h:%d,m:%d,tz:%d,"
+ "s:%.3f,validJD:%d,validYMS:%d,validHMS:%d,"
+ "nFloor:%d,rawS:%d,isError:%d,useSubsec:%d,"
+ "isUtc:%d,isLocal:%d}",
+ x.iJD, x.Y, x.M, x.D, x.h, x.m, x.tz,
+ x.s, x.validJD, x.validYMD, x.validHMS,
+ x.nFloor, x.rawS, x.isError, x.useSubsec,
+ x.isUtc, x.isLocal);
+ sqlite3_result_text(context, zJson, -1, sqlite3_free);
+ }
+}
+#endif /* !SQLITE_OMIT_DATETIME_FUNCS && SQLITE_DEBUG */
+
+
/*
** This function registered all of the above C functions as SQL
** functions. This should be the only routine in this file with
@@ -1611,6 +1799,9 @@ void sqlite3RegisterDateTimeFunctions(void){
PURE_DATE(datetime, -1, 0, 0, datetimeFunc ),
PURE_DATE(strftime, -1, 0, 0, strftimeFunc ),
PURE_DATE(timediff, 2, 0, 0, timediffFunc ),
+#ifdef SQLITE_DEBUG
+ PURE_DATE(datedebug, -1, 0, 0, datedebugFunc ),
+#endif
DFUNCTION(current_time, 0, 0, 0, ctimeFunc ),
DFUNCTION(current_timestamp, 0, 0, 0, ctimestampFunc),
DFUNCTION(current_date, 0, 0, 0, cdateFunc ),

View File

@@ -3,7 +3,7 @@ set -euo pipefail
cd -P -- "$(dirname -- "$0")"
curl -#OL "https://sqlite.org/2023/sqlite-amalgamation-3440200.zip"
curl -#OL "https://sqlite.org/2024/sqlite-amalgamation-3450200.zip"
unzip -d . sqlite-amalgamation-*.zip
mv sqlite-amalgamation-*/sqlite3* .
rm -rf sqlite-amalgamation-*
@@ -12,24 +12,25 @@ cat *.patch | patch --no-backup-if-mismatch
mkdir -p ext/
cd ext/
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/ext/misc/decimal.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/ext/misc/uint.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/ext/misc/uuid.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/ext/misc/base64.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/ext/misc/regexp.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/ext/misc/series.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/ext/misc/anycollseq.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/anycollseq.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/base64.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/decimal.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/ieee754.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/regexp.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/series.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/uint.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/ext/misc/uuid.c"
cd ~-
cd ../vfs/tests/mptest/testdata/
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/mptest/mptest.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/mptest/config01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/mptest/config02.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/mptest/crash01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/mptest/crash02.subtest"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/mptest/multiwrite01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/mptest/mptest.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/mptest/config01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/mptest/config02.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/mptest/crash01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/mptest/crash02.subtest"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/mptest/multiwrite01.test"
cd ~-
cd ../vfs/tests/speedtest1/testdata/
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.44.2/test/speedtest1.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.45.3/test/speedtest1.c"
cd ~-

View File

@@ -1,16 +1,57 @@
#include <stdbool.h>
#include <stddef.h>
#include "include.h"
#include "sqlite3.h"
void go_func(sqlite3_context *, int, sqlite3_value **);
void go_step(sqlite3_context *, int, sqlite3_value **);
void go_final(sqlite3_context *);
void go_value(sqlite3_context *);
void go_inverse(sqlite3_context *, int, sqlite3_value **);
void go_collation_needed(void *, sqlite3 *, int, const char *);
int go_compare(go_handle, int, const void *, int, const void *);
void go_func(sqlite3_context *, go_handle, int, sqlite3_value **);
void go_step(sqlite3_context *, go_handle *, go_handle, int, sqlite3_value **);
void go_final(sqlite3_context *, go_handle, go_handle);
void go_value(sqlite3_context *, go_handle);
void go_inverse(sqlite3_context *, go_handle *, int, sqlite3_value **);
void go_func_wrapper(sqlite3_context *ctx, int nArg, sqlite3_value **pArg) {
go_func(ctx, sqlite3_user_data(ctx), nArg, pArg);
}
void go_step_wrapper(sqlite3_context *ctx, int nArg, sqlite3_value **pArg) {
go_handle *agg = sqlite3_aggregate_context(ctx, 4);
go_handle data = NULL;
if (agg == NULL || *agg == NULL) {
data = sqlite3_user_data(ctx);
}
go_step(ctx, agg, data, nArg, pArg);
}
void go_final_wrapper(sqlite3_context *ctx) {
go_handle *agg = sqlite3_aggregate_context(ctx, 0);
go_handle data = NULL;
if (agg == NULL || *agg == NULL) {
data = sqlite3_user_data(ctx);
}
go_final(ctx, agg, data);
}
void go_value_wrapper(sqlite3_context *ctx) {
go_handle *agg = sqlite3_aggregate_context(ctx, 4);
go_value(ctx, *agg);
}
void go_inverse_wrapper(sqlite3_context *ctx, int nArg, sqlite3_value **pArg) {
go_handle *agg = sqlite3_aggregate_context(ctx, 4);
go_inverse(ctx, *agg, nArg, pArg);
}
int sqlite3_collation_needed_go(sqlite3 *db, bool enable) {
return sqlite3_collation_needed(db, /*arg=*/NULL,
enable ? go_collation_needed : NULL);
}
int sqlite3_create_collation_go(sqlite3 *db, const char *name, go_handle app) {
int rc = sqlite3_create_collation_v2(db, name, SQLITE_UTF8, app, go_compare,
go_destroy);
@@ -21,22 +62,22 @@ int sqlite3_create_collation_go(sqlite3 *db, const char *name, go_handle app) {
int sqlite3_create_function_go(sqlite3 *db, const char *name, int argc,
int flags, go_handle app) {
return sqlite3_create_function_v2(db, name, argc, SQLITE_UTF8 | flags, app,
go_func, /*step=*/NULL, /*final=*/NULL,
go_destroy);
go_func_wrapper, /*step=*/NULL,
/*final=*/NULL, go_destroy);
}
int sqlite3_create_aggregate_function_go(sqlite3 *db, const char *name,
int argc, int flags, go_handle app) {
return sqlite3_create_window_function(db, name, argc, SQLITE_UTF8 | flags,
app, go_step, go_final, /*value=*/NULL,
/*inverse=*/NULL, go_destroy);
return sqlite3_create_function_v2(db, name, argc, SQLITE_UTF8 | flags, app,
/*func=*/NULL, go_step_wrapper,
go_final_wrapper, go_destroy);
}
int sqlite3_create_window_function_go(sqlite3 *db, const char *name, int argc,
int flags, go_handle app) {
return sqlite3_create_window_function(db, name, argc, SQLITE_UTF8 | flags,
app, go_step, go_final, go_value,
go_inverse, go_destroy);
return sqlite3_create_window_function(
db, name, argc, SQLITE_UTF8 | flags, app, go_step_wrapper,
go_final_wrapper, go_value_wrapper, go_inverse_wrapper, go_destroy);
}
void sqlite3_set_auxdata_go(sqlite3_context *ctx, int i, go_handle aux) {

58
sqlite3/hooks.c Normal file
View File

@@ -0,0 +1,58 @@
#include <stdbool.h>
#include "sqlite3.h"
int go_progress_handler(void *);
int go_busy_handler(void *, int);
int go_commit_hook(void *);
void go_rollback_hook(void *);
void go_update_hook(void *, int, char const *, char const *, sqlite3_int64);
int go_wal_hook(void *, sqlite3 *, const char *, int);
int go_authorizer(void *, int, const char *, const char *, const char *,
const char *);
void go_log(void *, int, const char *);
unsigned int go_autovacuum_pages(void *, const char *, unsigned int,
unsigned int, unsigned int);
void sqlite3_progress_handler_go(sqlite3 *db, int n) {
sqlite3_progress_handler(db, n, go_progress_handler, /*arg=*/db);
}
int sqlite3_busy_handler_go(sqlite3 *db, bool enable) {
return sqlite3_busy_handler(db, enable ? go_busy_handler : NULL, /*arg=*/db);
}
void sqlite3_commit_hook_go(sqlite3 *db, bool enable) {
sqlite3_commit_hook(db, enable ? go_commit_hook : NULL, /*arg=*/db);
}
void sqlite3_rollback_hook_go(sqlite3 *db, bool enable) {
sqlite3_rollback_hook(db, enable ? go_rollback_hook : NULL, /*arg=*/db);
}
void sqlite3_update_hook_go(sqlite3 *db, bool enable) {
sqlite3_update_hook(db, enable ? go_update_hook : NULL, /*arg=*/db);
}
void sqlite3_wal_hook_go(sqlite3 *db, bool enable) {
sqlite3_wal_hook(db, enable ? go_wal_hook : NULL, /*arg=*/NULL);
}
int sqlite3_set_authorizer_go(sqlite3 *db, bool enable) {
return sqlite3_set_authorizer(db, enable ? go_authorizer : NULL, /*arg=*/db);
}
int sqlite3_config_log_go(bool enable) {
return sqlite3_config(SQLITE_CONFIG_LOG, enable ? go_log : NULL,
/*arg=*/NULL);
}
int sqlite3_autovacuum_pages_go(sqlite3 *db, go_handle app) {
int rc = sqlite3_autovacuum_pages(db, go_autovacuum_pages, app, go_destroy);
if (rc) go_destroy(app);
return rc;
}

View File

@@ -5,7 +5,7 @@
// https://github.com/JuliaLang/julia/blob/v1.9.4/src/julia.h#L67-L68
#define container_of(ptr, type, member) \
((type *)((char *)(ptr)-offsetof(type, member)))
((type *)((char *)(ptr) - offsetof(type, member)))
typedef void *go_handle;
void go_destroy(go_handle);

View File

@@ -1,34 +0,0 @@
# ISO week date specifiers.
# https://sqlite.org/forum/forumpost/73d99e4497e8e6a7
--- sqlite3.c.orig
+++ sqlite3.c
@@ -1373,6 +1373,29 @@ static void strftimeFunc(
sqlite3_str_appendchar(&sRes, 1, c);
break;
}
+ case 'V': /* Fall thru */
+ case 'G': {
+ DateTime y = x;
+ computeJD(&y);
+ y.validYMD = 0;
+ /* Adjust date to Thursday this week:
+ The number in parentheses is 0 for Monday, 3 for Thursday */
+ y.iJD += (3 - (((y.iJD+43200000)/86400000) % 7))*86400000;
+ computeYMD(&y);
+ if( cf=='G' ){
+ sqlite3_str_appendf(&sRes,"%04d",y.Y);
+ }else{
+ int nDay; /* Number of days since 1st day of year */
+ i64 tJD = y.iJD;
+ y.validJD = 0;
+ y.M = 1;
+ y.D = 1;
+ computeJD(&y);
+ nDay = (int)((tJD-y.iJD+43200000)/86400000);
+ sqlite3_str_appendf(&sRes,"%02d",nDay/7+1);
+ }
+ break;
+ }
case 'Y': {
sqlite3_str_appendf(&sRes,"%04d",x.Y);
break;

View File

@@ -1,7 +1,7 @@
# Use exclusive locking mode for WAL databases with v1 VFSes.
--- sqlite3.c.orig
+++ sqlite3.c
@@ -63210,7 +63210,9 @@
@@ -64209,7 +64209,9 @@
SQLITE_PRIVATE int sqlite3PagerWalSupported(Pager *pPager){
const sqlite3_io_methods *pMethods = pPager->fd->pMethods;
if( pPager->noLock ) return 0;

View File

@@ -4,14 +4,16 @@
#include "ext/anycollseq.c"
#include "ext/base64.c"
#include "ext/decimal.c"
#include "ext/ieee754.c"
#include "ext/regexp.c"
#include "ext/series.c"
#include "ext/uint.c"
#include "ext/uuid.c"
// Bindings
#include "column.c"
#include "func.c"
#include "hooks.c"
#include "pointer.c"
#include "progress.c"
#include "time.c"
#include "vfs.c"
#include "vtab.c"
@@ -22,6 +24,7 @@ __attribute__((constructor)) void init() {
sqlite3_initialize();
sqlite3_auto_extension((void (*)(void))sqlite3_base_init);
sqlite3_auto_extension((void (*)(void))sqlite3_decimal_init);
sqlite3_auto_extension((void (*)(void))sqlite3_ieee_init);
sqlite3_auto_extension((void (*)(void))sqlite3_regexp_init);
sqlite3_auto_extension((void (*)(void))sqlite3_series_init);
sqlite3_auto_extension((void (*)(void))sqlite3_uint_init);

View File

@@ -1,9 +0,0 @@
#include <stddef.h>
#include "sqlite3.h"
int go_progress(void *);
void sqlite3_progress_handler_go(sqlite3 *db, int n) {
sqlite3_progress_handler(db, n, go_progress, /*arg=*/NULL);
}

View File

@@ -9,10 +9,12 @@
#define HAVE_INT16_T 1
#define HAVE_INT32_T 1
#define HAVE_INT64_T 1
#define HAVE_INTPTR_T 1
#define HAVE_UINT8_T 1
#define HAVE_UINT16_T 1
#define HAVE_UINT32_T 1
#define HAVE_UINT64_T 1
#define HAVE_UINTPTR_T 1
#define HAVE_STDINT_H 1
#define HAVE_INTTYPES_H 1
@@ -20,6 +22,8 @@
#define HAVE_LOG10 1
#define HAVE_ISNAN 1
#define HAVE_STRCHRNUL 1
#define HAVE_USLEEP 1
#define HAVE_NANOSLEEP 1
@@ -37,6 +41,7 @@
#define SQLITE_DEFAULT_WAL_SYNCHRONOUS 1
#define SQLITE_LIKE_DOESNT_MATCH_BLOBS
#define SQLITE_MAX_EXPR_DEPTH 0
#define SQLITE_STRICT_SUBTYPE 1
#define SQLITE_USE_ALLOCA
#define SQLITE_OMIT_DEPRECATED
#define SQLITE_OMIT_SHARED_CACHE
@@ -52,8 +57,8 @@
#define SQLITE_ENABLE_ATOMIC_WRITE
#define SQLITE_ENABLE_BATCH_ATOMIC_WRITE
// Because WASM does not support shared memory,
// SQLite disables WAL for WASM builds.
// Because Wasm does not support shared memory,
// SQLite disables WAL for Wasm builds.
// We patch SQLite to use exclusive locking mode instead.
// https://sqlite.org/wal.html#noshm
#undef SQLITE_OMIT_WAL
@@ -68,18 +73,12 @@
#define SQLITE_ENABLE_MATH_FUNCTIONS 1
#define SQLITE_ENABLE_JSON1 1
#define SQLITE_ENABLE_FTS3 1
#define SQLITE_ENABLE_FTS3_PARENTHESIS 1
#define SQLITE_ENABLE_FTS4 1
#define SQLITE_ENABLE_FTS5 1
#define SQLITE_ENABLE_RTREE 1
#define SQLITE_ENABLE_GEOPOLY 1
#define SQLITE_SOUNDEX
// Session Extension
// #define SQLITE_ENABLE_SESSION
// #define SQLITE_ENABLE_PREUPDATE_HOOK
#define SQLITE_UNTESTABLE
// Implemented in vfs.c.
int localtime_s(struct tm *const pTm, time_t const *const pTime);

Some files were not shown because too many files have changed in this diff Show More