Compare commits

...

25 Commits

Author SHA1 Message Date
Nuno Cruces
9142e19d61 SQLite 3.50.0. 2025-05-31 01:14:16 +01:00
Daenney
4a76f2b064 README: Make the testing section simpler to read (#282) 2025-05-30 14:29:34 +01:00
Nuno Cruces
c9b364507e Avoid copy, alloc. 2025-05-27 11:35:43 +01:00
Nuno Cruces
2204b96ff6 Gorm. 2025-05-27 10:51:42 +01:00
Nuno Cruces
b46f480d79 FCNTL_NULL_IO. 2025-05-26 16:32:52 +01:00
Nuno Cruces
040a026925 More examples. 2025-05-26 12:26:14 +01:00
Michael Lynch
e678040a4e Remove duplicate link to 'custom VFSes' in README (#279) 2025-05-26 11:49:03 +01:00
Michael Lynch
f1cc12569c Link to example of incremental BLOB I/O usage (#280) 2025-05-26 11:48:53 +01:00
Nuno Cruces
721a987e0e Line numbers. 2025-05-26 11:48:02 +01:00
Michael Lynch
f3d65142cc Use sql.Named to clarify the blobio example (#281)
I find the example SQL queries a bit difficult to read at the callsite with the magic numbers and ? placeholders. I think sql.Named makes it more obvious to the reader what the different parts of the SQL query represent.
2025-05-25 19:02:19 +01:00
Nuno Cruces
93f711c77b More fuzzing. 2025-05-21 11:36:56 +01:00
Nuno Cruces
341bd063e8 More fuzzing. 2025-05-21 08:16:52 +01:00
Nuno Cruces
f765882670 Sunday's Quick Search. 2025-05-17 13:20:11 +01:00
Nuno Cruces
ff3676ff4a Case insensitive search, fixes. 2025-05-16 13:10:03 +01:00
Nuno Cruces
54877a53cd Differential fuzzing. 2025-05-15 14:55:34 +01:00
Nuno Cruces
fccc6c10a7 Issue #277. 2025-05-14 13:46:01 +01:00
Nuno Cruces
fc21ffcc71 Tweaks. 2025-05-14 01:10:44 +01:00
Nuno Cruces
687e643d7a Case insensitive compare. 2025-05-13 18:12:14 +01:00
Nuno Cruces
fc5ced209c Fix bcmp. 2025-05-13 14:28:21 +01:00
Nuno Cruces
c1bed07e3a Issue #276. 2025-05-13 14:27:04 +01:00
Nuno Cruces
a0771f2363 Relaxed SIMD. 2025-05-12 17:30:32 +01:00
Nuno Cruces
6bad547d3d Unknown haystack length. 2025-05-12 12:00:06 +01:00
Nuno Cruces
c2c1aea578 Handle repetitive needles. 2025-05-10 01:44:25 +01:00
Nuno Cruces
60ab485b29 Comments. 2025-05-09 14:21:23 +01:00
Nuno Cruces
e17a432fde Adds strstr and memmem. (#275) 2025-05-09 00:59:39 +01:00
41 changed files with 3868 additions and 904 deletions

View File

@@ -112,7 +112,7 @@ jobs:
version: '10.1'
flags: '-test.v -test.short'
- name: openbsd
version: '7.6'
version: '7.7'
flags: '-test.v -test.short'
runs-on: ubuntu-latest
needs: test
@@ -128,7 +128,7 @@ jobs:
run: .github/workflows/build-test.sh
- name: Test
uses: cross-platform-actions/action@v0.27.0
uses: cross-platform-actions/action@v0.28.0
with:
operating_system: ${{ matrix.os.name }}
architecture: ${{ matrix.os.arch }}

View File

@@ -30,10 +30,10 @@ db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
- [`github.com/ncruces/go-sqlite3`](https://pkg.go.dev/github.com/ncruces/go-sqlite3)
wraps the [C SQLite API](https://sqlite.org/cintro.html)
([example usage](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-package)).
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-package)).
- [`github.com/ncruces/go-sqlite3/driver`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver)
provides a [`database/sql`](https://pkg.go.dev/database/sql) driver
([example usage](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-package)).
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-package)).
- [`github.com/ncruces/go-sqlite3/embed`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/embed)
embeds a build of SQLite into your application.
- [`github.com/ncruces/go-sqlite3/vfs`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs)
@@ -44,12 +44,19 @@ db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
### Advanced features
- [incremental BLOB I/O](https://sqlite.org/c3ref/blob_open.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/blobio#example-package))
- [nested transactions](https://sqlite.org/lang_savepoint.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-Savepoint))
- [custom functions](https://sqlite.org/c3ref/create_function.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-Conn.CreateFunction))
- [virtual tables](https://sqlite.org/vtab.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-CreateModule))
- [custom VFSes](https://sqlite.org/vfs.html)
([examples](vfs/README.md#custom-vfses))
- [online backup](https://sqlite.org/backup.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#Conn))
- [JSON support](https://sqlite.org/json1.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-package-Json))
- [math functions](https://sqlite.org/lang_mathfunc.html)
- [full-text search](https://sqlite.org/fts5.html)
- [geospatial search](https://sqlite.org/geopoly.html)
@@ -57,7 +64,6 @@ db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
- [statistics functions](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/stats)
- [encryption at rest](vfs/adiantum/README.md)
- [many extensions](ext/README.md)
- [custom VFSes](vfs/README.md#custom-vfses)
- [and more…](embed/README.md)
### Caveats
@@ -77,10 +83,19 @@ It also benefits greatly from [SQLite's](https://sqlite.org/testing.html) and
[wazero's](https://tetrate.io/blog/introducing-wazero-from-tetrate/#:~:text=Rock%2Dsolid%20test%20approach)
thorough testing.
Every commit is [tested](https://github.com/ncruces/go-sqlite3/wiki/Support-matrix) on
Linux (amd64/arm64/386/riscv64/ppc64le/s390x), macOS (arm64/amd64),
Windows (amd64), FreeBSD (amd64/arm64), OpenBSD (amd64), NetBSD (amd64/arm64),
DragonFly BSD (amd64), illumos (amd64), and Solaris (amd64).
Every commit is tested on:
* Linux: amd64, arm64, 386, riscv64, ppc64le, s390x
* macOS: amd64, arm64
* Windows: amd64
* BSD:
* FreeBSD: amd64, arm64
* OpenBSD: amd64
* NetBSD: amd64, arm64
* DragonFly BSD: amd64
* illumos: amd64
* Solaris: amd64
Certain operating system and CPU combinations have some limitations. See the [support matrix](https://github.com/ncruces/go-sqlite3/wiki/Support-matrix) for a complete overview.
The Go VFS is tested by running SQLite's
[mptest](https://github.com/sqlite/sqlite/blob/master/mptest/mptest.c).
@@ -118,4 +133,4 @@ and features we're working on, planning to work on, or asking for help with.
- [`modernc.org/sqlite`](https://pkg.go.dev/modernc.org/sqlite)
- [`crawshaw.io/sqlite`](https://pkg.go.dev/crawshaw.io/sqlite)
- [`github.com/mattn/go-sqlite3`](https://pkg.go.dev/github.com/mattn/go-sqlite3)
- [`github.com/zombiezen/go-sqlite`](https://pkg.go.dev/github.com/zombiezen/go-sqlite)
- [`github.com/zombiezen/go-sqlite`](https://pkg.go.dev/github.com/zombiezen/go-sqlite)

View File

@@ -109,7 +109,7 @@ func (c *Conn) FileControl(schema string, op FcntlOpcode, arg ...any) (any, erro
default:
return nil, MISUSE
case FCNTL_RESET_CACHE:
case FCNTL_RESET_CACHE, FCNTL_NULL_IO:
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), 0))

View File

@@ -280,6 +280,7 @@ const (
FCNTL_DATA_VERSION FcntlOpcode = 35
FCNTL_RESERVE_BYTES FcntlOpcode = 38
FCNTL_RESET_CACHE FcntlOpcode = 42
FCNTL_NULL_IO FcntlOpcode = 43
)
// LimitCategory are the available run-time limit categories.

View File

@@ -177,12 +177,15 @@ func (ctx Context) ResultPointer(ptr any) {
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultJSON(value any) {
data, err := json.Marshal(value)
err := json.NewEncoder(callbackWriter(func(p []byte) (int, error) {
ctx.ResultRawText(p[:len(p)-1]) // remove the newline
return 0, nil
})).Encode(value)
if err != nil {
ctx.ResultError(err)
return // notest
}
ctx.ResultRawText(data)
}
// ResultValue sets the result of the function to a copy of [Value].

View File

@@ -1,6 +1,6 @@
# Embeddable Wasm build of SQLite
This folder includes an embeddable Wasm build of SQLite 3.49.2 for use with
This folder includes an embeddable Wasm build of SQLite 3.50.0 for use with
[`github.com/ncruces/go-sqlite3`](https://pkg.go.dev/github.com/ncruces/go-sqlite3).
The following optional features are compiled in:

Binary file not shown.

View File

@@ -13,8 +13,8 @@ mkdir -p build/ext/
cp "$ROOT"/sqlite3/*.[ch] build/
cp "$ROOT"/sqlite3/*.patch build/
# https://sqlite.org/src/info/9d6517e7cc8bf175
curl -# https://sqlite.org/src/tarball/sqlite.tar.gz?r=9d6517e7 | tar xz
# https://sqlite.org/src/info/54b8888080d99a87
curl -# https://sqlite.org/src/tarball/sqlite.tar.gz?r=54b8888080 | tar xz
cd sqlite
if [[ "$OSTYPE" == "msys" || "$OSTYPE" == "cygwin" ]]; then
@@ -46,9 +46,9 @@ cd ~-
-o bcw2.wasm build/main.c \
-I"$ROOT/sqlite3/libc" -I"build" \
-mexec-model=reactor \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-mmutable-globals -mnontrapping-fptoint \
-msimd128 -mbulk-memory -msign-ext \
-mreference-types -mmultivalue \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
@@ -61,6 +61,6 @@ cd ~-
"$BINARYEN/wasm-ctor-eval" -g -c _initialize bcw2.wasm -o bcw2.tmp
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
bcw2.tmp -o bcw2.wasm --low-memory-unused \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
--enable-mutable-globals --enable-nontrapping-float-to-int \
--enable-simd --enable-bulk-memory --enable-sign-ext \
--enable-reference-types --enable-multivalue

View File

@@ -14,9 +14,9 @@ trap 'rm -f sqlite3.tmp' EXIT
-o sqlite3.wasm "$ROOT/sqlite3/main.c" \
-I"$ROOT/sqlite3/libc" -I"$ROOT/sqlite3" \
-mexec-model=reactor \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-mmutable-globals -mnontrapping-fptoint \
-msimd128 -mbulk-memory -msign-ext \
-mreference-types -mmultivalue \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
@@ -28,6 +28,6 @@ trap 'rm -f sqlite3.tmp' EXIT
"$BINARYEN/wasm-ctor-eval" -g -c _initialize sqlite3.wasm -o sqlite3.tmp
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
sqlite3.tmp -o sqlite3.wasm --low-memory-unused \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
--enable-mutable-globals --enable-nontrapping-float-to-int \
--enable-simd --enable-bulk-memory --enable-sign-ext \
--enable-reference-types --enable-multivalue

View File

@@ -19,7 +19,7 @@ func Test_init(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if version != "3.49.2" {
if version != "3.50.0" {
t.Error(version)
}
}

Binary file not shown.

View File

@@ -1,6 +1,7 @@
package blobio_test
import (
"database/sql"
"io"
"log"
"os"
@@ -34,7 +35,8 @@ func Example() {
const message = "Hello BLOB!"
// Create the BLOB.
r, err := db.Exec(`INSERT INTO test VALUES (?)`, sqlite3.ZeroBlob(len(message)))
r, err := db.Exec(`INSERT INTO test VALUES (:data)`,
sql.Named("data", sqlite3.ZeroBlob(len(message))))
if err != nil {
log.Fatal(err)
}
@@ -45,15 +47,19 @@ func Example() {
}
// Write the BLOB.
_, err = db.Exec(`SELECT writeblob('main', 'test', 'col', ?, 0, ?)`,
id, message)
_, err = db.Exec(`SELECT writeblob('main', 'test', 'col', :rowid, :offset, :message)`,
sql.Named("rowid", id),
sql.Named("offset", 0),
sql.Named("message", message))
if err != nil {
log.Fatal(err)
}
// Read the BLOB.
_, err = db.Exec(`SELECT readblob('main', 'test', 'col', ?, 0, ?)`,
id, sqlite3.Pointer(os.Stdout))
_, err = db.Exec(`SELECT readblob('main', 'test', 'col', :rowid, :offset, :writer)`,
sql.Named("rowid", id),
sql.Named("offset", 0),
sql.Named("writer", sqlite3.Pointer(os.Stdout)))
if err != nil {
log.Fatal(err)
}

View File

@@ -209,8 +209,12 @@ func (d *ddl) renameTable(dst, src string) error {
return nil
}
func compileConstraintRegexp(name string) *regexp.Regexp {
return regexp.MustCompile("^(?i:CONSTRAINT)\\s+[\"`]?" + regexp.QuoteMeta(name) + "[\"`\\s]")
}
func (d *ddl) addConstraint(name string, sql string) {
reg := regexp.MustCompile("^CONSTRAINT [\"`]?" + regexp.QuoteMeta(name) + "[\"` ]")
reg := compileConstraintRegexp(name)
for i := 0; i < len(d.fields); i++ {
if reg.MatchString(d.fields[i]) {
@@ -223,7 +227,7 @@ func (d *ddl) addConstraint(name string, sql string) {
}
func (d *ddl) removeConstraint(name string) bool {
reg := regexp.MustCompile("^CONSTRAINT [\"`]?" + regexp.QuoteMeta(name) + "[\"` ]")
reg := compileConstraintRegexp(name)
for i := 0; i < len(d.fields); i++ {
if reg.MatchString(d.fields[i]) {
@@ -236,7 +240,7 @@ func (d *ddl) removeConstraint(name string) bool {
//lint:ignore U1000 ignore unused code.
func (d *ddl) hasConstraint(name string) bool {
reg := regexp.MustCompile("^CONSTRAINT [\"`]?" + regexp.QuoteMeta(name) + "[\"` ]")
reg := compileConstraintRegexp(name)
for _, f := range d.fields {
if reg.MatchString(f) {

View File

@@ -95,7 +95,7 @@ func parseAllColumns(in string) ([]string, error) {
}
return nil, fmt.Errorf("unexpected token: %s", string(s[i]))
case parseAllColumnsState_State_End:
break
continue // avoid SA4011
}
}
if state != parseAllColumnsState_State_End {

View File

@@ -313,6 +313,41 @@ func TestRemoveConstraint(t *testing.T) {
success: true,
expect: []string{"`id` integer NOT NULL"},
},
{
name: "lowercase",
fields: []string{"`id` integer NOT NULL", "constraint `fk_users_notes` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`))"},
cName: "fk_users_notes",
success: true,
expect: []string{"`id` integer NOT NULL"},
},
{
name: "mixed_case",
fields: []string{"`id` integer NOT NULL", "cOnsTraiNT `fk_users_notes` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`))"},
cName: "fk_users_notes",
success: true,
expect: []string{"`id` integer NOT NULL"},
},
{
name: "newline",
fields: []string{"`id` integer NOT NULL", "CONSTRAINT `fk_users_notes`\nFOREIGN KEY (`user_id`) REFERENCES `users`(`id`))"},
cName: "fk_users_notes",
success: true,
expect: []string{"`id` integer NOT NULL"},
},
{
name: "lots_of_newlines",
fields: []string{"`id` integer NOT NULL", "constraint \n fk_users_notes \n FOREIGN KEY (`user_id`) REFERENCES `users`(`id`))"},
cName: "fk_users_notes",
success: true,
expect: []string{"`id` integer NOT NULL"},
},
{
name: "no_backtick",
fields: []string{"`id` integer NOT NULL", "CONSTRAINT fk_users_notes FOREIGN KEY (`user_id`) REFERENCES `users`(`id`))"},
cName: "fk_users_notes",
success: true,
expect: []string{"`id` integer NOT NULL"},
},
{
name: "check",
fields: []string{"CONSTRAINT `name_checker` CHECK (`name` <> 'thetadev')", "`id` integer NOT NULL"},

View File

@@ -11,5 +11,4 @@ curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.7/error_translator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.7/migrator.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.7/sqlite.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.7/sqlite_test.go"
curl -#OL "https://github.com/go-gorm/sqlite/raw/v1.5.7/sqlite_test.go"
curl -#L "https://github.com/glebarez/sqlite/raw/v1.11.0/sqlite_error_translator_test.go" > error_translator_test.go

View File

@@ -2,7 +2,7 @@
# handle, and interrupt, sqlite3_busy_timeout.
--- sqlite3.c.orig
+++ sqlite3.c
@@ -183355,7 +183355,7 @@
@@ -183364,7 +183364,7 @@
if( !sqlite3SafetyCheckOk(db) ) return SQLITE_MISUSE_BKPT;
#endif
if( ms>0 ){

View File

@@ -3,7 +3,7 @@ set -euo pipefail
cd -P -- "$(dirname -- "$0")"
curl -#OL "https://sqlite.org/2025/sqlite-amalgamation-3490200.zip"
curl -#OL "https://sqlite.org/2025/sqlite-amalgamation-3500000.zip"
unzip -d . sqlite-amalgamation-*.zip
mv sqlite-amalgamation-*/sqlite3.c .
mv sqlite-amalgamation-*/sqlite3.h .
@@ -19,30 +19,30 @@ rm -rf sqlite-amalgamation-*
mkdir -p ext/
cd ext/
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/anycollseq.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/base64.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/decimal.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/ieee754.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/regexp.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/series.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/spellfix.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/ext/misc/uint.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/anycollseq.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/base64.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/decimal.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/ieee754.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/regexp.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/series.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/spellfix.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/ext/misc/uint.c"
cd ~-
cd ../vfs/tests/mptest/testdata/
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/mptest/config01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/mptest/config02.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/mptest/crash01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/mptest/crash02.subtest"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/mptest/multiwrite01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/mptest/config01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/mptest/config02.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/mptest/crash01.test"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/mptest/crash02.subtest"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/mptest/multiwrite01.test"
cd ~-
cd ../vfs/tests/mptest/wasm/
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/mptest/mptest.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/mptest/mptest.c"
cd ~-
cd ../vfs/tests/speedtest1/wasm/
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.49.2/test/speedtest1.c"
curl -#OL "https://github.com/sqlite/sqlite/raw/version-3.50.0/test/speedtest1.c"
cd ~-
cat *.patch | patch -p0 --no-backup-if-mismatch

View File

@@ -11,19 +11,20 @@ SRCS="${1:-libc.c}"
trap 'rm -f libc.c libc.tmp' EXIT
cat << EOF > libc.c
#include <string.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
EOF
"$WASI_SDK/clang" --target=wasm32-wasi -std=c23 -g0 -O2 \
-Wall -Wextra -Wno-unused-parameter -Wno-unused-function \
-o libc.wasm -I. "$SRCS" \
-mexec-model=reactor \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-mmutable-globals -mnontrapping-fptoint \
-msimd128 -mbulk-memory -msign-ext \
-mreference-types -mmultivalue \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,-z,stack-size=1024 \
-Wl,-z,stack-size=4096 \
-Wl,--stack-first \
-Wl,--import-undefined \
-Wl,--initial-memory=16777216 \
@@ -31,29 +32,34 @@ EOF
-Wl,--export=memchr \
-Wl,--export=memcmp \
-Wl,--export=memcpy \
-Wl,--export=memmem \
-Wl,--export=memmove \
-Wl,--export=memrchr \
-Wl,--export=memset \
-Wl,--export=stpcpy \
-Wl,--export=stpncpy \
-Wl,--export=strcasecmp \
-Wl,--export=strcasestr \
-Wl,--export=strchr \
-Wl,--export=strchrnul \
-Wl,--export=strcmp \
-Wl,--export=strcpy \
-Wl,--export=strcspn \
-Wl,--export=strlen \
-Wl,--export=strncasecmp \
-Wl,--export=strncat \
-Wl,--export=strncmp \
-Wl,--export=strncpy \
-Wl,--export=strrchr \
-Wl,--export=strspn \
-Wl,--export=strstr \
-Wl,--export=qsort
"$BINARYEN/wasm-ctor-eval" -g -c _initialize libc.wasm -o libc.tmp
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
libc.tmp -o libc.wasm \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
--enable-mutable-globals --enable-nontrapping-float-to-int \
--enable-simd --enable-bulk-memory --enable-sign-ext \
--enable-reference-types --enable-multivalue
"$BINARYEN/wasm-dis" -o libc.wat libc.wasm

Binary file not shown.

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,10 +1,10 @@
#include_next <math.h> // the system math.h
#ifndef _WASM_SIMD128_MATH_H
#define _WASM_SIMD128_MATH_H
#include <wasm_simd128.h>
#include_next <math.h> // the system math.h
#ifdef __cplusplus
extern "C" {
#endif
@@ -23,8 +23,8 @@ double fma(double x, double y, double z) {
const v128_t wx = wasm_f64x2_replace_lane(b, 0, x);
const v128_t wy = wasm_f64x2_splat(y);
const v128_t wz = wasm_f64x2_splat(z);
const v128_t wr = wasm_f64x2_relaxed_madd(wx, wy, wz);
return wasm_f64x2_extract_lane(wr, 0);
const v128_t wr = wasm_f64x2_relaxed_madd(wx, wy, wz);
return wasm_f64x2_extract_lane(wr, 0);
}
#endif // __wasm_relaxed_simd__

View File

@@ -1,11 +1,8 @@
#include_next <stdlib.h> // the system stdlib.h
#ifndef _WASM_SIMD128_STDLIB_H
#define _WASM_SIMD128_STDLIB_H
#include <stddef.h>
#include <stdint.h>
#include_next <stdlib.h> // the system stdlib.h
#ifdef __cplusplus
extern "C" {
#endif

View File

@@ -1,14 +1,14 @@
#include_next <string.h> // the system string.h
#ifndef _WASM_SIMD128_STRING_H
#define _WASM_SIMD128_STRING_H
#include <limits.h>
#include <stddef.h>
#include <ctype.h>
#include <stdint.h>
#include <strings.h>
#include <wasm_simd128.h>
#include <__macro_PAGESIZE.h>
#include_next <string.h> // the system string.h
#ifdef __cplusplus
extern "C" {
#endif
@@ -38,11 +38,9 @@ void *memmove(void *dest, const void *src, size_t n) {
#ifdef __wasm_simd128__
// SIMD implementations of string.h functions.
__attribute__((weak))
int memcmp(const void *v1, const void *v2, size_t n) {
// Baseline algorithm.
// Scalar algorithm.
if (n < sizeof(v128_t)) {
const unsigned char *u1 = (unsigned char *)v1;
const unsigned char *u2 = (unsigned char *)v2;
@@ -87,13 +85,13 @@ void *memchr(const void *v, int c, size_t n) {
// When n is zero, a function that locates a character finds no occurrence.
// Otherwise, decrement n to ensure sub_overflow overflows
// when n would go equal-to-or-below zero.
if (n-- == 0) {
if (!n--) {
return NULL;
}
// memchr must behave as if it reads characters sequentially
// and stops as soon as a match is found.
// Aligning ensures loads beyond the first match don't fail.
// Aligning ensures loads beyond the first match are safe.
uintptr_t align = (uintptr_t)v % sizeof(v128_t);
const v128_t *w = (v128_t *)((char *)v - align);
const v128_t wc = wasm_i8x16_splat(c);
@@ -111,7 +109,8 @@ void *memchr(const void *v, int c, size_t n) {
// If the mask is zero because of alignment,
// it's as if we didn't find anything.
if (mask) {
// We found a match, unless it is beyond the end of the object.
// Find the offset of the first one bit (little-endian).
// That's a match, unless it is beyond the end of the object.
// Recall that we decremented n, so less-than-or-equal-to is correct.
size_t ctz = __builtin_ctz(mask);
return ctz <= n + align ? (char *)w + ctz : NULL;
@@ -136,12 +135,13 @@ void *memrchr(const void *v, int c, size_t n) {
const v128_t cmp = wasm_i8x16_eq(wasm_v128_load(--w), wc);
// Bitmask is slow on AArch64, any_true is much faster.
if (wasm_v128_any_true(cmp)) {
// Find the offset of the last one bit (little-endian).
size_t clz = __builtin_clz(wasm_i8x16_bitmask(cmp)) - 15;
return (char *)(w + 1) - clz;
}
}
// Baseline algorithm.
// Scalar algorithm.
const char *a = (char *)w;
while (n--) {
if (*(--a) == (char)c) return (char *)a;
@@ -152,7 +152,7 @@ void *memrchr(const void *v, int c, size_t n) {
__attribute__((weak))
size_t strlen(const char *s) {
// strlen must stop as soon as it finds the terminator.
// Aligning ensures loads beyond the terminator don't fail.
// Aligning ensures loads beyond the terminator are safe.
uintptr_t align = (uintptr_t)s % sizeof(v128_t);
const v128_t *w = (v128_t *)(s - align);
@@ -167,6 +167,7 @@ size_t strlen(const char *s) {
// Knowing this helps the compiler.
__builtin_assume(mask || align);
if (mask) {
// Find the offset of the first one bit (little-endian).
return (char *)w - s + __builtin_ctz(mask);
}
}
@@ -175,6 +176,19 @@ size_t strlen(const char *s) {
}
}
static int __strcmp_s(const char *s1, const char *s2) {
// Scalar algorithm.
const unsigned char *u1 = (unsigned char *)s1;
const unsigned char *u2 = (unsigned char *)s2;
for (;;) {
if (*u1 != *u2) return *u1 - *u2;
if (*u1 == 0) break;
u1++;
u2++;
}
return 0;
}
static int __strcmp(const char *s1, const char *s2) {
// How many bytes can be read before pointers go out of bounds.
size_t N = __builtin_wasm_memory_size(0) * PAGESIZE - //
@@ -187,11 +201,10 @@ static int __strcmp(const char *s1, const char *s2) {
for (; N >= sizeof(v128_t); N -= sizeof(v128_t)) {
// Find any single bit difference.
if (wasm_v128_any_true(wasm_v128_load(w1) ^ wasm_v128_load(w2))) {
// The strings may still be equal,
// if the terminator is found before that difference.
// The terminator may come before the difference.
break;
}
// All characters are equal.
// We know all characters are equal.
// If any is a terminator the strings are equal.
if (!wasm_i8x16_all_true(wasm_v128_load(w1))) {
return 0;
@@ -200,28 +213,7 @@ static int __strcmp(const char *s1, const char *s2) {
w2++;
}
// Baseline algorithm.
const unsigned char *u1 = (unsigned char *)w1;
const unsigned char *u2 = (unsigned char *)w2;
for (;;) {
if (*u1 != *u2) return *u1 - *u2;
if (*u1 == 0) break;
u1++;
u2++;
}
return 0;
}
static int __strcmp_s(const char *s1, const char *s2) {
const unsigned char *u1 = (unsigned char *)s1;
const unsigned char *u2 = (unsigned char *)s2;
for (;;) {
if (*u1 != *u2) return *u1 - *u2;
if (*u1 == 0) break;
u1++;
u2++;
}
return 0;
return __strcmp_s((char *)w1, (char *)w2);
}
__attribute__((weak, always_inline))
@@ -247,11 +239,10 @@ int strncmp(const char *s1, const char *s2, size_t n) {
for (; n >= sizeof(v128_t); n -= sizeof(v128_t)) {
// Find any single bit difference.
if (wasm_v128_any_true(wasm_v128_load(w1) ^ wasm_v128_load(w2))) {
// The strings may still be equal,
// if the terminator is found before that difference.
// The terminator may come before the difference.
break;
}
// All characters are equal.
// We know all characters are equal.
// If any is a terminator the strings are equal.
if (!wasm_i8x16_all_true(wasm_v128_load(w1))) {
return 0;
@@ -260,7 +251,7 @@ int strncmp(const char *s1, const char *s2, size_t n) {
w2++;
}
// Baseline algorithm.
// Scalar algorithm.
const unsigned char *u1 = (unsigned char *)w1;
const unsigned char *u2 = (unsigned char *)w2;
while (n--) {
@@ -274,7 +265,7 @@ int strncmp(const char *s1, const char *s2, size_t n) {
static char *__strchrnul(const char *s, int c) {
// strchrnul must stop as soon as a match is found.
// Aligning ensures loads beyond the first match don't fail.
// Aligning ensures loads beyond the first match are safe.
uintptr_t align = (uintptr_t)s % sizeof(v128_t);
const v128_t *w = (v128_t *)(s - align);
const v128_t wc = wasm_i8x16_splat(c);
@@ -290,6 +281,7 @@ static char *__strchrnul(const char *s, int c) {
// Knowing this helps the compiler.
__builtin_assume(mask || align);
if (mask) {
// Find the offset of the first one bit (little-endian).
return (char *)w + __builtin_ctz(mask);
}
}
@@ -329,50 +321,57 @@ char *strrchr(const char *s, int c) {
return (char *)memrchr(s, c, strlen(s) + 1);
}
// SIMDized check which bytes are in a set
// http://0x80.pl/notesen/2018-10-18-simd-byte-lookup.html
#define _WASM_SIMD128_BITMAP256_T \
struct { \
uint8_t l __attribute__((__vector_size__(16), __aligned__(16))); \
uint8_t h __attribute__((__vector_size__(16), __aligned__(16))); \
}
typedef struct {
__u8x16 l;
__u8x16 h;
} __wasm_v128_bitmap256_t;
#define _WASM_SIMD128_SETBIT(bitmap, i) \
({ \
uint8_t _c = (uint8_t)(i); \
uint8_t _hi_nibble = _c >> 4; \
uint8_t _lo_nibble = _c & 0xf; \
bitmap.l[_lo_nibble] |= 1 << (_hi_nibble - 0); \
bitmap.h[_lo_nibble] |= 1 << (_hi_nibble - 8); \
})
__attribute__((always_inline))
static void __wasm_v128_setbit(__wasm_v128_bitmap256_t *bitmap, int i) {
uint8_t hi_nibble = (uint8_t)i >> 4;
uint8_t lo_nibble = (uint8_t)i & 0xf;
bitmap->l[lo_nibble] |= 1 << (hi_nibble - 0);
bitmap->h[lo_nibble] |= 1 << (hi_nibble - 8);
}
#define _WASM_SIMD128_CHKBIT(bitmap, i) \
({ \
uint8_t _c = (uint8_t)(i); \
uint8_t _hi_nibble = _c >> 4; \
uint8_t _lo_nibble = _c & 0xf; \
uint8_t _bitmask = 1 << (_hi_nibble & 0x7); \
uint8_t _bitset = (_hi_nibble < 8 ? bitmap.l : bitmap.h)[_lo_nibble]; \
_bitmask & _bitset; \
})
__attribute__((always_inline))
static int __wasm_v128_chkbit(__wasm_v128_bitmap256_t bitmap, int i) {
uint8_t hi_nibble = (uint8_t)i >> 4;
uint8_t lo_nibble = (uint8_t)i & 0xf;
uint8_t bitmask = 1 << (hi_nibble & 0x7);
uint8_t bitset = (hi_nibble < 8 ? bitmap.l : bitmap.h)[lo_nibble];
return bitmask & bitset;
}
#define _WASM_SIMD128_CHKBITS(bitmap, v) \
({ \
v128_t _w = v; \
v128_t _hi_nibbles = wasm_u8x16_shr(_w, 4); \
v128_t _lo_nibbles = _w & wasm_u8x16_const_splat(0xf); \
\
v128_t _bitmask_lookup = wasm_u8x16_const(1, 2, 4, 8, 16, 32, 64, 128, \
1, 2, 4, 8, 16, 32, 64, 128); \
\
v128_t _bitmask = wasm_i8x16_swizzle(_bitmask_lookup, _hi_nibbles); \
v128_t _bitsets = wasm_v128_bitselect( \
wasm_i8x16_swizzle(bitmap.l, _lo_nibbles), \
wasm_i8x16_swizzle(bitmap.h, _lo_nibbles), \
wasm_i8x16_lt(_hi_nibbles, wasm_u8x16_const_splat(8))); \
\
wasm_i8x16_eq(_bitsets & _bitmask, _bitmask); \
})
#ifndef __wasm_relaxed_simd__
#define wasm_i8x16_relaxed_laneselect wasm_v128_bitselect
#define wasm_i8x16_relaxed_swizzle wasm_i8x16_swizzle
#endif // __wasm_relaxed_simd__
__attribute__((always_inline))
static v128_t __wasm_v128_chkbits(__wasm_v128_bitmap256_t bitmap, v128_t v) {
v128_t hi_nibbles = wasm_u8x16_shr(v, 4);
v128_t lo_nibbles = v & wasm_u8x16_const_splat(0xf);
v128_t bitmask_lookup = wasm_u8x16_const(1, 2, 4, 8, 16, 32, 64, 128, //
1, 2, 4, 8, 16, 32, 64, 128);
v128_t bitmask = wasm_i8x16_relaxed_swizzle(bitmask_lookup, hi_nibbles);
v128_t bitsets = wasm_i8x16_relaxed_laneselect(
wasm_i8x16_relaxed_swizzle(bitmap.l, lo_nibbles),
wasm_i8x16_relaxed_swizzle(bitmap.h, lo_nibbles),
wasm_i8x16_lt(hi_nibbles, wasm_u8x16_const_splat(8)));
return wasm_i8x16_eq(bitsets & bitmask, bitmask);
}
#undef wasm_i8x16_relaxed_laneselect
#undef wasm_i8x16_relaxed_swizzle
__attribute__((weak))
size_t strspn(const char *s, const char *c) {
@@ -388,36 +387,38 @@ size_t strspn(const char *s, const char *c) {
const v128_t cmp = wasm_i8x16_eq(wasm_v128_load(w), wc);
// Bitmask is slow on AArch64, all_true is much faster.
if (!wasm_i8x16_all_true(cmp)) {
// Find the offset of the first zero bit (little-endian).
size_t ctz = __builtin_ctz(~wasm_i8x16_bitmask(cmp));
return (char *)w + ctz - s;
}
w++;
}
// Baseline algorithm.
// Scalar algorithm.
for (s = (char *)w; *s == *c; s++);
return s - a;
}
_WASM_SIMD128_BITMAP256_T bitmap = {};
__wasm_v128_bitmap256_t bitmap = {};
for (; *c; c++) {
_WASM_SIMD128_SETBIT(bitmap, *c);
__wasm_v128_setbit(&bitmap, *c);
// Terminator IS NOT on the bitmap.
}
for (; N >= sizeof(v128_t); N -= sizeof(v128_t)) {
const v128_t cmp = _WASM_SIMD128_CHKBITS(bitmap, wasm_v128_load(w));
const v128_t cmp = __wasm_v128_chkbits(bitmap, wasm_v128_load(w));
// Bitmask is slow on AArch64, all_true is much faster.
if (!wasm_i8x16_all_true(cmp)) {
// Find the offset of the first zero bit (little-endian).
size_t ctz = __builtin_ctz(~wasm_i8x16_bitmask(cmp));
return (char *)w + ctz - s;
}
w++;
}
// Baseline algorithm.
for (s = (char *)w; _WASM_SIMD128_CHKBIT(bitmap, *s); s++);
// Scalar algorithm.
for (s = (char *)w; __wasm_v128_chkbit(bitmap, *s); s++);
return s - a;
}
@@ -430,33 +431,234 @@ size_t strcspn(const char *s, const char *c) {
const v128_t *w = (v128_t *)s;
const char *const a = s;
_WASM_SIMD128_BITMAP256_T bitmap = {};
__wasm_v128_bitmap256_t bitmap = {};
for (;;) {
_WASM_SIMD128_SETBIT(bitmap, *c);
__wasm_v128_setbit(&bitmap, *c);
// Terminator IS on the bitmap.
if (!*c++) break;
}
for (; N >= sizeof(v128_t); N -= sizeof(v128_t)) {
const v128_t cmp = _WASM_SIMD128_CHKBITS(bitmap, wasm_v128_load(w));
const v128_t cmp = __wasm_v128_chkbits(bitmap, wasm_v128_load(w));
// Bitmask is slow on AArch64, any_true is much faster.
if (wasm_v128_any_true(cmp)) {
// Find the offset of the first one bit (little-endian).
size_t ctz = __builtin_ctz(wasm_i8x16_bitmask(cmp));
return (char *)w + ctz - s;
}
w++;
}
// Baseline algorithm.
for (s = (char *)w; !_WASM_SIMD128_CHKBIT(bitmap, *s); s++);
// Scalar algorithm.
for (s = (char *)w; !__wasm_v128_chkbit(bitmap, *s); s++);
return s - a;
}
#undef _WASM_SIMD128_SETBIT
#undef _WASM_SIMD128_CHKBIT
#undef _WASM_SIMD128_CHKBITS
#undef _WASM_SIMD128_BITMAP256_T
// SIMD-friendly algorithms for substring searching
// http://0x80.pl/notesen/2016-11-28-simd-strfind.html
// For haystacks of known length and large enough needles,
// Boyer-Moore's bad-character rule may be useful,
// as proposed by Horspool, Sunday and Raita.
//
// We augment the SIMD algorithm with Quick Search's
// bad-character shift.
//
// https://www-igm.univ-mlv.fr/~lecroq/string/node14.html
// https://www-igm.univ-mlv.fr/~lecroq/string/node18.html
// https://www-igm.univ-mlv.fr/~lecroq/string/node19.html
// https://www-igm.univ-mlv.fr/~lecroq/string/node22.html
static const char *__memmem(const char *haystk, size_t sh,
const char *needle, size_t sn,
uint8_t bmbc[256]) {
// We've handled empty and single character needles.
// The needle is not longer than the haystack.
__builtin_assume(2 <= sn && sn <= sh);
// Find the farthest character not equal to the first one.
size_t i = sn - 1;
while (i > 0 && needle[0] == needle[i]) i--;
if (i == 0) i = sn - 1;
// Subtracting ensures sub_overflow overflows
// when we reach the end of the haystack.
if (sh != SIZE_MAX) sh -= sn;
const v128_t fst = wasm_i8x16_splat(needle[0]);
const v128_t lst = wasm_i8x16_splat(needle[i]);
// The last haystack offset for which loading blk_lst is safe.
const char *H = (char *)(__builtin_wasm_memory_size(0) * PAGESIZE - i -
sizeof(v128_t));
while (haystk <= H) {
const v128_t blk_fst = wasm_v128_load((v128_t *)(haystk));
const v128_t blk_lst = wasm_v128_load((v128_t *)(haystk + i));
const v128_t eq_fst = wasm_i8x16_eq(fst, blk_fst);
const v128_t eq_lst = wasm_i8x16_eq(lst, blk_lst);
const v128_t cmp = eq_fst & eq_lst;
if (wasm_v128_any_true(cmp)) {
// The terminator may come before the match.
if (sh == SIZE_MAX && !wasm_i8x16_all_true(blk_fst)) break;
// Find the offset of the first one bit (little-endian).
// Each iteration clears that bit, tries again.
for (uint32_t mask = wasm_i8x16_bitmask(cmp); mask; mask &= mask - 1) {
size_t ctz = __builtin_ctz(mask);
// The match may be after the end of the haystack.
if (ctz > sh) return NULL;
// We know the first character matches.
if (!bcmp(haystk + ctz + 1, needle + 1, sn - 1)) {
return haystk + ctz;
}
}
}
size_t skip = sizeof(v128_t);
if (sh == SIZE_MAX) {
// Have we reached the end of the haystack?
if (!wasm_i8x16_all_true(blk_fst)) return NULL;
} else {
// Apply the bad-character rule to the character to the right
// of the righmost character of the search window.
if (bmbc) skip += bmbc[(unsigned char)haystk[sn - 1 + sizeof(v128_t)]];
// Have we reached the end of the haystack?
if (__builtin_sub_overflow(sh, skip, &sh)) return NULL;
}
haystk += skip;
}
// Scalar algorithm.
for (size_t j = 0; j <= sh; j++) {
for (size_t i = 0;; i++) {
if (sn == i) return haystk;
if (sh == SIZE_MAX && !haystk[i]) return NULL;
if (needle[i] != haystk[i]) break;
}
haystk++;
}
return NULL;
}
__attribute__((weak))
void *memmem(const void *vh, size_t sh, const void *vn, size_t sn) {
// Return immediately on empty needle.
if (sn == 0) return (void *)vh;
// Return immediately when needle is longer than haystack.
if (sn > sh) return NULL;
// Skip to the first matching character using memchr,
// thereby handling single character needles.
const char *needle = (char *)vn;
const char *haystk = (char *)memchr(vh, *needle, sh);
if (!haystk || sn == 1) return (void *)haystk;
// The haystack got shorter, is the needle now longer than it?
sh -= haystk - (char *)vh;
if (sn > sh) return NULL;
// Is Boyer-Moore's bad-character rule useful?
if (sn < sizeof(v128_t) || sh - sn < sizeof(v128_t)) {
return (void *)__memmem(haystk, sh, needle, sn, NULL);
}
// Compute Boyer-Moore's bad-character shift function.
// Only the last 255 characters of the needle matter for shifts up to 255,
// which is good enough for most needles.
size_t c = sn;
size_t i = 0;
if (c >= 255) {
i = sn - 255;
c = 255;
}
#ifndef _REENTRANT
static
#endif
uint8_t bmbc[256];
memset(bmbc, c, sizeof(bmbc));
for (; i < sn; i++) {
// One less than the usual offset because
// we advance at least one vector at a time.
bmbc[(unsigned char)needle[i]] = sn - i - 1;
}
return (void *)__memmem(haystk, sh, needle, sn, bmbc);
}
__attribute__((weak))
char *strstr(const char *haystk, const char *needle) {
// Return immediately on empty needle.
if (!needle[0]) return (char *)haystk;
// Skip to the first matching character using strchr,
// thereby handling single character needles.
haystk = strchr(haystk, *needle);
if (!haystk || !needle[1]) return (char *)haystk;
return (char *)__memmem(haystk, SIZE_MAX, needle, strlen(needle), NULL);
}
__attribute__((weak))
char *strcasestr(const char *haystk, const char *needle) {
// Return immediately on empty needle.
if (!needle[0]) return (char *)haystk;
// We've handled empty needles.
size_t sn = strlen(needle);
__builtin_assume(sn >= 1);
// Find the farthest character not equal to the first one.
size_t i = sn - 1;
while (i > 0 && needle[0] == needle[i]) i--;
if (i == 0) i = sn - 1;
const v128_t fst = wasm_i8x16_splat(tolower(needle[0]));
const v128_t lst = wasm_i8x16_splat(tolower(needle[i]));
// The last haystk offset for which loading blk_lst is safe.
const char *H =
(char *)(__builtin_wasm_memory_size(0) * PAGESIZE - i - sizeof(v128_t));
while (haystk <= H) {
const v128_t blk_fst = __tolower8x16(wasm_v128_load((v128_t *)(haystk)));
const v128_t blk_lst = __tolower8x16(wasm_v128_load((v128_t *)(haystk + i)));
const v128_t eq_fst = wasm_i8x16_eq(fst, blk_fst);
const v128_t eq_lst = wasm_i8x16_eq(lst, blk_lst);
const v128_t cmp = eq_fst & eq_lst;
if (wasm_v128_any_true(cmp)) {
// The terminator may come before the match.
if (!wasm_i8x16_all_true(blk_fst)) break;
// Find the offset of the first one bit (little-endian).
// Each iteration clears that bit, tries again.
for (uint32_t mask = wasm_i8x16_bitmask(cmp); mask; mask &= mask - 1) {
size_t ctz = __builtin_ctz(mask);
if (!strncasecmp(haystk + ctz + 1, needle + 1, sn - 1)) {
return (char *)haystk + ctz;
}
}
}
// Have we reached the end of the haystack?
if (!wasm_i8x16_all_true(blk_fst)) return NULL;
haystk += sizeof(v128_t);
}
// Scalar algorithm.
for (;;) {
for (size_t i = 0;; i++) {
if (sn == i) return (char *)haystk;
if (!haystk[i]) return NULL;
if (tolower(needle[i]) != tolower(haystk[i])) break;
}
haystk++;
}
return NULL;
}
// Given the above SIMD implementations,
// these are best implemented as
@@ -465,6 +667,7 @@ size_t strcspn(const char *s, const char *c) {
// Simple wrappers already in musl:
// - mempcpy
// - strcat
// - strlcat
// - strdup
// - strndup
// - strnlen
@@ -474,7 +677,6 @@ size_t strcspn(const char *s, const char *c) {
__attribute__((weak))
void *memccpy(void *__restrict dest, const void *__restrict src, int c, size_t n) {
void *memchr(const void *v, int c, size_t n);
const void *m = memchr(src, c, n);
if (m != NULL) {
n = (char *)m - (char *)src + 1;
@@ -484,6 +686,17 @@ void *memccpy(void *__restrict dest, const void *__restrict src, int c, size_t n
return (void *)m;
}
__attribute__((weak))
size_t strlcpy(char *__restrict dest, const char *__restrict src, size_t n) {
size_t slen = strlen(src);
if (n--) {
if (n > slen) n = slen;
memcpy(dest, src, n);
dest[n] = 0;
}
return slen;
}
__attribute__((weak))
char *strncat(char *__restrict dest, const char *__restrict src, size_t n) {
size_t strnlen(const char *s, size_t n);
@@ -513,6 +726,7 @@ char *stpcpy(char *__restrict dest, const char *__restrict src) {
return __stpcpy(dest, src);
}
__attribute__((weak, always_inline))
char *strcpy(char *__restrict dest, const char *__restrict src) {
__stpcpy(dest, src);
return dest;

View File

@@ -1,10 +1,12 @@
#include_next <strings.h> // the system strings.h
#ifndef _WASM_SIMD128_STRINGS_H
#define _WASM_SIMD128_STRINGS_H
#include <stddef.h>
#include <ctype.h>
#include <stdint.h>
#include <wasm_simd128.h>
#include_next <strings.h> // the system strings.h
#include <__macro_PAGESIZE.h>
#ifdef __cplusplus
extern "C" {
@@ -12,11 +14,16 @@ extern "C" {
#ifdef __wasm_simd128__
#ifdef __OPTIMIZE_SIZE__
// bcmp is the same as memcmp but only compares for equality.
int bcmp(const void *v1, const void *v2, size_t n);
#else // __OPTIMIZE_SIZE__
__attribute__((weak))
int bcmp(const void *v1, const void *v2, size_t n) {
// bcmp is the same as memcmp but only compares for equality.
// Baseline algorithm.
// Scalar algorithm.
if (n < sizeof(v128_t)) {
const unsigned char *u1 = (unsigned char *)v1;
const unsigned char *u2 = (unsigned char *)v2;
@@ -48,6 +55,113 @@ int bcmp(const void *v1, const void *v2, size_t n) {
return 0;
}
#endif // __OPTIMIZE_SIZE__
static v128_t __tolower8x16(v128_t v) {
__i8x16 i = v;
i = i + wasm_i8x16_splat(INT8_MAX - ('Z'));
i = i > wasm_i8x16_splat(INT8_MAX - ('Z' - 'A' + 1));
i = i & wasm_i8x16_splat('a' - 'A');
return v | i;
}
static int __strcasecmp_s(const char *s1, const char *s2) {
// Scalar algorithm.
const unsigned char *u1 = (unsigned char *)s1;
const unsigned char *u2 = (unsigned char *)s2;
for (;;) {
int c1 = tolower(*u1);
int c2 = tolower(*u2);
if (c1 != c2) return c1 - c2;
if (c1 == 0) break;
u1++;
u2++;
}
return 0;
}
static int __strcasecmp(const char *s1, const char *s2) {
// How many bytes can be read before pointers go out of bounds.
size_t N = __builtin_wasm_memory_size(0) * PAGESIZE - //
(size_t)(s1 > s2 ? s1 : s2);
// Unaligned loads handle the case where the strings
// have mismatching alignments.
const v128_t *w1 = (v128_t *)s1;
const v128_t *w2 = (v128_t *)s2;
for (; N >= sizeof(v128_t); N -= sizeof(v128_t)) {
v128_t v1 = __tolower8x16(wasm_v128_load(w1));
v128_t v2 = __tolower8x16(wasm_v128_load(w2));
// Find any single bit difference.
if (wasm_v128_any_true(v1 ^ v2)) {
// The terminator may come before the difference.
break;
}
// We know all characters are equal.
// If any is a terminator the strings are equal.
if (!wasm_i8x16_all_true(v1)) {
return 0;
}
w1++;
w2++;
}
return __strcasecmp_s((char *)w1, (char *)w2);
}
__attribute__((weak))
int strcasecmp(const char *s1, const char *s2) {
// Skip the vector search when comparing against small literal strings.
if (__builtin_constant_p(strlen(s2)) && strlen(s2) < sizeof(v128_t)) {
return __strcasecmp_s(s1, s2);
}
return __strcasecmp(s1, s2);
}
__attribute__((weak))
int strncasecmp(const char *s1, const char *s2, size_t n) {
// How many bytes can be read before pointers go out of bounds.
size_t N = __builtin_wasm_memory_size(0) * PAGESIZE - //
(size_t)(s1 > s2 ? s1 : s2);
if (n > N) n = N;
// Unaligned loads handle the case where the strings
// have mismatching alignments.
const v128_t *w1 = (v128_t *)s1;
const v128_t *w2 = (v128_t *)s2;
for (; n >= sizeof(v128_t); n -= sizeof(v128_t)) {
v128_t v1 = __tolower8x16(wasm_v128_load(w1));
v128_t v2 = __tolower8x16(wasm_v128_load(w2));
// Find any single bit difference.
if (wasm_v128_any_true(v1 ^ v2)) {
// The terminator may come before the difference.
break;
}
// We know all characters are equal.
// If any is a terminator the strings are equal.
if (!wasm_i8x16_all_true(v1)) {
return 0;
}
w1++;
w2++;
}
// Scalar algorithm.
const unsigned char *u1 = (unsigned char *)w1;
const unsigned char *u2 = (unsigned char *)w2;
while (n--) {
int c1 = tolower(*u1);
int c2 = tolower(*u2);
if (c1 != c2) return c1 - c2;
if (c1 == 0) break;
u1++;
u2++;
}
return 0;
}
#endif // __wasm_simd128__
#ifdef __cplusplus

41
sqlite3/strcasecmp.patch Normal file
View File

@@ -0,0 +1,41 @@
# Use strcasecmp and strncasecmp.
--- sqlite3.c.orig
+++ sqlite3.c
@@ -35685,35 +35685,15 @@
return sqlite3StrICmp(zLeft, zRight);
}
SQLITE_PRIVATE int sqlite3StrICmp(const char *zLeft, const char *zRight){
- unsigned char *a, *b;
- int c, x;
- a = (unsigned char *)zLeft;
- b = (unsigned char *)zRight;
- for(;;){
- c = *a;
- x = *b;
- if( c==x ){
- if( c==0 ) break;
- }else{
- c = (int)UpperToLower[c] - (int)UpperToLower[x];
- if( c ) break;
- }
- a++;
- b++;
- }
- return c;
+ return strcasecmp(zLeft, zRight);
}
SQLITE_API int sqlite3_strnicmp(const char *zLeft, const char *zRight, int N){
- register unsigned char *a, *b;
if( zLeft==0 ){
return zRight ? -1 : 0;
}else if( zRight==0 ){
return 1;
}
- a = (unsigned char *)zLeft;
- b = (unsigned char *)zRight;
- while( N-- > 0 && *a!=0 && UpperToLower[*a]==UpperToLower[*b]){ a++; b++; }
- return N<0 ? 0 : UpperToLower[*a] - UpperToLower[*b];
+ return strncasecmp(zLeft, zRight, N);
}
/*

View File

@@ -1,7 +1,7 @@
# Remove VFS registration. Go handles it.
--- sqlite3.c.orig
+++ sqlite3.c
@@ -26725,7 +26725,7 @@
@@ -26726,7 +26726,7 @@
sqlite3_free(p);
return sqlite3_os_init();
}
@@ -10,7 +10,7 @@
/*
** The list of all registered VFS implementations.
*/
@@ -26822,7 +26822,7 @@
@@ -26823,7 +26823,7 @@
sqlite3_mutex_leave(mutex);
return SQLITE_OK;
}

12
stmt.go
View File

@@ -367,11 +367,9 @@ func (s *Stmt) BindPointer(param int, ptr any) error {
//
// https://sqlite.org/c3ref/bind_blob.html
func (s *Stmt) BindJSON(param int, value any) error {
data, err := json.Marshal(value)
if err != nil {
return err
}
return s.BindRawText(param, data)
return json.NewEncoder(callbackWriter(func(p []byte) (int, error) {
return 0, s.BindRawText(param, p[:len(p)-1]) // remove the newline
})).Encode(value)
}
// BindValue binds a copy of value to the prepared statement.
@@ -751,3 +749,7 @@ func (s *Stmt) columns(count int64) ([]byte, ptr_t, error) {
return util.View(s.c.mod, typePtr, count), dataPtr, nil
}
type callbackWriter func(p []byte) (int, error)
func (fn callbackWriter) Write(p []byte) (int, error) { return fn(p) }

View File

@@ -114,6 +114,7 @@ func TestConn_FileControl(t *testing.T) {
t.Errorf("got %v, want MISUSE", err)
}
})
t.Run("FCNTL_RESET_CACHE", func(t *testing.T) {
o, err := db.FileControl("", sqlite3.FCNTL_RESET_CACHE)
if err != nil {
@@ -237,6 +238,16 @@ func TestConn_FileControl(t *testing.T) {
t.Errorf("got %v, want LOCK_EXCLUSIVE", o)
}
})
t.Run("FCNTL_NULL_IO", func(t *testing.T) {
o, err := db.FileControl("", sqlite3.FCNTL_NULL_IO)
if err != nil {
t.Fatal(err)
}
if o != nil {
t.Errorf("got %v, want nil", o)
}
})
}
func TestConn_Limit(t *testing.T) {

View File

@@ -295,6 +295,17 @@ func TestConn_Filename(t *testing.T) {
t.Parallel()
file := filepath.Join(t.TempDir(), "test.db")
f, err := os.Create(file)
if err != nil {
t.Fatal(err)
}
f.Close()
file, err = filepath.EvalSymlinks(file)
if err != nil {
t.Fatal(err)
}
db, err := sqlite3.Open(file)
if err != nil {
t.Fatal(err)

View File

@@ -14,9 +14,9 @@ trap 'rm -f sql3parse_table.tmp' EXIT
-o sql3parse_table.wasm main.c \
-I"$ROOT/sqlite3/libc" -I"$ROOT/sqlite3" \
-mexec-model=reactor \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-mmutable-globals -mnontrapping-fptoint \
-msimd128 -mbulk-memory -msign-ext \
-mreference-types -mmultivalue \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
@@ -25,6 +25,6 @@ trap 'rm -f sql3parse_table.tmp' EXIT
"$BINARYEN/wasm-ctor-eval" -c _initialize sql3parse_table.wasm -o sql3parse_table.tmp
"$BINARYEN/wasm-opt" --strip --strip-debug --strip-producers -c -Oz \
sql3parse_table.tmp -o sql3parse_table.wasm --low-memory-unused \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
--enable-mutable-globals --enable-nontrapping-float-to-int \
--enable-simd --enable-bulk-memory --enable-sign-ext \
--enable-reference-types --enable-multivalue

View File

@@ -234,6 +234,7 @@ const (
_FCNTL_CKSM_FILE _FcntlOpcode = 41
_FCNTL_RESET_CACHE _FcntlOpcode = 42
_FCNTL_NULL_IO _FcntlOpcode = 43
_FCNTL_BLOCK_ON_CONNECT _FcntlOpcode = 44
)
// https://sqlite.org/c3ref/c_shm_exclusive.html
@@ -246,6 +247,6 @@ const (
_SHM_EXCLUSIVE _ShmFlag = 8
_SHM_NLOCK = 8
_SHM_BASE = 120
_SHM_BASE = (22 + _SHM_NLOCK) * 4
_SHM_DMS = _SHM_BASE + _SHM_NLOCK
)

View File

@@ -13,22 +13,28 @@ import (
type vfsOS struct{}
func (vfsOS) FullPathname(path string) (string, error) {
path, err := filepath.Abs(path)
link, err := evalSymlinks(path)
if err != nil {
return "", err
}
return path, testSymlinks(filepath.Dir(path))
full, err := filepath.Abs(link)
if err == nil && link != path {
err = _OK_SYMLINK
}
return full, err
}
func testSymlinks(path string) error {
p, err := filepath.EvalSymlinks(path)
func evalSymlinks(path string) (string, error) {
var file string
_, err := os.Lstat(path)
if errors.Is(err, fs.ErrNotExist) {
path, file = filepath.Split(path)
}
path, err = filepath.EvalSymlinks(path)
if err != nil {
return err
return "", err
}
if p != path {
return _OK_SYMLINK
}
return nil
return filepath.Join(path, file), nil
}
func (vfsOS) Delete(path string, syncDir bool) error {

View File

@@ -28,9 +28,8 @@ func (memVFS) Open(name string, flags vfs.OpenFlag) (vfs.File, vfs.OpenFlag, err
//
// We refuse to open all other file types,
// but returning OPEN_MEMORY means SQLite won't ask us to.
const types = vfs.OPEN_MAIN_DB |
vfs.OPEN_TEMP_DB |
vfs.OPEN_TEMP_JOURNAL
const types = vfs.OPEN_MAIN_DB | vfs.OPEN_TEMP_DB |
vfs.OPEN_TRANSIENT_DB | vfs.OPEN_TEMP_JOURNAL
if flags&types == 0 {
// notest // OPEN_MEMORY
return nil, flags, sqlite3.CANTOPEN

View File

@@ -26,7 +26,6 @@ type vfsShm struct {
ptrs []ptr_t
stack [1]stk_t
fileLock bool
blocking bool
sync.Mutex
}

View File

@@ -10,9 +10,9 @@ WASI_SDK="$ROOT/tools/wasi-sdk/bin"
"$WASI_SDK/clang" --target=wasm32-wasi -std=c23 -g0 -O2 \
-o mptest.wasm main.c \
-I"$ROOT/sqlite3/libc" -I"$ROOT/sqlite3" \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-mmutable-globals -mnontrapping-fptoint \
-msimd128 -mbulk-memory -msign-ext \
-mreference-types -mmultivalue \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
@@ -27,7 +27,7 @@ WASI_SDK="$ROOT/tools/wasi-sdk/bin"
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
mptest.wasm -o mptest.tmp --low-memory-unused \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
--enable-mutable-globals --enable-nontrapping-float-to-int \
--enable-simd --enable-bulk-memory --enable-sign-ext \
--enable-reference-types --enable-multivalue
mv mptest.tmp mptest.wasm

Binary file not shown.

View File

@@ -10,9 +10,9 @@ WASI_SDK="$ROOT/tools/wasi-sdk/bin"
"$WASI_SDK/clang" --target=wasm32-wasi -std=c23 -g0 -O2 \
-o speedtest1.wasm main.c \
-I"$ROOT/sqlite3/libc" -I"$ROOT/sqlite3" \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-mmutable-globals -mnontrapping-fptoint \
-msimd128 -mbulk-memory -msign-ext \
-mreference-types -mmultivalue \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
@@ -22,7 +22,7 @@ WASI_SDK="$ROOT/tools/wasi-sdk/bin"
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
speedtest1.wasm -o speedtest1.tmp --low-memory-unused \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
--enable-mutable-globals --enable-nontrapping-float-to-int \
--enable-simd --enable-bulk-memory --enable-sign-ext \
--enable-reference-types --enable-multivalue
mv speedtest1.tmp speedtest1.wasm

View File

@@ -381,6 +381,10 @@ func vfsFileControlImpl(ctx context.Context, mod api.Module, file File, op _Fcnt
file.SetDB(ctx.Value(util.ConnKey{}))
return _OK
}
case _FCNTL_NULL_IO:
file.Close()
return _OK
}
return _NOTFOUND