Compare commits

...

28 Commits

Author SHA1 Message Date
Jorropo
69784e9dd7 chore: release v0.2.1 2022-09-04 13:03:33 +02:00
Rod Vagg
548365f3a7 fix: bring back, but deprecate CodecToStr and Codecs
use go-multicodec as the source of truth
2022-09-04 13:03:33 +02:00
web3-bot
5da28005a9 update .github/workflows/go-check.yml 2022-08-25 15:55:13 +10:00
web3-bot
7f6771bc54 update .github/workflows/go-test.yml 2022-08-25 15:55:13 +10:00
web3-bot
5b6a9dce73 run gofmt -s 2022-08-25 15:55:13 +10:00
web3-bot
890f55d9f3 bump go.mod to Go 1.18 and run go fix 2022-08-25 15:55:13 +10:00
Jorropo
085afa84d0 chore: release v0.3.0 2022-08-25 05:34:36 +02:00
Jorropo
0052a62190 fix: return nil Bytes() if the Cid in undef 2022-08-25 05:34:36 +02:00
GitHub
802b45594e chore: Update .github/workflows/stale.yml [skip ci] 2022-07-21 09:54:55 +00:00
Steven Allen
8f7d7ac18e Merge pull request #139 from mg98/feat/must-parse
Add MustParse
2022-06-27 08:55:37 -07:00
GitHub
b106e0883a Update .github/workflows/stale.yml 2022-06-27 10:45:49 +00:00
Marcel Gregoriadis
386c6cc18a Add MustParse 2022-06-24 17:35:23 +02:00
ipfs-mgmt-read-write[bot]
f4b3e66993 Add .github/workflows/stale.yml 2022-06-13 12:53:24 +00:00
Marcin Rataj
b2064d74a8 fix: remove invalid multicodec2string mappings (#137)
* refactor: remove Codecs table
* chore: go-cid 0.2.0

Codec table was missing dag-json and it had invalid code for dag-cbor.
It also had invalid string representation of dag-pb -- it was using
'protobuf' which is a totally different code.

This removes invalid mappings entirely.
https://github.com/multiformats/go-multicodec should be used instead.
2022-04-21 22:39:31 +02:00
web3-bot
ddd9ef7248 update .github/workflows/go-check.yml 2022-04-05 09:32:38 +10:00
web3-bot
6be8b631d0 update .github/workflows/go-test.yml 2022-04-05 09:32:38 +10:00
web3-bot
395d392889 update .github/workflows/automerge.yml 2022-04-05 09:32:38 +10:00
web3-bot
1d11990417 bump go.mod to Go 1.17 and run go fix 2022-04-05 09:32:38 +10:00
web3-bot
dc3bb41324 sync: update CI config files (#136)
* add version.json file

* update .github/workflows/go-test.yml

* update .github/workflows/go-check.yml

* add .github/workflows/releaser.yml

* add .github/workflows/release-check.yml

* add .github/workflows/tagpush.yml

Co-authored-by: web3-bot <web3-bot@users.noreply.github.com>
2021-12-13 12:26:14 +01:00
Masih H. Derkani
28f4a5eab6 Benchmark existing ways to check for IDENTITY CIDs
Add benchmarks that compare two ways of checking for
`multihash.IDENTITY` code:
1. `Cid.Prefix().MhType`
2. Decode of `Cid.Hash()`

This benchmark illustrates that using Cid.Prefix is more efficient than
`multihash.Decode`. Users wishing to perform such a check should use
`Cid.Prefix`.

Consider that `Cid.Prefix` is already efficient enough and introducing a
dedicated API for performing this check will likely result in small
gains.

Relates to #133
2021-11-08 14:52:45 +00:00
Daniel Martí
5640b0169f avoid double alloc in NewCidV1
We allocate once via "make([]byte, len)",
and again when that buffer is converted to a string.

Thankfully, since Go 1.10 we have strings.Builder,
designed specifically for this use case.

In a downstream benchmark in go-car,
which needs to reconstruct many CID values,
we see small but nice gains:

    name           old time/op    new time/op    delta
    ReadBlocks-16    1.09ms ± 4%    1.06ms ± 5%   -3.33%  (p=0.007 n=11+11)

    name           old speed      new speed      delta
    ReadBlocks-16   478MB/s ± 4%   494MB/s ± 5%   +3.46%  (p=0.007 n=11+11)

    name           old alloc/op   new alloc/op   delta
    ReadBlocks-16    1.30MB ± 0%    1.25MB ± 0%   -3.86%  (p=0.000 n=12+12)

    name           old allocs/op  new allocs/op  delta
    ReadBlocks-16     9.50k ± 0%     8.45k ± 0%  -11.05%  (p=0.000 n=12+12)
2021-09-12 14:08:59 +02:00
Steven Allen
cf76220258 Merge pull request #131 from ipfs/web3-bot/sync
sync: update CI config files
2021-08-30 08:37:22 -07:00
web3-bot
9e2855d9ff update .github/workflows/go-check.yml 2021-08-17 13:32:06 +00:00
web3-bot
b8eba8ea35 update .github/workflows/go-test.yml 2021-08-17 13:32:06 +00:00
web3-bot
2628583977 update .github/workflows/automerge.yml 2021-08-17 13:32:06 +00:00
web3-bot
6e96c56557 run gofmt -s 2021-08-17 13:32:05 +00:00
web3-bot
44cccd62db bump go.mod to Go 1.16 and run go fix 2021-08-17 13:32:05 +00:00
Daniel Martí
de6c03deae amend the CidFromReader slice extension math
The append+make slice extension idiom works, but note that append uses
the slice's length as its base. We need to append the number of bytes
required for length to reach cidLength, not the capacity.

The added test case panicked before this change, and works now:

	--- FAIL: TestReadCidsFromBuffer (0.00s)
	panic: runtime error: slice bounds out of range [:73] with capacity 64 [recovered]
		panic: runtime error: slice bounds out of range [:73] with capacity 64

	goroutine 37 [running]:
	testing.tRunner.func1.2({0x570d60, 0xc000016438})
		testing/testing.go:1203 +0x24e
	testing.tRunner.func1()
		testing/testing.go:1206 +0x218
	panic({0x570d60, 0xc000016438})
		runtime/panic.go:1038 +0x215
	github.com/ipfs/go-cid.CidFromReader({0x5b0e20, 0xc000010900})
		github.com/ipfs/go-cid/cid.go:803 +0x75f
	github.com/ipfs/go-cid.TestReadCidsFromBuffer(0xc00014ba00)
		github.com/ipfs/go-cid/cid_test.go:710 +0x625
	testing.tRunner(0xc00014ba00, 0x58af38)
		testing/testing.go:1253 +0x102
	created by testing.(*T).Run
		testing/testing.go:1300 +0x35a
	exit status 2
	FAIL	github.com/ipfs/go-cid	0.004s
2021-07-16 10:10:50 +01:00
19 changed files with 322 additions and 203 deletions

View File

@@ -1,51 +1,11 @@
# File managed by web3-bot. DO NOT EDIT. # File managed by web3-bot. DO NOT EDIT.
# See https://github.com/protocol/.github/ for details. # See https://github.com/protocol/.github/ for details.
# Automatically merge pull requests opened by web3-bot, as soon as (and only if) all tests pass.
# This reduces the friction associated with updating with our workflows.
on: [ pull_request ]
name: Automerge name: Automerge
on: [ pull_request ]
jobs: jobs:
automerge-check:
if: github.event.pull_request.user.login == 'web3-bot'
runs-on: ubuntu-latest
outputs:
status: ${{ steps.should-automerge.outputs.status }}
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Check if we should automerge
id: should-automerge
run: |
for commit in $(git rev-list --first-parent origin/${{ github.event.pull_request.base.ref }}..${{ github.event.pull_request.head.sha }}); do
committer=$(git show --format=$'%ce' -s $commit)
echo "Committer: $committer"
if [[ "$committer" != "web3-bot@users.noreply.github.com" ]]; then
echo "Commit $commit wasn't committed by web3-bot, but by $committer."
echo "::set-output name=status::false"
exit
fi
done
echo "::set-output name=status::true"
automerge: automerge:
needs: automerge-check uses: protocol/.github/.github/workflows/automerge.yml@master
runs-on: ubuntu-latest with:
if: ${{ needs.automerge-check.outputs.status == 'true' }} job: 'automerge'
steps:
- name: Wait on tests
uses: lewagon/wait-on-check-action@bafe56a6863672c681c3cf671f5e10b20abf2eaa # v0.2
with:
ref: ${{ github.event.pull_request.head.sha }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
wait-interval: 10
running-workflow-name: 'automerge' # the name of this job
- name: Merge PR
uses: pascalgn/automerge-action@741c311a47881be9625932b0a0de1b0937aab1ae # v0.13.1
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
MERGE_LABELS: ""
MERGE_METHOD: "squash"
MERGE_DELETE_BRANCH: true

View File

@@ -8,17 +8,28 @@ jobs:
unit: unit:
runs-on: ubuntu-latest runs-on: ubuntu-latest
name: All name: All
env:
RUNGOGENERATE: false
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v3
with: with:
submodules: recursive submodules: recursive
- uses: actions/setup-go@v2 - uses: actions/setup-go@v3
with: with:
go-version: "1.16.x" go-version: "1.19.x"
- name: Run repo-specific setup
uses: ./.github/actions/go-check-setup
if: hashFiles('./.github/actions/go-check-setup') != ''
- name: Read config
if: hashFiles('./.github/workflows/go-check-config.json') != ''
run: |
if jq -re .gogenerate ./.github/workflows/go-check-config.json; then
echo "RUNGOGENERATE=true" >> $GITHUB_ENV
fi
- name: Install staticcheck - name: Install staticcheck
run: go install honnef.co/go/tools/cmd/staticcheck@434f5f3816b358fe468fa83dcba62d794e7fe04b # 2021.1 (v0.2.0) run: go install honnef.co/go/tools/cmd/staticcheck@376210a89477dedbe6fdc4484b233998650d7b3c # 2022.1.3 (v0.3.3)
- name: Check that go.mod is tidy - name: Check that go.mod is tidy
uses: protocol/multiple-go-modules@v1.0 uses: protocol/multiple-go-modules@v1.2
with: with:
run: | run: |
go mod tidy go mod tidy
@@ -37,14 +48,26 @@ jobs:
fi fi
- name: go vet - name: go vet
if: ${{ success() || failure() }} # run this step even if the previous one failed if: ${{ success() || failure() }} # run this step even if the previous one failed
uses: protocol/multiple-go-modules@v1.0 uses: protocol/multiple-go-modules@v1.2
with: with:
run: go vet ./... run: go vet ./...
- name: staticcheck - name: staticcheck
if: ${{ success() || failure() }} # run this step even if the previous one failed if: ${{ success() || failure() }} # run this step even if the previous one failed
uses: protocol/multiple-go-modules@v1.0 uses: protocol/multiple-go-modules@v1.2
with: with:
run: | run: |
set -o pipefail set -o pipefail
staticcheck ./... | sed -e 's@\(.*\)\.go@./\1.go@g' staticcheck ./... | sed -e 's@\(.*\)\.go@./\1.go@g'
- name: go generate
uses: protocol/multiple-go-modules@v1.2
if: (success() || failure()) && env.RUNGOGENERATE == 'true'
with:
run: |
git clean -fd # make sure there aren't untracked files / directories
go generate ./...
# check if go generate modified or added any files
if ! $(git add . && git diff-index HEAD --exit-code --quiet); then
echo "go generated caused changes to the repository:"
git status --short
exit 1
fi

View File

@@ -10,38 +10,59 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ "ubuntu", "windows", "macos" ] os: [ "ubuntu", "windows", "macos" ]
go: [ "1.15.x", "1.16.x" ] go: [ "1.18.x", "1.19.x" ]
runs-on: ${{ matrix.os }}-latest env:
name: ${{ matrix.os}} (go ${{ matrix.go }}) COVERAGES: ""
runs-on: ${{ format('{0}-latest', matrix.os) }}
name: ${{ matrix.os }} (go ${{ matrix.go }})
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v3
with: with:
submodules: recursive submodules: recursive
- uses: actions/setup-go@v2 - uses: actions/setup-go@v3
with: with:
go-version: ${{ matrix.go }} go-version: ${{ matrix.go }}
- name: Go information - name: Go information
run: | run: |
go version go version
go env go env
- name: Use msys2 on windows
if: ${{ matrix.os == 'windows' }}
shell: bash
# The executable for msys2 is also called bash.cmd
# https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md#shells
# If we prepend its location to the PATH
# subsequent 'shell: bash' steps will use msys2 instead of gitbash
run: echo "C:/msys64/usr/bin" >> $GITHUB_PATH
- name: Run repo-specific setup
uses: ./.github/actions/go-test-setup
if: hashFiles('./.github/actions/go-test-setup') != ''
- name: Run tests - name: Run tests
uses: protocol/multiple-go-modules@v1.0 uses: protocol/multiple-go-modules@v1.2
with: with:
run: go test -v -coverprofile coverage.txt ./... # Use -coverpkg=./..., so that we include cross-package coverage.
# If package ./A imports ./B, and ./A's tests also cover ./B,
# this means ./B's coverage will be significantly higher than 0%.
run: go test -v -shuffle=on -coverprofile=module-coverage.txt -coverpkg=./... ./...
- name: Run tests (32 bit) - name: Run tests (32 bit)
if: ${{ matrix.os != 'macos' }} # can't run 32 bit tests on OSX. if: ${{ matrix.os != 'macos' }} # can't run 32 bit tests on OSX.
uses: protocol/multiple-go-modules@v1.0 uses: protocol/multiple-go-modules@v1.2
env: env:
GOARCH: 386 GOARCH: 386
with: with:
run: go test -v ./... run: |
export "PATH=${{ env.PATH_386 }}:$PATH"
go test -v -shuffle=on ./...
- name: Run tests with race detector - name: Run tests with race detector
if: ${{ matrix.os == 'ubuntu' }} # speed things up. Windows and OSX VMs are slow if: ${{ matrix.os == 'ubuntu' }} # speed things up. Windows and OSX VMs are slow
uses: protocol/multiple-go-modules@v1.0 uses: protocol/multiple-go-modules@v1.2
with: with:
run: go test -v -race ./... run: go test -v -race ./...
- name: Collect coverage files
shell: bash
run: echo "COVERAGES=$(find . -type f -name 'module-coverage.txt' | tr -s '\n' ',' | sed 's/,$//')" >> $GITHUB_ENV
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
uses: codecov/codecov-action@a1ed4b322b4b38cb846afb5a0ebfa17086917d27 # v1.5.0 uses: codecov/codecov-action@81cd2dc8148241f03f5839d295e000b8f761e378 # v3.1.0
with: with:
file: coverage.txt files: '${{ env.COVERAGES }}'
env_vars: OS=${{ matrix.os }}, GO=${{ matrix.go }} env_vars: OS=${{ matrix.os }}, GO=${{ matrix.go }}

11
.github/workflows/release-check.yml vendored Normal file
View File

@@ -0,0 +1,11 @@
# File managed by web3-bot. DO NOT EDIT.
# See https://github.com/protocol/.github/ for details.
name: Release Checker
on:
pull_request:
paths: [ 'version.json' ]
jobs:
release-check:
uses: protocol/.github/.github/workflows/release-check.yml@master

11
.github/workflows/releaser.yml vendored Normal file
View File

@@ -0,0 +1,11 @@
# File managed by web3-bot. DO NOT EDIT.
# See https://github.com/protocol/.github/ for details.
name: Releaser
on:
push:
paths: [ 'version.json' ]
jobs:
releaser:
uses: protocol/.github/.github/workflows/releaser.yml@master

26
.github/workflows/stale.yml vendored Normal file
View File

@@ -0,0 +1,26 @@
name: Close and mark stale issue
on:
schedule:
- cron: '0 0 * * *'
jobs:
stale:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 7 days.'
close-issue-message: 'This issue was closed because it is missing author input.'
stale-issue-label: 'kind/stale'
any-of-labels: 'need/author-input'
exempt-issue-labels: 'need/triage,need/community-input,need/maintainer-input,need/maintainers-input,need/analysis,status/blocked,status/in-progress,status/ready,status/deferred,status/inactive'
days-before-issue-stale: 6
days-before-issue-close: 7
enable-statistics: true

12
.github/workflows/tagpush.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# File managed by web3-bot. DO NOT EDIT.
# See https://github.com/protocol/.github/ for details.
name: Tag Push Checker
on:
push:
tags:
- v*
jobs:
releaser:
uses: protocol/.github/.github/workflows/tagpush.yml@master

View File

@@ -59,10 +59,17 @@ fmt.Println("Got CID: ", c)
#### Creating a CID from scratch #### Creating a CID from scratch
```go ```go
import (
cid "github.com/ipfs/go-cid"
mc "github.com/multiformats/go-multicodec"
mh "github.com/multiformats/go-multihash"
)
// Create a cid manually by specifying the 'prefix' parameters // Create a cid manually by specifying the 'prefix' parameters
pref := cid.Prefix{ pref := cid.Prefix{
Version: 1, Version: 1,
Codec: cid.Raw, Codec: mc.Raw,
MhType: mh.SHA2_256, MhType: mh.SHA2_256,
MhLength: -1, // default length MhLength: -1, // default length
} }

View File

@@ -39,7 +39,8 @@ type Cid interface {
// and the Multihash length. It does not contains // and the Multihash length. It does not contains
// any actual content information. // any actual content information.
// NOTE: The use -1 in MhLength to mean default length is deprecated, // NOTE: The use -1 in MhLength to mean default length is deprecated,
// use the V0Builder or V1Builder structures instead //
// use the V0Builder or V1Builder structures instead
type Prefix struct { type Prefix struct {
Version uint64 Version uint64
Codec uint64 Codec uint64

View File

@@ -12,14 +12,14 @@ import (
// //
// Sample results on linux amd64 go1.11beta: // Sample results on linux amd64 go1.11beta:
// //
// BenchmarkCidMap_CidStr-8 100000 16317 ns/op // BenchmarkCidMap_CidStr-8 100000 16317 ns/op
// BenchmarkCidMap_CidIface-8 100000 20516 ns/op // BenchmarkCidMap_CidIface-8 100000 20516 ns/op
// //
// With benchmem on: // With benchmem on:
// //
// BenchmarkCidMap_CidStr-8 100000 15579 ns/op 11223 B/op 207 allocs/op // BenchmarkCidMap_CidStr-8 100000 15579 ns/op 11223 B/op 207 allocs/op
// BenchmarkCidMap_CidIface-8 100000 19500 ns/op 12824 B/op 307 allocs/op // BenchmarkCidMap_CidIface-8 100000 19500 ns/op 12824 B/op 307 allocs/op
// BenchmarkCidMap_StrPlusHax-8 200000 10451 ns/op 7589 B/op 202 allocs/op // BenchmarkCidMap_StrPlusHax-8 200000 10451 ns/op 7589 B/op 202 allocs/op
// //
// We can see here that the impact of interface boxing is significant: // We can see here that the impact of interface boxing is significant:
// it increases the time taken to do the inserts to 133%, largely because // it increases the time taken to do the inserts to 133%, largely because
@@ -36,7 +36,6 @@ import (
// re-arranges itself, it involves more or less an O(n) copy of the content // re-arranges itself, it involves more or less an O(n) copy of the content
// in addition to the alloc itself). This isn't topical to the question of // in addition to the alloc itself). This isn't topical to the question of
// whether or not interfaces are a good idea; just for contextualizing. // whether or not interfaces are a good idea; just for contextualizing.
//
func BenchmarkCidMap_CidStr(b *testing.B) { func BenchmarkCidMap_CidStr(b *testing.B) {
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
mp := map[CidStr]int{} mp := map[CidStr]int{}

View File

@@ -114,7 +114,7 @@ func NewCidStr(version uint64, codecType uint64, mhash mh.Multihash) CidStr {
// //
// For CidV1, the data buffer is in the form: // For CidV1, the data buffer is in the form:
// //
// <version><codec-type><multihash> // <version><codec-type><multihash>
// //
// CidV0 are also supported. In particular, data buffers starting // CidV0 are also supported. In particular, data buffers starting
// with length 34 bytes, which starts with bytes [18,32...] are considered // with length 34 bytes, which starts with bytes [18,32...] are considered

View File

@@ -110,7 +110,7 @@ func (c CidStruct) Prefix() Prefix {
// //
// For CidV1, the data buffer is in the form: // For CidV1, the data buffer is in the form:
// //
// <version><codec-type><multihash> // <version><codec-type><multihash>
// //
// CidV0 are also supported. In particular, data buffers starting // CidV0 are also supported. In particular, data buffers starting
// with length 34 bytes, which starts with bytes [18,32...] are considered // with length 34 bytes, which starts with bytes [18,32...] are considered

61
benchmark_test.go Normal file
View File

@@ -0,0 +1,61 @@
package cid_test
import (
"math/rand"
"testing"
"github.com/ipfs/go-cid"
"github.com/multiformats/go-multihash"
)
// BenchmarkIdentityCheck benchmarks two ways of checking whether a CIDv1 has multihash.IDENTITY
// code:
// 1. Cid.Prefix(), and
// 2. decoding the Cid.Hash().
//
// This benchmark illustrates that using Cid.Prefix is more efficient than multihash.Decode.
// Users wishing to perform such a check should use Cid.Prefix.
//
// Consider that `Cid.Prefix` is already efficient enough and introducing a dedicated API for
// performing this check will likely result in small gains.
func BenchmarkIdentityCheck(b *testing.B) {
rng := rand.New(rand.NewSource(1413))
data := make([]byte, rng.Intn(100)+1024)
if _, err := rng.Read(data); err != nil {
b.Fatal(err)
}
mh, err := multihash.Sum(data, multihash.IDENTITY, -1)
if err != nil {
b.Fatal(err)
}
cv1 := cid.NewCidV1(cid.Raw, mh)
b.SetBytes(int64(cv1.ByteLen()))
b.ReportAllocs()
b.ResetTimer()
b.Run("Prefix", func(b *testing.B) {
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
if cv1.Prefix().MhType != multihash.IDENTITY {
b.Fatal("expected IDENTITY CID")
}
}
})
})
b.Run("MultihashDecode", func(b *testing.B) {
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
dmh, err := multihash.Decode(cv1.Hash())
if err != nil {
b.Fatal(err)
}
if dmh.Code != multihash.IDENTITY {
b.Fatal("expected IDENTITY CID")
}
}
})
})
}

143
cid.go
View File

@@ -10,7 +10,7 @@
// //
// A CIDv1 has four parts: // A CIDv1 has four parts:
// //
// <cidv1> ::= <multibase-prefix><cid-version><multicodec-packed-content-type><multihash-content-address> // <cidv1> ::= <multibase-prefix><cid-version><multicodec-packed-content-type><multihash-content-address>
// //
// As shown above, the CID implementation relies heavily on Multiformats, // As shown above, the CID implementation relies heavily on Multiformats,
// particularly Multibase // particularly Multibase
@@ -22,6 +22,7 @@ package cid
import ( import (
"bytes" "bytes"
"encoding" "encoding"
"encoding/binary"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
@@ -29,6 +30,7 @@ import (
"strings" "strings"
mbase "github.com/multiformats/go-multibase" mbase "github.com/multiformats/go-multibase"
"github.com/multiformats/go-multicodec"
mh "github.com/multiformats/go-multihash" mh "github.com/multiformats/go-multihash"
varint "github.com/multiformats/go-varint" varint "github.com/multiformats/go-varint"
) )
@@ -46,19 +48,21 @@ var (
ErrInvalidEncoding = errors.New("invalid base encoding") ErrInvalidEncoding = errors.New("invalid base encoding")
) )
// These are multicodec-packed content types. The should match // Consts below are DEPRECATED and left only for legacy reasons:
// the codes described in the authoritative document: // <https://github.com/ipfs/go-cid/pull/137>
// https://github.com/multiformats/multicodec/blob/master/table.csv // Modern code should use consts from go-multicodec instead:
// <https://github.com/multiformats/go-multicodec>
const ( const (
Raw = 0x55 // common ones
Raw = 0x55
DagProtobuf = 0x70 // https://ipld.io/docs/codecs/known/dag-pb/
DagCBOR = 0x71 // https://ipld.io/docs/codecs/known/dag-cbor/
DagJSON = 0x0129 // https://ipld.io/docs/codecs/known/dag-json/
Libp2pKey = 0x72 // https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#peer-ids
DagProtobuf = 0x70 // other
DagCBOR = 0x71 GitRaw = 0x78
Libp2pKey = 0x72 DagJOSE = 0x85 // https://ipld.io/specs/codecs/dag-jose/spec/
GitRaw = 0x78
DagJOSE = 0x85
EthBlock = 0x90 EthBlock = 0x90
EthBlockList = 0x91 EthBlockList = 0x91
EthTxTrie = 0x92 EthTxTrie = 0x92
@@ -81,61 +85,22 @@ const (
) )
// Codecs maps the name of a codec to its type // Codecs maps the name of a codec to its type
var Codecs = map[string]uint64{ // Deprecated: modern code should use consts from go-multicodec instead:
"v0": DagProtobuf, // <https://github.com/multiformats/go-multicodec>
"raw": Raw, var Codecs map[string]uint64
"protobuf": DagProtobuf,
"cbor": DagCBOR,
"libp2p-key": Libp2pKey,
"git-raw": GitRaw,
"eth-block": EthBlock,
"eth-block-list": EthBlockList,
"eth-tx-trie": EthTxTrie,
"eth-tx": EthTx,
"eth-tx-receipt-trie": EthTxReceiptTrie,
"eth-tx-receipt": EthTxReceipt,
"eth-state-trie": EthStateTrie,
"eth-account-snapshot": EthAccountSnapshot,
"eth-storage-trie": EthStorageTrie,
"bitcoin-block": BitcoinBlock,
"bitcoin-tx": BitcoinTx,
"zcash-block": ZcashBlock,
"zcash-tx": ZcashTx,
"decred-block": DecredBlock,
"decred-tx": DecredTx,
"dash-block": DashBlock,
"dash-tx": DashTx,
"fil-commitment-unsealed": FilCommitmentUnsealed,
"fil-commitment-sealed": FilCommitmentSealed,
"dag-jose": DagJOSE,
}
// CodecToStr maps the numeric codec to its name // CodecToStr maps the numeric codec to its name
var CodecToStr = map[uint64]string{ // Deprecated: modern code should use consts from go-multicodec instead:
Raw: "raw", // <https://github.com/multiformats/go-multicodec>
DagProtobuf: "protobuf", var CodecToStr map[uint64]string
DagCBOR: "cbor",
GitRaw: "git-raw", func init() {
EthBlock: "eth-block", Codecs = make(map[string]uint64)
EthBlockList: "eth-block-list", CodecToStr = make(map[uint64]string)
EthTxTrie: "eth-tx-trie", for _, code := range multicodec.KnownCodes() {
EthTx: "eth-tx", Codecs[code.String()] = uint64(code)
EthTxReceiptTrie: "eth-tx-receipt-trie", CodecToStr[uint64(code)] = code.String()
EthTxReceipt: "eth-tx-receipt", }
EthStateTrie: "eth-state-trie",
EthAccountSnapshot: "eth-account-snapshot",
EthStorageTrie: "eth-storage-trie",
BitcoinBlock: "bitcoin-block",
BitcoinTx: "bitcoin-tx",
ZcashBlock: "zcash-block",
ZcashTx: "zcash-tx",
DecredBlock: "decred-block",
DecredTx: "decred-tx",
DashBlock: "dash-block",
DashTx: "dash-tx",
FilCommitmentUnsealed: "fil-commitment-unsealed",
FilCommitmentSealed: "fil-commitment-sealed",
DagJOSE: "dag-jose",
} }
// tryNewCidV0 tries to convert a multihash into a CIDv0 CID and returns an // tryNewCidV0 tries to convert a multihash into a CIDv0 CID and returns an
@@ -173,16 +138,24 @@ func NewCidV0(mhash mh.Multihash) Cid {
// Panics if the multihash is invalid. // Panics if the multihash is invalid.
func NewCidV1(codecType uint64, mhash mh.Multihash) Cid { func NewCidV1(codecType uint64, mhash mh.Multihash) Cid {
hashlen := len(mhash) hashlen := len(mhash)
// two 8 bytes (max) numbers plus hash
buf := make([]byte, 1+varint.UvarintSize(codecType)+hashlen) // Two 8 bytes (max) numbers plus hash.
n := varint.PutUvarint(buf, 1) // We use strings.Builder to only allocate once.
n += varint.PutUvarint(buf[n:], codecType) var b strings.Builder
cn := copy(buf[n:], mhash) b.Grow(1 + varint.UvarintSize(codecType) + hashlen)
b.WriteByte(1)
var buf [binary.MaxVarintLen64]byte
n := varint.PutUvarint(buf[:], codecType)
b.Write(buf[:n])
cn, _ := b.Write(mhash)
if cn != hashlen { if cn != hashlen {
panic("copy hash length is inconsistent") panic("copy hash length is inconsistent")
} }
return Cid{string(buf[:n+hashlen])} return Cid{b.String()}
} }
var ( var (
@@ -228,10 +201,19 @@ func Parse(v interface{}) (Cid, error) {
} }
} }
// MustParse calls Parse but will panic on error.
func MustParse(v interface{}) Cid {
c, err := Parse(v)
if err != nil {
panic(err)
}
return c
}
// Decode parses a Cid-encoded string and returns a Cid object. // Decode parses a Cid-encoded string and returns a Cid object.
// For CidV1, a Cid-encoded string is primarily a multibase string: // For CidV1, a Cid-encoded string is primarily a multibase string:
// //
// <multibase-type-code><base-encoded-string> // <multibase-type-code><base-encoded-string>
// //
// The base-encoded string represents a: // The base-encoded string represents a:
// //
@@ -287,7 +269,7 @@ func ExtractEncoding(v string) (mbase.Encoding, error) {
// Cast takes a Cid data slice, parses it and returns a Cid. // Cast takes a Cid data slice, parses it and returns a Cid.
// For CidV1, the data buffer is in the form: // For CidV1, the data buffer is in the form:
// //
// <version><codec-type><multihash> // <version><codec-type><multihash>
// //
// CidV0 are also supported. In particular, data buffers starting // CidV0 are also supported. In particular, data buffers starting
// with length 34 bytes, which starts with bytes [18,32...] are considered // with length 34 bytes, which starts with bytes [18,32...] are considered
@@ -416,7 +398,13 @@ func (c Cid) Hash() mh.Multihash {
// Bytes returns the byte representation of a Cid. // Bytes returns the byte representation of a Cid.
// The output of bytes can be parsed back into a Cid // The output of bytes can be parsed back into a Cid
// with Cast(). // with Cast().
//
// If c.Defined() == false, it return a nil slice and may not
// be parsable with Cast().
func (c Cid) Bytes() []byte { func (c Cid) Bytes() []byte {
if !c.Defined() {
return nil
}
return []byte(c.str) return []byte(c.str)
} }
@@ -497,7 +485,7 @@ func (c *Cid) UnmarshalJSON(b []byte) error {
// MarshalJSON procudes a JSON representation of a Cid, which looks as follows: // MarshalJSON procudes a JSON representation of a Cid, which looks as follows:
// //
// { "/": "<cid-string>" } // { "/": "<cid-string>" }
// //
// Note that this formatting comes from the IPLD specification // Note that this formatting comes from the IPLD specification
// (https://github.com/ipld/specs/tree/master/ipld) // (https://github.com/ipld/specs/tree/master/ipld)
@@ -554,7 +542,8 @@ func (c Cid) Prefix() Prefix {
// and the Multihash length. It does not contains // and the Multihash length. It does not contains
// any actual content information. // any actual content information.
// NOTE: The use -1 in MhLength to mean default length is deprecated, // NOTE: The use -1 in MhLength to mean default length is deprecated,
// use the V0Builder or V1Builder structures instead //
// use the V0Builder or V1Builder structures instead
type Prefix struct { type Prefix struct {
Version uint64 Version uint64
Codec uint64 Codec uint64
@@ -593,7 +582,7 @@ func (p Prefix) Sum(data []byte) (Cid, error) {
// Bytes returns a byte representation of a Prefix. It looks like: // Bytes returns a byte representation of a Prefix. It looks like:
// //
// <version><codec><mh-type><mh-length> // <version><codec><mh-type><mh-length>
func (p Prefix) Bytes() []byte { func (p Prefix) Bytes() []byte {
size := varint.UvarintSize(p.Version) size := varint.UvarintSize(p.Version)
size += varint.UvarintSize(p.Codec) size += varint.UvarintSize(p.Codec)
@@ -793,7 +782,7 @@ func CidFromReader(r io.Reader) (int, Cid, error) {
if cidLength > cap(br.dst) { if cidLength > cap(br.dst) {
// If the multihash digest doesn't fit in our initial 64 bytes, // If the multihash digest doesn't fit in our initial 64 bytes,
// efficiently extend the slice via append+make. // efficiently extend the slice via append+make.
br.dst = append(br.dst, make([]byte, cidLength-cap(br.dst))...) br.dst = append(br.dst, make([]byte, cidLength-len(br.dst))...)
} else { } else {
// The multihash digest fits inside our buffer, // The multihash digest fits inside our buffer,
// so just extend its capacity. // so just extend its capacity.

View File

@@ -1,4 +1,4 @@
// +build gofuzz //go:build gofuzz
package cid package cid

View File

@@ -15,37 +15,6 @@ import (
mh "github.com/multiformats/go-multihash" mh "github.com/multiformats/go-multihash"
) )
// Copying the "silly test" idea from
// https://github.com/multiformats/go-multihash/blob/7aa9f26a231c6f34f4e9fad52bf580fd36627285/multihash_test.go#L13
// Makes it so changing the table accidentally has to happen twice.
var tCodecs = map[uint64]string{
Raw: "raw",
DagProtobuf: "protobuf",
DagCBOR: "cbor",
Libp2pKey: "libp2p-key",
GitRaw: "git-raw",
EthBlock: "eth-block",
EthBlockList: "eth-block-list",
EthTxTrie: "eth-tx-trie",
EthTx: "eth-tx",
EthTxReceiptTrie: "eth-tx-receipt-trie",
EthTxReceipt: "eth-tx-receipt",
EthStateTrie: "eth-state-trie",
EthAccountSnapshot: "eth-account-snapshot",
EthStorageTrie: "eth-storage-trie",
BitcoinBlock: "bitcoin-block",
BitcoinTx: "bitcoin-tx",
ZcashBlock: "zcash-block",
ZcashTx: "zcash-tx",
DecredBlock: "decred-block",
DecredTx: "decred-tx",
DashBlock: "dash-block",
DashTx: "dash-tx",
FilCommitmentUnsealed: "fil-commitment-unsealed",
FilCommitmentSealed: "fil-commitment-sealed",
DagJOSE: "dag-jose",
}
func assertEqual(t *testing.T, a, b Cid) { func assertEqual(t *testing.T, a, b Cid) {
if a.Type() != b.Type() { if a.Type() != b.Type() {
t.Fatal("mismatch on type") t.Fatal("mismatch on type")
@@ -61,25 +30,42 @@ func assertEqual(t *testing.T, a, b Cid) {
} }
func TestTable(t *testing.T) { func TestTable(t *testing.T) {
if len(tCodecs) != len(Codecs)-1 { // test some known codecs, hard-wired here to make them a fixture that would
t.Errorf("Item count mismatch in the Table of Codec. Should be %d, got %d", len(tCodecs)+1, len(Codecs)) // need to be updated if they change elsewhere
} for k, v := range map[uint64]string{
Raw: "raw",
for k, v := range tCodecs { DagProtobuf: "dag-pb",
DagCBOR: "dag-cbor",
DagJSON: "dag-json",
Libp2pKey: "libp2p-key",
GitRaw: "git-raw",
EthBlock: "eth-block",
EthBlockList: "eth-block-list",
EthTxTrie: "eth-tx-trie",
EthTx: "eth-tx",
EthTxReceiptTrie: "eth-tx-receipt-trie",
EthTxReceipt: "eth-tx-receipt",
EthStateTrie: "eth-state-trie",
EthAccountSnapshot: "eth-account-snapshot",
EthStorageTrie: "eth-storage-trie",
BitcoinBlock: "bitcoin-block",
BitcoinTx: "bitcoin-tx",
ZcashBlock: "zcash-block",
ZcashTx: "zcash-tx",
DecredBlock: "decred-block",
DecredTx: "decred-tx",
DashBlock: "dash-block",
DashTx: "dash-tx",
FilCommitmentUnsealed: "fil-commitment-unsealed",
FilCommitmentSealed: "fil-commitment-sealed",
DagJOSE: "dag-jose",
} {
if Codecs[v] != k { if Codecs[v] != k {
t.Errorf("Table mismatch: 0x%x %s", k, v) t.Errorf("Table mismatch: 0x%x %s", k, v)
} }
} }
} }
// The table returns cid.DagProtobuf for "v0"
// so we test it apart
func TestTableForV0(t *testing.T) {
if Codecs["v0"] != DagProtobuf {
t.Error("Table mismatch: Codecs[\"v0\"] should resolve to DagProtobuf (0x70)")
}
}
func TestPrefixSum(t *testing.T) { func TestPrefixSum(t *testing.T) {
// Test creating CIDs both manually and with Prefix. // Test creating CIDs both manually and with Prefix.
// Tests: https://github.com/ipfs/go-cid/issues/83 // Tests: https://github.com/ipfs/go-cid/issues/83
@@ -667,6 +653,7 @@ func TestReadCidsFromBuffer(t *testing.T) {
"k2cwueckqkibutvhkr4p2ln2pjcaxaakpd9db0e7j7ax1lxhhxy3ekpv", "k2cwueckqkibutvhkr4p2ln2pjcaxaakpd9db0e7j7ax1lxhhxy3ekpv",
"Qmf5Qzp6nGBku7CEn2UQx4mgN8TW69YUok36DrGa6NN893", "Qmf5Qzp6nGBku7CEn2UQx4mgN8TW69YUok36DrGa6NN893",
"zb2rhZi1JR4eNc2jBGaRYJKYM8JEB4ovenym8L1CmFsRAytkz", "zb2rhZi1JR4eNc2jBGaRYJKYM8JEB4ovenym8L1CmFsRAytkz",
"bafkqarjpmzuwyzltorxxezjpkvcfgqkfjfbfcvslivje2vchkzdu6rckjjcfgtkolaze6mssjqzeyn2ekrcfatkjku2vowseky3fswkfkm2deqkrju3e2",
} }
var cids []Cid var cids []Cid

15
go.mod
View File

@@ -2,9 +2,20 @@ module github.com/ipfs/go-cid
require ( require (
github.com/multiformats/go-multibase v0.0.3 github.com/multiformats/go-multibase v0.0.3
github.com/multiformats/go-multicodec v0.5.0
github.com/multiformats/go-multihash v0.0.15 github.com/multiformats/go-multihash v0.0.15
github.com/multiformats/go-varint v0.0.6 github.com/multiformats/go-varint v0.0.6
golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf // indirect
) )
go 1.15 require (
github.com/klauspost/cpuid/v2 v2.0.4 // indirect
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1 // indirect
github.com/minio/sha256-simd v1.0.0 // indirect
github.com/mr-tron/base58 v1.2.0 // indirect
github.com/multiformats/go-base32 v0.0.3 // indirect
github.com/multiformats/go-base36 v0.1.0 // indirect
golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf // indirect
golang.org/x/sys v0.0.0-20210309074719-68d13333faf2 // indirect
)
go 1.18

7
go.sum
View File

@@ -13,6 +13,8 @@ github.com/multiformats/go-base36 v0.1.0 h1:JR6TyF7JjGd3m6FbLU2cOxhC0Li8z8dLNGQ8
github.com/multiformats/go-base36 v0.1.0/go.mod h1:kFGE83c6s80PklsHO9sRn2NCoffoRdUUOENyW/Vv6sM= github.com/multiformats/go-base36 v0.1.0/go.mod h1:kFGE83c6s80PklsHO9sRn2NCoffoRdUUOENyW/Vv6sM=
github.com/multiformats/go-multibase v0.0.3 h1:l/B6bJDQjvQ5G52jw4QGSYeOTZoAwIO77RblWplfIqk= github.com/multiformats/go-multibase v0.0.3 h1:l/B6bJDQjvQ5G52jw4QGSYeOTZoAwIO77RblWplfIqk=
github.com/multiformats/go-multibase v0.0.3/go.mod h1:5+1R4eQrT3PkYZ24C3W2Ue2tPwIdYQD509ZjSb5y9Oc= github.com/multiformats/go-multibase v0.0.3/go.mod h1:5+1R4eQrT3PkYZ24C3W2Ue2tPwIdYQD509ZjSb5y9Oc=
github.com/multiformats/go-multicodec v0.5.0 h1:EgU6cBe/D7WRwQb1KmnBvU7lrcFGMggZVTPtOW9dDHs=
github.com/multiformats/go-multicodec v0.5.0/go.mod h1:DiY2HFaEp5EhEXb/iYzVAunmyX/aSFMxq2KMKfWEues=
github.com/multiformats/go-multihash v0.0.15 h1:hWOPdrNqDjwHDx82vsYGSDZNyktOJJ2dzZJzFkOV1jM= github.com/multiformats/go-multihash v0.0.15 h1:hWOPdrNqDjwHDx82vsYGSDZNyktOJJ2dzZJzFkOV1jM=
github.com/multiformats/go-multihash v0.0.15/go.mod h1:D6aZrWNLFTV/ynMpKsNtB40mJzmCl4jb1alC0OvHiHg= github.com/multiformats/go-multihash v0.0.15/go.mod h1:D6aZrWNLFTV/ynMpKsNtB40mJzmCl4jb1alC0OvHiHg=
github.com/multiformats/go-varint v0.0.6 h1:gk85QWKxh3TazbLxED/NlDVv8+q+ReFJk7Y2W/KhfNY= github.com/multiformats/go-varint v0.0.6 h1:gk85QWKxh3TazbLxED/NlDVv8+q+ReFJk7Y2W/KhfNY=
@@ -22,14 +24,9 @@ golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWP
golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf h1:B2n+Zi5QeYRDAEodEu72OS36gmTWjgpXr2+cWcBW90o= golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf h1:B2n+Zi5QeYRDAEodEu72OS36gmTWjgpXr2+cWcBW90o=
golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8= golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210309074719-68d13333faf2 h1:46ULzRKLh1CwgRq2dC5SlBzEqqNCi8rreOZnNrbqcIY= golang.org/x/sys v0.0.0-20210309074719-68d13333faf2 h1:46ULzRKLh1CwgRq2dC5SlBzEqqNCi8rreOZnNrbqcIY=
golang.org/x/sys v0.0.0-20210309074719-68d13333faf2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210309074719-68d13333faf2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=

3
version.json Normal file
View File

@@ -0,0 +1,3 @@
{
"version": "v0.3.1"
}