Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

`load` is a Go CLI tool in the senzing-tools suite that pulls records from message queues (RabbitMQ, AWS SQS) and inserts them into the Senzing entity resolution database. It validates records against the Generic Entity Specification.

## Prerequisites

Senzing C library must be installed at:

- `/opt/senzing/er/lib` - shared objects
- `/opt/senzing/er/sdk/c` - SDK header files
- `/etc/opt/senzing` - configuration

See [How to Install Senzing for Go Development](https://github.com/senzing-garage/knowledge-base/blob/main/HOWTO/install-senzing-for-go-development.md) for setup instructions.

## Common Commands

```bash
# One-time dev setup (installs golangci-lint, gotestfmt, etc.)
make dependencies-for-development

# Install/update Go dependencies
make dependencies

# Lint (runs golangci-lint, govulncheck, cspell)
make lint

# Build binary (output in target/<os>-<arch>/)
make build

# Run tests (requires setup first)
make clean setup test

# Run single test
go test -v -run TestName ./path/to/package

# Coverage report (opens in browser)
make clean setup coverage

# Build Docker image
make docker-build

# Auto-fix lint issues
make fix
```

## Architecture

```console
cmd/ CLI layer (cobra/viper)
root.go Main command definition with flags and execution
context_*.go Platform-specific context variables

load/ Core loading logic
load.go BasicLoad struct implementing Load interface
Orchestrates input reading and stats monitoring

input/ Input source adapters
selector.go URL-based routing to appropriate queue reader
rabbitmq/ RabbitMQ consumer (amqp:// URLs)
sqs/ AWS SQS consumer (sqs:// or https:// URLs)
```

### Key Dependencies

- `github.com/senzing-garage/go-cmdhelping` - CLI option handling
- `github.com/senzing-garage/go-queueing` - Queue consumers for RabbitMQ/SQS
- `github.com/senzing-garage/sz-sdk-go` - Senzing SDK interface
- `github.com/senzing-garage/go-sdk-abstract-factory` - Factory for Senzing engine instances

### Input URL Routing

The `input/selector.go` routes based on URL scheme:

- `amqp://` → RabbitMQ
- `sqs://` or `https://` → AWS SQS

## Linting Configuration

Uses golangci-lint v2 with extensive linter set. Config at `.github/linters/.golangci.yaml`. Notable exclusions:

- `exhaustruct` excludes `cobra.Command` and `load.BasicLoad`
- `ireturn` allows `logging.Logging` interface returns

## Testing

Tests require the Senzing database setup:

```bash
make setup # Copies testdata/sqlite/G2C.db to /tmp/sqlite/
```

Test output uses gotestfmt for formatted display. Tests run sequentially (`-p 1`) due to shared resources.

## Environment Variables

- `LD_LIBRARY_PATH` - Path to Senzing libraries (default: `/opt/senzing/er/lib`)
- `SENZING_TOOLS_DATABASE_URL` - Database connection string
- `SENZING_LOG_LEVEL` - Logging level (TRACE, DEBUG, INFO, WARN, ERROR, FATAL, PANIC)
3 changes: 0 additions & 3 deletions .claude/commands/senzing-code-review.md

This file was deleted.

3 changes: 3 additions & 0 deletions .claude/commands/senzing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Senzing

- Perform the steps specified by <https://raw.githubusercontent.com/senzing-factory/claude/refs/tags/v1/commands/senzing.md>
2 changes: 1 addition & 1 deletion .github/linters/.jscpd.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{}
{}
27 changes: 19 additions & 8 deletions .github/linters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,12 @@
- [jscpd configuration]
- Example:

```json
{
"ignore": [
"**/*.go,**/go-test*.yaml"
],
```json
{
"ignore": ["**/*.go,**/go-test*.yaml"],
"threshold": 10
}
```
}
```

## .yaml-lint.yml

Expand All @@ -40,10 +38,23 @@
- [yaml-lint]
- [yaml-lint configuration]

[.checkov.yaml]: .checkov.yaml
## bearer.yml

- [bearer.yml]
- Used by [bearer.yaml]
- [bearer]
- [bearer repository]
- [bearer configuration]

[.checkov.yaml]: .checkov.yaml
[.golangci.yaml]: .golangci.yaml
[.jscpd.json]: .jscpd.json
[.yaml-lint.yml]: .yaml-lint.yml
[bearer configuration]: https://docs.bearer.com/reference/config/
[bearer repository]: https://github.com/Bearer/bearer/tree/main
[bearer.yaml]: ../workflows/README.md#beareryaml
[bearer.yml]: bearer.yml
[bearer]: https://docs.bearer.com/
[checkov configuration]: https://www.checkov.io/2.Basics/CLI%20Command%20Reference.html
[checkov]: https://www.checkov.io/
[golangci linters]: https://golangci-lint.run/usage/linters/
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/bearer.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
name: bearer
name: Bearer

on:
push:
branches-ignore: [main]
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
rule_check:
permissions:
contents: read
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Checkout repository
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/claude-pr-review.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
name: Claude PR Review

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

on:
pull_request:
types: [opened, synchronize]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/dependabot-approve-and-merge.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/docker-build-container.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,21 @@ name: Docker build container

on:
pull_request:
branches:
- main
branches: [main]
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
docker-build-container:
permissions:
contents: read
runs-on: ubuntu-latest
timeout-minutes: 30

steps:
- name: Get repository name
Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/docker-verify-refreshed-at-updated.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Docker verify REFRESHED_AT Updated

# Required status check: Verify Dockerfiles REFRESHED_AT Updated / Dockerfile Check Status
# The other status checks beginning with "Verify Dockerfiles REFRESHED_AT Updated / ..." are not required.

on:
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
verify-dockerfiles:
name: Verify Dockerfiles REFRESHED_AT Updated
permissions:
contents: read
uses: senzing-factory/build-resources/.github/workflows/verify-dockerfile-refreshed-at-updated.yaml@v3
1 change: 1 addition & 0 deletions .github/workflows/go-proxy-pull.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ jobs:
permissions:
contents: write
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Pull new module version
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/go-test-darwin.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ jobs:
go: ["1.24"]
os: [macos-latest]
senzingsdk-version: [production-v4, staging-v4]
timeout-minutes: 30

steps:
- name: Checkout repository
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/go-test-linux.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,15 @@ name: Go test linux

on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
- cron: "15 7 * * *"
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

env:
SENZING_LOG_LEVEL: TRACE
Expand All @@ -28,6 +32,7 @@ jobs:
go: ["1.24"]
os: [ubuntu-latest]
senzingsdk-version: [production-v4, staging-v4]
timeout-minutes: 30

steps:
- name: Checkout repository
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/go-test-windows.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,15 @@ on:
- cron: "15 7 * * *"
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

env:
SENZING_LOG_LEVEL: TRACE
SENZING_TOOLS_DATABASE_URL: "sqlite3://na:na@nowhere/C:\\Temp\\sqlite\\G2C.db"

permissions:
contents: read
permissions: {}

jobs:
go-test-windows:
Expand All @@ -27,6 +30,7 @@ jobs:
matrix:
go: ["1.24"]
senzingsdk-version: [production-v4, staging-v4]
timeout-minutes: 30

steps:
- name: Checkout repository
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/golangci-lint.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
name: Golangci lint

on:
push:
branches-ignore: [main]
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
golangci:
name: lint
name: Lint
permissions:
# Required: allow read access to the content for analysis.
contents: read
Expand All @@ -22,6 +24,7 @@ jobs:
matrix:
go: ["1.24"]
senzingsdk-version: [production-v4, staging-v4]
timeout-minutes: 10

steps:
- name: Checkout repository
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/govulncheck.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
name: govulncheck
name: Govulncheck

on:
push:
branches-ignore: [main]
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
Expand All @@ -18,6 +20,7 @@ jobs:
matrix:
go: ["1.24"]
senzingsdk-version: [production-v4, staging-v4]
timeout-minutes: 10

steps:
- name: Checkout repository
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/lint-workflows.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
name: Lint workflows

on:
push:
branches-ignore: [main]
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

permissions: {}

jobs:
Expand Down
Loading
Loading