Skip to content

Commit 70babd2

Browse files
committed
perf: eliminates splitAndTrim bottleneck and replace map+RWMutex with sync.Map
- Replace map + RWMutex with sync.Map for better concurrent cache access performance. - Eliminated the major splitAndTrim bottleneck and achieved significant performance gains. The key insight was parsing comma-separated values directly instead of creating intermediate string allocations. Replace entire split and trims mechanism into single parseSliceFromStrings function. - Enhance time.Time parsing. - New benchmark method - Add Parallel benchmark
1 parent 14c9fa5 commit 70babd2

File tree

7 files changed

+450
-191
lines changed

7 files changed

+450
-191
lines changed

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,5 @@
11
coverage.*
2+
*.prof
3+
*.test
4+
*.bak
5+
personal-note.md

README.md

Lines changed: 64 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
[![Go Report Card](https://goreportcard.com/badge/github.com/prawirdani/qparser)](https://goreportcard.com/report/github.com/prawirdani/qparser)
44
![Build Status](https://github.com/prawirdani/qparser/actions/workflows/ci.yml/badge.svg)
55

6-
`qparser` is a simple package that helps parse query parameters into structs in Go. It is inspired by [gorilla/schema](https://github.com/gorilla/schema) with a main focus on query parameters. Built on top of Go stdlib, it uses a custom struct tag `qp` to define the query parameter key.
6+
`qparser` is a lightweight Go package designed to parse URL query parameters directly into Go structs. It is built on the Go standard library, offering a simple and focused solution for handling incoming query strings.
77

88
## Table of Contents
99
- [Installation](#installation)
@@ -79,6 +79,7 @@ func main() {
7979
// Do something with pagination
8080
}
8181
```
82+
8283
### Multiple Values Query & Nested Struct
8384
To support multiple values for a single query parameter, use a slice type. For nested structs, utilize the qp tag within the fields of the nested struct to pass the query parameters. It's important to note that the parent struct containing the nested/child struct **should not have its own qp tag**. Here's an example:
8485
```go
@@ -107,9 +108,14 @@ func GetMenus(w http.ResponseWriter, r *http.Request) {
107108
}
108109
```
109110
There are three ways for the parser to handle multiple values query parameters:
110-
1. Comma-separated values: `/menus?categories=desserts,beverages,sides`
111-
2. Repeated Keys: `/menus?categories=desserts&categories=beverages&categories=sides`
112-
3. Combination of both: `/menus?categories=desserts,beverages&categories=sides`
111+
<div align="center">
112+
113+
| Pattern | URL |
114+
| :---------------------|:-----------------------------------------------------------------------|
115+
| Comma-separated | /menus?`categories`=desserts,beverages,sides |
116+
| Repeated Keys | /menus?`categories`=desserts&`categories`=beverages&`categories`=sides |
117+
| Combination of both | /menus?`categories`=desserts,beverages&`categories`=sides |
118+
</div>
113119

114120
Simply ensure that the qp tags are defined appropriately in your struct fields to map these parameters correctly.
115121

@@ -119,6 +125,8 @@ Supports time.Time, *time.Time, and type aliases. Handles a variety of standard
119125

120126
| Format Description |Layout Example |
121127
| :------------------------------------------|:-------------------------------------|
128+
| RFC3339 (Z or offset) |`2006-01-02T15:04:05Z07:00` |
129+
| RFC3339Nano (Z or offset, nanosecond prec) |`2006-01-02T15:04:05.999999999Z07:00` |
122130
| Time only |`15:04:05` |
123131
| Date only |`2006-01-02` |
124132
| Date & time (space separated) |`2006-01-02 15:04:05` |
@@ -129,11 +137,9 @@ Supports time.Time, *time.Time, and type aliases. Handles a variety of standard
129137
| Date & time + milliseconds + TZ |`2006-01-02T15:04:05.000-07:00` |
130138
| Date & time + microseconds + TZ |`2006-01-02T15:04:05.000000-07:00` |
131139
| Date & time + nanoseconds + TZ |`2006-01-02T15:04:05.999999999-07:00` |
132-
| RFC3339 (Z or offset) |`2006-01-02T15:04:05Z07:00` |
133-
| RFC3339Nano (Z or offset, nanosecond prec) |`2006-01-02T15:04:05.999999999Z07:00` |
134140
| Space separator + TZ |`2006-01-02 15:04:05-07:00` |
135141
| Space separator + TZ (+ offset) |`2006-01-02 15:04:05+07:00` |
136-
| Space separator + fractional + TZ |`2006-01-02 15:04:05.123456789-07:00`|
142+
| Space separator + fractional + TZ |`2006-01-02 15:04:05.123456789-07:00` |
137143

138144
</div>
139145

@@ -145,8 +151,9 @@ Example:
145151
```go
146152
type ReportFilter struct {
147153
From time.Time `qp:"from"`
148-
To time.Time `qp:"to"`
154+
To time.Time `qp:"to"`
149155
}
156+
150157
func main() {
151158
var filter ReportFilter
152159

@@ -245,23 +252,62 @@ This allows you to:
245252

246253

247254
## Notes
248-
- Empty query values are not validated by default. For custom validation (including empty value checks), implement your own validation method on the struct or use a third-party validator such as [go-playground/validator](https://github.com/go-playground/validator).
255+
256+
- Empty query values are not validated by default. For custom validation (including empty value checks), implement your own validation method on the struct or use a third-party validator such as go-playground/validator.
249257
- Missing query parameters:
250-
- Primitive fields keep their zero values (0, "", false, etc.).
251-
- Pointer fields are always initialized, even when the parameter is missing. They will contain the zero value of the underlying type.
252-
- Slice fields become an empty slice ([]T{}), not nil.
253-
- Pointer nested structs are also always initialized. Missing fields inside them receive their zero values.
258+
- Primitive fields keep their zero values (0, "", false, etc.).
259+
- Pointer-to-primitive fields (e.g., `*string`, `*int`) remain `nil` when the parameter is missing. They are only allocated when the parameter is provided.
260+
- Slice fields (`[]T`) remain `nil` when the parameter is missing. They are allocated only when at least one value is successfully decoded.
261+
- Pointer-to-slice fields (`*[]T`) remain `nil` when the parameter is missing. They are allocated only when the parameter is provided.
262+
- Pointer-to-struct fields are **always initialized**, even when the nested parameters are missing. They contain the zero value of the struct.
254263
- For repeated query parameters, the value is appended to the slice every time. If you want deduplication or sanitization, implement a post-processing method on your struct.
255-
- The qp tag is case-sensitive and must match the query parameter key exactly.
264+
- The `qp` tag is case-sensitive and must match the query parameter key exactly.
265+
- Pointer-to-struct fields offer no practical benefit because they are always initialized and never `nil`, you cannot rely on `nil` checks to detect whether a nested parameter group was supplied. If you need that behavior, inspect field values or apply custom post-processing.
266+
256267

257268

258269
## Benchmarks
270+
259271
```text
260272
goos: linux
261273
goarch: amd64
262274
cpu: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
263-
Benchmark/Small-8 224376 4602 ns/op 2680 B/op 13 allocs/op
264-
Benchmark/Medium-8 246818 4851 ns/op 2720 B/op 13 allocs/op
265-
Benchmark/Large-8 173422 5867 ns/op 3056 B/op 21 allocs/op
266-
Benchmark/LargeWithDate-8 160401 6543 ns/op 3104 B/op 23 allocs/op
275+
=== Sequential ===========================================================================================
276+
Benchmark/Seq/Minimal-8 1204855 983.8 ns/op 224 B/op 1 allocs/op
277+
Benchmark/Seq/1-date-8 930016 1233 ns/op 248 B/op 2 allocs/op
278+
Benchmark/Seq/2-dates-8 626414 1599 ns/op 272 B/op 3 allocs/op
279+
Benchmark/Seq/slices-string-1*50-8 348960 2924 ns/op 1144 B/op 3 allocs/op
280+
Benchmark/Seq/slices-int-1*50-8 292065 3613 ns/op 664 B/op 3 allocs/op
281+
Benchmark/Seq/slices-2*50-8 202550 5514 ns/op 1584 B/op 5 allocs/op
282+
Benchmark/Seq/slices-2*100-8 123055 9351 ns/op 2960 B/op 5 allocs/op
283+
Benchmark/Seq/2*25-slices-and-2-dates-8 257817 4158 ns/op 944 B/op 7 allocs/op
284+
=== Parallel =============================================================================================
285+
Benchmark/Par/Minimal-8 4529499 261.3 ns/op 224 B/op 1 allocs/op
286+
Benchmark/Par/1-date-8 3590239 356.3 ns/op 248 B/op 2 allocs/op
287+
Benchmark/Par/2-dates-8 2865769 420.1 ns/op 272 B/op 3 allocs/op
288+
Benchmark/Par/slices-string-1*50-8 1261774 963.2 ns/op 1144 B/op 3 allocs/op
289+
Benchmark/Par/slices-int-1*50-8 1136395 1025 ns/op 664 B/op 3 allocs/op
290+
Benchmark/Par/slices-2*50-8 684651 1758 ns/op 1584 B/op 5 allocs/op
291+
Benchmark/Par/slices-2*100-8 410012 2982 ns/op 2960 B/op 5 allocs/op
292+
Benchmark/Par/2*25-slices-and-2-dates-8 990036 1244 ns/op 944 B/op 7 allocs/op
267293
```
294+
### Noticeable Behaviors
295+
296+
#### Allocation Behavior
297+
- **Base cost**: 1 allocation for parsing infrastructure
298+
- **Dates**: Each successful parse creates exactly 1 allocation
299+
- **Slices**: Each slice parsing creates exactly 2 allocations
300+
301+
#### Scaling Behavior
302+
- **Linear growth**: Time and memory scale proportionally with item count
303+
- **Predictable**: 2× items = ~2× time, exactly 2× memory
304+
- **Consistent**: Same allocation count regardless of data size
305+
306+
#### Cache Behavior
307+
- **First use**: Reflection builds struct metadata cache
308+
- **Subsequent uses**: Zero-allocation cache lookups
309+
- **Thread-safe**: sync.Map enables concurrent access
310+
- **Persistent**: Cache lives for application lifetime
311+
312+
#### Concurrency
313+
Parallel execution leverages sync.Map for effective caching, significantly improving performance under concurrent workloads like HTTP handlers by reducing per-operation time

0 commit comments

Comments
 (0)