Skip to content

Conversation

@yash-sangwan
Copy link

Problem

Running npx convex function-spec crashes when a function uses v.literal(BigInt(1)) in its validator:
This happens because:

  1. BigInt literals were serialized using the same format as regular BigInt values: {"$integer": "base64"}
  2. When function-spec returns validator metadata as a query result, it goes through jsonToConvex
  3. jsonToConvex rejects any object fields starting with $ (reserved for Convex internal formats)

Solution

Changed BigInt literal serialization in validator metadata to use a new format:

  • New format: {"__convexBigIntLiteral": "1"}
  • Old format (still used for actual Convex values): {"$integer": "AQAAAAAAAAA="}

This separates validator metadata serialization from Convex value serialization.

Changes

Rust (crates/common/src/schemas/json.rs)

  • Modified LiteralValidator::Int64 serialization to use __convexBigIntLiteral
  • Maintained backward compatibility: still deserializes old $integer format
  • Added tests for new serialization and backward compatibility

TypeScript (npm-packages/convex/src/values/validators.ts)

  • Updated VLiteral.json getter to use new format for BigInt values
  • Avoids calling convexToJson for BigInt to prevent $integer format

Tests

  • Updated test expectations to match new format
  • Added integration test functions with BigInt literals

Testing

All Rust tests passing (5/5 in schemas::json::tests)
All TypeScript tests passing (27/27 in schema.test.ts)
Package builds successfully
Manual verification: new format works with jsonToConvex

Backward Compatibility

The old $integer format is still accepted when deserializing, so existing validator metadata will continue to work.

Fixes #212

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The problem: When you use v.literal(BigInt(1)) in a validator and run
function-spec, it crashes with 'Field name $integer starts with a $'.

This happened because BigInt literals were being serialized the same way
as regular BigInt values (using {$integer: ...}), but when these come back
from function-spec queries, jsonToConvex rejects any fields starting with $.

The fix: BigInt literals in validators now use a different format:
{__convexBigIntLiteral: "1"} instead of {$integer: "base64"}.

The old $integer format is still for actual Convex values, not validator
metadata. We still read the old format for backwards compatibility.

Changes:
- Rust: Updated Int64 literal serialization and added tests
- TypeScript: Updated VLiteral.json to use new format for bigints
- Tests: Updated expectations and added integration tests

All tests pass.

Fixes get-convex#212
}
return {
type: this.kind,
value: convexToJson(this.value as string | boolean | number | bigint),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if we didn't do convexToJson here, so the value remained a bigint? Then the single convexToJson for the overall validator json would encode it, but other uses of .json() would need to get updated to not assume it's been convex-encoded

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, VLiteral.json() calls convexToJson(this.value), which converts:

BigInt(1)  { "$integer": "1" }

This nested object ends up causing a crash.

Proposed Change

The proposal is to skip the convexToJson call inside VLiteral.json() and instead:

  • Keep the value as a raw bigint
  • Perform encoding once, at a higher level, when serializing the entire validator

Confusion

My main question is about where this single, top-level convexToJson call would happen.

When I trace the code today, I see:

  • validator.json() being called in places like:

    • the CLI
    • system UDFs
  • The returned object is then passed directly to JSON.stringify()

However, JSON.stringify() cannot handle bigint values and throws:

TypeError: Do not know how to serialize a BigInt

Questions

  1. Is there already an existing serialization layer that applies convexToJson to the entire validator object?

  2. If not, would we need to introduce one?

  3. Which other uses of .json() would need to be updated to safely handle raw bigint values?

  4. Am I missing something about the architecture here?

    • Specifically, where does overall validator serialization actually happen?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

npx convex function-spec crashes on literal BigInt

2 participants