Debug does not work with opa extensions

[gusega]

Hi,

I try to debug my test that uses the policies with the custom function.

When I press debug,

I get the message that my function is undefined. I set the opa path to the path of my customized opa.

[charlieegan3]

Interesting use case! This is actually because the evaluation is done in the regal language server. For this to work, you'd need to build https://github.com/open-policy-agent/regal with the customised OPA and point the opa.dependency_paths.regal to that.

Would that work for your use case?

[gusega]

@charlieegan3 how do I do that? My custom opa is a module that depends on a certain opa version.
I tried replace the opa depependency in the regal with my module, but it does not contain certain opa modules, so this does not work.

[charlieegan3]

Interesting, can you share the error message and the go.mod you have tried?

Is your OPA fork based on an old version of OPA?

[gusega]

@charlieegan3 I don't have an opa fork, I have an opa extension as describe in the extension documentation: https://www.openpolicyagent.org/docs/extensions

[gusega]

my extension adds a builtin function.

[charlieegan3]

Oh I see, In that case you'd need to build a custom Regal with the function registered there instead. I will move this issue to Regal so we can continue to discuss there.

I am about to finish for the day here, but it'd be interesting to know what the custom functions do so we have the full picture here.

[gusega]

build a custom Regal with the function registered there instead

How do I do that?

what the custom functions do so we have the full picture

I have a function UnzipFiles , which accepts base64 encoded zip archive. That zip archive contains multiple text files. The function creates a map[filename]= filecontent and then serializes that map as a json.

If archive contains

foo.txt, baz.txt

then the resulting map is

{
foo.txt: "foo.txt_content",
baz.txt: "baz.txt_content"
}

then function returns that map in a variable.

In general, I want to create different custom functions and then debug, validate my code with Regal.

[anderseknert]

I don't think there's much to do in Regal about this, as this is simply how Go works. OPA + custom functions written in Go is no longer "OPA" as distributed by the project. So like Charlie mentioned, you can only pull this off by also making Regal no longer "Regal" in the same sense — i.e. by compiling it from source with the custom distribution of OPA you have also compiled from source as a local dependency.

I don't think there's much we can do about this in Regal other than possibly document how this could be done.

[gusega]

@anderseknert but how can I do this? What do I need to do? I have my opa extension, what do I need to do to compile regal with those changes?

Regal depends on OPA. But I only have Extension.

[anderseknert]

I tried replace the opa depependency in the regal with my module, but it does not contain certain opa modules, so this does not work.

That's how you do it. What do you mean it does not contain certain OPA modules? What issue are you seeing?

[gusega]

@anderseknert so, what I did, I added to regal go.mod:

replace github.com/open-policy-agent/opa => /Users/gusega/path-to-sources-of-my-opa-extension/opa/

That opa/ directory contains main.go, go.mod and other files from my opa extension, my opa extension looks like the one described in the documentation.

When I run go mod tidy in regal, I get the following error:

go: github.com/open-policy-agent/regal/bundle imports
        github.com/open-policy-agent/opa/v1/bundle: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/bundle
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/ast: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/ast
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/ast/location: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/ast/location
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/cover: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/cover
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/debug: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/debug
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/loader: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/loader
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/logging: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/logging
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/metrics: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/metrics
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/storage: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/storage
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/storage/inmem: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/storage/inmem
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/tester: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/tester
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/topdown: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/topdown
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/util: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/util
go: github.com/open-policy-agent/regal/cmd imports
        github.com/open-policy-agent/opa/v1/version: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/version
go: github.com/open-policy-agent/regal/internal/explorer imports
        github.com/open-policy-agent/opa/v1/compile: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/compile
go: github.com/open-policy-agent/regal/internal/explorer imports
        github.com/open-policy-agent/opa/v1/ir: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/ir
go: github.com/open-policy-agent/regal/internal/io imports
        github.com/open-policy-agent/opa/v1/loader/filter: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/loader/filter
go: github.com/open-policy-agent/regal/internal/lsp imports
        github.com/open-policy-agent/opa/v1/ast/oracle: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/ast/oracle
go: github.com/open-policy-agent/regal/internal/lsp imports
        github.com/open-policy-agent/opa/v1/format: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/format
go: github.com/open-policy-agent/regal/internal/lsp imports
        github.com/open-policy-agent/opa/v1/rego: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/rego
go: github.com/open-policy-agent/regal/internal/lsp imports
        github.com/open-policy-agent/opa/v1/topdown/print: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/topdown/print
go: github.com/open-policy-agent/regal/internal/lsp/hover imports
        github.com/open-policy-agent/opa/v1/types: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/types
go: github.com/open-policy-agent/regal/internal/lsp/rego/query imports
        github.com/open-policy-agent/opa/v1/resolver: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/resolver
go: github.com/open-policy-agent/regal/internal/metrics imports
        github.com/open-policy-agent/opa/v1/profiler: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/profiler
go: github.com/open-policy-agent/regal/pkg/builtins imports
        github.com/open-policy-agent/opa/v1/topdown/builtins: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/topdown/builtins
go: github.com/open-policy-agent/regal/internal/io tested by
        github.com/open-policy-agent/regal/internal/io.test imports
        github.com/open-policy-agent/opa/v1/util/test: module github.com/open-policy-agent/opa@latest found (v1.13.1, replaced by /Users/gusega/path-to-sources-of-my-opa-extension/opa/), but does not contain package github.com/open-policy-agent/opa/v1/util/test

the go.mod of my opa extension looks like this:

module opa++

go 1.24.7

require github.com/open-policy-agent/opa v1.12.2

// and then the bunch of other dependencies

So it looks like when I build regal, go does not see packages from the github.com/open-policy-agent/opa dependency of opa++

[anderseknert]

Ah, I see. You're extending OPA in a new "application", not a fork of OPA. My bad. Not sure what the best way would be to accomplish this. I suppose you could have your custom built-in functions referenced in Regal's go.mod without replacing the OPA dependency. And then add a few lines somewhere in Regal to have those built-in functions loaded on init?

@srenatus what would you do?

[gusega]

@anderseknert i think i will create an new application out of regal, just like I did with opa and then load those functions in it. I think I will make those functions a part of the library in my opa application. This way, I can simply add dependency to my "custom opa" application in regal, ocp etc. Without having to duplicate the functions declaration and definition in regal, ocp.