Skip to content

chore: let interop testing launch N tasks instead of N^2 #789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ajewellamz merged 3 commits into from
Jul 17, 2025
Merged

chore: let interop testing launch N tasks instead of N^2 #789

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e7a0e1d
chore: let interop testing launch N tasks instead of N^2
ajewellamz Jul 16, 2025
c54e984
m
ajewellamz Jul 17, 2025
05de2ba
Merge branch 'mainline' into ajewell/linear-interop
ajewellamz Jul 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 160 additions & 2 deletions .github/workflows/library_interop_keyring_test_vectors.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,6 @@ jobs:
# windows-latest,
ubuntu-22.04,
]
encrypting_language: [java, net, rust, python, go, c]
decrypting_language: [java, net, rust, python, go, c]
# https://taskei.amazon.dev/tasks/CrypTool-5284
dotnet-version: ["6.0.x"]
Expand Down Expand Up @@ -549,10 +548,169 @@ jobs:
CORES=$(node -e 'console.log(os.cpus().length)')
make transpile_python CORES=$CORES

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 7200

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_java_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language != 'c'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os == 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_unix

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os != 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_mac

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 7200

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_net_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language != 'c'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os == 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_unix

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os != 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_mac

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 7200

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_rust_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language != 'c'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os == 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_unix

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os != 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_mac

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 7200

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_python_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language != 'c'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os == 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_unix

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os != 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_mac

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 7200

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_go_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language != 'c'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os == 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_unix

- name: Decrypt Encrypt Manifest
if: matrix.decrypting_language == 'c' && matrix.os != 'ubuntu-22.04'
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_c_mac

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 7200

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_${{matrix.encrypting_language}}_${{github.sha}}
name: ${{matrix.os}}_vector_artifact_c_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
Expand Down
126 changes: 124 additions & 2 deletions .github/workflows/library_interop_mkp_test_vectors.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,6 @@ jobs:
# windows-latest,
ubuntu-22.04,
]
encrypting_language: [java, net, rust, python, python_mkp, java_mkp, go]
decrypting_language: [java, net, rust, python, python_mkp, java_mkp, go]
# https://taskei.amazon.dev/tasks/CrypTool-5284
dotnet-version: ["6.0.x"]
Expand Down Expand Up @@ -404,10 +403,133 @@ jobs:
CORES=$(node -e 'console.log(os.cpus().length)')
make transpile_python CORES=$CORES

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 3600

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_${{matrix.encrypting_language}}_legacy_format_${{github.sha}}
name: ${{matrix.os}}_vector_artifact_java_legacy_format_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 3600

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_net_legacy_format_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 3600

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_rust_legacy_format_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 3600

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_python_legacy_format_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 3600

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_python_mkp_legacy_format_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 3600

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_java_mkp_legacy_format_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
working-directory: ./${{ matrix.library }}
run: make test_decrypt_encrypt_vectors_${{matrix.decrypting_language}}

# TestVectors will call KMS
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-Public-ESDK-Dafny-Role-us-west-2
role-session-name: InterOpTests
role-duration-seconds: 3600

- name: Download Encrypt Manifest Artifact
uses: actions/download-artifact@v4
with:
name: ${{matrix.os}}_vector_artifact_go_legacy_format_${{github.sha}}
path: ./${{matrix.library}}/runtimes/${{matrix.decrypting_language}}

- name: Decrypt Encrypt Manifest
Expand Down
4 changes: 4 additions & 0 deletions TestVectors/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,7 @@ test_encrypt_vectors_c_mac: c_test_vectors
rm -rf local; \
mkdir -p local; \
LD_LIBRARY_PATH=../../install/lib/ ./test_vectors encrypt --manifest-path ./from-dafny --decrypt-manifest-path ./local
rm -rf ./runtimes/c/
mkdir -p ./runtimes/c/
cp aws-encryption-sdk-c/tests/TestVectors/local/*.json ./runtimes/c/
mv ./runtimes/c/decrypt-manifest.json ./runtimes/c/manifest.json
Expand All @@ -376,6 +377,7 @@ test_encrypt_vectors_c_unix: c_test_vectors_unix
rm -rf local; \
mkdir -p local; \
LD_LIBRARY_PATH=../../install/lib/ ./test_vectors encrypt --manifest-path ./from-dafny --decrypt-manifest-path ./local
rm -rf ./runtimes/c/
mkdir -p ./runtimes/c/
pwd
ls -l aws-encryption-sdk-c/tests/TestVectors/
Expand All @@ -386,6 +388,7 @@ test_encrypt_vectors_c_unix: c_test_vectors_unix
cp -r aws-encryption-sdk-c/tests/TestVectors/local/ciphertexts/ ./runtimes/c/ciphertexts/

test_decrypt_encrypt_vectors_c_mac: c_test_vectors
rm -rf aws-encryption-sdk-c/tests/TestVectors/local/
mkdir -p aws-encryption-sdk-c/tests/TestVectors/local/
cp ./runtimes/c/*.json aws-encryption-sdk-c/tests/TestVectors/local/
cp -r ./runtimes/c/plaintexts/ aws-encryption-sdk-c/tests/TestVectors/local/plaintexts/
Expand All @@ -394,6 +397,7 @@ test_decrypt_encrypt_vectors_c_mac: c_test_vectors
LD_LIBRARY_PATH=../../install/lib/ ./test_vectors decrypt --manifest-path ./local --manifest-name manifest.json

test_decrypt_encrypt_vectors_c_unix: c_test_vectors_unix
rm -rf aws-encryption-sdk-c/tests/TestVectors/local/
mkdir -p aws-encryption-sdk-c/tests/TestVectors/local/
pwd
ls -l ./runtimes/c/
Expand Down
Loading