Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c2c0e74
Refactor Project for LogicApp, add vscode files
karlrissland Jan 11, 2024
2da70b9
added logic app
karlrissland Jan 11, 2024
c9d609e
using uniquestring for IaC
karlrissland Jan 11, 2024
db94e7d
added workspace file
karlrissland Jan 11, 2024
95ee735
refactored docs to fit with workspaces
karlrissland Jan 12, 2024
9d1c11e
added logicapp and refactored iac
karlrissland Jan 12, 2024
c98fd16
added telemetry to function
karlrissland Jan 18, 2024
f863285
added http trigger function
karlrissland Jan 23, 2024
7c3efbc
infra as code changes
karlrissland Feb 9, 2024
f9a625b
Some infra as code cleanup
karlrissland Feb 9, 2024
bc12b6e
got the function key flowing
karlrissland Feb 13, 2024
330f4eb
Added app insights to apim via bicep
karlrissland Feb 13, 2024
85a3679
Added workflow and removed function
karlrissland Feb 13, 2024
31aa145
fixed some iac and enabled enhanced telem
karlrissland Feb 14, 2024
f0a121b
some iac cleanup
karlrissland Feb 14, 2024
9a12b68
track callertrackid in apim policy
karlrissland Feb 14, 2024
cac0bf4
tracing callertrackingid from APIM
karlrissland Feb 14, 2024
eb020a8
messing with tracing in the function
karlrissland Feb 14, 2024
75690d3
messed with tracking info in function
karlrissland Feb 14, 2024
1410f22
more fiddling
karlrissland Feb 14, 2024
6bd4347
logging tests
karlrissland Feb 15, 2024
3141f3d
getting closer
karlrissland Feb 15, 2024
c3467ba
almost... i can feel it
karlrissland Feb 15, 2024
3d2bfec
adding event id to see if trace pops
karlrissland Feb 15, 2024
9e7dee1
just one more change
karlrissland Feb 16, 2024
012128b
configuring tracking for logicapp
karlrissland Feb 16, 2024
36c69eb
Updated Workflow to remove spliton
karlrissland Feb 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions IntegrationSample.code-workspace
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"folders": [
{
"name": "Logic App",
"path": "src/LogicApp"
},
{
"name": "Function",
"path": "src/FunctionApp"
},
{
"name": "Infrastructure as Code",
"path": "infra"
},
{
"name":"Documentation",
"path":"docs"
}
],
"settings": {
"azurite.location": "C:\\Users\\karlriss\\.azurelogicapps\\.azurite",
"terminal.integrated.env.windows": {
"PATH": "C:\\Users\\karlriss\\.azurelogicapps\\dependencies\\DotNetSDK;${env:PATH}"
},
"omnisharp.dotNetCliPaths": [
"C:\\Users\\karlriss\\.azurelogicapps\\dependencies\\DotNetSDK"
]
}
}
98 changes: 98 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Integration using Azure Service Bus and API Management

This is a sample integration app template that walks through setting up API Management policy for sending data to Azure Service Bus. The API Management uses Managed Identity to access the Service Bus REST APIs. A Function is triggered when a message is queued in Service Bus, and it will write message data to Cosmos DB. The Function App uses Managed Identity to get access to Service Bus. This is a typical integration scenario leveraging APIs.

> Refer to the [App Templates](https://github.com/microsoft/App-Templates) repo Readme for more samples that are compatible with [Azure Developer CLI (azd)](https://github.com/Azure/azure-dev/)

## Architecture
Below architecture is deployed in this demonstration.

![Integration Architecture](media/s8.png)

Azure Services used:

1. API Management
1. Service Bus
1. Function App
1. Application Insights (Function Execution)
1. Storage Account
1. Cosmos DB

The client can be simulated using curl, or any other tool that can send HTTP request to APIM gateway.

## Benefits of this Architecture

Below are benefits and potential extension scenarios for this architecture.

1. Integrate backend systems using message broker to decouple services for scalability and reliability.
1. Allows work to be queued when backend systems are unavailable.
1. API Management provides the publishing capability for HTTP APIs, to promote reuse and discoverability. It can manage other cross-cutting concerns such as authentication, throughput limits, and response caching.
1. Provide load leveling to handle bursts in workloads and broadcast messages to multiple consumers.

In the above architecture, Azure Function App processes the messages by simply writing the data to the Cosmos DB.
Other potential extensions of this architecture are:

1. The function can be converted to a durable function that orchestrates normalization and correlation of data prior to persisting to the Cosmos DB or persisting to other storage.
1. Instead of a Function App, other consumers can process the messages in Service Bus. Services such as Logic Apps to orchestrate workflows, or Microservices running in Container Apps/AKS to process the workload.
1. An Azure EventGrid could be integrated with Service Bus for cost optimization in cases where messages are received occasionally.
1. The APIM can be configured to expose other synchronous REST APIs.
1. The Service bus could be replaced by other queueing technology such as EventHub and EventGrid.

## Deploy solution to Azure

### Prerequisites

1. Local bash shell with Azure CLI or [Azure Cloud Shell](https://ms.portal.azure.com/#cloudshell/)
1. Azure Subscription. [Create one for free](https://azure.microsoft.com/en-us/free/).
1. Clone or fork of this repository.

### Deploy

Login to your Azure in your terminal.

```bash
az login
```

To check your subscription.

```bash
az account show
```

Run the deployment. The deployment will create the resource group "rg-\<Name suffix for resources\>". Make sure you are in the 'app-templates-integration-services' directory.

```bash
cd app-templates-integration-services

az deployment sub create --name "<unique deployment name>" --location "<Your Chosen Location>" --template-file infra/main.bicep --parameters name="<Name suffix for resources>" publisherEmail="<Publisher Email for APIM>" publisherName="<Publisher Name for APIM>"
```

The following deployments will run:

![deployment times](media/s9.png)

>**NOTE**: The APIM deployment can take over an hour to complete.

## Validate Deployment

1. Use Curl or another tool to send a request as shown below to the "demo-queue" created during deployment. Make sure to send in the API key in the header "Ocp-Apim-Subscription-Key".

```bash
curl -X POST https://<Your APIM Gateway URL>/sb-operations/demo-queue -H 'Ocp-Apim-Subscription-Key:<Your APIM Subscription Key>' -H 'Content-Type: application/json' -d '{ "date" : "2022-09-17", "id" : "1", "data" : "Sending data via APIM->Service Bus->Function->CosmosDB" }'
```
If using PowerShell use Invoke-WebRequest:

```
Invoke-WebRequest -Uri "https://<Your APIM Gateway URL>/sb-operations/demo-queue" -Headers @{'Ocp-Apim-Subscription-Key' = '<Your APIM Subscription Key>'; 'Content-Type' = 'application/json'} -Method 'POST' -Body '{ "date" : "2022-09-17", "id" : "1", "data" : "Sending data via APIM->Service Bus->Function->CosmosDB" }'
```

1. Go to your deployment of Cosmos DB in Azure Portal, click on Data Explorer, select "demo-database" and the "demo-container”, click Items. Select the first item and view the content. It will match the data submitted to the APIM gateway in step 1.

![Data in Cosmos DB](media/s10.png)

## Disclaimer

The code and deployment biceps are for demonstration purposes only.


File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
139 changes: 91 additions & 48 deletions infra/main.bicep
Original file line number Diff line number Diff line change
@@ -1,103 +1,139 @@
targetScope = 'subscription'

@minLength(1)
@maxLength(16)
@description('Prefix for all resources, i.e. {name}storage')
param name string

@minLength(1)
@description('Primary location for all resources')
param location string = deployment().location

@description('The email address of the owner of the service')
@description('The email address of the owner of the APIM service')
@minLength(1)
param publisherEmail string
param publisherEmail string = 'integrationSampleUser@sample.com'

@description('The name of the owner of the service')
@description('The name of the owner of the APIM service')
@minLength(1)
param publisherName string
param publisherName string = 'Integration Sample User'

param deploymentRepositoryUrl string = 'https://github.com/aarthiem/app-templates-integration-services.git'
param deploymentBranch string = 'main'

var templatename = 'IntegrationSample'
var uniqueSuffix = substring(uniqueString(concat(subscription().id),templatename, location),0,6)

resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' = {
name: 'rg-${name}'
name: 'rg-${templatename}-${uniqueSuffix}'
location: location
tags: {
apptemplate: 'IntegrationSample'
}
}


module apim './modules/apim.bicep' = {
name: '${rg.name}-apim'
module appInsights './modules/appinsights.bicep' = {
name: '${rg.name}-appinsights'
scope: rg
params: {
apimServiceName: 'apim-${toLower(name)}'
publisherEmail: publisherEmail
publisherName: publisherName
applicationInsightsName: 'appinsights-${toLower(uniqueSuffix)}'
logAnalyticsWorkspaceName: 'loganalytics-${toLower(uniqueSuffix)}'
location: rg.location
}
}

module servicebus './modules/service-bus.bicep' = {
name: '${rg.name}-servicebus'
module logicApp './modules/logicapp.bicep' = {
name: '${rg.name}-logicapp'
scope: rg
dependsOn: [
appInsights
servicebus
cosmosdb
]
params: {
nameSpace: 'sb-${toLower(name)}'
appName: 'logicapp-${toLower(uniqueSuffix)}'
location: rg.location
appInsightsInstrumentationKey: appInsights.outputs.appInsightsInstrumentationKey
}
}

module cosmosdb './modules/cosmosdb.bicep' = {
name: '${rg.name}-cosmosdb'
module function './modules/function.bicep' = {
name: '${rg.name}-function'
scope: rg
dependsOn: [
appInsights
servicebus
cosmosdb
]
params: {
accountName: 'cosmos-${toLower(name)}'
appName: 'func-${toLower(uniqueSuffix)}'
location: rg.location
applicationInsightsConnectionsString: appInsights.outputs.appInsightsConnectionString
applicationInsightsInstrumentationkey: appInsights.outputs.appInsightsInstrumentationKey
}
}

module function './modules/function.bicep' = {
name: '${rg.name}-function'
module apim './modules/apim.bicep' = {
name: '${rg.name}-apim'
scope: rg
params: {
appName: 'func-${toLower(name)}'
apimServiceName: 'apim4-${toLower(uniqueSuffix)}' //testing, remove the # later - Soft Delete issue
publisherEmail: publisherEmail
publisherName: publisherName
location: rg.location
appInsightsLocation: rg.location
//functionKey: function.outputs.functionKey //update to secure string at some point
appInsightsName: appInsights.outputs.applicationInsightsName
functionName: function.outputs.functionAppName
}
}

module roleAssignmentAPIMSenderSB './modules/configure/roleAssign-apim-service-bus.bicep' = {
name: '${rg.name}-roleAssignmentAPIMSB'
module servicebus './modules/service-bus.bicep' = {
name: '${rg.name}-servicebus'
scope: rg
params: {
apimServiceName: apim.outputs.apimServiceName
sbNameSpace: servicebus.outputs.sbNameSpace
nameSpace: 'sb-${toLower(uniqueSuffix)}'
location: rg.location
}
dependsOn: [
apim
servicebus
]
}

module roleAssignmentFcuntionReceiverSB './modules/configure/roleAssign-function-service-bus.bicep' = {
name: '${rg.name}-roleAssignmentFunctionSB'
module cosmosdb './modules/cosmosdb.bicep' = {
name: '${rg.name}-cosmosdb'
scope: rg
params: {
functionAppName: function.outputs.functionAppName
sbNameSpace: servicebus.outputs.sbNameSpace
accountName: 'cosmos-${toLower(uniqueSuffix)}'
location: rg.location
}
dependsOn: [
function
servicebus
]
}

// module roleAssignmentAPIMSenderSB './modules/configure/roleAssign-apim-service-bus.bicep' = {
// name: '${rg.name}-roleAssignmentAPIMSB'
// scope: rg
// params: {
// apimServiceName: apim.outputs.apimServiceName
// sbNameSpace: servicebus.outputs.sbNameSpace
// }
// dependsOn: [
// apim
// servicebus
// ]
// }

// module roleAssignmentFcuntionReceiverSB './modules/configure/roleAssign-function-service-bus.bicep' = {
// name: '${rg.name}-roleAssignmentFunctionSB'
// scope: rg
// params: {
// functionAppName: function.outputs.functionAppName
// sbNameSpace: servicebus.outputs.sbNameSpace
// }
// dependsOn: [
// function
// servicebus
// ]
// }

module configurFunctionAppSettings './modules/configure/configure-function.bicep' = {
name: '${rg.name}-configureFunction'
scope: rg
params: {
functionAppName: function.outputs.functionAppName
cosmosAccountName: cosmosdb.outputs.cosmosDBAccountName
sbHostName: servicebus.outputs.sbHostName
deploymentRepositoryUrl: deploymentRepositoryUrl
deploymentBranch: deploymentBranch
sbConnString: servicebus.outputs.sbConnString
}
dependsOn: [
function
Expand All @@ -106,21 +142,28 @@ module configurFunctionAppSettings './modules/configure/configure-function.bicep
]
}

module configurAPIM './modules/configure/configure-apim.bicep' = {
name: '${rg.name}-configureAPIM'
module configurLogicAppSettings './modules/configure/configure-logicapp.bicep' = {
name: '${rg.name}-configureLogicApp'
scope: rg
params: {
apimServiceName: apim.outputs.apimServiceName
sbEndpoint: servicebus.outputs.sbEndpoint
logicAppName: logicApp.outputs.logicappAppName
cosmosAccountName: cosmosdb.outputs.cosmosDBAccountName
sbHostName: servicebus.outputs.sbHostName
deploymentRepositoryUrl: deploymentRepositoryUrl
deploymentBranch: deploymentBranch
sbConnString: servicebus.outputs.sbConnString
}
dependsOn: [
apim
logicApp
servicebus
cosmosdb
]
}

// Telemetry Deployment
@description('Enable usage and telemetry feedback to Microsoft.')
param enableTelemetry bool = true

var telemetryId = '69ef933a-eff0-450b-8a46-331cf62e160f-apptemp-${location}'
resource telemetrydeployment 'Microsoft.Resources/deployments@2021-04-01' = if (enableTelemetry) {
name: telemetryId
Expand Down
Loading