CloudIQ Webhook Integration: BigPanda Example

CloudIQ is a cloud-based AIOps application that utilizes machine learning and predictive analytics to provide intelligent insights for your Dell IT Infrastructure. By providing north bound interfaces such as Webhooks and REST API, CloudIQ  can integrate with other IT tools to assist in your autonomous operations journey.  This article demonstrates how to use CloudIQ Webhooks to integrate CloudIQ health notifications with BigPanda, the event management processing tool.  We will show how to create a REST API Integration in BigPanda and provide an example of intermediate code that processes the Webhook using Google Cloud Functions.

BigPanda Overview

BigPanda offers a solution that has a modern twist on event management process. The main product consists of a fully customizable Cloud Hosted event management console for event integration, reporting, correlation, enrichment, etc.

Webhook Overview

A CloudIQ Webhook is a notification that is sent upon a health issue change. Notifications are sent when an issue is detected or resolved. A Webhook is an HTTP post composed of a header and JSON payload that is sent to a user configured destination. Webhooks are available under the Admin > Integrations menu in the CloudIQ UI. Users must have the CloudIQ DevOps role to access the Integrations menu.

Webhook Event Details

The Webhook consists of data in Headers  and Payload . The Headers include control information; the Payload is a JSON data structure which includes the useful details for the notification and the health issue. Examples of the header and payload JSON files can be found here.

BigPanda Integration

In CloudIQ, we enable webhook integration by configuring a name, destination, and the secret to sign the payload.

In BigPanda, we have a couple of possibilities for 3rd-party integration.

1. BigPanda's Open Integration Hub allows users to implement mapping of elements from a graphical interface.
2. The raw REST API allows users to integrate with BigPanda programmatically.

In our example, we use the REST API. Note that some of the requirements of the Open Integration Hub (alert severity, configurable application key, etc.) are not configurable today in CloudIQ Webhooks.

Architecture

The main challenge when integrating CloudIQ health events to BigPanda alerts is implementing a mapping function to translate CloudIQ fields to BigPanda fields.

To do this, we will use a serverless function to:

• receive the health event from CloudIQ Webhook trigger
• convert the CloudIQ health event into a BigPanda alert
• post that alert to BigPanda

In this integration, the serverless function will be a Google Cloud Function. Any other serverless framework can work.

 

Create BigPanda REST Application

The first step is to create an application for integration in BigPanda. This can be done by doing the following: 

1. Login
2. Click on the `Integrations` button on the top of the console

3. Click on the blue "New Integration" button

   
4. Select "Alerts Rest API" (first card)

    

   
5. Set an integration name and click on the blue "Generate App Key" button

    

Save the generated App Key and Bearer token

 

 

Note that the “application key” and “token” will be needed later to configure the trigger to post data to that endpoint. If you forget to save the “application key” or “token”, you can obtain them later by clicking on "Review Instructions".

Create the GCP Cloud Function

This step is very similar to what has been presented in the CloudIQ to Slack tutorial. The only changes are that we are using a golang runtime and we store the authentication token in a secret instead of a plain text environment variable.

1. Select Create Secret from the Secret Manager.
2. Provide a name, BP_TOKEN in this example.
3. Paste the Authorization token from the HTTP headers section of the BigPanda integration into the secret value field.

    

   

    

    

4. Select Create Function and provide a function name.
5. Under the Trigger section, keep a trigger type of HTTP and select Allow unauthenticated invocations.
6. Take note of the Trigger URL because it will be used as the Payload URL when configuring the Webhook in CloudIQ.
7. Select SAVE.
8. Expand the RUNTIME, BUILD AND CONNECTIONS SETTINGS section.
9. Under the RUNTIME tab, click the + ADD VARIABLE button to create the following variable:
   BP_APP_KEY. The value is set to the application key obtained after creating the BigPanda integration.
   
10. Select the SECURITY AND IMAGE REPO tab.
11. Select REFERENCE A SECRET.
12. Select the BP_TOKEN secret from the pulldown.
13. Select Exposed as environment variable from the Reference Method pulldown.
14. Enter BP_TOKEN as the environment variable name.
15. Select DONE.
16. Select Next.
17. Select Go 1.16 from the Runtime pulldown.
18. Change the Entry point to CiqEventToBigPandaAlert.
19. Replace the code for function.go with the example function.go code.
20. Replace the go.mod with the example go.mod code.
21. Select DEPLOY.

Implement the Mapping

Using Go's static typing first approach, we have clearly defined `struct` for the input (`CiqHealthEvent`) and output (`BigPandaAlerts`).

Most of the logic consists of mapping one field to the other.

func CiqEventMapping(c *CiqHealthEvent, bp *BigPandaClient) *BigPandaAlerts {log.Println("mapping input CloudIQ event: ")log.Printf("%+v", c)alert := BigPandaAlerts{AppKey:  bp.AppKey,Cluster: "CloudIQ",Host:    c.SystemName,}if len(c.NewIssues) > 0 {for _, v := range c.NewIssues {alert.Alerts = append(alert.Alerts, BigPandaAlert{Status:             statusForScore(c.CurrentScore),Timestamp:          c.Timestamp,Host:               c.SystemName,Description:        v.Description,Check:              v.RuleID,IncidentIdentifier: v.ID,})}}return &alert}

Two things to note here:

1. CloudIQ doesn't have the notion of severity; therefore, we convert the score to a status using the code below.
2. CloudIQ has an event identifier that will help to deduplicate the alert in BigPanda or reopen a closed event in case of a re-notify.

// BigPanda status values: ok,ok-suspect,warning,warning-suspect,critical,critical-suspect,unknown,acknowledged,oksuspect,warningsuspect,criticalsuspect,ok_suspect,warning_suspect,critical_suspect,ok suspect,warning suspect,critical suspectfunc statusForScore(s int) string {if s == 100 {return "ok"} else if s <= 99 && s > 95 {return "ok suspect"} else if s <= 95 && s > 70 {return "warning"} else if s <= 70 {return "critical"} else {return "unknown"}}

Build

Behind the scenes, the GCP Cloud Functions are built and executed as a container. To develop and test the code locally (instead of doing everything in GCP Console), we can develop locally and then build the package using buildpack like GCP does :

pack build \--builder gcr.io/buildpacks/builder:v1 \--env GOOGLE_RUNTIME=go \--env GOOGLE_FUNCTION_SIGNATURE_TYPE=http \--env GOOGLE_FUNCTION_TARGET=CiqEventToBigPandaAlert \ciq-webhook-to-bigpanda-alert

Run

After the build is successful, we can test it with something similar to:

docker run --rm -p 8080:8080 -e BP_TOKEN=xxxxx -e BP_APP_KEY=yyyyy ciq-webhook-to-bigpanda-alert

Alternatively, you can create a `main.go` and run it with:

FUNCTION_TARGET=CiqEventToBigPandaAlert go run cmd/main.go

Deploy

Users can choose to deploy the function outside of the GCP console. You can publish it with:

cloud functions deploy ciq-webhook-to-bigpanda-alert --runtime go116 --entry-point CiqEventToBigPandaAlert --trigger-http --allow-unauthenticated

Configure CloudIQ

It is time to point the CloudIQ webhook to the GCP Function trigger URL. From the Admin > Integrations menu in CloudIQ, go to the Webhooks tab.

Click on Add Webhook.

Enter a Name for the Webhook.

Enter the Payload URL. This is the Trigger URL from the GCP Function.

As we did not utilize a Webhook secret, enter any text.

Click ADD WEBHOOK to save the configuration.

Testing

From CloudIQ

To ease the simulation of a webhook event, you can go to the Integrations menu in the CloudIQ UI and click on the "TEST WEBHOOK" button. This sends a ping request to the destination.

Another option is to go to the Integrations menu in CloudIQ and redeliver an existing event.

Easy Post Script

For an actual event and not just a "ping", you can use the "easy_post.sh" script after configuring the appropriate ENDPOINT.

Conclusion

If everything flows correctly, you will see the health alerts delivered to the BigPanda console.

 

References

https://docs.bigpanda.io/reference/alerts

https://cloud.google.com/functions/docs/concepts/go-runtime

https://developer.dell.com/apis/4138/versions/1.0/docs/Tutorials/02-webhook-integration-svcnow-slack.md