In Part I of this series, we saw the value in adding webhooks for managing builds at the project level. In this blog article, we will examine a thorough example for managing releases based on Jira and Bitbucket events. By the end, Jira webhooks will be configured to manage release life cycle and packages in releases, along with Bitbucket webhooks to manage package content.
Creating Webhook Providers
To start, we will create Bitbucket and Jira webhook providers. The Bitbucket script verifies a webhook by the user-agent header and by a token query parameter. Bitbucket user and password properties are required for our Bitbucket function. The Jira script evaluates a webhook by the user-agent header and IP address.
Bitbucket Provider
Match Script
LOG.fine("Evaluating Bitbucket for incoming message");
def match = false;
// validating based on token and user agent headers
def userAgent = HTTP_HEADERS.get('user-agent');
def token = QUERY_PARAMS.get('token');
if (token && userAgent)
{
LOG.fine("Using token ${token} and user agent ${userAgent}");
if (token.equals(BITBUCKET_TOKEN))
{
if (userAgent.toLowerCase().equals('bitbucket-webhooks/2.0'))
{
match = true;
}
}
}
LOG.info("Bitbucket provider is a match: ${match}");
return match;
Jira Provider
Match Script
LOG.fine('Jira - evaluating');
def matchHeader = HTTP_HEADERS.get(JIRA_MATCH_HEADER);
def ip = HTTP_HEADERS.get('x-forwarded-for');
if (ip && matchHeader)
{
if (ip.contains(JIRA_IP_BLOCK))
{
LOG.fine('Jira - IP match');
if (matchHeader.toLowerCase().equals(JIRA_HEADER_VALUE.toLowerCase()))
{
LOG.info('Jira - success!');
return true;
}
}
}
LOG.info('Jira - no match');
return false;
When a webhook is received by FlexDeploy, the provider match scripts are run based on the provider order until a script returns true. It’s likely that we’ll receive webhooks from Bitbucket most often, so Bitbucket is listed first in the provider list for better performance.
Creating Webhook Functions
Our next step is creating functions which will define what to do when we receive a webhook from our provider. Once a webhook is matched to a provider, it will be matched to a function or multiple functions by URI (excluding query parameters). You may choose to divide functions by event. For simplicity, I have one function for each provider.
Bitbucket Function Details
Bitbucket Function Script
The Bitbucket function takes care of managing packages at the file level. It will create a stream once a Bitbucket branch is created and inactivate the stream once the branch is deleted. We will never build from these streams, but they will be added so we can populate files from these streams before merging a pull request to a main branch. For each Bitbucket push, the associated package (package name = branch name) will be updated with all files in the push. Lastly, a build of this package will happen once a pull request for this branch is merged. Note that this function is expecting Bitbucket Cloud payloads.
// assuming FlexDeploy project name matches repository name
def projectName = PAYLOAD.repository.name;
def repoName = PAYLOAD.repository.full_name;
def projectId = FLEXDEPLOY.findProjectId(projectName);
if (PAYLOAD.pullrequest)
{
// pull request merged, build from destination branch
def sourceBranch = PAYLOAD.pullrequest.source.branch;
def destBranch = PAYLOAD.pullrequest.destination.branch;
// get package name from feature branch name
def packageName = sourceBranch.subString(sourceBranch.indexOf("/") + 1);
def projectPackage = new ProjectPackagePojo(projectId, sourceBranch, null);
def streamId = FLEXDEPLOY.findStreamId(projectId, destBranch);
FLEXDEPLOY.buildPackage(streamId, projectPackage);
LOG.setMessage("Building package ${packageName}");
}
else
{
// push event, manage streams and update packages
def branch;
def isNewBranch = PAYLOAD.push.changes[0].old == null;
def isDeletedBranch = PAYLOAD.push.changes[0].new == null;
def streamCache = [:];
if (isDeletedBranch)
{
branch = PAYLOAD.push.changes[0].old.name;
LOG.info("Branch ${branch} has been deleted. Inactivating associated stream for project ${projectName}");
FLEXDEPLOY.inactivateStream(projectId, branch);
LOG.setMessage("Inactivated stream ${branch}");
}
else
{
branch = PAYLOAD.push.changes[0].new.name;
LOG.info("Running BitbucketPush function: ${projectName}, ${branch}");
if (isNewBranch)
{
streamCache = tryCreateStreams(projectName, branch);
}
// find the changed files for the push and use to update package
def logs = BITBUCKET.getChangeLogs(PAYLOAD, BITBUCKET_USER, BITBUCKET_PASSWORD);
def streamId = findStreamId(projectId, branch, streamCache, isNewBranch);
// get package name from branch name
def packageName = branch.subString(branch.indexOf("/") + 1);
FLEXDEPLOY.updatePackage(projectId, packageName, streamId, logs);
}
}
// if this commit was on a new branch, get from cache, otherwise check on server
def findStreamId(projectId, streamName, streamCache, isNewBranch)
{
def streamId = null;
if (isNewBranch)
{
streamId = streamCache[projectId];
}
else
{
streamId = FLEXDEPLOY.findStreamId(projectId, streamName);
}
return streamId;
}
// this event is only received once (on creation of a new branch)
def tryCreateStreams(repoName, branch)
{
def allProjects = FLEXDEPLOY.findProjectsForChange(repoName, null, null);
def streams = [:];
for (def project in allProjects)
{
LOG.info(String.format("Creating stream %s on project %s", branch, project));
def streamId = FLEXDEPLOY.createStream(project, branch);
streams[project] = streamId;
}
return streams;
}
Jira Function Details
Jira Function Script
The Jira function is divided by event. When a sprint is created in Jira, a release is created in FlexDeploy. When a sprint is started/completed in Jira, the corresponding FlexDeploy release will be started/ended. Packages are created for each issue as issues are created. As issues are moved from sprint to sprint, the associated FlexDeploy package will be moved to the sprint’s release
// manage FD release based on Jira events
def event = PAYLOAD.webhookEvent;
LOG.fine("Processing event ${event}");
// create new release for created sprint
if (event.equals("sprint_created"))
{
def pipelineName = 'Simple Pipeline';
def sprintName = PAYLOAD.sprint.name;
def sprintId = QUERY_PARAMS.sprintId;
LOG.info("Creating Release for Jira sprint: ${sprintId}");
FLEXDEPLOY.createRelease(sprintName,pipelineName,PAYLOAD.sprint.self, "0 0/30 * 1/1 * ? *");
LOG.setMessage("Created Release ${sprintName}");
}
// start associated release
else if (event.equals("sprint_started"))
{
def sprintName = PAYLOAD.sprint.name;
FLEXDEPLOY.startRelease(sprintName);
LOG.setMessage("Started Release ${sprintName}");
}
// end associated release
else if (event.equals("sprint_closed"))
{
def sprintName = PAYLOAD.sprint.name;
FLEXDEPLOY.endRelease(sprintName);
LOG.setMessage("Ended Release ${sprintName}");
}
// create package for jira issue
else if (event.equals("jira:issue_created"))
{
def issueKey = QUERY_PARAMS.issueKey;
def projectName = QUERY_PARAMS.projectName;
def projectId = FLEXDEPLOY.findProjectId(projectName);
FLEXDEPLOY.createPackage(projectId, issueKey, "Package created from webhooks for Jira issue " + issueKey, null);
LOG.setMessage("Created package ${issueKey}");
}
// add package to release/remove package from release if issue was moved to/from sprint
else if (event.equals("jira:issue_updated"))
{
def issueKey = QUERY_PARAMS.issueKey;
def projectName = QUERY_PARAMS.projectName;
def projectId = FLEXDEPLOY.findProjectId(projectName);
def changeLogs = PAYLOAD.changelog;
if (changeLogs)
{
for (def item : changeLogs.items)
{
LOG.fine("Processing change log item: ${item.toString()}");
if ("Sprint".equals(item.field))
{
// remove package from release
if (item.from)
{
LOG.info("Removing package ${issueKey} from release ${item.fromString}");
def relProject = new ReleaseProjectsPojo(projectId, issueKey, false);
def targetRelease = item.toString.contains(',') ? item.toString.substring(item.toString.lastIndexOf(',') + 1) : item.toString;
FLEXDEPLOY.removeProjectsFromRelease(targetRelease.trim(),[relProject]);
LOG.info("Removed package ${issueKey} from release ${targetRelease.trim()}");
}
// add package to release
if (item.to)
{
LOG.info("Adding package ${issueKey} to release ${item.toString}");
def relProject = new ReleaseProjectsPojo(projectId, issueKey, false);
def targetRelease = item.toString.contains(',') ? item.toString.substring(item.toString.lastIndexOf(',') + 1) : item.toString;
FLEXDEPLOY.addProjectsToRelease(targetRelease.trim(),[relProject]);
LOG.setMessage("Added package ${issueKey} to release ${targetRelease.trim()}");
}
}
}
}
else
{
LOG.info('No change logs');
}
}
External Webhook Setup
Bitbucket
Now, let’s look at setup required for our repository in Bitbucket. The URL here should match whatever URI is listed in the webhook function in FlexDeploy. The URL should also contain the token query parameter if validating by token in the provider match script. I checked triggers repository push and pull request merged.
Jira
Similarly, we will have to set up webhooks in Jira. When creating a webhook, the first thing to configure is the URL. Again, the end needs to match the URI in the Jira webhook function, but we will also add query parameters here for issue webhooks which are used by the function. We will create webhooks to subscribe to the following events: sprint created, sprint started, sprint closed, issue created, and issue updated. Webhooks in Jira are defined globally, so we need to make sure we aren’t receiving webhooks more than needed. We can limit webhooks sent to only our project with JQL.
Try it Out
Now we are ready to test the new webhooks. First, I will create a sprint, add an issue to it, and start my sprint. As we go along, any webhooks received by FlexDeploy can be viewed from the webhook messages page. I can see the release was created and started and it contains the package for this issue.
Currently, the package is empty since we haven’t made any changes in SCM for this issue. Now, I can create a branch with a matching name and make changes to it. Once I push this change, the files will be added to my package, and any new files will automatically be populated.
After a pull request is merged for this branch, a build will be triggered. My release has a Snapshot Schedule which automatically creates a snapshot every half hour. As these snapshots are created, they will pick up on the most recent webhook build, so I don’t need to manage this.
We can see the entire sequence of received webhooks and more details about their execution on the webhook messages page.
This is just one example of using webhooks for release management, and can be customized to many use cases. All in all, this kind of automation could provide a number of benefits. It prevents some common mistakes of missing a file in the package, forgetting to populate a new file, or not adding a package to the release. It standardizes your software delivery process. Most importantly, it saves valuable time for teams to focus on work that’s important and cut down on time they spend doing repetitive tasks.
