In this blog series, I will introduce FlexDeploy’s DevOps solution for Oracle E-Business Suite, walking through several use cases and deployment models. Using the crawling/walking/running metaphor, we will explore how FlexDeploy can support escalating DevOps maturity requirements. The challenges are real, but why should ERP teams be left in the cold?
- FlexDeploy Loves Oracle E-Business Suite: Series Overview
- DevOps Best Practices for Oracle EBS
- FlexDeploy Loves Oracle E-Business Suite: Crawling
- FlexDeploy Loves Oracle E-Business Suite: Walking
- FlexDeploy Loves Oracle E-Business Suite: Running!
In order to implement a highly effective DevOps solution it is critical that best practices are established, starting with the very core of the development practices. Without them, you are sub-optimizing the process and setting yourself up for disappointment in the end solution. In this blog article I will describe the best practices that are the common denominator of the most successful FlexDeploy implementations across our customer base.
Source Control Management
Yes, you need one!
One of the most critical components of a successful DevOps practice is to use s Source Control Management (SCM) system to host your source code and configuration. At its core, SCM keeps track of changes (revisions) made to your source code – when it was changed, who changed it, and even comments about why the particular change was made.
While maybe obvious to companies who use SCM, far too many organizations do not, and instead rely on network drives, backups of files, and backups of the backups. SCMs ensure that all revisions of every file is retained, have very efficient and reliable storage mechanisms, and provide tools to manage it all. All DevOps tools provide integrations with the most popular SCMs. FlexDeploy provides such integrations for Git, Subversion, Microsoft TFS, CVS, Perforce, and PVCS.
Version ALL Source Code
The next best practice is to place all source code under version control. Not most of it, not some of it, but all of it. That fact is, you will want full visibility into changes that were made, when, and by who, with ability to trace that across systems in which you deploy it. The revision number provides this data lineage.
For Oracle E-Business Suite, one practice we encounter frequently is to use the source stored in the Development environment as the master source. While convenient from a development standpoint, the downfall is that this information is not versioned. You will lose record of incremental changes over time, making it difficult to rollback changes or identify issues caused by a particular source code modification.
Develop Branching Strategies
A majority of organizations follow the best practice of using SCM, and a vast majority even follow the practice of source controlling all of their source. The one fairly common practice we encounter from customers is not employing an effective branching strategy. Branching is a powerful feature of every SCM, and is used to manage different development tracks. It is paramount in today’s reality of concurrent release development and hotfixes. In fact, DevOps embraces and encourages agile and continuous delivery methodologies.
In a nutshell, a branch is a “copy” of the source code. SCM is aware of the revision the branch was created from and provides tools for merging changes between them. This is the most effective way to support parallel development paths, which is a reality in every organization. Without branching, you sub-optimize your process by either holding on development for a release until the other one completes, or are faced with the nightmare of keeping track of specific revisions which should be deployed for a particular release. This often gets managed with complex spreadsheets and highly manually processes.
One of the most common reasons I have heard from customers that they do not employ branching is because they either do not understand it, or are intimidated by merging and resolving conflicts. In the end, working through any complexities and educating the development staff is worth the investment, as it provides far greater advantages over its alternatives.
Another reason customers do not employ branching roots in the fact that many SCMs have documented practices of “branch when needed” or “always branch”. Those who have adopted “branch when needed”, have along the way incorrectly determined that it was not needed, and rather employed manual circumvention around its downfall. In 2019, Git has become the most popular SCM in the market. Git embraces the “always branch” methodology, and tools such as Atlassian’s Jira and BitBucket provide features which have helped users adopt it.
Organize with Purpose
Don’t let your SCM repository look like the “My Documents” folder on my laptop! There should be a method to the madness.
Start by organizing files for related systems into their own repositories. For large organizations, avoid creating a single enterprise repository which hosts everything, to avoid running into storage related issues later.
Next, create a folder structure which makes it easy for everyone with a vested interest to locate the source for a particular system and application. For Oracle E-Business Suite, source control your customization files in a structure that closely resembles your Custom TOP. This not only makes it easy to locate and correlate the files with the EBS server, but enables automation for deployment. FlexDeploy recommends a source control structure as follows:
/EBS
/<Logical Instance Name> (e.g. GlobalEBS)
/<TOP Name> (e.g. XXAP)
/import (contains .ltd files for AOLs)
/java (contains XML and Java files for OAF)
/forms
/reports
/sql
etc. (folders for other object types)
Note: For Subversion, place the trunk/branch/tags folders under the <TOP Name> folder for each TOP.
Use SCM Commit Messages
Commit messages are indispensable when trying to establish the cause of a particular issue. You don’t need to write a novel, or even a paragraph. Simply provide a small amount of context to identify what changed. DevOps tools like FlexDeploy provide insight into the revisions that are part of a build, and are ultimately deployed across your instances. Commit messages provide context, making it easier to facilitate code reviews, approvals, and aid in root cause analysis of inevitable issues. Annotating your commit messages with references to issue/defect number can also provide valuable context, and even enable integration with other tools (e.g. Jira Issue #).
Build/Packaging Paradigms
Build and Deployment automation is a core tenant of any DevOps solution. When building a web application, the natural build artifact is an Enterprise Application Archive (EAR) file or a Web Application Archive (WAR) file. But what should the output of a build consist of for an ERP, like E-Business Suite. As described in the introduction for this blog series, an ERP is a bit different due to the ad hoc nature of what is being deployed. There is no concept of an application, but rather a collection of some subset of the customization files.
An effective approach is to collect those customization files during the build and organize them into what FlexDeploy terms a package. A package can then be versioned, deployed, and tracked across any number of target instances. We will dive into the specific FlexDeploy implementation in the next three blog articles in this series.
Continuous Integration & Continuous Delivery
Continuous Integration is a development practice that requires developers to integrate code into a shared repository several times a day. CI is enabled by web-hooks or periodic polling of the SCM to notify/identify when the source code changes, triggering an automated build. This practice is essential to identifying problems early and giving immediate feedback to the development team. The sooner an integration issue is identified, the easier it is to resolve it, since changes are fresh in the developers mind and the number of changes since the last build/deployment are smaller.
Continuous Delivery is the ability to get changes of all types, into production, and into the hands of users, safely, quickly, and in a sustainable manner. The goal is to make deployments predictable, routine, and with minimal overhead such that they can be performed on-demand. Whether releasing changes daily, weekly, monthly, or quarterly, the principles are the same. The more frequent the delivery, the more critical it is to have automation in place. For example, if releasing daily or weekly it is impossible to run hundreds of test cases manually, or be afforded lost time due to breakdowns in email communication and other orchestration. This is especially true when much of the testing and orchestration falls on the same resources which are responsible for the development.
Builds and Deployments can occur dozens or hundreds of times from the start of development through when the changes are deployed to production. This typically occurs across a series of environments for the purpose of integration testing, performance testing, user acceptance testing, and ultimately delivered to the end user in production.
This iterative process is a large overhead for teams when performing builds and deployment manually. A pipeline is a DevOps construct which orchestrates the propagation of changes across the environments, injecting approvals, automated testing, and other process steps. In the next three blog articles we will explore how FlexDeploy enables continuous integration, and pipelines for continuous delivery, empowering IT teams to focus on innovation and remove the roadblocks to faster delivery.
Another facet to continuous delivery is automating as much of the process as possible. Sometimes manual steps cannot be avoided, but they should be orchestrated within the pipeline. For Oracle E-Business Suite, examples of steps that can be automated beyond core build and deploy include ADOP phases for online patching and server restarts.
Configuration by Convention
When replacing manual steps with automated ones, you are often faced with challenges of being able to derive information to make effective decisions. The tendency can be to add volumes of metadata to the process to facilitate the orchestration. This metadata management itself can add overhead. One effective technique is to follow consistent standards, such that assumptions can be made, and values can be derived. Examples include consistent directory structures and naming conventions. Another example would be deriving the pipeline you will use based on the naming convention of the SCM branch.
Tools Integration
There are many dimensions to an effective DevOps solution, including Source Control Management, Test Automation, Change Management, Issue Tracking, build and deployment automation, and CI/CD. Leveraging and integrating a core set of tools together, playing to their strengths, is essential. Examples include Git/Subverson for source control management, Jira for agile development, ServiceNow for change management, and Oracle ATS or HP Unified Functional Test for test automation. As a leading build, deployment, and release automation solution, FlexDeploy offers integration with these tools and more.
Summary
A high performing DevOps solution that meets the demands of the business and all stakeholders does not just happen. If requires thoughtful design which includes representation of all stakeholders, reflection on positive and negative past experiences, a desire to strive for greatness, and implementing of the right tooling. In this article I shared some of the best practices when implementing an effective DevOps practice for Oracle E-Business Suite. In the next three articles we will dive into the details of a winning approach for DevOps (EBS style), exploring basic and advanced techniques using FlexDeploy.
Previous Post: FlexDeploy Loves Oracle E-Business Suite: Series Overview
Next Post: FlexDeploy Loves Oracle E-Business Suite: Crawling
