1 – Introduction
Deploying in Salesforce is a non-negligible part of your project lifecycle, it can take quite a lot of time and effort.
Salesforce offers two main tools to support deployment activities. Change sets are very good for small one off deployments, and when you need a more robust solution for bigger and more frequent deployments, you’re usually better off using Force.com Migration Tool.
To reduce the time and effort required for each deployment, we started writing our own functionalities with ANT on top of the Force.com Migration Tool. Over time we ended up with a pretty useful framework that allows for easier and more flexible deployments.
In this article I’m going to present the deployment framework.
I assume that you are already familiar with Salesforce, the Force.com Migration Tool and ANT.
2 – Deployment Framework Overview
A typical deployment is composed of 3 main actions:
– Extract: extract from the source org the metadata to be deployed
– Tweak: tweak the extracted metadata if required
– Deploy: deploy the metadata in the destination org
The deployment framework is a set of batch files calling various ANT tasks that perform these 3 main actions.
The high level workflow for a typical deployment is as follow:
3 – Watch the Demo
You can watch the demo by clicking here: Coming Soon.
4 – Features Overview
4.1 – Extract
Extract is the first thing to do for a deployment. It consists of extracting from the source org (e.g. the DEV org) the metadata of the elements to deploy. This is pretty straightforward and all you need to do is specify which elements to include in the extract. For a list of all the metadata types you can extract/deploy with the Force.com Migration Tool, click here.
Once the extract is complete, you get a set of metadata files representing all the elements extracted.
4.2 – Specify Target Org
Deployments often require deploying the same thing to different target orgs (e.g. TEST, UAT, FULL, LIVE, etc). In order to simplify and automate deployments to multiple orgs, we added a functionality that lets you choose the target org you wish to deploy to.
All you need is to define all the available target orgs in the Deploy.build.properties file. You do this once, and then, every time you run a deployment, you’re asked to enter which target org you want to deploy to. It’s a much quicker and safer way to select amongst predefined target orgs than having to switch or modify the properties file every time. You can see how it works in the demo video.
4.3 – Tweak Metadata Files
It often happens that only partial elements from the source org should be deployed to a target org or that some values should be changed from one org to another. For instance, in DEV, you might have a few experimental fields in an object that shouldn’t be deployed anywhere else or some DEV values should be modified before being deployed to LIVE. The best solution for this is to extract the elements as is from the source org, and before deploying, to remove or change certain parts.
Removing or changing certain parts is done in what we call the tweak step. The tweak step is an ANT task that you can configure to perform the required changes on the metadata files before they’re deployed to the target org.
For instance, if in a specific object, you want to remove the field called TestField1, you would use an ANT RegEx to modify the contact metadata file. A lot of commonly used ANT tasks are already defined in the deployment framework to perform these type of changes. You can see how it works in the demo video.
4.4 – Backup Before Deploy
One very useful feature of the deployment framework is the ability to automatically backup elements from the target org before deploying new ones. This feature basically allows you to undo a deployment, something that is lacking in change sets and the Force.com Migration Tool.
Let’s say you’re deploying the Account object the Account layout. In your package.xml file, you have the list of elements to extract and deploy (i.e. Account and Account Layout). Just before deploying, the deployment framework will automatically perform an extract from the target org of all the elements referenced in the package.xml file. This will take a backup of the elements you’re about to deploy in the state they were before the deployment. You can see how it works in the demo video.
4.5 – Deploy
The deploy part is actually pretty straightforward. After all the extracted metadata files have been tweaked and a backup taken before deploying, all there is to do is to deploy the tweaked metadata files and optionally run some Apex tests.
This is done by either calling a generic ANT deploy task or creating a custom one if Apex tests are required.
4.6 – Backup Files with GIT
For additional safety/backup, the deployment framework can automatically push the metadata saved in the “Backup Before Deploy” step into a GIT repository. This is usually done only for deployments to a LIVE org.
4.7 – Deploy Data with Data Loader
For some deployments, you also need to deploy actual data along with the metadata. The deployment framework has the capability to deploy data using Salesforce data loader. This feature is discussed in another article.
4.8 – Modify Only Certain Components
One good thing about the deployment framework (or in this case the Force.com Migration Tool), is the flexibility you have to modify only certain elements in the metadata files (something you can’t do with change sets).
Let’s say you want to deploy the contact object and a custom visualforce page called contactTest.page. You extract the contact object and the contactTest page and deploy them to your target org. Now let’s say that later on, you want to deploy a modified version of the page but keep the current version of the contact object (maybe because someone is currently modifying it in DEV and hasn’t finalised the changes). You can simply do that by changing just the contactTest page in your extracted metadata files, and deploy it with the previous version of the contact object.
Of course you’d need to extract the contactTest page separately but the above scenario frequently occurs in real life so it’s good to know the feature is available.
5 – What’s in the Deployment Framework?
The deployment framework is composed of a set of batch files to launch processes (mostly ANT targets) and a set of config/xml files to configure the processes.
5.1 – Package.xml files
Package.xml files located in the 11-Extract\00-Extract_Package_XML folder and contain the list of elements to extract from the source org and deploy to a target org.
You need to create one package.xml file per deployment (e.g. extract.package.CRM_RELEASE_13.xml to extract and deploy filed for CRM Release 13).
The package.xml files’ content must follow the format understood by the Force.com Migration Tool. For more information, visit the Salesforce website: Constructing a Project Manifest or Sample package.xml Manifest Files
5.2 – Extract batch files
For each deployment, you need to create an extract batch file. Extract batch files are very simple. All they do is call the common extract batch file with some parameters (the common extract batch file is already written).
Below is an example of an extract batch file for a deployment called CRM_RELEASE_13
The common 00.b-Extract.bat file that will do the extraction for us i.e.
- Create a folder called Metadata_CRM_RELEASE_13 under the 11-Extract folder (see the tree structure below)
- Extract all the metadata files for CRM Release 13 and place them in the folder created above. The components to be extracted are defined in the package.CRM_RELEASE_13.xml file located in the …\11-Extract\00-Extract_Package_XML folder.
5.3 – Deploy batch files
For each deployment, you need to create a deploy batch file. Just like extract batch files, deploy batch files call a common deploy batch file with some parameters (the common deploy batch file is already written).
Below is an example of a deploy batch file for a deployment called CRM_RELEASE_13.
Deploy batch files call the common 00.c-Deploy.bat file that takes a bit more parameters than the extract one. In the example above, the parameters are as follow:
- /DeployName: The name of the deployment. Here it’s CRM_RELEASE_13
- /AntDeployTask: The name of the ANT task to run to deploy the metadata files
- /AntTweakTask: (Optional) The name of the ANT task to run to tweak the metadata files before deploying.
- /BackupBeforeDeployTweakTask: (Optional) The name of the ANT task to run to tweak the backup metadata files taken before the deployment.
The common 00.c-Deploy.bat file that will do the deployment for us i.e.
- Copy metadata files: Copy the extracted metadata files from the 11-Extract\Metadata_CRM_RELEASE_13 folder into the 12-Deploy folder (see the tree structure below)
- Ask for target org: Ask for the target org in which to deploy to (available target orgs are set up separately in a specific config file)
- Backup before deploy: If a backup before deploy is requested (e.g. for LIVE orgs), a backup of the deployed metadata files is taken from the target org.
- Backup before deploy tweak: If a Backup Before Deploy Tweak Task is specified, run it to tweak the backup metadata files extracted from the target org.
- Metadata files tweak: If a tweak ANT task is specified, run it to tweak the metadata files before deploying. Tweaks are specific to each deployment and/or target org. It can be things like removing certain fields or objects, changing certain values (e.g. emails), etc.
- Deploy metadata: Deploy all the (tweaked) metadata files to the target org.
- Commit backup metadata file to GIT: If requested, backup metadata files taken in the previous steps are committed to a GIT repository
5.4 – Build.xml files
There are 2 build.xml files, one for extracts and one for deploys. These files contain the definition of the Extract and Deploy ANT tasks.
The Extract build file (build.extract.xml) located in the 11-Extract folder is very simple and contains one ANT task called Extract. The Extract task is called from the common batch 00.b-Extract.bat and perform the extraction of metadata files by calling the sf:retrieve ANT task defined in the Force.com Migration Tool. You should never have to modify the extract build file.
The Deploy build file (build.deploy.xml) located in the 12-Deploy folder is a bit more complex and contains all the ANT tweak and deploy tasks. For each deployment, there is at least one deploy task and potentially several tweak tasks. All the deploy and tweak tasks are called from the common batch 00.c-Deploy.bat. You need to modify the deploy build file for each new deployment (e.g. CRM Release 13, CRM Release 14, etc).
The deploy tasks performs the deployment of (tweaked) metadata files by calling the sf:deploy ANT task defined in the Force.com Migration Tool. In each deploy task, you mainly have to define which parameters to use when calling sf:deploy (e.g. test level and which test you want to run, etc).
The tweak tasks are completely custom and dependent on each deployment requirements. However, they can use global ANT tasks for certain generic tasks (e.g. remove elements from metadata files, etc).
An example of the content of build.deploy.xml for CRM_RELEASE_13 is as follow:
5.5 – Build.properties files
There are 2 build.properties files, one for extracts and one for deploys. These files contain the definitions of global ANT variables used in the ANT processes (e.g. sf.username, sf.password, etc).
The extract properties file (11-Extract\Extract.build.properties) contains one set of global variables: sf.username, sf.password and sf.serverurl. They contain the connection credentials to the source org from where to extract the metadata (usually the main Development org).
The deploy (12-Deploy\Deploy.build.properties) contains one set of global variables per available target org. These variables contain mainly connection credentials to a target org. Their names are suffixed by the target org name for easy identification (e.g. sf.username.FULL, sf.username.LIVE, sf.password.FULL, sf.password.LIVE, ect). The deploy targets in build.deploy.xml use the variables matching the selected target org (i.e. sf.username.FULL and sf.password.FULL when deploying to FULL target org).
An extract of the content of Deploy.build.properties for the FULL org is as follow:
5.6 – Tree Structure
All the files composing the deployment framework are organised in the following tree structure.
Releases is the root folder that you can place anywhere on the hard disk. All the folders specified below are located within the Releases folder.
- 01-Additional Release Files
All additional files required for a deployment/release are located here. You can organise these files the way you want. One subfolder per release named with the deployment name / release name usually works well.
- 05-Release Batches
Folder where all the batch files are located.
It doesn’t directly contain any files, but only sub folders.
Contains all the common batch files used by all deployments (e.g. 00.b-Extract.bat and 00.c-Deploy.bat).
Contains all the test batch files (e.g. 03.a-Extract – TEST.bat)
- 01-Hot Fix
Contains all hot fix batches (created for each hot fix)
- 02-CRM Release
Folder for CRM release batch files. We separate CRM and Service Cloud releases but you don’t have to, as long as you create one subfolder per release.
- 13-Release 13
Contains batches for Release 13.
- 13-Release 13
- 03-SCI Release
Folder for Service Cloud release batch files.
Folder where all the extract files are located.
Folder where all package.xml files are located (they contain the elements to extract).
g. extract.package.CRM_RELEASE_13.xml is the file to extract CRM Release 13 files.
Folder where log files are automatically generated
When an extract is run, a folder called with the release/deployment name is automatically created under the 11-Extract MetaData_CRM_RELEASE_13 is the folder containing CRM Release 13 files.
Folder where all the deploy files are located.
Contains metadata files from a target org automatically backed up before a deployment.
- Custom Files
Contains custom files to be used for some deployments.
Folder where log files are automatically generated
When a deploy is run is run, the folder called with the release/deployment name is automatically copied from the 11-Extract folder into the 12-Deploy MetaData_CRM_RELEASE_13 is the folder containing CRM Release 13 files.
- 13-Data Loader
Folder where all the Data Loader files are located.
The data loader part of the deployment framework is covered in another article (to be published soon).
6 – Deployment Framework Source Code
If you’d like the source code for the deployment framework, just get in touch with us.
7 – Improvements
7.1 – Deployment History
It’s a work in progress for the moment but a good thing would be to store in the target org, a history of all deployments (e.g. deployment name, version number, date, deployed files, etc).
7.2 – Removing Objects
Some deployments require removing some objects from a target org. This is handled by what’s called destructive package files (destructive pre or destructive post deployments). The framework currently supports destructive package files but the functionality requires improvement because running the same deployment several time cause the destructive step to fail. What’s required is a way to remove elements only if they are present in the target org.
7.3 – Automatically Handling New Target Orgs
In the current version of the deployment framework, when you want to add a new target org, you need to modify the framework files in 2 places (in the Deploy.build.properties file and in the build.deploy.xml file). An improvement would be to only have to add the new credentials into the Deploy.build.properties file.
8 – Conclusion
As we’ve seen in this article, with a little bit of upfront effort, you can build a deployment framework that will cover most of your deployment needs.
This will save you a lot of time and effort in the long run.