Setting up Personal Packages and Deployments

Rational Team Concert 4.0.1 has released and with it comes a bunch of great new enhancements!  Packaging now has the ability to override the user requesting the package as well as build properties during package creation time.  With this its now possible to set up personal packages and then use the personal deployment capability that came with RTC 4.0.  In most customer environments you won’t just be able to enter your user id and password in the build agent overrides section and be good to go because files required are locked to you.  Here we will go through the steps required to get personal packaging and deployment up and running, with specific notes on the differences for IBM i and z/OS.

To begin the Package Definition needs to be modified to make certain values properties, so we can override them at package creation time.  At a minimum the Package root directory should be set to a property.  For example if you create a property package.root.dir with the value you currently have the package root directory set to, then in the Package Root Directory field enter: ${package.root.dir}

IBM i Note: Make sure to also set the IBM i Intermediate Save File Library to a property, or make sure that users who will run personal packaging have access to the library.

z/OS Note: Make sure all users have access to the ISPF Gateway script.

Now that the package definition is configured for personal packaging, you can bring up the Create Package wizard and create a personal package.  To do that in the Build Agent Authentication section fill in the details for your user id and password and in the Build Properties section override the properties with values for resources you have access to.  Then you can change the include/exclude ship lists and the restore mapping files to be what you want for your personal package and submit the request.

We now have a personal package created and want to deploy it.  In the Deployment Definition we need to make similar changes to what we did for the Package Definition.  This time for the Deployed Package Root Directory and Original Package Root Directory.  In addition if you are using FTP to transfer the package you need to make sure the user id specified has permissions to the original package root directory, or set the user id and password file fields as properties as well.

IBM i Note: User running deployment must have permissions to the RSTOBJ Command.  Best practice is to create a group profile, update the RSTOBJ command authority for the group profile to have *USE, and then for all users that should be allowed to use deployment have the group profile added to their user account.  Also in the Deployment Definition you may need to set the IBM i Intermediate Save File library as a property if you did in the Package Definition.

Once the Deployment Definition is configured, we can go about deploying the personal package we created earlier.  In the Request Deployment Dialog expand the Build Agent Authentication Override section and enter your user id/password.  In the Deployment Options section select the personal package you created and select Personal Deploy if you did not enter the restore mapping information when you created the package. Otherwise the information on where to restore the package to is already contained in the package.  As the final step expand the Build Properties section and fill in the properties with values for resources you have access to.  Make sure the original package root directory property matches the value you set when you created the package in the personal package request.  Click Submit and watch your personal package get deployed into your personal libraries/PDS’s.

Differences between IBM i Build Definitions explained

This blog post goes over the different build definitions specific for IBM i and the common scenarios they are used in.  When looking at moving Builds over to Rational Team Concert it can be a bit confusing to figure out which build definition template to use.

To start off, the build definitions are differentiated by the engines they use. Rational Team Concert has multiple engines, the ones supported by IBM i are the Jazz Build Engine and Rational Build Agent.  The Jazz Build Engine is used by our older build definitions, while our newer build definitions use the Rational Build Agent.  The Rational Build Agent is the build agent used by Build Forge, which means if you already are using it today you can reuse the same agent with RTC.

IBM i Continuous Load
The IBM i Continuous Load build definition uses the Jazz Build Engine.  This definition will not actually build(compile) anything.  It is used specifically for loading the contents of your Streams to some mapped Libraries on the IBM i.  This is used by some teams when first adopting RTC to allow them to reuse their old build/compilation process.  With this definition they can get the source out of the RTC Streams and run their own build/compilation process.  You can setup a schedule so the continuous load runs every 5 mins, every hour, or any time amount you desire and each time it runs it will load only what has changed since the last load.  Find out more in the InfoCenter.

IBM i Command
Like the IBM i Continuous Load definition this definition uses the Jazz Build Engine.  It is used by teams that have a certain command they run to build/compile their applications.  For example if you have a CL command you can use that with this definition.  You have the ability to have a command run for every changed member or to run one command for all members that have changed.  The options for what to load are to load the entire application each time or just load the changes since the last build.  Find out more in the InfoCenter.

IBM i Build Specification (Deprecated as of RTC 4.0)
The IBM i Build Specification build was the first build definition to use the Rational Build Agent.  This was the beginning of our builds to start doing dependency analysis.  It uses a Build Specification created in every IBM i Project that defines the commands to run for various inputs.  In addition to that we have scanners that run in the background to analyze certain dependencies, so we will rebuild for example all RPGLE members if the include file they use has changed.  We deprecated this style of build in RTC 4.0, as we introduced the IBM i Dependency Build.  Read below for details on it.  This does not mean we will remove the IBM i Build Specification build definition in the next release it just means we do not recommend it be used for dependency based builds, as the recommended approach is now the IBM i Dependency Build.  If you are currently using the IBM i Build Specification build you should start investigating the IBM i Dependency Build as it has more dependency analysis available in it and more.  For more details read the end-to-end article.

IBM i Dependency Build
The IBM i Dependency Build uses the Rational Build Agent.  This build definition is very similar to the z/OS Dependency Build definition as we are now using the same framework between IBM i and z/OS for our dependency builds instead of having separate technologies like we did before with the Build Specification.  This means when we do improvements you will likely see it for both platforms!  This style of definition uses System Definitions instead of a Build Specification for specifying how to compile your application.  More information on System Definitions can be found in this great article that explains it all.  This style of build is used by teams that want to take advantage of dependency analysis instead of having in-depth knowledge of various parts of their application and having to know when it needs to be recompiled.  Refer to this wiki page for details on what scanners are available and the types of dependencies they can find.

Additional Links:
Starting and Stopping the Jazz Build Engine
Installing and Configuring the Rational Build Agent

RTC Enterprise Extensions Useful Links

It can be hard to find the information you are looking for sometimes and over time we have built up a great list of bookmarks that we are constantly sending to everyone when they have questions or referring back to ourselves.  To make these bookmarks accessible to everyone we have started a Useful Links page, so everyone can easily find information they are looking for.  We have broken the page down into links by functionality.

This page is a work in progress, but if you find we are missing any links that you use and find helpful let us know by commenting and we will add it to the list.  We will be continuing to keep it updated as the team produces more content.

Auto Clean Demystified

The Auto Clean option available in the Package Definition is used to cleanup unwanted packages or packages that have already been deployed from the package root directory. This is done by deleting package results. When a package result is deleted it gets marked to delete the package from the package root directory, on the next create package request.

Steps for setting up and using auto clean:

1. The package definition includes an Auto Clean option in the Options tab that should be selected.
2. The packaging result associated with the package that needs to be cleaned should be deleted, thereby marking the actual package for deletion.
3. Under the default setting the user will have to wait up to 30 minutes after the package result is deleted and then on the next Create Package… request, the Auto Clean will take place and all marked packages will be deleted.

Note: This happens because the “Build Result Pruner Task Fixed Delay (seconds)” runs every 30 minutes by default and checks for any package results that have been deleted and marks those packages for deletion. If a Create Package is requested before the pruner task has run, the packages will not be deleted and a subsequent Create Package will have to be requested.

Additional information on Auto Clean

1. The 30 minute delay can be changed by changing the advanced property “Build Result Pruner Task Fixed Delay (seconds)” on the server in the Change and Configuration Management Advanced Properties page. Go to the Build System section and change the Current Value field of the Build Result Pruner Task Fixed Delay property and Click Save.
2. A failed package is automatically removed from the package root directory even if you do not select the Auto clean option.

Introduction to RTC Source Control for IBM i Users

One of the most common questions and confusion about RTC from new users is how the Source Control Management(SCM) works and where is the code base of my IBM i code in RTC.  Today IBM i developers are used to looking in libraries to find their source, but with RTC that is no longer the case as the official source is stored in the RTC database.  Lets walk through a typical scenario to understand how this all fits together.

• The team as a first and one time step takes your current source residing in your IBM i Libraries and using the tooling in Rational Developer for Power Systems converts it into 1 or many IBM i Projects.  An IBM i Project is a collection of the source in an Eclipse project style which mimics the structure of an IBM i Library.  Learn more about IBM i Projects by watching this video. By using an Eclipse project style to store your source there is no difference between the way this source is stored on your local machine as compared to a Java project, so all the powerful SCM capabilities in RTC apply to IBM i Projects as well like History, Annotate, Suspend, etc.  Once the projects are defined they are then shared with a stream.  At this point your official source code base is stored in the RTC Database not in any IBM i Libraries.  This is why you can have the RTC server running on any platform, it doesn’t have to be running on IBM i.
• As a developer I create a Repository Workspace, connected to the stream we shared the projects with in the previous step, in which I load IBM i Projects from to my local workstation.
• I work on the development of my IBM i source and use the capabilities in Rational Developer for Power Systems Software.  When I am ready to test my changes I use the push(load)/compile capabilities in Rational Developer for Power Systems or Personal Builds in Rational Team Concert to load and build(compile) the source in my own personal libraries.  When I am confident with my changes I deliver them to the stream. Notice the developer is not using the green screen or an emulator to modify the code, they are using Rational Developer for Power Systems Software for making all source code changes.
• A team build is run based on a schedule or someone requests a build (using one of the IBM i build definitions in RTC) that will load the source from the RTC Database to IBM i Libraries defined in the build definition and then compile the source.

To recap the official code base of your applications is stored in the RTC database, developers can use there own personal libraries to compile there changes, and official team builds are the mechanism to load the source from the RTC database to the IBM i Libraries and then compile(build) the application.  To learn more about the SCM capabilities in the RTC Eclipse client read the help topic on just that.

What’s new in RTC 4.0 from an IBM i Perspective

During the development for RTC 4.0 we remember a customer asking us what happened to RTC for IBM i?  We all were puzzled by the question as RTC 4.0 has the most new features for IBM i customers since the initial release of RTC on IBM i.  In a future blog post we will detail how to navigate the plan items for future releases to tell what is for IBM i or not to avoid this type of question/confusion.  We like to think of RTC 4.0 as the release where we dramatically closed the gap between z/OS and IBM i.  In previous releases we had for example Promotion capabilities and work item packaging for z/OS but not for IBM i.  With the 4.0 release we have closed the gap considerably and now have for IBM i along with z/OS the main three functionalities of: Dependency Build, Promotion, and Deployment.

Dependency Build
In past releases (RTC 2.0 and newer) we introduced the IBM i Build Specification build which is a dependency build, but did timestamp checking to determine everything that needed to be built then based on what was built determined what else needed to be rebuilt.  In RTC 4.0 we introduced the IBM i Dependency Build Definition template which works the same way the old Dependency Build template did for z/OS (now renamed z/OS Dependency Build).  No longer do we do timestamp checking but instead look directly at what has changed and the database of dependencies that have been scanned to determine what to build, which will result in much faster builds for all of you!  In addition we have added new scanners to look for dependencies in DDS, CL, and PF files.  New for both z/OS and IBM i is a new build report that is much enhanced from the previous build report in the IBM i Build Specification builds.  You will be able to see everything that was built, with the reason why it was built, and more.  To learn more about the new IBM i Dependency Build go to the Information Center, read an article, and/or watch a video on it.

Promotion
In previous releases you had to manually promote your source from one stream to another and then run builds or package/deploy to get the built binaries in that new environment.  It is now possible to create Promotion definitions to promote from one level to another.  These promotion definitions will promote your source and potentially your binaries to the new level in your environment.  As we have already gotten questions related to the Promotion capabilities we want to highlight some specifics.  Promotion definitions run on the same machine the Source build ran on, this means when you think of promoting binaries they are all on the same machine as the build.  If you want the binaries on a different machine you will want to refer to the Deployment capabilities after.  Robin Yehle has a great blog post talking about this from a z/OS perspective, but all the same logistics apply to IBM i.

Deployment
In previous releases we had sequential packaging and deployment, where the packages deployed had to be deployed in the sequence the packages were created. It is now possible to deploy packages in a non-sequential order by selecting the package to deploy. You can also deploy a package multiple times. New for IBM i is the introduction of the work-item based packaging feature. This includes packaging of one or more work items, thereby packaging the built output objects from the change sets associated with the work items. In addition, the UI for packaging and deployment has been completely changed. The ship list and restore mapping files can be now created in a new UI for the IBM i package definitions. Objects can be added manually to your ship list or a query can be run to find outputs from the Enterprise Dependency Build (IBM i)  build definitions. You can map libraries to the libraries to use during deployment in the package and deployment definitions. Lastly we still provide backwards compatibility for 3.0 and 3.0.1 versions. Package and Deployment definitions using 3.0.1 format can be successfully run using the 4.0 server and build system toolkit. To learn more about the new Deployment capability go to the Information Center.

Those are the highlights of the new functionality available specifically to IBM i users, but many more improvements went into RTC 4.0.  For a full list see the New and Noteworthy.