README/manual¶

Kubuntu Automation, for obsoleting people [http://dilbert.com/strip/2015-12-09]

Configuration pre-requisites¶

Add the following to your .ssh/config and make sure you have an ssh-agent running and the key unlocked
```
Host tinami.kde.org
    User ftpubuntu
```

Make sure you have set this environment variables in your ~/.bashrc

export DEBEMAIL="user@mailhost.com"
export DEBFULLNAME="John Doe"

Packaging a new upstream release of KDE Frameworks¶

Edit ~/ka-metadata/versions.json and set the proper value

If you didn’t already, clone all the repositories with git-clone-all -r frameworks. This will create by default a layout of directories like this:

user@host:~/kubuntu-dev/frameworks$ tree -L 3
.
├── attica
│   └── git
│       └── debian
├── baloo
│   └── git
│       └── debian
├── bluez-qt
│   └── git
│       └── debian
├── breeze-icons
│   └── git
│       └── debian
├── extra-cmake-modules
│   └── git
│       └── debian
[...]

You may configure a different directory layout in your ~/.kubuntu-automation.conf file customizing the git-subdirectory, build-area and upload-area options.

Merge any KCI branch you consider appropiate ( origin/kubuntu_unstable, origin/kubuntu_stable, …). You can do it with:
```
user@host:~/kubuntu-dev/frameworks$ do-all git merge <branch>
```
Warning

If you are merging debian branches, please don’t do it with do-all git merge, see the section Merging from Debian branches below
Update the package metadata with
```
user@host:~/kubuntu-dev/frameworks$ ka-update-metadata -d <distribution> -r frameworks
```
Note

This metadata is going to be used later by other parts of Kubuntu Automation, such as the scripts which generate the build pages, or gbp-newrelease, so it’s very important to update it properly.

Among the metadata there is a build dependencies map which is going to be used for the build depends version bumping automatically done by gbp-newrelease.

If the Frameworks release in question needs an specific version of Qt you can update the Qt map first and then merge it into the frameworks map with
```
user@host:~/kubuntu-dev/frameworks$ ka-update-metadata -d <distribution> -r frameworks -m qt
```
Note

Remember to commit and push the resulting files to the ka-metadata git repository.
(OPTIONAL) If you want, before building the source packages, you can download all the orig tarballs in advance with
```
$ download-tarballs -r frameworks
```
If you don’t do this, any gbp-<something> command will download the tarball automatically if it wasn’t already downloaded.
Do the actual updating to the new upstream release
```
user@host:~/kubuntu-dev/frameworks$ do-all gbp-newrelease
```
This will make the appropiate changes and will build the initial version for the staging PPA.

If some source package fails to build, it will report it at the end of the do-all output. So, let’s say attica fails to build and we want to investigate the problem; we can do
```
user@host:~/kubuntu-dev/frameworks/attica$ gbp-ppa
```
to see again what failed, change things (such as updating patches) and build again and again with gbp-ppa until we fix the issue.

If at this point any source package failed to build because a patch doesn’t apply anymore, to fix that you might need to use quilt and unpack the upstream KDE source code, you can do this with:
```
user@host:~/staging/frameworks/kcoreaddons/git$ unpack-source
```
once you are done you can remove the upstream code with
```
user@host:~/staging/frameworks/kcoreaddons/git$ git clean -xdff
```
Hint

Be careful when doing git clean -xdff because if you have new, git untracked files under the debian/ directory it will remove them.

Once you get everything building fine, you will be ready to upload the source packages you have just created to the staging PPA’s
```
user@host:~/kubuntu-dev/frameworks$ do-all uploadsource
```

Push the changes to the git repositories with

user@host:~/kubuntu-dev/frameworks$ do-all git push

You can check the build status and health of the packages you uploaded to the PPA with ppa-build-status and ka-iron-hand

The first tool, ppa-build-status is meant to generate HTML pages with a general health status report. You can generate a report for the KDE Frameworks packages in the staging PPA with something like this:
```
user@host:~/kubuntu-dev$ mkdir build-status
user@host:~/kubuntu-dev$ cd build-status
user@host:~/kubuntu-dev/build-status$ cp /usr/share/kubuntu-automation/kubuntu-ppa-build-status.css .
user@host:~/kubuntu-dev/build-status$ ppa-build-status -v <version> \
-p ppa:kubuntu-ppa/staging-frameworks \
-d <dist> \
-r frameworks \
> frameworks_status.html
```
The second tool, ka-iron-hand is mostly meant to generate graphs, so we can have a good overview of everything and also if something fails to build we can see where the build process is stuck.

To use the ka-iron-hand you need to compose a ~/.ka-iron-hand.conf (you can use /etc/ka/ka-iron-hand.conf as template), once you have it you can just execute it from any directory
```
user@host:~$ ka-iron-hand
```
The execution of this tool may take several minutes and once it’s done, by default, it will place the generated reports in the ~/ka-iron-hand_reports/ directory.
If you need to retry builds you can do it with
```
$ kubuntu-retry-builds -d <distribution> -r frameworks
```
Note that Launchpad sometimes marks incorrectly as ‘Failed to build’ packages which should be actually in ‘Dependency wait’ status, if that’s the case you can pass --force to kubuntu-retry-builds so it would also retry the packages which failed to build and not just the ones in ‘Dependency wait’ status.

Warning

Please don’t abuse the --force option, use it only when it’s actually needed.

Prepare KDE Frameworks packaging for backports¶

Let’s see how to backport the packages we prepared in the previous section. Let’s supose cosmic is the current Ubuntu development release and we want to backport packages for bionic.

In a set of frameworks clones created with git-clone-all (see the second step of Packaging a new upstream release of KDE Frameworks):
- Checkout the corresponding kubuntu_bionic_backports branch.
- If the branches doesn’t exist in your local clones but they already exist in the git server you can create and checkout them with
```
$ do-all git checkout -b kubuntu_bionic_backports origin/kubuntu_bionic_backports
```
- If the branches doesn’t exist neither on your local clones nor in the server you can create and checkout them with
```
$ do-all git checkout -b kubuntu_bionic_backports kubuntu_cosmic_staging
```
  (But make sure the kubuntu_cosmic_staging branches are up-to-date first)
Merge the branches from the current development release (i.e. kubuntu_cosmic_staging) into kubuntu_bionic_backports with
```
$ do-all git checkout kubuntu_bionic_backports
$ do-all git merge --no-edit origin/kubuntu_cosmic_staging
```
Hint

Don’t miss the --no-edit option for git merge. Otherwise the editor may pop up for any and all of the dozens of packages you are working on.
Update the package metadata with
```
user@host:~/kubuntu-dev/frameworks$ ka-update-metadata -d bionic -r frameworks
```
Note

This is needed by the kubuntu-retry-builds script.

Build the source packages from that branches with

user@host:~/kubuntu-dev/frameworks$ do-all gbp-ppa -d bionic

Push the changes to the git repositories with

user@host:~/kubuntu-dev/frameworks$ do-all git push

Upload the source packages to the staging PPA with

user@host:~/kubuntu-dev/frameworks$ do-all uploadsource

You can check the build status and health of the packages you uploaded to the PPA with ppa-build-status and ka-iron-hand

The first tool, ppa-build-status is meant to generate HTML pages with a general health status report. You can generate a report for the KDE Frameworks packages in the staging PPA with something like this:
```
user@host:~/kubuntu-dev$ mkdir build-status
user@host:~/kubuntu-dev$ cd build-status
user@host:~/kubuntu-dev/build-status$ cp /usr/share/kubuntu-automation/kubuntu-ppa-build-status.css .
user@host:~/kubuntu-dev/build-status$ ppa-build-status -v <version> \
-p ppa:kubuntu-ppa/staging-frameworks \
-d <dist> \
-r frameworks \
> frameworks_status.html
```
The second tool, ka-iron-hand is mostly meant to generate graphs, so we can have a good overview of everything and also if something fails to build we can see where the build process is stuck.

To use the ka-iron-hand you need to compose a ~/.ka-iron-hand.conf (you can use /etc/ka/ka-iron-hand.conf as template), once you have it you can just execute it from any directory
```
user@host:~$ ka-iron-hand
```
The execution of this tool may take several minutes and once it’s done, by default, it will place the generated reports in the ~/ka-iron-hand_reports/ directory.
If you need to retry builds you can do it with
```
$ kubuntu-retry-builds -d bionic -r frameworks
```
Note that Launchpad sometimes marks incorrectly as ‘Failed to build’ packages which should be actually in ‘Dependency wait’ status, if that’s the case you can pass --force to kubuntu-retry-builds so it would also retry the packages which failed to build and not just the ones in ‘Dependency wait’ status.

Warning

Please don’t abuse the --force option, use it only when it’s actually needed.

Upload KDE Frameworks to the archive¶

First of all, let’s get our local git clones ready.

If you are re-using the same clones used for staging, you would just need to checkout the _archive branches with
```
user@host:~/kubuntu-dev/frameworks$ do-all git checkout kubuntu_<dist>_archive
```
You can of course start fresh with a new set of local clones, you just need to pass -b kubuntu_<dist>_archive
```
user@host:~/kubuntu-archive/frameworks$ git-clone-all -r frameworks -b kubuntu_<dist>_archive
```

Merge the _staging branges into the _archive ones

user@host:~/kubuntu-dev/frameworks$ do-all git merge --no-edit origin/kubuntu_<dist>_staging

Build the source packages to be uploaded with

user@host:~/kubuntu-dev/frameworks$ do-all gbp-archive

Then do the upload with

user@host:~/kubuntu-dev/frameworks$ do-all uploadsource -t ubuntu

And finally push the git changes …

user@host:~/kubuntu-dev/frameworks$ do-all git push

… and the git tags

user@host:~/kubuntu-dev/frameworks$ do-all git push --tags

You can check the build status and health of the packages you uploaded to the archive with ppa-build-status and ka-iron-hand

The first tool, ppa-build-status is meant to generate HTML pages with a general health status report. You can generate a report for the KDE Frameworks packages in the staging PPA with something like this:
```
user@host:~/kubuntu-dev$ mkdir build-status
user@host:~/kubuntu-dev$ cd build-status
user@host:~/kubuntu-dev/build-status$ cp /usr/share/kubuntu-automation/kubuntu-ppa-build-status.css .
user@host:~/kubuntu-dev/build-status$ ppa-build-status -v <version> \
-d <dist> \
-r frameworks \
> frameworks_status.html
```
The second tool, ka-iron-hand is mostly meant to generate graphs, so we can have a good overview of everything and also if something fails to build we can see where the build process is stuck.

To use the ka-iron-hand you need to compose a ~/.ka-iron-hand.conf (you can use /etc/ka/ka-iron-hand.conf as template), once you have it you can just execute it from any directory
```
user@host:~$ ka-iron-hand
```
The execution of this tool may take several minutes and once it’s done, by default, it will place the generated reports in the ~/ka-iron-hand_reports/ directory.
If you need to retry builds in the main archive you can do it with
```
$ kubuntu-retry-builds --archive -d <distribution> -r frameworks
```
Note that Launchpad sometimes marks incorrectly as ‘Failed to build’ packages which should be actually in ‘Dependency wait’ status, if that’s the case you can pass --force to kubuntu-retry-builds so it would also retry the packages which failed to build and not just the ones in ‘Dependency wait’ status.

Warning

Please don’t abuse the --force option, use it only when it’s actually needed.

Working with Plasma or KDE Applications packaging¶

The process for packaging new upstream releases, backporting or uploading to the archive for Plasma and KDE Applications is pretty much the same than the one explained in:

Packaging a new upstream release of KDE Frameworks
Prepare KDE Frameworks packaging for backports
Upload KDE Frameworks to the archive

you will just need to replace the word frameworks with plasma or applications. For instance, if we want to perform the second step of Packaging a new upstream release of KDE Frameworks (i.e. cloning all the repositories) we would need to execute git-clone-all -r plasma for Plasma or git-clone-all -r applications for KDE Applications.

Merging from Debian branches¶

You shouldn’t merge Debian branches doing git merge directly because:

Our Vcs-* fields are different than the ones from Debian packaging.
Our versioned Build-Depends might be different compared with Debian.
Our Standards-Version fields might be different compared with Debian.
Maybe some things from Debian should be kept out of our packaging. For instance, right now Debian is adding Breaks against all reverse dependencies for KDE Frameworks and Plasma packages, probably to avoid partial migrations by Britney to Debian testing. This is very inconvenient for us because:
- It pollutes the apt-get debug output when you do
```
$ apt-get -o debug::pkgproblemresolver=1 dist-upgrade
```
- It makes harder to package managers to find a solution for dist-upgrades increasing the risk of getting an apt-get dist-upgrade removing things it shouldn’t remove or not being able to upgrade or install packages.

So you could end up having unneccesary conflicts due to 1. 2. and 3. which you would have to resolve. You also might merge some undesired things like that one pointed out in 4.

To perform Debian merges we have some scripts to automate some tasks to do a proper merge, fixing some issues in advance.

How to do the merges with the KA tooling? we will show an example for one particular package. All the commands pointed out below must be executed in the git repository of the package in question.

Check the debian revision: in case debian has a higher version than our packaging the changelog merge won’t work properly, for instance if they had on top of their changelog the version 5.26.0-1 and we have the 5.26.0-0ubuntu1 their changelog entry would end up on top of the resulting changelog after the merge, so to avoid this you can bump the version of our top changelog block with:

$ bump-ubuntu-revision

This will create a commit on kubuntu_xx_archive like this:

$ git show
commit 8acef268219982c0b37ece31b51b4020b2ad6552
Author: José Manuel Santamaría Lema <panfaust@gmail.com>
Date:   Thu Oct 27 21:28:45 2016 +0200

    Bump ubuntu revision to -1ubuntu1 to make the merges from debian easier.

diff --git a/debian/changelog b/debian/changelog
index c4743f7..bb85aee 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,4 +1,4 @@
-extra-cmake-modules (5.27.0-0ubuntu1) UNRELEASED; urgency=low
+extra-cmake-modules (5.27.0-1ubuntu1) UNRELEASED; urgency=low

   [ Rik Mills ]
   * New upstream release (5.27.0)

Merge from the latest Debian tag with

$ ka-debian2kubuntu-merge

This will create a couple of temporary preprocessed branches with some commits on top of it, so if you have conflicts at this point you have two options:

a) Drop the merge to do it in another time
b) Fix the conflicts and go ahead with the merge

You can do a) this way

$ git merge --abort
$ ka-debian2kubuntu-clean # This will remove debian's temporary branches
$ git checkout kubuntu_<distribution>_archive
$ git branch -D kubuntu_<distributions>_debian-merge

You can do b) this way (once you fixed the conflicts)

$ git add <file1 with conflicts>
$ git add <file2 with conflicts>
$ git commit
$ dch "Merge with debian/master."
$ git commit -am "Update changelog."
$ ka-debian2kubuntu-clean # This will remove any temporary branch

If you are satisfied with your work so far, push and upload the changed package to the staging PPA

$ git checkout kubuntu_xx_archive
$ git merge kubuntu_xx_debian-merge
$ git push origin kubuntu_xx_archive
$ gbp-ppa
$ uploadsource

Finally you may want to remove the kubuntu_<distribution>_debian-merge branch and any other temporary branches

$ git branch -d kubuntu_<distribution>_debian-merge
$ ka-debian2kubuntu-clean

FAQ¶

I don’t have my PGP key in the host where I’m building my packages. How can I make gbp-<something> not sign the packages after building the source?
- You can use gbp-whatever -- -us -uc, the options after -- are passed to gbp buildpackage (which will be invoked by gbp-whatever to build the source package)
- You can configure gbp-<whatever> to not sign the packages having this in your ~/.kubuntu-automation.conf:
```
[gbp-* signing]
_default = off
```
  or
```
[gbp-* signing]
<whatever> = off
_default = on
```
  to have it signing by default but not signing only for gbp-<whatever>
How can I have a single upload directory instead of having one per package?
- Make sure you have this in your ~/.kubuntu-automation.conf
```
[areas]
upload-area = ../../upload
```
How to execute 2 commands with do-all? Typing do-all command1 && command2 doesn’t do what I expect…
- You can try this
```
$ do-all bash -c "command1 && command2"
```