OBS with Git/scmsync SCM bridge

How to exercise Git-backed OBS on build.opensuse.org

Introduction

The Open Build Service (OBS) has its own internal source code management (SCM) system. Together with the command-line client osc, the user-facing behavior is roughly equivalent to Subversion (SVN), wherein a working copy (local directory where you do work and which contains a (partial) copy of the sources) traditionally contains only data from one chosen revision at any one time, necessiting the use of the network if one wishes to navigate forth and back in the commit history.

The rest of this document is organized somewhat chronologically, which should help build understanding as why choices were made the way there were.

Related documentation

There is a bunch of other documents floating around which are either noteworthy, either because they are related, complementary, or outright confusing.

• RELATED: "SCM/CI Workflow Integration" looks at how one can trigger OBS builds in the context of Continuous Integration (CI). While the scmsync mechanism as discussed below could likely be used as a novel way to pull this off, this inai.de document is not looking at scmsync with a CI focus.
• COMPLEMENTARY: "SCM Bridge" is a high-level overview for what we will dive into in section "Synchronization from git" anyway.
• WARNING: scm-staging user guide. This document contains misleading information. See a section "Warning about scm-staging user guide" (much) further below. It is recommended to continue reading this inai.de document linearly, and visit the warning section at the end.

Synchronization to Git

There exists a mechanism (name??) which copies package sources from OBS to Git (just this direction). This synchronization service is not user-controllable, so just know that it exists. Packages from Factory are copied to https://src.opensuse.org/pool/pkgname. This does not happen for all packages for some reason, e.g. openSUSE:Factory/gfan has not been copied to https://src.opensuse.org/pool/gfan as of 2024-09-30.

The pool/pkgname repositories typically contain a single branch called factory. The delay for this service's synchronization service can be many weeks. There are no permissions set. Altogether, the pool/pkgname repositories have overall limited use.

Synchronization from Git

Because the OBSSCM system is entrenched everywhere, it is perhaps not easy to directly replace. Instead, the SCM Bridge / obs_scm_bridge / scmsync mechanism was added which can copy package sources from an arbitrary Git repository to an OBS package entity (again, just this very direction).

The software running next to other OBS server processes is called obs_scm_bridge. Configuration happens via an OBS package entity's metadata (osc meta pkg -e):

<package name="descent3" project="games">
<title/>
<description/>
<scmsync>https://src.opensuse.org/jengelh/descent3#master</scmsync>
<person userid="jengelh" role="maintainer"/>
</package>

The <scmsync> tag specifies the remote location. A branch may be specified using the URI fragment syntax, e.g. the “master” branch is referenced here. It is conceivable an arbitrary Git ref may be specified.

As soon as the scmsync tag is added, the package is watched by obs_scm_bridge. Commit synchronization runs asynchronously in the background. Whenever obs_scm_bridge detects a change in the Git repository, a new OBSSCM commit is created from the files in the referenced Git revision. The Git history in its own right is not transferred; for example, Git force pushes simply result in a new OBSSCM commit on-top. (Hypothesis: it makes a --depth=1 shallow Git clone, and uses that.)

The synchronization is usually quick to complete (a few seconds), but slow networks or huge repositories might delay the completion. If and when it is done, the OBSSCM history will have gained a new commit with a terse commit message of the form `[info=825f56ff6136e3c958676fb574b502d157b152c361e7dd43761752cb12e926d2]` (Git commit hash). The OBSSCM history can be viewed by the well-known `osc log` command:

$ osc log games/descent3
----------------------------------------------------------------------------
r4 | unknown | 2024-09-23 13:54:57 | 880e81b6fa73957de8f254d65997d709 | 1.6.0~g226.g616f921e |

[info=825f56ff6136e3c958676fb574b502d157b152c361e7dd43761752cb12e926d2]
...

880e81b6.. is the so-called srcmd5, kind of like an OBSSCM commit hash.

Another artifact of Git-backed OBS packages is that the OBS revisions contains an extra file, _scmsync.obsinfo.

Checking out git-backed OBS packages

Even after switching an OBS package to scmsync, the user should not stop using the osc command-line utility. Repositories downloaded with just Git are not osc-enabled and one cannot use commands like `osc r` without further arguments.

Furthermore, using `osc up` to update an existing working copy whose metadata has just been scmsync-enabled skips setting up the local Git repository. It is therefore recommended to discard working copies where a scmsync switch occurred and re-download with `osc co`.In git-backed working copies, the following commands no longer work, or have a replacement, or still work with caveats:

• osc branch, osc bco: The command is deactivated in favor of git branch — more on that in the next section.
• osc commit: Deactivated in favor of git commit.
• osc log: Shows the OBSSCM history. You mostly want git log now.
• osc sr: Works as before for submissions to Git-disabled OBS package entities. Keep in mind that this will be submitting the OBSSCM state, and no Git history. You would use osc sr when submitting from e.g. games/descent3 (develpkg) to openSUSE:Factory/descent3. (As of 2024-09-30, openSUSE:Factory is not git-backed, so sr is exactly right.)

Branching/Contributing

Git repositires on their own store just code/history. Unlike OBS package entities, there is no implicit build trigger and no conceptual build artifact storage defined. As a result, Git branches on their own do not lead to any implicit server-side builds being performed. But this is not necessarily a bad thing. Trivial changes like fixing up changelog spellos now no longer incur a mandatory build cycle in some home: project. A local build via `osc build` may also turn out to be faster — OBS workers are often configured with just 4-way parallelism, whereas locally, I can exercise all 32 threads at once.

CEFP: Checkout–Edit–Fork–Publish (TBD Title for a method 1)

My answer to: “How would I go about branching a package?”

1. osc co X11:Waylang/glslang && cd X11/Wayland/glslang
2. edit glslang.spec
3. Perform local build to test for sanity: `osc build`
4. `git add $changed_files`, `git commit -m "Whatever"`
5. Open the scmsync URL (https://src.opensuse.org/jengelh/glslang) in a browser and create a fork, e.g. at https://src.opensuse.org/contributor/glslang .
6. Because the default Git remote, called origin is almost always using the readonly URL portal, add the writable URL portal as another git remote: `git remote add my gitea@src.opensuse.org:contributor/glslang`
7. `git push my`
8. If and only if you want to do a server-side build:
   1. Vivify a dummy project to use for testing, e.g. `osc meta prj -e home:contributor:test`
   2. Vivify a package entity to use for testing, e.g. `osc meta pkg -e home:contributor:test/glslang`
   and add a <scmsync> line pointing to https://src.opensuse.org/contributor/glslang.
   3. Wait for build to succeed, do whatever
9. Open a pull request via browser

The order is not particularly important; one might as well create the fork before the first build:

FCEP: Fork–Checkout–Edit–Publish (Title for a method 2)

1. Fork at https://src.opensuse.org/jengelh/glslang
2. Vivify a dummy project to use, e.g. `osc meta prj -e home:contributor:test`
3. Vivify a package entity to use, e.g. `osc meta pkg -e home:contributor:test/glslang`
add a <scmsync> line
4. `osc co home:contributor:test/glslang && cd home/contributor/test/glslang`
5. `git remote add my gitea@src.opensuse.org:contributor/glslang`
6. edit glslang.spec
7. `git add` / `git commit` / `git push my`
8. Open a pull request via browser

Warning about scm-staging user guide

I will now elaborate on everything that is wrong with the that document, paragraph by paragraph. I will use games/descent3 as a placeholder package so you, the reader, can better imagine what I am talking about.

Unwritable repository

The document asks the maintainer to change the OBS package entity metadata (osc meta pkg -e X11:Wayland/glslang) and employ <scmsync>https://src.opensuse.org/pool/glslang#factory</scmsync>. This leads to the first group of issues:

The /pool/glslang repository does not have any write permissions. The maintainer is unable to perform git pushes. The maintainer is also unable to accept any pull requests made towards /pool/glslang.

Abolishment of development projects

Opening a pull request against /pool/glslang creates an OBS project entity by the name of e.g. devel:Factory:git-workflow:staging:contributor:glslang:1 and creates an OBSSCM submit-typed request (SR) from …glslang:1/glslang to openSUSE:Factory/glslang. This is the “will automatically get forwarded” part mentioned in the scm-staging user guide.

The maintainer(s) of X11:Wayland/glslang are not added to this SR in any way. In essence, develprjs/develpkgs are effectively abolished.

Now, if that was the intent of Git-backed packages, then we would immediately run into the next onset of rhetoric questions: Why do we still need X11:Wayland/glslang at all? Why can't we make openSUSE:Factory/glslang scmsync-point to src.opensuse.org instead?

Contributor guide

Though there were problems with the section "Maintainer guide", the contributor guide looks accurate though, and seems to mirror what I have written out in section "Branching/contributing", namely forking the URL that is in the scmsync line.

This subpage was last modified 2024-10-01 by ej • contact