How to exercise Git-backed OBS on build.opensuse.org

Introduction

The Open Build Service (OBS) has its own internal source code management (SCM) system. Together with the command-line client osc, the user-facing behavior is roughly equivalent to Subversion (SVN), wherein a working copy (local directory where you do work and which contains a (partial) copy of the sources) traditionally contains only data from one chosen revision at any one time, necessiting the use of the network if one wishes to navigate forth and back in the commit history.

The rest of this document is organized somewhat chronologically, which should help build understanding as why choices were made the way there were.

Synchronization to Git

There exists a mechanism [if I had the script's name I would use that name] which copies package sources from OBS to Git (just this direction). This synchronization service is not user-controllable, so just know that it exists. Packages in openSUSE:Factory get synchronized, somewhat infrequently, to https://src.opensuse.org/pool/pkgname. This is good news for the "offline browsing" usecase alluded to earlier. Packages which are already "git-ified" are exempt from the synchronization. The pool/pkgname repositories typically contain a single branch called factory. There are no permissions set. Altogether, the pool/pkgname Git repositories have overall limited use.

Synchronization from Git

Because the OBSSCM system is entrenched everywhere, it is not easy to directly replace. Instead, the SCM Bridge / obs_scm_bridge / scmsync mechanism was added which can copy package sources from an arbitrary Git repository to an OBS package entity (again, just this very direction).

The software running next to other OBS server processes is called obs_scm_bridge. Configuration happens via an OBS package entity's metadata (osc meta pkg -e):

<package name="descent3" project="games"> <title/> <description/> <scmsync>https://src.opensuse.org/jengelh/descent3#master</scmsync> <person userid="jengelh" role="maintainer"/> </package>

The <scmsync> tag specifies the remote location. A branch may be specified using the URI fragment syntax, e.g. the “master” branch is referenced here. It is conceivable that an arbitrary Git ref may be specified.

As soon as the scmsync tag is added, the package is watched by obs_scm_bridge. Commit synchronization runs asynchronously in the background. Whenever obs_scm_bridge detects a change in the Git repository, a new OBSSCM commit is created from the files in the referenced Git revision. The Git history in its own right is not transferred; for example, Git force pushes simply result in a new OBSSCM commit on-top. (Hypothesis: it makes a --depth=1 shallow Git clone, and uses that.)

The synchronization is usually quick to complete (a few seconds), but slow networks or huge repositories might delay the completion. If and when it is done, the OBSSCM history will have gained a new commit with a terse commit message of the form `[info=825f56ff6136e3c958676fb574b502d157b152c361e7dd43761752cb12e926d2]` (Git commit hash). The OBSSCM history can be viewed by the well-known `osc log` command:

$ osc log games/descent3 ---------------------------------------------------------------------------- r4 | unknown | 2024-09-23 13:54:57 | 880e81b6fa73957de8f254d65997d709 | 1.6.0~g226.g616f921e | [info=825f56ff6136e3c958676fb574b502d157b152c361e7dd43761752cb12e926d2] ...

880e81b6.. is the so-called srcmd5, kind of like an OBSSCM commit hash. Another artifact of Git-backed OBS packages is that the OBS revisions contains an extra file, _scmsync.obsinfo.

The games/descent3 package is now, for all intents and purposes, stored in, and to be managed with, Git. This has no immediate effect on openSUSE:Factory/descent3 or home:contributor/descent3, as sources can amicably be copied since they all still have associated OBSSCM copies in any case.

Checking out git-backed OBS packages

Even after switching an OBS package to scmsync, the user should not stop using the osc command-line utility. Repositories downloaded with just Git are not osc-enabled and one cannot use commands like `osc r` without further arguments.

Furthermore, using `osc up` to update an existing working copy whose metadata has just been scmsync-enabled skips setting up the local Git repository. It is therefore recommended to discard working copies where a scmsync switch occurred and re-download with `osc co`.In git-backed working copies, the following commands no longer work, or have a replacement, or still work with caveats:

osc branch, osc bco: The command is deactivated in favor of git branch — more on that in the next section.
osc commit: Deactivated in favor of git commit.
osc log: Shows the OBSSCM history. You mostly want git log now.
osc sr: Works as before for submissions to Git-disabled OBS package entities. Keep in mind that this will be submitting the OBSSCM state, and no Git history. You would use osc sr when submitting from e.g. games/descent3 (develpkg) to openSUSE:Factory/descent3. (As of 2024-09-30, openSUSE:Factory is not git-backed, so sr is exactly right.)

Branching/Contributing

Git repositories store just code/history. Unlike OBS package entities, there is no implicit build trigger and no conceptual build artifact storage defined. As a result, Git branches on their own do not lead to any implicit server-side builds being performed. But this is not necessarily a bad thing. Trivial changes like fixing up changelog spellos now no longer incur a mandatory build cycle in some home: project. A local build via `osc build` may also turn out to be faster than workers from build.opensuse.org, as they are often configured with just 4-way parallelism, whereas locally, I can exercise all umpteenth threads at once (e.g. 32).

CEFP: Checkout–Edit–Fork–Publish (TBD Title for a method 1)

My answer to: “How would I go about branching a package?”

osc co games/descent3 && cd games/descent3
edit descent3.spec
Perform local build to test for sanity: `osc build`
`git add $changed_files`, `git commit -m "Whatever"`
Open the scmsync URL (https://src.opensuse.org/jengelh/descent3) in a browser and create a fork, e.g. at https://src.opensuse.org/contributor/descent3 .
Because the default Git remote, called origin is almost always using the readonly URL portal, add the writable URL portal as another git remote: `git remote add my gitea@src.opensuse.org:contributor/descent3`
`git push my`
If and only if you want to do a server-side build:
1. Vivify a dummy project to use for testing, e.g. `osc meta prj -e home:contributor:test`
2. Vivify a package entity to use for testing, e.g. `osc meta pkg -e home:contributor:test/descent3`
3. Wait for build to succeed, do whatever
Open a pull request via browser

The order is not particularly important; one might as well create the fork before the first build:

FCEP: Fork–Checkout–Edit–Publish (Title for a method 2)

Fork at https://src.opensuse.org/jengelh/descent3
Vivify a dummy project to use, e.g. `osc meta prj -e home:contributor:test`
Vivify a package entity to use, e.g. `osc meta pkg -e home:contributor:test/descent3`

<scmsync>

`osc co home:contributor:test/descent3 && cd home/contributor/test/descent3`
`git remote add my gitea@src.opensuse.org:contributor/descent3`
edit descent3.spec
`git add` / `git commit` / `git push my`
Open a pull request via browser

Notes for maintainers

src.opensuse.org uses git-lfs. This has some implications for pull request analysis. On the command line, wishing to examine repository-foreign branches needs some extra steps. Consider this workflow:

$ osc co devel:tools/grpc && cd devel/tools/grpc/ $ git ls-remote origin ed54753dfc5efcea1f4f678be907436e7b76449d377b1060a469da9d2ea7c084 refs/pull/1/head $ git fetch origin refs/pull/1/head * branch refs/pull/1/head -> FETCH_HEAD $ git checkout FETCH_HEAD Downloading v1.68.2.tar.gz (17 MB) Error downloading object: v1.68.2.tar.gz (afbc5d7): Smudge error: Error downloading v1.68.2.tar.gz (afbc5d78d6ba6d509cc6e264de0d49dcd7304db435cbf2d630385bacf49e066c): missing protocol: "No such OID\n"

The act of making a merge request on collaborative forge software (such as GitHub, Gitea, etc.) copies the source-side commit into the target repository. This includes internal data structures like commits, trees and blobs. However, LFS objects seem to be exempt, thus the smudge error report. Anecdotally, this feels unusual, because if and when a merge request is accepted via the webinterface, Gitea does ultimately copy LFS objects. Anyway, to workaround the missing LFS objects prior to merge, one needs to explicitly retrieve LFS objects from the sending repository like so:

$ git remote add r2 https://src.opensuse.org/badshah400/grpc $ git lfs fetch r2 ed54753dfc5efcea1f4f678be907436e7b76449d377b1060a469da9d2ea7c084 $ git checkout FETCH_HEAD HEAD is now at ed54753 Update to version 1.68.2.

Warning about scm-staging user guide

I will now elaborate on everything that is wrong with the that document, paragraph by paragraph. I will use games/descent3 as a placeholder package so you, the reader, can better imagine what I am talking about.

Unwritable repository

The document asks the maintainer to change the OBS package entity metadata (osc meta pkg -e games/descent3) and employ <scmsync>https://src.opensuse.org/pool/descent3#factory</scmsync>. This leads to the first group of issues:

The /pool/descent3 repository does not have any write permissions. The maintainer is unable to perform git pushes. The maintainer is also unable to accept any pull requests made towards /pool/descent3.

Abolishment of development projects

Opening a pull request against /pool/descent3 creates an OBS project entity by the name of e.g. devel:Factory:git-workflow:staging:contributor:descent3:1 and creates an OBSSCM submit-typed request (SR) from …descent3:1/descent3 to openSUSE:Factory/descent3. This is the “will automatically get forwarded” part mentioned in the scm-staging user guide.

The maintainer(s) of games/descent3 are not added to this SR in any way and cannot intervene in case an evil/innocent user makes a malicious/stupid pull request. In essence, develprjs/develpkgs are abolished.

Contributor guide

Though there were problems with the section "Maintainer guide", the contributor guide looks accurate though, and seems to mirror what I have written out in section "Branching/contributing", namely forking the URL that is in the scmsync line.

OBS with Git/scmsync SCM bridge