Skip to content

Commit

Permalink
umpf: add built-in support for synchronizing topic branches
Browse files Browse the repository at this point in the history 
While umpf was explicitly developed with multiple developers adding
utags and sharing topic branches in mind, it is less than ideal when
there are multiple developers working on the same topic branches.

And his frequently leads to one of two issues:

  - The branch is pushed before the utag is accepted into the BSP
    repository and other developers get an unexpected addition to
    their umpf

  - The branch is not pushed after the utag is accepted into the BSP
    repository and other developers get an unexpected removal from
    their umpf

Every time this happens, it wastes a bit of time to identify what went
wrong and thus a solution built into umpf is appropriate:

  - CI will call umpf --remote=downstream --force $BSP/series.inc
    when a PR touching a useries is accepted

  - Developers will call umpf pull to synchronize their topic branches
    or to find out when difference they have to the now upstream
    version

Signed-off-by: Ahmad Fatoum <[email protected]>
  • Loading branch information
@a3f
a3f committed Jan 25, 2025
1 parent 10a5951 commit 9f35a0f
Show file tree
Hide file tree
Showing 3 changed files with 244 additions and 2 deletions.
2 changes: 1 addition & 1 deletion bash_completion
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ _umpf_completion()
"")
COMPREPLY=( $( compgen -W "${completion_cmds[*]} help" -- $cur ) )
;;
diff|show|tag|tig|build)
diff|show|tag|tig|build|push|pull)
local -a refs
refs=( $( compgen -W "$( git for-each-ref --format='%(refname:short)' refs/tags refs/heads refs/remotes)" -- $cur ) )
if [ ${#refs[@]} -eq 0 ]; then
Expand Down
41 changes: 41 additions & 0 deletions doc/getting-started.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,47 @@ Or tell umpf to rebase onto a new *umpf-base* when creating a fresh *utag*::
# umpf-topic-range: 8bae5bbec8cb4599c141405e9755b7c0e42e064f..19cdc2b857e662a38c712b41ce610000a5ddc6ae
# umpf-end

Synchronizing umpf topic branch
-------------------------------

Due to Git's distributed nature, checked out topic branches can get
out-of-sync. To compare local topic branches against those referenced
in a *utag*, ``umpf pull`` can be used::

umpf --dry-run pull 5.0/special-customer-release/20190311-1
umpf: Using series from commit message...
* [new branch] 02fb74aa381080855a57080138b29ecc96586788 -> v5.0/topic/most-fixes
! [rejected] f0693b782dd026f2adc4d3c336d9ac6dfb352a73 -> v5.0/topic/more-fixes (non-fast-forward)

Following options are supported:

- ``--dry-run``: compare the branches, but stop short of actually updating
them
- ``--force``: reset local branches that are not checked-out to the
``umpf-hashinfo`` in the ``utag``
- ``--update``: restrict updates to only branches available locally

The counterpart to publish topic branches to a remote after creating a new
``utag`` is ``umpf push``:

umpf --dry-run --remote=downstream push 5.0/special-customer-release/20190311-1
umpf: Using series from commit message...
To ssh://downstream
* [new branch] 02fb74aa381080855a57080138b29ecc96586788 -> v5.0/topic/most-fixes
! [rejected] f0693b782dd026f2adc4d3c336d9ac6dfb352a73 -> v5.0/topic/more-fixes (non-fast-forward)
error: failed to push some refs to 'ssh:/downstream'

It supports the same options as ``umpf pull``, but instead of doing local
changes, it operates on the specified remote.

``umpf push`` is especially useful when multiple developers are creating
`utags` for the same project in parallel. Each developer will initially
only push their `utag` to the common repository. Once the changes
introduced by a `utag` are accepted, all topic branches can be force
updated on the remote to this most recent `utag`, possibly via
a server-side pull-request post-merge hook running, e.g.::

umpf --remote=downstream --force .../linux/patches/series.inc

Overview
--------
Expand Down
203 changes: 202 additions & 1 deletion umpf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ PATCH_DIR="umpf-patches"
IDENTICAL=false
STABLE=false
FORCE=false
DRYRUN=false
UPDATE=false
VERBOSE=false
VERSION_SEPARATOR=-
Expand Down Expand Up @@ -178,6 +179,8 @@ usage() {
--nix with format-patch: write patch series nix
-h, --help
-f, --force
--dry-run with push/pull: Do everything except actually send
the updates.
--flags specify/override umpf-flags
-i, --identical use exact commit hashes, not tip of branches
-s, --stable create a 'stable' tag from a branch based on an
Expand All @@ -194,6 +197,7 @@ usage() {
specified, it's interpreted as
<topic>=[<remote>/]<topic>
-u, --update with --patchdir: update existing patches in <path>
with push/pull: update only existing branches
-v, --version <version> with tag: overwrite version number [default: 1]
Commands:
Expand All @@ -218,6 +222,8 @@ usage() {
build <umpf> build an umerge from another umpf
distribute <commit-ish> push patches not yet in any topic branch
upstream
push [<umpf>] push topic branches to the given remote
pull [<umpf>] pull topic branches into the local repository
continue continue a previously interrupted umpf command
abort abort a previously started umpf command
Expand Down Expand Up @@ -245,7 +251,7 @@ setup() {
fi

o="fhilsub:n:p:r:v:"
l="auto-rerere,bb,nix,flags:,force,help,identical,stable,update,base:,name:,patchdir:,relative:,override:,remote:,local,version:"
l="auto-rerere,bb,nix,flags:,dry-run,force,help,identical,stable,update,base:,name:,patchdir:,relative:,override:,remote:,local,version:"
if ! args="$(getopt -n umpf -o "${o}" -l "${l}" -- "${@}")"; then
usage
exit 1
Expand All @@ -271,6 +277,9 @@ setup() {
-f|--force)
FORCE=true
;;
--dry-run)
DRYRUN=true
;;
--flags)
FLAGS="${1}"
shift
Expand Down Expand Up @@ -1855,6 +1864,198 @@ do_distribute() {
run_distribute
}
### namespace: push ###
push_topic() {
echo "${content}" >> "${STATE}/topic-names"
}
push_hashinfo() {
echo "${content}" >> "${STATE}/topics"
}
push_release() {
echo "${content}" >> "${STATE}/tagname"
}
push_topic_range() {
[ ! -e "${STATE}/tagname" ] && return
[ -e "${STATE}/tagrev-flat" ] && abort "more than one 'topic-range' after 'release'!"
echo "${content##*..}" > "${STATE}/tagrev-flat"
}
### command: push ###
resolve_commitish() {
${GIT} rev-parse --revs-only "$@" 2>/dev/null
}
shorten_commitish() {
resolve_commitish --short ${1}
}
resolve_tag() {
local remote=$1 tag=$2 commit
if [ -n "$remote" ]; then
# handles conflicting tags on remote
commit=$(git ls-remote -q $remote refs/tags/$tag 2>/dev/null | \
sed 's/\s\+.*$//')
else
commit="refs/tags/$tag"
fi
resolve_commitish "${commit}^{}"
}
update_local() {
local success=false args="${1}"
local opts
${FORCE} && opts+="--force"
local line
while read -r line; do
local prefix="" suffix=""
eval set -- ${line}
[ ${#} -eq 0 ] && continue
local refparsed=$(shorten_commitish $3)
if [ -z "${refparsed}" ]; then
prefix=" * [new branch]"
elif [[ "${1}" = *~* ]]; then
prefix=" ${1}..$(shorten_commitish ${2})"
elif $FORCE; then
prefix=" + ${refparsed}..$(shorten_commitish ${2})"
suffix=" (forced update)"
else
prefix=" ! [rejected]"
suffix=" (non-fast-forward)"
fi
printf "%-40s %s -> %s%s\n" "$prefix" $2 $3 "$suffix"
done <<< "$args"
while read -r line; do
eval set -- ${line}
[ ${#} -eq 0 ] && continue
if $DRYRUN || git branch $opts $3 $2; then
success=true
fi
done <<< "$args"
$success || abort
}
do_push () {
local opts args remote
local -a branches branch_names
local -A topics
if [ -z "${GIT_REMOTE}" ]; then
info "Git remote must be specified. Cannot continue."
exit 1
fi
if [ "${GIT_REMOTE}" != "refs/heads/" ]; then
remote=${GIT_REMOTE%/}
fi
prepare_persistent push "${@}"
parse_series push "${STATE}/series"
local tagname="$(<"${STATE}/tagname")"
local tagrevf="$(<"${STATE}/tagrev-flat")"
mapfile -t branches < "${STATE}/topics"
mapfile -t branch_names < "${STATE}/topic-names"
if [ -n "${remote}" ]; then
# Needed, so git rev-parse below can check for existent branches
git fetch --quiet --no-tags ${remote} 2>/dev/null
fi
local rtagrev="$(resolve_tag "${remote}" ${tagname})"
local rtagrevf="$(resolve_commitish ${rtagrev}^)"
if [ "$tagrevf" != "$rtagrevf" ]; then
if [ -z "$rtagrevf" ]; then
abort "${remote}${remote:+/}refs/tags/$tagname not found"
else
abort "${remote}${remote:+/}refs/tags/$tagname" \
"has unexpected commit-ish $rtagrev"
fi
fi
for i in "${!branch_names[@]}"; do
local branch=${branch_names[$i]}
local rbranchrev="$(resolve_commitish "${GIT_REMOTE}${branch}")"
[ -z "${remote}" ] && [ "${branches[$i]}" = "${rbranchrev}" ] && continue
# Don't touch local branches that are already on the correct revision.
# For remote branches, we let git push handle it.
[ -z "${remote}" ] && [ "${branches[$i]}" = "${rbranchrev}" ] && continue
$UPDATE && [ -z "${rbranchrev}" ] && continue
topics[${branch}]=$(resolve_commitish ${branches[$i]})
done
if [ -n "${remote}" ]; then
${FORCE} && opts+="--force-with-lease"
${DRYRUN} && opts+="--dry-run"
for topic in "${!topics[@]}"; do
args+="${topics[$topic]}:refs/heads/${topic} "
done
if [ -z "$args" ]; then
info "No branches to push"
cleanup
return
fi
${GIT} push $opts ${remote} -- $args
else # local
for topic in "${!topics[@]}"; do
local ref=$topic rev=${topics[$topic]}
local oldval
# The old value being computed below is not needed to
# create the branch. We compute a suitable one anyway,
# so we can show how a ref's commit-ish has changed in
# the pull case like we do in the push case.
if git merge-base --is-ancestor $ref $rev &>/dev/null; then
local ancestors=$(git rev-list $rev ^$ref --count)
args+="$(shorten_commitish "$rev")~$ancestors"
elif $FORCE; then
args+="$(shorten_commitish "$ref")"
else
args+='""'
fi
args+=" $rev $ref"
args+=$'\n'
done
if [ -z "$args" ]; then
info "No branches to push"
cleanup
return
fi
update_local "$args"
fi
cleanup
}
### command: pull ###
do_pull () {
GIT_REMOTE=refs/heads/ do_push "$@"
}
### command: continue ###
do_continue() {
Expand Down

0 comments on commit 9f35a0f

Please sign in to comment.