changeset 4:0bf6d77bfb0b

more robust build instructions
author jrose
date Tue, 27 Jan 2009 19:15:47 -0800
parents 81daeabfb5ac
children 41bc9c25b6d0
files README.txt make/Makefile make/current-release.sh make/each-patch-repo.sh make/link-patch-dirs.sh
diffstat 5 files changed, 210 insertions(+), 39 deletions(-) [+]
line wrap: on
line diff
--- a/README.txt	Mon Sep 29 19:30:36 2008 -0700
+++ b/README.txt	Tue Jan 27 19:15:47 2009 -0800
@@ -47,9 +47,11 @@
 
 === Patch Guards
 
-The series file contains guard annotations for each patch.  Patches are guarded with the OpenJDK release tags that they apply to.  In this way, as the OpenJDK release advances, patches can be rebased independently from each other.  Patches must be rebased in a special-purpose "rebasing patch commit" which includes a change to the patch guard in the series file also.  Actual development must be placed in patch commits that are *not* rebasing.
+The series file contains guard annotations for each patch.  Patches are guarded with Mercurial revision hashes that they are known apply to.  (They may additionally be guarded with OpenJDK release tags or other tags.)  In this way, as the OpenJDK repositories advance, patches can be rebased independently from each other.  Patches must be rebased in a special-purpose "rebasing patch commit" which includes a change to the patch guard in the series file also.  Actual development must be placed in patch commits that are *not* rebasing.
 
-Each patch must have one or more positive guards, and each must be the tag of an OpenJDK build, such as "jdk7-b25".  If a patch is guarded by such a tag, it is guaranteed to apply, without rejects, to that particular OpenJDK build, and to build successfully.
+Each patch must have one or more positive guards.  At last one guard must be a twelve-digit hexadecimal mercurial revision hash, such as "7836be3e92d0".  If a patch is guarded by such a revision, it is guaranteed to apply, without rejects, to that particular OpenJDK build, and to build successfully.
+
+Other tags may be present, especially tags of OpenJDK builds, such as "jdk7-b25".  As it happens, "7836be3e92d0" and "jdk7-b25" refer to the same revision in the hotspot repository.  These symbolic tags are informational and approximate, and (being less accurate than hashes) do not guarantee clean application of patches.
 
 Each patch must have a negative guard which names that patch with a "slash" prefix.  This allows developers to control individual entries in the patch queue without editing the series file.  Editing the series file is risky, since it is under version control and shared by all developers.  By contrast, the guards file is not under version control.
 
@@ -59,14 +61,17 @@
 
 For normal development, the guards 'buildable' and 'testable' should be present in the guard file, as well as the OpenJDK release in use.
 
-Example series file entries:
-	anonk.patch #-/anonk #+jdk7-b25
-	meth.patch  #-/meth  #+jdk7-b25
+Example series file contents:
+	# base = f6c0827e5919 in http://hg.openjdk.java.net/jdk7/hotspot/hotspot
+	anonk.patch #-/anonk #+7836be3e92d0 #+jdk7-b25
+	meth.patch  #-/meth  #+7836be3e92d0 #+jdk7-b25
 
 The 'qselect' command can be used to control these patches:
-	hg qselect jdk7-b25  # select both patches, plus any other jdk7-b25 ones
+	hg qselect 7836be3e92d0  # select both patches
 	hg qselect jdk7-b25 /meth  # select anonk but not meth
 
+The first line of each series file must contain a twelve-digit hexadecimal number, which declares the base Mercurial revision for the patch series as a whole.  The script patches/make/current-release.sh will scan this revision automatically.  By convention, the first line of the series file can be a handy human-readable description of that base.  If it were missing, the hexadecimal number would be extracted from the tag on the first patch in the series.
+
 References:
 * more on guards: http://hgbook.red-bean.com/hgbookch13.html
 * patch rebasing procedures: http://www.selenic.com/mercurial/wiki/index.cgi/MqMerge
@@ -74,12 +79,16 @@
 
 === Multi-based Patches
 
+(This technique is untested; maybe we don't need it.)
+
 If a patch must be split so as to apply to several OpenJDK builds, the latest patch in the series must be a complete patch for the most recent build, and for each previous build, a temporary patches must simply track the relevant changes up to the most recent build, so as to make the newest patch apply correctly in all cases.
 
 For example, to support builds 25 and 28 behind build 30 two fixup patches are needed:
-	anonk.jdk7-b25-b30.patch #-/anonk #+jdk7-b25
-	anonk.jdk7-b28-b30.patch #-/anonk #+jdk7-b28
-	anonk.patch #-/anonk #+jdk7-b30 #+jdk7-b28 #+jdk7-b25
+	anonk.jdk7-b28.patch #-/anonk #+7836be3e92d0 #+jdk7-b25
+	anonk.jdk7-b30.patch #-/anonk #+c14dab40ed9b #+jdk7-b28
+	anonk.patch          #-/anonk #+d1605aabd0a1 #+jdk7-b30
+
+If you have build jdk7-b28, you must set the guards for all three revisions, so that all three patches are enabled.
 
 Normally this will not be necessary, unless the patch provides functionality which several other patches depend on, and those patches are in different stages of rebasing.
 
@@ -91,17 +100,31 @@
 	$ cd davinci
         $ hg fclone http://hg.openjdk.java.net/jdk7/hotspot sources
 	$ hg fclone http://hg.openjdk.java.net/mlvm/mlvm patches
+	$ (cd patches/make; gnumake setup)  # will probably fail
+	$ (cd patches/make; gnumake force)  # force "hg update" to fix
+	$ (cd patches/make; gnumake)
+	$ (cd patches/make; gnumake FORCE_VERSIONS=1) # include the force by default
+
+The "gnumake setup" command is likely to fail, if the source version of each sub-repository is not exactly the same as the version advertised in the first line of its series file.  If it fails, the "force" target cleans up by running "hg update" (a.k.a. "hg checkout") to force the repository back to the required revision.
+
+Instead of using the makefile, the following shell commands will work as well:
 	$ bash patches/make/link-patch-dirs.sh sources patches
 	+ ln -s ../../../patches/hotspot sources/hotspot/.hg/patches
 	+ ln -s ../../../patches/jdk sources/jdk/.hg/patches
 	$ ls -il patches/hotspot/series sources/hotspot/.hg/patches/series
 	(should be identical files)
-	$ guards="buildable testable $(sh patches/make/current-release.sh)"
-	$ sh patches/make/each-patch-repo.sh hg qselect --pop $guards
+	$ export davinci=$(pwd) guards="buildable testable"
+	$ sh patches/make/each-patch-repo.sh "hg qselect --pop $guards" \
+		'$(sh $davinci/patches/make/current-release.sh)'
+	$ sh patches/make/each-patch-repo.sh "hg qselect; hg qunapplied"
+	$ sh patches/make/each-patch-repo.sh "hg update -r" \
+		'$(sh $davinci/patches/make/current-release.sh)'
 	$ sh patches/make/each-patch-repo.sh hg qpush -a
 
-The last command may produce output about patches being skipped.  This is correct, because the setting of $guards will ensure that only buildable and testable patches will be applied.
+The last command may produce output about patches being skipped.  This is correct, because the setting of $guards will ensure that only buildable and testable patches will be applied.  It may also produce messages about the patches being already applied, in which case a non-zero exit status is normal.
+
+If you have not forced the repository to the expected base revision (via "hg update"), the last command may also produce output about failed patch application.  Such failures must be fixed in an ad hoc manner, as you merge the upstream changes with the patch.
+
+If you have forced the repository, the last command may produce a warning about "working directory not at tip".  This is normal.  It reminds you that you are working with an older version of the software.
 
 (To apply other sets of patches, adjust the guard settings and/or use qpush to refer to specific desired patches.  Do not make permanent edits to the series file, unless they reflect true changes of development status.)
-
-TO DO: Put the details for setup and build in patches/Makefile, with a simple build target.
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/make/Makefile	Tue Jan 27 19:15:47 2009 -0800
@@ -0,0 +1,87 @@
+# Copyright 2008-2009 Sun Microsystems, Inc.  All Rights Reserved.
+# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+#
+# This code is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 2 only, as
+# published by the Free Software Foundation.  Sun designates this
+# particular file as subject to the "Classpath" exception as provided
+# by Sun in the LICENSE file that accompanied this code.
+#
+# This code is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# version 2 for more details (a copy is included in the LICENSE file that
+# accompanied this code).
+#
+# You should have received a copy of the GNU General Public License version
+# 2 along with this work; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
+# CA 95054 USA or visit www.sun.com if you need additional information or
+# have any questions.
+
+RUN_FROM_DIR = ../..
+
+SH = ksh  # works everywhere
+
+GLOBAL_GUARDS = buildable testable
+
+# RELAX_CHECKS |= FORCE_VERSIONS
+RELAX_CHECKS$(RELAX_CHECKS) = $(FORCE_VERSIONS)
+
+# if (RELAX_CHECKS) disable check
+MAYBE_CHECK$(RELAX_CHECKS) = check
+
+# if (!FORCE_VERSIONS) disable force
+MAYBE_FORCE = force
+MAYBE_FORCE$(FORCE_VERSIONS) = #off
+
+all: setup $(MAYBE_FORCE) $(MAYBE_CHECK) patch build
+
+setup: link-patch-dirs mq-select-guards
+check: hg-check-versions
+force: hg-force-versions
+unforce: hg-unforce-versions
+patch: mq-push-patches
+build:
+	cd $(RUN_FROM_DIR); $(SH) patches/make/each-patch-repo.sh \
+		"(cd make; $(MAKE))"
+
+link-patch-dirs: $(RUN_FROM_DIR)/sources $(RUN_FROM_DIR)/patches
+	cd $(RUN_FROM_DIR); bash patches/make/link-patch-dirs.sh sources patches
+	@echo "# Should be identical files:"
+	cd $(RUN_FROM_DIR); ls -il patches/hotspot/series sources/hotspot/.hg/patches/series
+
+mq-select-guards:
+	cd $(RUN_FROM_DIR); $(SH) patches/make/each-patch-repo.sh \
+		"hg qpop -a; hg qselect $(GLOBAL_GUARDS)" \
+			"\$$($(SH) `pwd`/patches/make/current-release.sh)"
+	# report what happened:
+	cd $(RUN_FROM_DIR); $(SH) patches/make/each-patch-repo.sh 'hg qselect; hg qunapplied'
+
+mq-push-patches:
+	cd $(RUN_FROM_DIR); $(SH) patches/make/each-patch-repo.sh \
+		'(! hg qunapplied | read something) || hg qpush -a'
+
+hg-check-versions:
+	@echo "# If this command fails, consider setting RELAX_CHECKS=true or FORCE_VERSIONS=true."
+	cd $(RUN_FROM_DIR); $(SH) patches/make/each-patch-repo.sh \
+		hg diff --git -I src -r "\$$($(SH) `pwd`/patches/make/current-release.sh)" \
+		"> current-diffs.txt;" \
+		"if [ ! -s current-diffs.txt ]; then rm current-diffs.txt;" \
+		"else grep '^diff' current-diffs.txt; exit 1; fi"
+
+hg-force-versions:
+	cd $(RUN_FROM_DIR); $(SH) patches/make/each-patch-repo.sh \
+		hg update -r "\$$($(SH) `pwd`/patches/make/current-release.sh)"
+
+hg-unforce-versions:
+	cd $(RUN_FROM_DIR); $(SH) patches/make/each-patch-repo.sh "hg update"
+
+.PHONY: \
+	setup check force unforce patch build \
+	link-patch-dirs mq-select-guards mq-push-patches \
+	hg-check-versions hg-force-versions hg-unforce-versions \
+	# end
+
--- a/make/current-release.sh	Mon Sep 29 19:30:36 2008 -0700
+++ b/make/current-release.sh	Tue Jan 27 19:15:47 2009 -0800
@@ -1,6 +1,6 @@
 #! /bin/sh
 #
-# Copyright 2008 Sun Microsystems, Inc.  All Rights Reserved.
+# Copyright 2008-2009 Sun Microsystems, Inc.  All Rights Reserved.
 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
 #
 # This code is free software; you can redistribute it and/or modify it
@@ -23,14 +23,48 @@
 # CA 95054 USA or visit www.sun.com if you need additional information or
 # have any questions.
 
-# Report the last tag in sources/.hgtags.
+# Report the last tag in sources/.../.hgtags.
 # Hopefully, that will be the most recent one also.
+# Also report any hashcode in the first line of source/.../series.
 
-sources=${1-sources}
+command=${0##*/}
 
-sed -n < "$sources/.hgtags" '
-  s/#.*//; s/^  *//; s/  *$//
-  s/.* //p
-' \
- |
-tail -1
+hflag=
+tflag=
+
+while true; do
+  case $# in
+  0) break;;
+  esac
+  case ${1-} in
+    -h) hflag=$1;;
+    -t) tflag=$1;;
+    *)  >&2 echo "Usage: sh $command [ -h | -t ]"; exit 2;;
+  esac
+  shift
+done
+
+print_hash() {
+  d='[0-9a-fA-F]'; not_d='[^'${d#'['}
+  d12=$d$d$d$d$d$d$d$d$d$d$d$d
+  sed -n < .hg/patches/series "
+    s/^.*${not_d}\(${d12}\)/\1/
+    s/^\(${d12}\)${not_d}.*/\1/
+    /^${d12}$/p
+    q
+  "
+}
+
+print_tag() {
+  sed -n < .hgtags "
+    s/#.*//; s/^  *//; s/  *$//
+    s/.* //p
+  " | tail -1
+}
+
+case $hflag,$tflag in
+'','') print_hash;;
+-h,'') print_hash;;
+-h,-t) print_hash; print_tag;;
+'',-t) print_tag;;
+esac
--- a/make/each-patch-repo.sh	Mon Sep 29 19:30:36 2008 -0700
+++ b/make/each-patch-repo.sh	Tue Jan 27 19:15:47 2009 -0800
@@ -1,6 +1,6 @@
 #! /bin/sh
 #
-# Copyright 2008 Sun Microsystems, Inc.  All Rights Reserved.
+# Copyright 2008-2009 Sun Microsystems, Inc.  All Rights Reserved.
 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
 #
 # This code is free software; you can redistribute it and/or modify it
@@ -23,8 +23,15 @@
 # CA 95054 USA or visit www.sun.com if you need additional information or
 # have any questions.
 
+qflag=
+case ${1-} in
+-q) qflag=$1; shift;;
+esac
+
 sources=sources
 
+exit_status=0
+
 for d in $(cd "$sources"; echo *); do
   srepo=$sources/$d
   # Do not mention a repo if there is no 'series' file.
@@ -32,6 +39,14 @@
   case $# in
   0) echo "$srepo"; continue;;
   esac
-  >&2 echo "+ (cd $srepo; $@)"
-  (cd "$srepo"; "$@")
+  case $qflag in
+  '') >&2 echo "+ (cd $srepo; $@)";;
+  esac
+  (cd "$srepo"; eval "$@") \
+  || {
+    exit_status=$?
+    >&2 echo "*** Exit status $exit_status."
+  }
 done
+
+exit $exit_status
\ No newline at end of file
--- a/make/link-patch-dirs.sh	Mon Sep 29 19:30:36 2008 -0700
+++ b/make/link-patch-dirs.sh	Tue Jan 27 19:15:47 2009 -0800
@@ -1,4 +1,4 @@
-#! /bin/bash
+#! /bin/ksh
 #
 # Copyright 2008 Sun Microsystems, Inc.  All Rights Reserved.
 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
@@ -23,6 +23,11 @@
 # CA 95054 USA or visit www.sun.com if you need additional information or
 # have any questions.
 
+nflag=false
+case ${1-} in
+-n) nflag=true; shift;;
+esac
+
 sources=${1-sources}
 patches=${2-patches}
 case $# in
@@ -35,7 +40,6 @@
   exit 2
 }
 
-shopt -s extglob
 
 here=$(pwd -P)
 sources=$(cd "$sources"; pwd -P)
@@ -48,27 +52,35 @@
   # Do not link a repo if there is no 'series' file.
   [ ! -f "$prepo/series" ] && continue
   srepo=$sources/${prepo#"$patches/"}
-  [ ! -d "$srepo/.hg" ] && {
-    >&2 echo "*** Skipping: ${prepo#$here/}"
-    continue
-  }
-  [ -h "$srepo/.hg/patches" ] && {
-    # silently remove previous sym-links
-    rm -f "$srepo/.hg/patches"
-  }
-  [ -r "$srepo/.hg/patches" ] && {
-    >&2 echo "*** Already exists; please remove: ${srepo#$here/}/.hg/patches"; exit 2
+  $nflag || {
+    [ ! -d "$srepo/.hg" ] && {
+      >&2 echo "*** Skipping: ${prepo#$here/}"
+      continue
+    }
+    [ -h "$srepo/.hg/patches" ] && {
+      # silently remove previous sym-links
+      rm -f "$srepo/.hg/patches"
+    }
+    [ -r "$srepo/.hg/patches" ] && {
+      >&2 echo "*** Already exists; please remove: ${srepo#$here/}/.hg/patches"; exit 2
+    }
   }
   linkname="$prepo"
   [[ "$prepo" == "$here/"* && "$srepo" == "$here"/* ]] && {
     stepup=${srepo#"$here/"}/.hg
-    stepup=${stepup//+([^\/])/'..'}  # turn all path components to '..'
+    # turn all path components to '..'
+    ##bash-isms: shopt -s extglob; stepup=${stepup//+([^\/])/'..'}
+    stepup=$(echo "$stepup" | sed 's:[^/][^/]*:..:g')
     linkname=$stepup/${linkname#"$here/"}
     ( cd "$srepo/.hg"; [ -d "$linkname/." ] ||
         echo "*** Problem linking: $linkname" )
   }
   linkdest=$srepo/.hg
   linkdest=${linkdest#"$here/"}
+  $nflag && {
+    echo ln -s "$linkname" "$linkdest/patches"
+    continue
+  }
   [ -d "$linkdest" ] || >&2 echo "*** Problem naming: $linkdest"
   echo + \
   ln -s "$linkname" "$linkdest/patches"