Skip to content
Snippets Groups Projects
Unverified Commit 6a81aa2c authored by Issy Long's avatar Issy Long
Browse files

Maintainer's guide for Homebrew on Linux (linuxbrew-core) 

- There are a lot more manual steps in Homebrew on Linux (Linuxbrew)
  land than there are in Homebrew (core), it feels like.
- This is a first attempt at a maintainer's guide. I have basically
  tried to brain-dump everything that Michka and Shaun have taught me
  over the months into something consumable for future Homebrew on Linux
  maintainers!
- Some of it is overly verbose - the merging docs are taken from
  [Linuxbrew/homebrew-developer's docs](https://github.com/Linuxbrew/homebrew-developer/blob/master/docs/merge-homebrew-core.md)
  which were recently updated to reflect reality but probably don't need
  to have so many words: just some instructions. :-)
parent 2c0d25da
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs/Homebrew-linuxbrew-core-Maintainer-Guide.md 0 → 100644
+ 282
− 0
View file @ 6a81aa2c
# Homebrew linuxbrew-core Maintainer Guide
## Merging formulae updates from Homebrew/homebrew-core
Linuxbrew-core is a fork of Homebrew-core and, therefore, it has to periodically
merge changes made by Homebrew developers and contributors. Below we
describe the steps required to merge `Homebrew/homebrew-core` into
`Linuxbrew/homebrew-core`, possible conflicts and ways to resolve
them. Note, that instructions below have been written for a "clean"
environment and you might be able to skip some of the steps if you
have done them in the past.
### Preparation
First of all, we want to enable developer commands and prevent
automatic updates while we do the merge:
```bash
export HOMEBREW_DEVELOPER=1
export HOMEBREW_NO_AUTO_UPDATE=1
```
Once we've done that, we need to get access to the `merge-homebrew`
command that will be used for the merge. To do that we have to tap the
[`linuxbrew/homebrew-developer`](https://github.com/Linuxbrew/homebrew-developer)
repo:
```bash
brew tap linuxbrew/developer
```
Next, we have to navigate to the repository where we want to do the
merge and make sure that there are 3 remotes:
* a remote named `origin` pointing to Linuxbrew-core,
* a remote named `homebrew` pointing to Homebrew-core, and
* a remote pointing to your GitHub fork of Linuxbrew-core.
Remote names `origin` and `homebrew` are hard-coded in
`merge-homebrew`, while the remote pointing to your fork must be the
same as your GitHub username, as it will be used to submit a pull
request for the merge. Set the name to the `$GITHUB_USER` environment
variable, or let `hub fork` add a remote for you.
```bash
brew install hub
cd $(brew --repo homebrew/core)
git remote add homebrew https://github.com/Homebrew/homebrew-core.git
hub fork --remote-name=$GITHUB_USER
```
Now, let's make sure that our local branch `master` is clean and that
your fork is up-to-date with Homebrew/linuxbrew-core:
```bash
git checkout master
git fetch origin master
git reset --hard origin/master
git push --force $GITHUB_USER master
```
Strictly speaking, there is no need for `git reset --hard
origin/master` and simple `git merge origin/master` would have been
sufficient if you didn't mess with your local `master` branch.
However, hard reset makes sure that these instructions are correct
even if you did mess something up. The same is true for the `--force`
flag for the `git push` command above.
By default, the following command will attempt to merge all the
changes that the upstream Homebrew developers have made.
```bash
brew merge-homebrew --core
```
Merging all the changes from upstream in one go is usually
undesireable since our build servers will time out. Instead, attempt
to only merge 8-10 modified formulae.
`git log --oneline master..homebrew/master` will show a list of all
the upstream commits since the last merge, from oldest to newest.
Pick a commit SHA-1 that will merge between 8-10 formulae (16-20 commits
including bottles). Once you're satisfied with the list of updated
formulae, begin the merge:
```bash
brew merge-homebrew --core --skip-style <sha>
```
The `--skip-style` argument skips running `brew style`, which saves
time and in some cases avoids errors. The style errors can be fixed in
bottle PRs later in the process when CI flags them.
#### Simple Conflicts
Once you issue the above command, the merge will begin and in the very
end you will see the list of (conflicting) formulae that
`merge-homebrew` could not merge automatically:
```bash
==> Conflicts
Formula/git-lfs.rb Formula/gnutls.rb Formula/godep.rb
```
Note, that you can also get a list of unmerged files (*i.e.* files with conflicts) using:
```sh
git diff --name-only --diff-filter=U
```
Of course, conflicts will be different every merge. You have to
resolve these conflicts either manually in a text editor, or by using
tools like `diffuse`, `tkdiff`, or `meld`, some of which are available
from Homebrew. Frequently, conflicts are caused by the new versions
of macOS bottles and look like:
```ruby
<<<<<<< HEAD
sha256 "bd66be269cbfe387920651c5f4f4bc01e0793034d08b5975f35f7fdfdb6c61a7" => :sierra
sha256 "7071cb98f72c73adb30afbe049beaf947fabfeb55e9f03e0db594c568d77d69d" => :el_capitan
sha256 "c7c0fe2464771bdcfd626fcbda9f55cb003ac1de060c51459366907edd912683" => :yosemite
sha256 "95d4c82d38262a4bc7ef4f0a10ce2ecf90e137b67df15f8bf8df76e962e218b6" => :x86_64_linux
=======
sha256 "ee6db42174fdc572d743e0142818b542291ca2e6ea3c20ff6a47686589cdc274" => :sierra
sha256 "e079a92a6156e2c87c59a59887d0ae0b6450d6f3a9c1fe14838b6bc657faefaa" => :el_capitan
sha256 "c334f91d5809d2be3982f511a3dfe9a887ef911b88b25f870558d5c7e18a15ad" => :yosemite
>>>>>>> homebrew/master
```
For such conflicts, simply remove the "HEAD" (Linuxbrew's) part of the
conflict along with `<<<<<<< HEAD`, `=======`, and `>>>>>>>
homebrew/master` lines. Later, we will submit a request to rebuild
bottles for Linux for such formulae.
The `merge-homebrew` script will stage resolved conflicts for you.
#### Complex Conflicts
Of course, from time to time conflicts are more complicated and you
have to look carefully into what's going on. An example of a slightly
more complex conflict is below:
```ruby
<<<<<<< HEAD
if OS.mac?
lib.install "out-shared/libleveldb.dylib.1.19" => "libleveldb.1.19.dylib"
lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.dylib"
lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.1.dylib"
system "install_name_tool", "-id", "#{lib}/libleveldb.1.dylib", "#{lib}/libleveldb.1.19.dylib"
else
lib.install Dir["out-shared/libleveldb.so*"]
end
=======
lib.install "out-shared/libleveldb.dylib.1.19" => "libleveldb.1.19.dylib"
lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.dylib"
lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.1.dylib"
MachO::Tools.change_dylib_id("#{lib}/libleveldb.1.dylib", "#{lib}/libleveldb.1.19.dylib")
>>>>>>> homebrew/master
```
Note, that in the "HEAD" (Linuxbrew's) part we see previous code of
the Homebrew's formula wrapped in `if OS.mac?`. To resolve such a
conflict you have to replace the contents of `if OS.mac?` part up
until `else` with the contents of the bottom part of the conflict
("homebrew/master"). You also have to check if there are any obvious
modifications that have to be made to the `else` part of the code that
deals with non-macOS-related code.
#### Finishing the merge
Once all the conflicts have been resolved, a text editor will open
with pre-populated commit message title and body:
```text
Merge branch homebrew/master into linuxbrew/master
# Conflicts:
# Formula/git-lfs.rb
# Formula/gnutls.rb
# Formula/godep.rb
```
Leave the title of the message unchanged and uncomment all the
conflicting files. Your final commit message should be:
```text
Merge branch homebrew/master into linuxbrew/master
Conflicts:
Formula/git-lfs.rb
Formula/gnutls.rb
Formula/godep.rb
```
#### Submitting a PR
The `merge-homebrew` command will create a pull-request for you, using `hub`.
Once the PR successfully passes the tests and/or is approved by other
Homebrew developers, you can finalize the merge with:
```bash
brew pull --clean <PR-NUMBER>
git push origin master
```
The merge is now complete. Don't forget to update your GitHub fork by
running `git push your-fork master`
## Building bottles for updated formulae
After merging changes, we must rebuild bottles for all the PRs that
had conflicts.
To do this, tap `Linuxbrew/homebrew-developer` and run the following
command where the merge commit is `HEAD`:
```sh
for formula in $(brew find-formulae-to-bottle); do
brew build-bottle-pr --remote=$GITHUB_USER $formula
done
```
The `find-formulae-to-bottle` command outputs a list of formulae
parsed from the merge commit body. It also performs some checks
against the formulae:
And it skips formulae if any of the following are true:
- it doesn't need a bottle
- it already has a bottle
- the formula depends on macOS to build
- the formula's tap is Homebrew/homebrew-core (the upstream macOS repo)
- there is already an open PR for the formula's bottle
- the current branch is not master
If a formula you are expecting to bottle is skipped, there may be an
error; by default, this script won't output the errors. To see them,
run `brew find-formulae-to-bottle --verbose` separate to the `for`
loop above.
The `build-bottle-pr` script creates a branch called `bottle-<FORMULA>`, adds `# Build a bottle
for Linuxbrew` to the top of the formula, pushes the branch to GitHub
at the specified remote (default: `origin`), and opens a pull request using `hub
pull-request`.
## Common build failures and how to handle them
### Bottling errors
## Handling `brew bump-formula-pr` PRs
### Formulae that exist in Homebrew/homebrew-core
The `brew bump-formula-pr` command will raise PRs against the Linux
formula repo for upstream Mac formulae when running on Linux. This
isn't how version bumps are done for _most_ formulae. Until
[Homebrew/brew issue
6341](https://github.com/Homebrew/brew/issues/6341) is implemented - a
feature that will raise PRs against the correct repo for macOS
formulae bumps on Linux - we have to redirect users to raise their PRs
in Homebrew/homebrew-core:
> Thanks for your PR.
>
> However, this formula is not Linux-specific. Its new versions are merged from the [Homebrew/homebrew-core](https://github.com/Homebrew/homebrew-core) repository daily [as documented in CONTRIBUTING.md](https://github.com/Homebrew/linuxbrew-core/blob/master/CONTRIBUTING.md). Please submit this change as a PR to that repository.
>
> We look forward to your PR against Homebrew/homebrew-core for the next version bump!
### Linux-only formulae
If the formula is a Linux-only formula, it either:
- will contain the line `# tag "linuxbrew"`
- won't have macOS bottles
These formulae are fine for users to bump with `brew bump-formula-pr`,
but you should request that they remove the existing `x86_64_linux`
bottle SHA line so that CI will build a bottle for the new version
correctly. If the bottle SHA isn't removed, CI will fail with the
following error:
> `--keep-old` was passed but there are changes in `sha256 => x86_64_linux`
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment