fetch from merge directive to stacked branch unable to fill in chk pages

On 15 February 2011 00:19, David I <email address hidden> wrote:
> Public bug reported:
>
> I'm getting a problem using bzr 2.3.0 with the sequence
>
> bzr branch --stacked <patch file>
> bzr merge <patch file>
>
> while the same sequence works if I leave out the --stacked.

Is this a new problem in bzr 2.3.0, or can you reproduce it using
whatever you were running before? (And which version was that?)

When you say <patch file> do you mean you're trying to branch
--stacked from a merge proposal? If so, that's very creative.

Martin Pool wrote:
> On 15 February 2011 00:19, David I <email address hidden> wrote:
>
>> Public bug reported:
>>
>> I'm getting a problem using bzr 2.3.0 with the sequence
>>
>> bzr branch --stacked <patch file>
>> bzr merge <patch file>
>>
>> while the same sequence works if I leave out the --stacked.
>>
>
> Is this a new problem in bzr 2.3.0, or can you reproduce it using
> whatever you were running before? (And which version was that?)
>
> When you say <patch file> do you mean you're trying to branch
> --stacked from a merge proposal? If so, that's very creative.
>
>

Good morning Martin,

First a reminder of the history of bug 375013

period 1) no stacked branches
milestone 1) Support for stacked branches implemented
period 2) stacked branches used.
milestone 2) bug 375013 found with commits to stacked branches,
'workaround' code introduced to disallow such commits. (about 2 years ago).
period 3) stacked branches have limited value (for me). Normal branches
used.
milestone 3) bug 375013 marked as solved and fix released in 2.3.0.
today) try to use the same workflow that worked in period 2 --> bug 718723!

To answer your 2 questions:

1) In period 3 it was not possible to commit to a stacked branch. The
sequence I described would have continued with a commit if it had
succeeded. With the fix for the infamous bug 375013 it should now be
possible. As I said, the same sequence works correctly if the
"--stacked" is removed. It is therefore likely that either:

    * Fix didn't resolve all the facets of bug 375013
    * Through the fix some other bug has been uncovered that was
      previously hidden by 375013 and that arose during period 3.

2) Oops! The creativity lies in my cut-and paste email creation. It
should have read "bzr branch --stacked <path>". My apologies. The log I
included does correctly describe the actual circumstances under which
the bug occurs.
The merge /is/ from a patch file. And that has previously worked without
any problems /without/ stacked branches in period 3 and /with/ stacked
branches in period 2 - and we do about 40 such merges a week.

I do not know if the patch file was created from a normal or a stacked
branch. Either way it would have been created from the branch after a
commit. I will try to find out today once they involved people get out
of bed. [I saw 2 separate examples of the crash, and no successes]
--

* David*

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2/15/2011 12:45 AM, David I wrote:
> Martin Pool wrote:
>> On 15 February 2011 00:19, David I <email address hidden> wrote:
>>
>>> Public bug reported:
>>>
>>> I'm getting a problem using bzr 2.3.0 with the sequence
>>>
>>> bzr branch --stacked <patch file>
>>> bzr merge <patch file>
>>>
>>> while the same sequence works if I leave out the --stacked.
>>>

you should be able to:

bzr branch --stacked $OTHER_BRANCH target
cd target
bzr pull --overwrite <patch_file>

The issue is that we cannot stack *on top* of a patch file, but only
another branch. I can agree that the UI should be cleaned up around this.

I don't remember 'bzr branch <patch file>' working before. I wonder if
it is actually finding the branch in the containing directory, and then
ignoreing 'patch-file'. So it is, effectively doing:

bzr branch --stacked . $TARGET

And possibly defaulting the name of the target to the base name of the
patch file? I don't really know.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1ammoACgkQJdeBCYSNAAMzBwCgwIPL15ugssQ/OBrPWNzWqppW
1mgAoNTYclPfDjsZkqW7CaJBaIos7bjf
=RBSG
-----END PGP SIGNATURE-----

John Arbash Meinel wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 2/15/2011 12:45 AM, David I wrote:
>
>> Martin Pool wrote:
>>
>>> On 15 February 2011 00:19, David I <email address hidden> wrote:
>>>
>>>
>>>> Public bug reported:
>>>>
>>>> I'm getting a problem using bzr 2.3.0 with the sequence
>>>>
>>>> bzr branch --stacked <patch file>
>>>> bzr merge <patch file>
>>>>
>>>> while the same sequence works if I leave out the --stacked.
>>>>
>>>>
>
> you should be able to:
>
> bzr branch --stacked $OTHER_BRANCH target
> cd target
> bzr pull --overwrite <patch_file>
>
>
> The issue is that we cannot stack *on top* of a patch file, but only
> another branch. I can agree that the UI should be cleaned up around this.
>
> I don't remember 'bzr branch <patch file>' working before. I wonder if
> it is actually finding the branch in the containing directory, and then
> ignoreing 'patch-file'. So it is, effectively doing:
>
> bzr branch --stacked . $TARGET
>
> And possibly defaulting the name of the target to the base name of the
> patch file? I don't really know.
>
> John
> =:->
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Cygwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk1ammoACgkQJdeBCYSNAAMzBwCgwIPL15ugssQ/OBrPWNzWqppW
> 1mgAoNTYclPfDjsZkqW7CaJBaIos7bjf
> =RBSG
> -----END PGP SIGNATURE-----
>
John,
thanks for the reply. I'm sorry for the confusion, there was a mistake
in my original posting. The stacked branch was actually from a regular
branch - I'd never have imagined doing a branch from a patch file!
Please see my response in the bug trail to Martin's questions. I'll do a
quick check with your suggestion to use pull --overwrite.

Also a crash with pull --overwrite

Non-stacked works:

$ bzr branch bzr+ssh://${BRANCH_PATH} ${LOCAL_TARGET}
Branched 750 revision(s).
$ cd ${LOCAL_TARGET}/
$ bzr pull --overwrite ../13619.patch
  <list of updates>
All changes applied successfully.
Now on revision 749.

Cleanup:

$ cd ..
$ rm -rf ${LOCAL_TARGET}/

With stacked - doesn't work:

$ bzr branch --stacked bzr+ssh://${BRANCH_PATH} ${LOCAL_TARGET}
Created new stacked branch referring to bzr+ssh://${BRANCH_PATH}/.
$ cd ${LOCAL_TARGET}/
$ bzr pull --overwrite /data/id/CmsRoot/tasks/patches/13619.patch
bzr: failed to report crash using apport:
     AssertionError()
bzr: ERROR: bzrlib.errors.BzrCheckError: Internal check failed: Cannot add revision(s) to repository: missing referenced chk root keys: [StaticTuple('sha1:1cf87f3b340e298218a852fee5ae97783670c8dd',)]

Traceback (most recent call last):
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/commands.py", line 923, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/commands.py", line 1123, in run_bzr
    ret = run(*run_argv)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/commands.py", line 691, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/commands.py", line 710, in run
    return self._operation.run_simple(*args, **kwargs)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/builtins.py", line 985, in run
    mergeable.install_revisions(branch_to.repository)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/merge_directive.py", line 278, in install_revisions
    info.install_revisions(target_repo, stream_input=False)
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/bundle/serializer/v4.py", line 471, in install_revisions
    return ri.install()
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/bundle/serializer/v4.py", line 543, in install
    self._repository.commit_write_group()
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/repository.py", line 1640, in commit_write_group
    result = self._commit_write_group()
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/repofmt/pack_repo.py", line 2333, in _commit_write_group
    hint = self._pack_collection._commit_write_group()
  File "${BZR_DISTR_PATH}/lib/python/bzrlib/repofmt/pack_repo.py", line 2171, in _commit_write_group
    "Cannot add revision(s) to repository: " + problems_summary)
BzrCheckError: Internal check failed: Cannot add revision(s) to repository: missing referenced chk root keys: [StaticTuple('sha1:1cf87f3b340e298218a852fee5ae97783670c8dd',)]

bzr 2.3.0 on python 2.5.2 (Linux-2.6.24-22-generic-i686-with-debian-lenny-sid)
arguments: ['${BZR_DISTR_PATH}/bin/bzr', 'pull', '--overwrite', '/data/id/CmsRoot/tasks/patches/13619.patch']
encoding: 'ANSI_X3.4-1968', fsenc: 'ANSI_X3.4-1968', lang: None
plugins:
  bash_completion ${BZR_DISTR_PATH}/lib/python/bzrlib/plugins/bash_completion [2.3.0]
  bzrtools ${BZR_DISTR_PATH}/lib/python/bzrlib/p...

I think I have a handle on this now. And I think this is actually a dupe, but I don't have the old bug number handy.

After doing your first stacked branch, you end up with a branch pointer, but no content in the repository. When you go to merge the bundle (or pull, or whatever), it creates revision X (with parent Y). However, the bundle only contains enough information to *just* represent X, and not recreate the inventory of Y. Our stacking logic says "for any revision in the repository, we must hold that revision's information, plus its parent inventory, so that we can compute the delta."

To do this from a merge-directive means that we need to:
a) Put more info into the merge-directive itself. We'd like to avoid bloating it, but if we want to treat it as a self-contained unit, then we need to.
b) Be able to fetch the extra data from our stacked-on location, or possibly from the extra meta-information in the merge-directive. (Often there will be a 'public' url where we can fetch extra info from.)

I've added a launchpad bugtask here, as I believe this bug is to blame for breaking the emailing of bundles to <email address hidden> for 2a format branches.

Evidence for this:

* A mailed bundle for a pack-0.92 format trunk works just fine and creates a branch.
* A mailed merge directive with neither bundle nor patch, addressing already-pushed source and target 2a branches works fine.
* A mailed bundle for a 2a format trunk dies. OOPS-1886CMP6. Also probably OOPS-1886CMP2 is the same, another user's problem which led me to be investigating this.

Launchpad no longer supports merge directives.