Bazaar

Bazaar

 

Wiki Tools

  • Find Page
  • Recent Changes
  • Page History
  • Attachments

Contents

  1. Overview
  2. Method
    1. Bazaar version
    2. Network connection
    3. Data set
  3. Tests
    1. Case 1: Push of new standalone branch
      1. Results
    2. Case 2: Push 1 new revision onto existing standalone branch (format 5)
      1. Results
    3. Case 3: Push 1 new revision onto existing standalone branch (format 6)
      1. Results
    4. Case 4: Push a diverged branch, resulting in an error
      1. Results
    5. Case 5: Push a diverged branch using --overwrite
      1. Results
    6. Case 6: Push new branch into shared repo, only 1 new revision
      1. Results
    7. Case 7: Push new branch into empty shared repo
      1. Results

Overview

This page analyses how "bzr push bzr://..." spends its time.

This is by no means a totally comprehensive suite of tests. There are lots of potentially interesting situations, at the moment I'm just looking at fairly simple cases that I think are fairly representative of many typical use cases.

The raw ~/.bzr.log files from these tests are here: HPSS-push-bzr-logs.tar.gz

Method

Bazaar version

revision: 3390
revid: pqm@pqm.ubuntu.com-20080429155137-0w5littcl831pq4m

Network connection

I used the loopback interface on my laptop, configured to have 500ms latency (thus 1000ms for a full round trip) using the following command: 

sudo tc qdisc add dev lo root netem delay 500ms

Normal system behaviour is restored with 

sudo tc qdisc del dev lo root

A more precise version that only filters traffic to port 4155 is: 

tc qdisc add dev lo root handle 1: prio
tc qdisc add dev lo parent 1:3 handle 30: netem delay 500ms 
tc qdisc add dev lo parent 30:1 handle 40: prio
tc filter add dev lo protocol ip parent 1:0 prio 3 u32 match ip dport 4155 0xffff flowid 1:3 handle 800::800
tc filter add dev lo protocol ip parent 1:0 prio 3 u32 match ip sport 4155 0xffff flowid 1:3 handle 800::801

and to remove this: 

tc filter del dev lo protocol ip parent 1: pref 3 u32
tc qdisc del dev lo root handle 1:

Data set

I used bzr.dev at r3390 (pqm@pqm.ubuntu.com-20080429155137-0w5littcl831pq4m). Revision 3390 happens to be a very small revision, in terms of changed lines versus the previous revision.

The repositories and branches used are generally in the current default format (pack-0.92/format 6) unless otherwise noted.

Tests

Case 1: Push of new standalone branch

On the server: 

mkdir empty_dir
cd empty_dir
bzr serve --allow-writes

On the client: 

bzr -Dhpss push bzr://localhost/bzr.dev

Note: source branch was in format 5

Results

Total time: 10m 9s

Rough analysis:

  • (note: format 5 branch)
  • 20s: created new control dir
  • 44s: created new repo
  • 45-506s: pushing pack file in ~1MB pieces, (total size: 71264612), about 3-9s per chunk (TCP windows?). First chunk took 28s, fairly consistent after that.
  • 507-548s: pushing rix, iix, tix, six files. (~9MB total)
  • 548-559s: rename pack, lock repo (w/vfs), push new pack-names, unlock (w/vfs)
  • 559-561s: 1 readv of rix file!
  • 561-576s: create branch
  • 577-580: re-read pack-names from repo!
  • 581-602s: set revision-history and parent in branch (locking and unlocking around each!)
  • 602-609s: reopen repo
  • 609s: done.

Some points of interest:

  • empty put took 3s (i.e. 3 round trips!). Ditto the following append of 42 bytes.
  • TCP caps us to about 150kB/s, and overall we got about 130kB/s.

Case 2: Push 1 new revision onto existing standalone branch (format 5)

On the server: 

bzr branch -r 3389 ~/code/bzr bzr-r3389
cd bzr-r3389
bzr serve --allow-writes

On the client: 

bzr -Dhpss push bzr://localhost/

Results

Total time: 1m 37s

Rough analysis:

  • 0-21s: opening existing branch, reading revision-history
  • 21-29s: readvs of rix (single pack repo)
  • 29-31s: starts a new pack file
  • 31-42s: readvs of tix
  • 42s: pushed 3474 bytes to pack file (it's a small revision!)
  • 43-47s: push rix, iix, tix, six
  • 47-58s: rename pack, lock repo, write new pack-names, unlock
  • 58s: does one readv of the new rix (all 314 bytes)
  • 59-89s: many get_parent_map calls
  • 89-95s: put the revision-history
  • 95s: call Branch.last_revision_info
  • 96s: unlock branch

Case 3: Push 1 new revision onto existing standalone branch (format 6)

Same setup as case 2, apart from branch format on the server.

Results

Total time: 1m 11s

Rough analysis:

  • 0-18s: opening existing branch, get last revision info
  • 18-27s: readvs of rix
  • 27-29s: start a new pack file
  • 29-43s: readvs of tix
  • 43-44s: pushed 3474 bytes...
  • 44-48s: push rix, iix, tix, six
  • 49-59s: rename pack, lock repo, new pack-names, unlock
  • 59s: single readv of new rix
  • 60s-65: 2 get_parent_maps
  • 65: get branch.conf
  • 66-69: put last-revision file (again, why 3s for this?)
  • 69-71: last_revision_info, unlock.

Case 4: Push a diverged branch, resulting in an error

On the server: 

bzr branch ~/code/bzr bzr-diverged
cd bzr-diverged
bzr commit --unchanged -m "New remote commit."
bzr serve --allow-writes

On the client: 

bzr commit --unchanged -m "New local commit."
bzr -Dhpss push bzr://localhost/

Results

Total time: 55s

Rough analysis:

  • at 17s it has the last_revision_info, which should be enough to know that it has diverged
  • it pushes a pack before raising an error
  • follows similar pattern as incremental push, but doesn't read tix. It's up to the single readv of new rix at 47s.

Case 5: Push a diverged branch using --overwrite

Set up: same as case 4, but push with --overwrite.

Results

Total time: 2m 2s

Rough analysis:

  • identical pattern to incremental case up to "59s: single readv of new rix"
  • 60-116s: many get_parent_map calls (why? we already pushed the the 4.3k pack file!)
  • 117-120s: Branch.set_last_revision
  • 120-122s: Branch.last_revision_info, Branch.unlock

Case 6: Push new branch into shared repo, only 1 new revision

On the server: 

bzr init-repo --no-trees shared-repo
cd shared-repo
bzr branch ~/code/bzr devel-r3389 -r -2
bzr serve --allow-writes

On the client: 

bzr -Dhpss push bzr://localhost/devel

Results

Total time: 2m 5s

Rough analysis:

  • 0-4s: hello, find there is no branch at that URL
  • 4-20s: make bzrdir, unlock it
  • 20s: fail to open repository in new bzrdir

  • 21s: BzrDir.open

  • 22s: BzrDir.find_repositoryV2

  • 23s: Repository.is_shared (why doesn't find_repository return this?)
  • 24s-28s: VFS open of bzrdir and repository.
  • 28s-36s: readvs of rix
  • 36s-38s: start a new pack upload file
  • 38s-50s: readvs of tix
  • 50s-51s: push 3474 bytes of data into upload pack
  • 51-54s: upload rix, tix, iix, six for new pack file
  • 55-66s: rename pack, lock repo, upload new pack-names, unlock
  • 66s: single readv of new rix (entire contents)
  • 67-79s: create branch skeleton
  • 70-83s: lock branch, upload [format, last-revision, branch.conf, tags] for empty branch, unlock
  • 84s: VFS attempt to open repo in branch dir (fails)

  • 85-88s: BzrDir.open, BzrDir.find_repositoryV2, Repository.is_shared, Repository.lock_write

  • 89s-99s: lock_write branch (via VFS), get last-revision, get branch.conf, put new last-revision, unlock
  • 100s: Repository.lock_write (why? the last one hasn't been unlocked, and no token passed, yet it succeeds!)
  • 101-111s: lock_write branch (via VFS), get branch.conf, put branch.conf, unlock
  • 112-115s: 84-88s repeated!

  • 116-125s : opens repo (via vfs), checks no-working-trees, opens branch (via vfs), fails to open repo in branch (via vfs), BzrDir.open, BzrDir.find_repositoryV2, Repository.is_shared.

Case 7: Push new branch into empty shared repo

On the server: 

bzr init-repo --no-trees shared-repo
cd shared-repo
bzr serve --allow-writes

On the client: 

bzr -Dhpss push bzr://localhost/devel

Results

Total time: 9m 31s

Rough analysis:

  • Time is consistent with Case 1 (note that case 1 was format 5, and this was format 6. The lack of revision-history file probably explains why this case is ~30s faster than case 1.)