Posts Tagged hg
Using Gmail with hg’s Patchbomb Extension
Posted by Sid in Open Source on February 24th, 2009
I recently created a patch for the Mercurial Project and in order to submit the patch I had to use hg’s patchbomb extension. Unfortuantely that requires me to have SMTP on my system. I didn’t have that so either I needed to install SMTP or figure out a way to use Gmail. I decided to use Gmail but it took me a while to get it working because I couldn’t really find much help on the web. Since nobody seems to have explained how to do this properly I decided I’ll just put out a simple guide. Here it is:
- Open up your repo’s .hgrc file
- Add the following:
[extensions]
hgext.patchbomb =
[email]
from=[your_name] [<your_email>]
method=smtp
[smtp]
host=smtp.gmail.com
port=587
username=[gmail_email_address]
password=[gmail_password]
tls=True
v0.6 Release - Refactoring to Fix the Bitrotting Issue with Bug 459727
Posted by Sid in DPS911, Mercurial Project, Open Source on February 21st, 2009
I had mentioned in my previous blog post that hgpoller/pushlog-feed.py had bitrotted. One of my goals for this release was to make changes to the current version of pushlog-feed.py so that my patch is no longer broken for bug 459727. I’ve finally made those changes, which mainly occur in pushes_worker(). The following is what this method looks like with my changes:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | def pushes_worker(query, repo): """Given a PushlogQuery, return a data structure mapping push IDs to a map of data about the push.""" pushes = {} for id, user, date, node in query.entries: mergeData = [] ctx = repo.changectx(node) if len(ctx.parents()) > 1: for cs in ctx.parents(): mergeData.append(hex(cs.node()) + '|-|' + clean(person(cs.user())) + '|-|' + clean(cs.description())) if id in pushes: # we get the pushes in reverse order pushes[id]['changesets'].insert(0, node) pushes[id]['mergeData'].append(mergeData) else: pushes[id] = {'user': user, 'date': date, 'changesets': [node], 'formattedDate': util.datestr(localdate(date)), 'individualChangeset': hex(ctx.node()), 'author': clean(person(ctx.user())), 'desc': clean(ctx.description()), 'mergeData': mergeData, 'max': gettotalpushlogentries(conn) } return pushes |
Basically I had to pass in repo (web.repo) so that I could have access to repo.changectx(node). This now allows me access to ctx.parents() which I need to retrieve merge changeset data. I also went through the whole file and changed every instance where pushes_worker() was called so that repo was being passed in as a paramater along with query.
These are all the changes I needed to make to the server side code. Now I’ll have to examine the changes that were made to the client side which caused my patch to bitrot.
v0.6 Release - Examining the Changes with hgpoller/pushlog-feed.py
Posted by Sid in DPS911, Mercurial Project, Open Source on February 20th, 2009
I had mentioned in my v0.6 goals blog post that my patch for bug 459727 had bitrotted. Unfortunately significant changes were made to hgpoller which apparently broke my patch. I need to remedy this situation because all my server side functionality for this patch is in the pushlog-feed.py file.
I downloaded the latest hgpoller source code and had a look at the changes that had been made. The file has changed in quite a few places. It seems that the function that my patch alters has been changed as well. I’m talking about pushes_worker(), which is repsonsible for passing the data from the server side to the client side.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | def pushes_worker(query): """Given a PushlogQuery, return a data structure mapping push IDs to a map of data about the push.""" pushes = {} for id, user, date, node in query.entries: if id in pushes: # we get the pushes in reverse order pushes[id]['changesets'].insert(0, node) else: pushes[id] = {'user': user, 'date': date, 'changesets': [node] } return pushes |
Now, the problem is that I need access to web.repo within pushes_worker() so that I can call repo.changectx(node) but right now, I don’t have access to repo within the method. I’ll have to figure out a way to do that somehow.
v0.6 Release - Fixing Annotate for the Paper Theme
Posted by Sid in DPS911, Mercurial Project, Open Source on February 17th, 2009
In my last release I had put out an hg annotate fix for the gitweb_mozilla theme. Dirkjan Ochtman, a Mercurial Project developer, noticed my release and asked me to come up with a similar fix for the Mercurial Project’s paper theme. I decided to take up the task and see if I could get similar results for the paper theme as I did with gitweb_mozilla.
The paper theme uses a HUGE table to display the results just like gitweb_mozilla. I tested the current version of the theme using a large 10,000 line cpp file. It gave me a loading time of ~30sec, which is ~10sec longer than gitweb_mozilla.
The following is the code to fix annotate:
map
annotateline = '<div class="l#parity#"><div class="codeauthor"><a href="{url}annotate/{node|short}/{file|urlescape}{sessionvars%urlparameter}#{targetline}" title="{node|short}: {desc|escape|firstline}">{author|user}@{rev}</a></div><a class="codeline" href="#{lineid}" id="{lineid}">{linenumber}</a>{line|escape}</div>'style-paper.css
div.codeauthor { display:inline-block; width:16ch; text-align: right; color:#999999; text-decoration:none; margin-right: 25em; } a.codeline { color:#999999; text-decoration:none; margin:0 10px; } div.l0 { background-color:#f6f6f0; } div.l0, div.l1 { display:block; } pre.completecodeline { font-size: 90%; line-height:1.4em; font-family: monospace; white-space: pre; } div.headrev { float: left; margin-right: 32em; margin-left: 8em; font-size: 90%; font-weight: bolder; } div.headline { font-weight: bolder; font-size: 90%; }
I changed the code (see above) and tested annotate again to examine the difference in loading time. I got some surprising results. The reduction in file size was not significant at all. The fix for gitweb_mozilla brought down the file size by 25%. However, in this case the reduction in file size wasn’t nearly as significant. Also the reduction in loading time for gitweb_mozilla was ~15sec but for the paper theme the loading time was only reduced by ~12secs. Currently, on my machine the loading time has gone down from ~30secs to ~18secs.
The speed increase isn’t as signficant for the paper theme as it was for gitweb_mozilla. Why is that? I don’t exactly know. Obviously there are other factors coming into play that aren’t allowing a similar speed boost. Nonetheless, there is a noticeble increase in loading times.
v0.5 Release Complete
Posted by Sid in DPS911, Mercurial Project, Open Source on February 7th, 2009
I’ve finally completed my 0.5 release. This release I tackled an interesting problem with hg annotate. Trying to improve efficiency and loading time of an application isn’t something I had tackled before. It was a good experience trying to figure out the solution to this problem. I’ll be putting up a new patch on the bug page very soon. The following are some of the important links for this release:
- The release goals
- Understanding the problem
- Examining other templates
- Examining two older patches
- My fix for the problem
- Running an experiment
View the project page for more details.
EDIT: I’ve posted the patch for this release. Have a look…
v0.5 Release - Running an Experiment
Posted by Sid in DPS911, Mercurial Project, Open Source on February 7th, 2009
In my last post I revealed my fix for the hg annotate loading issues. My fix reduced the loading time to a relatively reasonable ~8sec considering the fact that currently the loading time is ~20sec. However, it was still bugging me that Mat’s implementation was producing faster loading times than my fix. His implementation is ~1sec faster.
Mat’s Implementation
The problem is that this implementation doesn’t use valid HTML. Mat uses two non-standard tags, x and l#parity#. Although, I must say that this is a very unorthodox and smart solution to this problem:
annotateline = '<l#parity#><x><a href="#url#diff/#node|short#/#file|urlescape#{sessionvars%urlparameter}">#author|obfuscate#@#rev#</a></x><a href="##lineid#" id="#lineid#">#linenumber#</a>#line|escape#</l#parity#>'Altering Mat’s Code into Valid HTML
As an experiment I decided to test what would happen if I took Mat’s code and replaced the x and l#parity# tags with divs. Would the implemenation still remain as fast? Would it still be faster than mine? The following is the altered code:
annotateline = '<div class="l#parity#"><div class="codeauthor"><a href="#url#diff/#node|short#/#file|urlescape#{sessionvars%urlparameter}">#author|obfuscate#@#rev#</a></div><a class="codeline" href="##lineid#">#linenumber#</a>#line|escape#</div>'1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | div.codeauthor { display:inline-block; width:16ch; text-align: right; color:#999999; text-decoration:none; margin-right: 25em; } a.codeline { color:#999999; text-decoration:none; margin:0 10px; } div.l0 { background-color:#f6f6f0; } div.l0, div.l1 { display:block; } pre.completecodeline { font-size:12px; line-height:1.4em; } a.codeline:hover, a.codeline:visited, a.codeline:active { color: #880000; } |
Results
I tried testing the altered code and I was getting a loading time of ~7sec for this cpp file. However, I was never able to get into the ~5sec region which Mat’s original code was sometimes able to achieve. This altered version was only able to reach a minimum time of ~6sec.
The point is that this altered version is a bit faster than my fix but slower than Mat’s original implementation. I compared the file sizes of the 3 versions:
- My fix: 2.4MB
- Mat’s Fix:1.8MB
- Altered version of Mat’s Fix: 2.2MB
It is interesting to note that using div tags instead of an x and l#parity# tag increases the file size by 18%. I don’t know why that happens but somehow the x and l#parity# tags are more efficient than div tags. Nonetheless the reasons don’t matter, the altered version of Mat’s fix seems to be the best solution to this problem at this time.
v0.5 Release - My Fix for the hg Annotate Problem
Posted by Sid in DPS911, Mercurial Project, Open Source on February 6th, 2009
In my last post I had examined a couple of patches that other people had posted for bug 459823. This time I wrote my own code to see if I could make an improvement on what others have done. I did borrow some things but my implementation will be a bit different.
HTML
Instead of using one large table to hold all the rows I’ve decided to use a single div that holds two links and another div:
annotateline = '<a class="codelineauthor" href="#url#diff/#node|short#/#file|urlescape#{sessionvars%urlparameter}">#author|obfuscate#@#rev#</a><a class="codelinenumber" href="##lineid#" id="#lineid#">#linenumber#</a><div class="codeline parity#parity#">#line|escape#</div>'The HTML is pretty simple, nothing fancy. All the formatting will be done by the CSS instead of a table.
CSS
I’ve decided to place my CSS in the style-gitweb.css style sheet instead of just leaving it in the fileannotate.tmpl file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | a.codelineauthor { width: 50ch; float: left; } a.codelinenumber:hover, a.codelinenumber:visited, a.codelinenumber:active { color: #880000; } a.codelinenumber { text-decoration: none; margin-right: 0.5em; float: left; color:#999999; } div.completecodeline { font-size: 12px; line-height: 1.4em; font-family: monospace; white-space: pre; display: block; } |
The float property is used to align the two links and the div so that it looks like a table.
Result
The result is quite positive. This implementation cuts the loading time by ~12sec when testing a 10,000 line cpp file in mozilla-central. The loading time currently on live is ~20sec and my implementation cuts it to ~8sec. This is around the same amount of time that Bonsai takes to load the same file. The browser doesn’t freeze and the initial screenful appears immediately just as jorendorff specified.
It is interesting to note that the size of the page on live (with the HUGE table) is 3.2MB while my solution cuts the file size down to 2.4MB, a 25% reduction.
v0.5 Release Goals
Posted by Sid in DPS911, Mercurial Project, Open Source on January 28th, 2009
My v0.4 release is over and done with. In that release I mainly focused on bug 459727 but for my v0.5 release I’ll be taking a look at a problem with hg annotate, bug 459823. The problem here is that hg annotate takes FOREVER to load large files such as ten thousand line cpp files found in mozilla-central. People have already spent a bit of time working on this bug so this gives me some insight intto what I’m required to do.
I had a chat with jorendorff and we discussed what should happen when using hg annotate:
- Show the first screenful in a second or less
- Don’t lock up the browser while the rest loads
- Don’t take forever to load the rest of the file, because I often want to search for a snippet of code
This is an interesting task for me as I’ve never really looked at how firefox renders things on the page. In order to make hg annotate load things faster I’ll have to look at what is more efficient. For example would a page with a table of 10,000+ rows load faster or a page with 10,000+ divs? Right now, I don’t know but I’ll have to find out because as jorendorff put it “the current code just fails all day long” and something must be done about that.