v0.3 Release Complete!

Posted on December 7, 2008 by Sid

My v0.3 release has finally come to an end along with the course. Throughout this release I have put out a total of 7 patches spread over 3 bugs and wrote 13 blog posts (not including this one). I have put in hours and hours of work and communicated with various Mozilla community members linked to my project. It has been grueling and fun at the same time. Sometimes I struggled while other times I triumphed. Some of the better moments in the past couple of weeks was the news that my patch for bug 448707 was approved on my very first try and my patch for bug 445560 was also approved after a few adjustments. I felt extremely happy to see something that I had worked on go live. Everyday Mozilla users will utilize these features and that gives me a sense of pleasure. This sort of feeling is the reason I put in all those hours of blood and tears these past few weeks and throughout the length of this project.

For this release I think I have fulfilled the goals I set out to complete. I blogged regularly, kept my project page up to date and put out various patches for all my bugs that required them. I went back and fixed problems with the patches for my v0.2 release. I was also able to implement a new feature for hgweb, the graph view, which the community wanted to see. It was the type of thing that was very different from the other bugs that I worked on. This one required knowledge of drawing graphics with JavaScript, something I hadn’t even thought about doing before I took this on. I think I definitely challenged myself with this release as the means to this feature were entirely alien to me beforehand. To view all the features implemented/fixed for with my v0.3 visit my project page.

I’ve released my final version for this course and now everything is in the hands of the Mozilla elders. They will have a look at my patches and decide if they are worthy. Only time will tell if they are…

Posted in Fitness Journal, Mercurial Project | Tagged , , | Leave a comment

Another Patch Approved, for bug 445560

Posted on December 5, 2008 by Sid

I’m happy to report that another patch of mine has been approved by ted! This is for bug 445560. For this patch I had to make some changes after I initially put up the patch but they weren’t major ones. For more details about this bug and what my patch does visit my project page.

After my first patch that was successful, this is the second patch that I have gotten approved from the work I have done throughout the semester. Hopefully pushlog users will be utilizing this feature very soon! It just gives me a sense of delight when something I have worked hard on gets approved so that it is available to the users. I feel like doing a little dance right now!

Posted in Mercurial Project, Open Source | Tagged , , | Leave a comment

v0.3 Release – Improved bug link for bug 459727

Posted on December 4, 2008 by Sid

The pushlog has a buglink feature that changes a string that contains reference to a bug number into a link to that bug on BugZilla. This is done on the server side by hgpoller/buglink.py file. However, when loading more entries on scroll the bug links do not work as buglink.py is not run on them. I have to make the links on the client side. I’ve streamlined by bug link implementation by using a regex.

Previous Implementation

The following is the bug link code used previously:

  //Create buglink
  var bugInDesc = (pushData[i].desc).toUpperCase().indexOf("bug"); 
  if(bugInDesc != -1) {
    var bugLinkName = (pushData[i].desc).substring(bugInDesc, bugInDesc + 10);
    var bugNumber = bugLinkName.substring(4, 10);
    var bugLink = 
      (pushData[i].desc).substring(0, bugInDesc) + 
      '<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=' + 
      bugNumber + 
      '">' + 
      bugLinkName + 
      '</a>' + 
    (pushData[i].desc).substring(bugInDesc + 10, (pushData[i].desc).length);
    } else { //No bug provided
        var bugLink = pushData[i].desc;
    }

With this implementation an indexOf() is performed to find the string, “bug” in the description string. If the string, “bug” is found then it is turned into a bug link. However, the problem is things like “debugger” also contain the string, “bug” are changed into a bug link as well which is obviously wrong. We want to avoid this.

New Implementation

The following is the new bug link code:

//Create buglink
re = new RegExp("[bB][uU][gG]\s\d{6}");
var bugInDesc = (pushData[i].desc).search(re); 
if(bugInDesc != -1) {
  var bugLinkName = (pushData[i].desc).substring(bugInDesc, bugInDesc + 10);
  var bugNumber = bugLinkName.substring(4, 10);
  var bugLink = 
    (pushData[i].desc).substring(0, bugInDesc) + 
    '<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=' + 
    bugNumber + 
    '">' + 
    bugLinkName + 
    '</a>' + 
    (pushData[i].desc).substring(bugInDesc + 10, (pushData[i].desc).length);
  } else { //No bug provided
      var bugLink = pushData[i].desc;
  }

Now with this regex strings like “debugger” are not turned into a bug link. Only valid strings like “bug 123456″ which match the regex pattern are turned into a link.

Posted in Mercurial Project, Open Source | Tagged , , | Leave a comment

v0.3 Release – Further Problems with Loading More Items, bug 459727

Posted on December 3, 2008 by Sid

I put up a couple of patches today to fix problems with an earlier patch I had submitted for bug 459727. It seems that my fix for the problem of the initial results not loading enough entries to fill up the window does not work. Justin Wood pointed that out on the bug page, comment 15. He is right it seems.

I’ve been trying to figure out ways to make this work and the following is what I have come up with.

The Fix

var start;
function setup()
{
  if ('getElementsByClassName' in document) {
    var l = document.getElementById("localize");
    l.style.display = "";
    l.onclick = localizeDates;
 
    start = 0;
    getMaxEntries();
 
    var docHeight = document.body.clientHeight;
    while(docHeight <= window.innerHeight) {
      if(start > 0) {
        renderMorePushLogResults();
        docHeight += 500;
      }
    }
  }
}

The setup() function is called once the inital entries have loaded. Then we measure the document height and set it to the variable, docHeight. A while loop is run until docHeight is greater than the window height. Each time you enter the while loop a new set of entries are loaded and the docHeight’s value is increased by 500 pixels (the approximate height of the new entries). This should load new entries according to the size of every users individual window height.

The Problem

Now, with this new code I am getting the following script error when I load the page:


 
I think this may be because of the while loop. Firefox maybe waiting for the while loop to finish before it does anything else and then this error occurs. I’m not exactly sure, this is just a poke in the dark.

It is important to note that if you click continue it works and more entries are loaded properly to fill the rest of the page.

This new code for the pushlog.tmpl can be found here

I don’t know if this is a major problem or not but I can’t see a way around this for now.

Posted in Mercurial Project, Open Source | Tagged , , , | Leave a comment

v0.3 Release – Fixing the Problems with my Patch for bug 459727

Posted on December 3, 2008 by Sid

I’ve been busy trying to fix problems that had arised regarding the patch I submitted for bug 459727. All that is being done right now is fixing various small problems but the overall functionality remains the same. So the following are the solutions to the problems that jorendorff identified:

To calculate the max SELECT COUNT(*) FROM … should be used

Changed my implementation to use gettotalpushlogentries(conn) which performs a select statement. I’ve gotten rid of getMaxEntries() since it is no longer required.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

def pushes_worker(repo, startID=0, endID=None):
    stmt = 'SELECT id, user, date, rev, node from pushlog INNER JOIN changesets ON id = pushid WHERE id > ? %s ORDER BY id ASC, rev ASC'
 
    args = (startID,)
    if endID is not None:
        stmt = stmt % 'and id <= ?'
        args = (startID, endID)
    else:
        stmt = stmt % ""
    if os.path.basename(repo.path) != '.hg':
        repo.path = os.path.join(repo.path, '.hg')
    conn = sqlite.connect(os.path.join(repo.path, 'pushlog2.db'))
    pushes = {}
    for id, user, date, rev, node in conn.execute(stmt, args):
        mergeData = []
        ctx = repo.changectx(node)
        if len(ctx.parents()) > 1:
          for cs in ctx.parents():
            mergeData.append(hex(cs.node()) + '|-|' + clean(person(cs.user())) + '|-|' + clean(cs.description()))
        if id in pushes:
            pushes[id]['changesets'].append(node)
            pushes[id]['mergeData'].append(mergeData)
        else:
            pushes[id] = {'user': user,
                          'date': date,
                          'changesets': [node],
                          'formattedDate': util.datestr(localdate(date)),
                          'individualChangeset': hex(ctx.node()),
                          'author': clean(person(ctx.user())),
                          'desc': clean(ctx.description()),
                          'mergeData': mergeData,
                          'max': gettotalpushlogentries(conn)
                          }
    return pushes

`new Function(“return ” + entries.responseText) ()` is saying
`eval(entries.responseText)`, perhaps use `JSON.parse (entries.responseText)`

I haven’t implemented this yet because I want more information on why the way I have done is not the appropriate way. I’ve asked on the bug page (comment 12)but haven’t heard back yet

The page loads more results when the user scrolls.  What if the initial results don’t fill up the window?

I added the following CSS to always show a scrollbar no matter what. This should allow the user to scroll even if the initial entries don’t fill up the page.

html { 
  overflow: scroll; overflow-x: auto; 
}

getMaxEntriesis called, then ‘start’ is used once the user scrolls.  But the user might scroll before the first result comes back, while start is still zero — a race condition

I changed the implementation to check the value of the start variable before more entries are allowed to load onScroll. If the value of start is still 0 then no entries are allowed to load. The following is the code:

$(window).scroll(function() {
  if(start > 0) {
    if($(window).scrollTop() == $(document).height() - $(window).height()) {
      renderMorePushLogResults();
    }
  }
});

Better to move CSS and formatting to stylesheets where possible

I removed explicitly setting the styles of the rows. Now I am just setting the class name of the row so that it uses the preexisting styles for that class name. View lines 37-43 for the code

The JS code isn’t consistently indented

It should be correctly formatted now. View here

This patch seems to contain some non-ASCII characters.  Bugzilla doesn’t render it properly

Used &mdash in the places where non-ASCII characters were being used. View lines 73 and 121

In a few places the b tag is used where the server uses the strong tag

Changed the b tags to use strong. View lines 73 and 121

I will be putting up a new patch very soon…

EDIT:

The new patches have been posted here

Posted in Mercurial Project, Open Source | Tagged , , , , , , , , | Leave a comment

v0.3 Release – Problems with my Patch for bug 459727

Posted on December 2, 2008 by Sid

Lately there has been some activity on bug 459727 concerning the patch I had submitted to implement an OnScroll feature for the pushlog. View comments 8 and 9 to see the problems that jorendorff as identified. So far there hasn’t been an official review but it is inevitable that I will have to make changes to the patch I had submitted.

In comments 10 and 12 I underlined what steps I will be taking to fix the problems that jorendorff has identified so far. They don’t seem to be major changes although I am worried about using JSON.parse(). I don’t know if that will work or not. I guess I will have to try it out and see what I can do.

Also I don’t really understand what is wrong with using `new Function(“return ” + entries.responseText) ()` as jorendorff mentioned in comment 9. In comment 11, Justin mentioned that it may cause vulnerabilities to XSS attacks but he couldn’t explain why that was. This has me interested and I would love it if someone could tell me why.

This is the biggest patch I submitted for my v0.2 so I’m expecting to go through several iterations before the patch is accepted. Look out for a new patch soon…

Posted in Mercurial Project, Open Source | Tagged , , , , | Leave a comment

v0.3 Release – 1st Iteration of the Graph View

Posted on December 2, 2008 by Sid

I’ve been working hard trying to get the graph view working for the gitweb_mozilla template. Earlier, I documented the problems I was having. That is all resolved now and I’m moving on to bigger and better things.

So my goal was to get the graph to the stage of the coal template, which I am using as a reference and then make further improvements from there to make it look similar to the rest of the gitweb_mozilla pages.

To do this I created a new template called graph.tmpl which can be viewed by typing {url}/graph in the address bar. This is where the graph will be drawn. You can view the code it has here. However, with just this code my graph wasn’t rendering properly. The following is what it looked like: http://sidkalra.com/files/mercurial/graphView1.png. I couldn’t quite figure out why this was happening. It seemed like the graph was drawing properly and the data was appearing properly too and yet they weren’t coming together. Something was missing from the puzzle…

I showed my problem to a friend, Tiago Moreira who happened to be sitting right next to me as the time. He was able to point out that it might be a CSS problem, something I failed to consider. This was definately a lead and I had a look at the coal template’s css file and the answer hit me in the face straight away. The canvas tag was assigned a z-index of 5.

canvas {
	position: absolute;
	z-index: 5;
	top: -0.7em;
	margin: 0;
}

This means that anything in the canvas tag will have a higher priority over everything else and it will also allow the data in other tags (in this case the two ul tags with the ids graphnodes and nodebgs) to slip in underneath the canvas giving the following result: http://sidkalra.com/files/mercurial/graphView2.png

Now my graph view looks similar to the one that the coal template has. My next objectives are to improve it greatly by making it look similar to the other pages in the gitweb_mozilla theme as I mentioned above and adding more changeset information to the each row. Basically the end goal is for it to look similar to the pushlog but with the graph drawn in as well.

Posted in Mercurial Project, Open Source | Tagged , , , , , , | 1 Comment

v0.3 Release – A Problem with Getting data for the Graph View

Posted on December 1, 2008 by Sid

hgweb has a graph function found in mercurial/hgweb/webcommands.py which returns a template. It also returns a variable called jsdata which contains all information that I want to display on the page along with the graph. Now, my problem was that I wasn’t quite understanding how I can could call graph() so that I could pass the data it returns to the client side.Should I have a new template or can I pass the data to an existing template.

For the longest time I was stumped on this issue. I didn’t understand whether I should be calling this function in my python code or could I call it on the client side using JavaScript? Did I have to play around with the template’s map file? I guess my lack of experience working with web applications that use templates and themes was rearing its ugly head. I was stumped!

I tried various methods via trial and error to see if I could figure it out but all in vain. Eventually I decided it was time for some help. I asked djc who told me that I would have to create a new template file which gets its data from the graph function in webcommands.py . But I still didn’t get how to actually pass the data to the client side.

So, I decided to ask for help in freenode’s mercurial channel. There, Brendan was able to point out that all I needed to do was add the following to hg_templates/gitweb_mozilla/map…

graph = graph.tmpl

The answer was staring me right in the face all along. I had glanced over this line many times while trying to figure out how this works but I guess I just didn’t realize that this is how you create a new tmpl file. Now I can access jsdata by adding {jsdata|json} in my template file.

Now I can actually start working on rendering the graph on the client side.

Posted in Mercurial Project, Open Source | Tagged , , , | Leave a comment

My patch for Bug 448707 Approved!

Posted on December 1, 2008 by Sid

Some good news to report today. My patch for bug 448707 has been approved by jorendorff. He did find some problems but they were minor enough that they could be ignored for now.

Getting an r+ on the first go is pleasently surprising. Hopefully I can keep moving along and get a patch up for the graph view very soon.

Posted in Mercurial Project, Open Source | Tagged , , , | Leave a comment

v0.3 Release – Understanding djc’s Graph View

Posted on November 25, 2008 by Sid

I’m starting work on a graph view for the pushlog. Frankly, I didn’t know where to begin since I have never really done anything like this before, then again I could say the same thing about the whole project. After having a discussion with jorendorff I was pointed towards djc’s (Dirkjan Ochtman) Graph view for Mercurial. He already has it working and my job is to now understand what he has done and get it working for the pushlog now.

The Source

I downloaded the source from http://hg.intevation.org/mercurial/crew/ using hg clone.

Getting it Running

Getting it running on my browser was pretty simple. Almost identical to what I had to do for the pushlog. Just open up a command prompt and navigate to the directory where you saved the source and call the hg serve command. Then fire up a browser and type in http://localhost:8000 and you should be able to see the graph view.

Understanding the Code

Mercurial crew comes with Mercurial and hgweb. Some of the functionality is implemented in crew/templates/static/graph.js which is what runs on the client side. Also the tmpl file used to render the page is crew/templates/coal/graph.tmpl which imports graph.js to utilize its methods. The template also takes in a parameter, jsdata which contains the graph data. jsdata is populated on the server side by the graph method of the crew/mercurial/hgweb/webcommands.py file. It contains graphmod.graph() which is apparently what does the actual graph layout.

I don’t fully understand this code at the moment. There are many questions going through my mind. For example it seems that the server side code (mercurial.hgweb.webcommands.graph) returns a tmpl so I don’t know whether I should be working from a new tmpl file or can I put my client side code in hg_templates/gitweb_mozilla/pushlog.tmpl like I have been doing all along?

The client side code receives the data from the server side through {jsdata|json} which is all the changeset data. But right now with the pushlog all this is processed through pushlog-feed.py which gives the data to hg_templatesgitweb_mozillamap that renders the page. So should I be messing with this map file? If not then how can I pass the data to the client side so that I can draw the graph and render it on the page?

The above 2 examples are only some of the questions going through my mind right now. I have told you about them because they are ones that must be answered urgently. It is important that I understand the basic functionality of djc’s Graph view so that I can deduce a way to apply it to my problem.

It is still early days for this feature and I’m still trying to feel my way through it. Hopefully things will pick up as my understanding of this feature increases.

Posted in Mercurial Project, Open Source | Tagged , , , , , , , , , , | Leave a comment