v0.2 Release – Fix to the Current Patch for bug 445560

Posted on November 8, 2008 by Sid

One month ago ted put up a patch to fix bug 445560 to implement expand/collapse functionality for merge changesets for the pushlog. The patch works properly except for one issue. When a user clicks on a particular [Expand] link it should only expand the changesets belonging to it and not all merge changesets on the page. Right now all merge changesets are expanding. For example the following are 2 merge changesets that have [Expand] links:

With ted’s patch when the user clicks on the first Expand link the following happens:

Both merge changesets expand when only the first one should.

The Fix

jQuery is used to implement this expand/collapse feature which allows us to perform hide() and toggle() to hide and show the appropriate merge changesets. Each row which should be hidden is given the class name “hidden” which then allows one to go $(‘.hidden’).hide() and $(‘.hidden’).toggle(). But the problem is that all rows have the same class name so no matter which Expand link one clicks ALL rows with the class name “hidden” will get toggled.

Somehow each set of merge changesets have to be given their own unique identifier. In this case I decided to use the date, which is common for a set of changesets that are grouped together. To do this I made some back-end changes to changelist() in the pushlog-feed.py file. Now, it also returns a dateId:

def changelist(limit=0, **map):
        #pdb.set_trace()
        allentries = []
        lastid = None
        ch = None
        l = []
        mergehidden = ""
        p = 0
        currentpush = None
        for id, user, date, node in entries:
            ctx = web.repo.changectx(node)
            n = ctx.node()
            entry = {"author": ctx.user(),
                     "desc": ctx.description(),
                     "files": web.listfilediffs(tmpl, ctx.files(), n),
                     "rev": ctx.rev(),
                     "node": hex(n),
                     "tags": nodetagsdict(web.repo, n),
                     "branches": nodebranchdict(web.repo, ctx),
                     "inbranch": nodeinbranch(web.repo, ctx),
                     "hidden": "",
                     "push": [],
                     "mergerollup": [],
                     "dateId": localdate(date)
                     }

I then passed this dateId to the map file to make it the id of each hidden row and the row which holds the expand link:

pushlogentry = '<tr id="date#dateId#" class="parity#parity# #hidden# #dateId#">
mergehidden = '<br/>← #count# hidden changesets <a id="hidedate#dateId#" class="expand" href="#">[Expand]</a>'

Now each grouped set of hidden rows has a unique id, which will allow me to hide the right set of changesets according to which Expand link the user clicks. To do this I changed some code in the pushlog.tmpl file:

1
2
3
4
5
6
7
8
9
10
11
12
13

// hide things that should be hidden
  $('.hidden').hide();
  // add click handler to unhide hidden things
  $('.expand').click(function (idDate) {
    if ($(this).text() == "[Expand]")
      $(this).text("[Collapse]");
    else
      $(this).text("[Expand]");
    //XXX: don't toggle all hidden, just the ones following this row
    var id = $(this).attr("id");
    id = '#' + id.substring(4, id.length);
    $(id).nextAll(id).toggle();
    return false;

I replaced the line $(‘.hidden’).toggle() with lines 10-12. Basically:

  1. I get the id of the Expand row that the user clicked.
  2. Then, I manipulate the string to get the id of the hidden rows
  3. Lastly, I toggle the id of the hidden rows

The above code gives the appropriate result when the first expand link is clicked:

Posted in Mercurial Project, Open Source | Tagged , , , , , , | 1 Comment

v0.2 Release – Adding a loader GIF to Pushloghtml’s OnScroll feature

Posted on November 8, 2008 by Sid

These days it seems like every AJAX application has some sort of loader GIF to let the user know that new data is being loaded. It has almost become a standard and lets one know that they are using an AJAX application. I wanted to add this functionality to pushloghtml’s OnScroll feature, bug 459727

So, I went hunting for some a good loader GIF that I could use and in the process I found this nifty little site: http://www.ajaxload.info/, which lets one create a custom loader GIF.

The next step was to decide where to place the actual GIF on the server so that I could call it in the pushlog.tmpl file. This had me stumped for a while. I tried placing the GIF in various places but nothing seemed to work. Then, I noticed there were pre-existing images on the pushloghtml page and they must be stored somewhere on the server so I did a little search for *.jpeg and found the location of the images.

To show the GIF before new data is loaded I placed the following code in my OnScroll function:

1
2

var loader = document.getElementById("loader");
      loader.innerHTML = '<img src="{url}static/ajax-loader.gif" align="right" />';

Then, right before the very first row is displayed on the screen with appendChild() I remove the GIF by doing the following:

1
2

loader.innerHTML = "";
        document.getElementById("titlePush").appendChild(trScroll);

The above code shows the GIF OnScroll (bottom-right corner): http://sidkalra.com/files/mercurial/loader.png

Posted in Mercurial Project, Open Source | Tagged , , , , | Leave a comment

A Puzzling Problem with ctx.parents()[i].description()

Posted on November 7, 2008 by Sid

I’m using a JSON script to pass data from the server side (written in Python) to the client side (in JavaScript) so that I can format the data and show it on the pushlog onScroll. I’m trying to add functionality so that merge changesets show up properly for the newly loaded data like the the image below:

Now, to do this I need to add extra data to the json-pushes script which means making changes server side. The functionality of the script is in pushes_worker() in the pushlog-feed.py file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

def pushes_worker(repo, startID=0, endID=None):
    #pdb.set_trace()
    stmt = 'SELECT id, user, date, rev, node from pushlog INNER JOIN changesets ON id = pushid WHERE id &gt; ? %s ORDER BY id ASC, rev ASC'
 
    args = (startID,)
    if endID is not None:
        stmt = stmt % 'and id &lt;= ?'
        args = (startID, endID)
    else:
        stmt = stmt % ""
    if os.path.basename(repo.path) != '.hg':
        repo.path = os.path.join(repo.path, '.hg')
    conn = sqlite.connect(os.path.join(repo.path, 'pushlog2.db'))
    pushes = {}
    for id, user, date, rev, node in conn.execute(stmt, args):
        ctx = repo.changectx(node)
        if id in pushes:
            pushes[id]['changesets'].append(node)
        else:
            pushes[id] = {'user': user,
                          'date': date,
                          'formattedDate': util.datestr(localdate(date)),
                          'changesets': [node],
                          'individualChangeset': hex(ctx.node()),
                          'author': ctx.user(),
                          'desc': ctx.description()
                          }
    return pushes

Now, a row is a merge row if len(ctx.parents()) is greater than 1. Then we can get the required data for the merge changesets (changeset, user, description) by going ctx.parents()[0].user() or ctx.parents()[0].description() etc.

Then, I needed to decide which data structure to use so that I can get the changeset, user and description for each merge row. I thought of using a 3D array or a character delimited string inserted into an array or a dictionary within a dictionary but the problem is that I’m getting a weird server error when I call ctx.parents()[0].description()

The following is my code right now:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

def pushes_worker(repo, startID=0, endID=None):
    #pdb.set_trace()
    stmt = 'SELECT id, user, date, rev, node from pushlog INNER JOIN changesets ON id = pushid WHERE id &gt; ? %s ORDER BY id ASC, rev ASC'
 
    args = (startID,)
    if endID is not None:
        stmt = stmt % 'and id &lt;= ?'
        args = (startID, endID)
    else:
        stmt = stmt % ""
    if os.path.basename(repo.path) != '.hg':
        repo.path = os.path.join(repo.path, '.hg')
    conn = sqlite.connect(os.path.join(repo.path, 'pushlog2.db'))
    pushes = {}
    mergeData = []
    for id, user, date, rev, node in conn.execute(stmt, args):
        ctx = repo.changectx(node)
        if len(ctx.parents()) &gt; 1:
          #pdb.set_trace()
          for cs in ctx.parents():
            #uc = hex(cs.node()) + '||' + cs.user()
            #mergeData.append(hex(cs.node()) + '||' + cs.user() + '||' + cs.description())
            #mergeData.append(str(cs.description()))
            #mergeData[uc] = hex(cs.description())
            #mergeData = cs.description()
            #mergeData[cs.user()] = cs.description()
            mergeData = cs.user()
        if id in pushes:
            pushes[id]['changesets'].append(node)
        else:
            pushes[id] = {'user': user,
                          'date': date,
                          'formattedDate': util.datestr(localdate(date)),
                          'changesets': [node],
                          'individualChangeset': hex(ctx.node()),
                          'author': ctx.user(),
                          'desc': ctx.description(),
                          'isMerge': len(ctx.parents()),
                          'mergeData': mergeData
                          }
    return pushes

I’m looping through each ctx.parent() and then performing cs.user() or cs.description() to get the required data. Then I am giving the data (either in array or dictionary form) to the script by going ‘mergeData’: mergeData

Now, here’s the problem, if I do what I have on line 22, mergeData.append(… and then give that array to the script I get the following error… http://pastebin.com/f1b4e6981

BUT…

if I package the data in a dictionary like on line 26 I get no error and the script renders it with no problem. I’ve narrowed the problem down to cs.description(). If I invoke cs.description() and put it in an array I get the error but if I put the same cs.description() within a dictionary I get no error. This is very puzzling and as you can probably tell from all the various comments I’ve tried multiple different ways to pass cs.description() but only the dictionary way works, which is not ideal for what I want to be able to do

I’ve spent 8-9 hrs on this issue but I still keep hitting the same wall and I just don’t see why this is happening…

EDIT: The only way I have been able to pass this information to the client side is by putting the user and description in a dictionary and then putting the changeset into a seperate array. This is not a preferred solution but it is the only way I have been able to make this work so far. So the following is how this would work…

1
2
3
4
5
6
7

mergeData = {}
mergeChangesets = []
ctx = repo.changectx(node)
if len(ctx.parents()) > 1:
  for cs in ctx.parents():
    mergeData[person(cs.user())] = cs.description()
    mergeChangesets.append(hex(cs.node()))

EDIT:Thanks to some great work by jorendorff I have managed to solve this problem. Let me explain…hg stores everything in 8 bit strings (it is Unicode ignorant). When trying to convert to json it attempts to convert to JavaScript strings which are 16 bit. I was getting a UTF-8 error because the 8 bit strings used by hg are not valid.

So to check where the error was happening I printed out the results of cs.description() to a file (http://pastebin.com/m249f0ef5). I showed it to jorendorff and he pointed out line 146, which read:

“”Back out Dxe3o Gottwald’s patch from bug 380960 due to JS exception.”

The name should read Däo Gottwald. The ä wasn’t being rendered properly and thus the UTF-8 error was occuring. A great catch by jorendorff! 8 hrs or so of work and it all came down to one character. I think jorendorff put it best, “some people are UTF-8 impaired” and I couldn’t agree more (Just kidding Däo, if you are reading).

The Fix

OK, so we now know why the error was happening, now how can we fix it? Well, jorendorff recommended that I use clean()

1
2
3
4
5

def clean(s):
    try:
        return s.decode('utf-8')
    except UnicodeDecodeError, exc:
        return s.decode('latin-1')

Then, I can simply use the above function like so:

1

clean(cs.description())

and the problem is solved! The error is gone!

Posted in Mercurial Project, Open Source | Tagged , , , , | Leave a comment

Looking Forward to v0.2 Release

Posted on November 7, 2008 by Sid

The date for my v0.2 Release is approaching rapidly and it’s time that I decide on what exactly I will be working on. After a good bit of thought I have decided that my best option is to work on the following:

  • Continue working on bug 459727 to make pushloghtml show more entries onScroll
  • Add a fix to ted’s patch for bug 445560 so that when the user clicks on a particular [Expand] link it only expands the changesets belonging to it and not all changesets on the page.
  • Add functionality to show which files were touched by a changeset and roughly the size of the diffs. Maybe add onMouseOver functionality to show it in an unobtrusive way.

I think adding these 3 features will be more than enough for v0.2 release in terms of work load. I have already spent countless hours working on the  first 2 features. Also I think these 3 features will benefit the community the most by improving the core features of the pushlog. I’ve consulted with jorendorff on this and he seems to agree that these 3 features would be good enough for my next release.

Posted in Mercurial Project, Open Source | Tagged , , , , , , , | Leave a comment

A Simple Extension to fix Firefox’s tab functionality

Posted on November 5, 2008 by Sid

For this week’s lab humph asked us to create a simple extension that fixes the tab functionality of Firefox. Right now without any extensions when one opens a new tab for firefox it gets placed at the end, not right beside the current tab. The goal of the extension is to make it so that when one opens a new tab it gets placed right next to the current tab.

This is a very simple extesion and thus only has 4 files:

  • addtabbeside.js – Performs the needed functionality
  • install.rdf – Needed extension file
  • chrome.manifest – Needed extension file
  • overlay.xul

Frankly, Firefox should have this functionality built in. Most people have tons and tons of tabs open at any one time and having to go to the end of the list to get the new tab is just annoying. Even Internet Explorer has the same functionality but the good thing about Firefox and open source is that one can fix/change something that one dislikes just as I have done with this extension.

You download this extension here. Just click and drag it into firefox and the install sequence should start up. Please make sure to rename the file to Addtabbeside.xpi instead of Addtabbeside.zip

Posted in Open Source | Tagged , , , | Leave a comment

Further Progress with pushloghtml’s OnScroll Feature

Posted on November 5, 2008 by Sid

I’ve been working hard on bug 459727 to get it working. Right now it’s starting to look like it should but there are still some problems that need to be fixed.

var start = -19;
var end = start + 19;
 
function renderMorePushLogResults() {
  $('test').html('<img src="D:CRULSHORUKH_MobileSENECASEMESTER_7DPS909Mercurialajax-loader.gif">');
  start += 20;
  end = start + 19;
  var pushCheckins = new XMLHttpRequest();
  pushCheckins.open('GET', '/json-pushes?startID=' + (start - 1) + '&endID=' + end, true);
  pushCheckins.onreadystatechange = function() {
  if(pushCheckins.readyState == 4)  {
    if(pushCheckins.status != 404) {
      var pushData = new Function("return " + pushCheckins.responseText) ();
      var counter = 0;
      for(var i = end; i >= start; i--) {
        var trScroll = document.createElement("tr");
 
        if(counter == 0) {
          counter = 1;
        } else {
            trScroll.style.backgroundColor = "#f6f6f0";
            counter = 0;
        }  
 
        var tdScrollUser = document.createElement("td");
        tdScrollUser.width = "184px";
        tdScrollUser.innerHTML += '<i>' + pushData[i].user + '<br />' + pushData[i].formattedDate + '</i>';
 
        var tdScrollChangeset = document.createElement("td");
        tdScrollChangeset.innerHTML += 
         '<a href="/rev/' + 
         pushData[i].individualChangeset.substring(0, 12) + 
         '">' + 
         pushData[i].individualChangeset.substring(0, 12) + 
         '</a>';
 
        //Create buglink
        var bugInDesc = (pushData[i].desc).indexOf("Bug");
        if(bugInDesc == -1)
          bugInDesc = (pushData[i].desc).indexOf("bug");
 
        if(bugInDesc != -1) {
          var bugLinkName = (pushData[i].desc).substring(bugInDesc, bugInDesc + 10);
          var bugNumber = bugLinkName.substring(4, 10);
          var bugLink = 
           (pushData[i].desc).substring(0, bugInDesc) + 
           '<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=' + 
           bugNumber + 
           '">' + 
           bugLinkName + 
           '</a>' + 
           (pushData[i].desc).substring(bugInDesc + 10, (pushData[i].desc).length);
        } else { //No bug provided
            var bugLink = pushData[i].desc;
        }
        var tdScrollAuthorDesc = document.createElement("td");
        tdScrollAuthorDesc.innerHTML += '<b>' + pushData[i].author + ' - ' + bugLink + '</b>';
 
        trScroll.appendChild(tdScrollUser);
        trScroll.appendChild(tdScrollChangeset);
        trScroll.appendChild(tdScrollAuthorDesc);
 
        document.getElementById("titlePush").appendChild(trScroll);
      }    
    } 
   }
  }
  pushCheckins.send(null);
}

The above code gives the following result. This is a marked improvement from my previous results wouldn’t you say? Now how did I work all this out? If you read my previous blog post you would know how many problems I was having with the formatting and getting the right data. Let me explain how I have rectified the situation:

Formatting

Currently all the formatting is done by the template files map and style-gitweb.css, which get data from python-feed.py and format the data. Now when I load items when the browser registers the OnScroll event there is no way to render the page again so that map and style-gitweb.css can format this new data (as far as I could tell at least).

I have decided to do all the formatting on the JavaScript side for all the new entries. It is a bit tedious and clumsy I suppose but it gets the job done. Take for example the bugzilla links. For the normal entries that appear onPageLoad the bugzilla links are setup by a server side service called buglink.py but again the new entries don’t have access to this service so I have to do it manually. Basically:

  • I take the description string
  • Get the position of “bug” or “Bug” in the description string by using indexOf()
  • Setup the link which will always be “https://bugzilla.mozilla.org/show_bug.cgi?id=” + the bugId
  • Reassemble the description string but this time with the link of the bug

Acquiring the Right Data

As I mentioned in a previous post I didn’t have all the data I needed. To acquire the needed data I had to change the functionality of the server side script I was using, json-pushes. The following are the changes I made:

def pushes_worker(repo, startID=0, endID=None):
    #pdb.set_trace()
    stmt = 'SELECT id, user, date, rev, node from pushlog INNER JOIN changesets ON id = pushid WHERE id > ? %s ORDER BY id ASC, rev ASC'
 
    args = (startID,)
    if endID is not None:
        stmt = stmt % 'and id <= ?'
        args = (startID, endID)
    else:
        stmt = stmt % ""
    if os.path.basename(repo.path) != '.hg':
        repo.path = os.path.join(repo.path, '.hg')
    conn = sqlite.connect(os.path.join(repo.path, 'pushlog2.db'))
    pushes = {}
    mergeChangesets = {}
    for id, user, date, rev, node in conn.execute(stmt, args):
        ctx = repo.changectx(node)
        if len(ctx.parents()) == 2:
          #code to deal with merge changesets, u can access stuff by going ctx.parents()[0].description() or ctx.parents()[0].user()
          par = ctx.parents()
        if id in pushes:
            pushes[id]['changesets'].append(node)
        else:
            pushes[id] = {'user': user,
                          'date': date,
                          'formattedDate': util.datestr(localdate(date)),
                          'changesets': [node],
                          'individualChangeset': hex(ctx.node()),
                          'author': ctx.user(),
                          'desc': ctx.description(),
                          'isMerge': len(ctx.parents())
                          }
    return pushes

I had to alter pushes_worker() so that it recieved web.repo so that I could perform ctx.web.repo.changectx(node), which gives me the information I need like ctx.user() and ctx.description(). To clarify in the above code the passed in variable repo = web.repo

I was also able to format date in the appropriate way by calling util.datestr(localdate(date)). This required me to get the util module.

Further improvements

I’m not done yet, there are still things that need to be done:

  • Chronological Order: This has me a bit confused. Right now I have it set that for the very first time the user scrolls to the bottom of the page json-pushes loads with startid=0&endid=20. The next time it loads startid=20&endid=40 and so on. This doesn’t have the same effect of going from pushloghtml to pushloghtml/2 to pushloghtml/3 and so on, which is what I want to happen, right?
  • Merge Changesets: This functionality doesn’t work right now. My problem is how do I know whether something is a merge changeset or not? Well, I’ve answered that question now thanks to djc. I need to use len(ctx.parents()) which returns 2 if a node is merge changeset. Then I can access the description and user data by going ctx.parents()[0].description() or ctx.parents()[0].user()
  • loader-gif: I want to add a loader gif animation so that users know that the entries are loading, which is quite a standard feature for AJAX applications. But my problem is that I don’t know where in my file structure I should be placing the gif file. I’ve tried something like 
    $('test').html('<img src="{url}ajax-loader.gif">');

    and placed my gif file everywhere possible but nothing has worked yet

  • Keeping expand/collapse functionality: I want to make sure that my new entries keep the expand/collapse functionality for merge changesets.
Posted in Mercurial Project, Open Source | Tagged , , , , , , | Leave a comment

Building Firefox from Source

Posted on November 2, 2008 by Sid

This is coming a bit late on my part since my desktop was having some hard drive issues. Well, I finally got that cleared up (ended up having to reformat my computer) and proceeded to build Firefox from source. So I headed over to the build instructions and started following the steps for the Windows platform…

Requirements

So after all the prerequisites were ready I got one of the bash scripts found in C:mozilla-build running. It passed all the initial checkups and gave me the command prompt. Awesome!

Source

Already had that downloaded and ready to go!

Configuring Build Options

Now .mozconfig needed to be setup. This can be tricky from what I hear. This file will hold all my build options. The following is how my mozconfig file looks like…

ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@

Build

I ran the build by typing make -f client.mk build. My build ramped up and started to run but eventually I got an error…

“configure: error: –enable-application=APP was not specified and is required”

I ran the command ./configure –enable-application=browser to fix this error. It’s weird though since I have –enable-application=browser in my .mozconfig file already so why do I have to explicitly invoke it from the command line?

The build took around an hour as various unreadable compile data zoomed by as I looked on. At the end I didn’t know whether the build succeeded or not. It would have been nice to see a message clearly telling me that the build had been successful or not like. Nonetheless, I checked my source folder and found the minefield exe ready to run.

Posted in Open Source | Tagged , , , | Leave a comment

FSOSS 2008 – Thoughts and Experiences

Posted on October 24, 2008 by Sid

Today was the first day of the annual FSOSS (Free Software and Open Source Symposium) held at Seneca College, Toronto. The day began with registration which started at 8:15am. I got there at around 8:50 (just in time since the first round of presentations were slated to begin at 9am). Each person attending the conference was given a Seneca handbag with various things inside (including an FSOSS T-shirt, schedule booklet and the latest Ubuntu CD). The last couple of years there has been at least one key note speaker but that wasn’t the case this time around. Everybody was ushered into the various tracks and all the presentations began immediately.

Software Security

This time around the first day had four tracks instead of the usual three. This new track carried the same theme throughout as various Seneca people gave their thoughts relating to open source. I chose not to attend any of these tracks since I already have some idea of what they were going to talk about. The very first presentation that I attended, The Most Important Thing – How Mozilla Does Security, and What You can Steal was excellent. The speaker, Johnathan Nightingale was clear and confident, he knew his stuff. I found it interesting how he called browsers “stupid” since they download unknown code 3rd party and run it. I agree with him, it is “stupid” but that is what makes a browser so useful. The ability to run other people’s code situated anywhere on web makes the browser the most used application on a person’s desktop. Throughout the presentation Johnathan kept emphasizing the importance of writing tests. I think everybody agrees that they are needed, it’s just that it’s difficult finding people who want or like writing them. Tests are one of those things that are essential but are always hated. They are not fun or “cool” to most people. They are like taking medicine, we might hate the procedure but the results are beneficial in every way. The presenter stated that the most important thing is to learn from your mistakes so that you never make them again. I think this can’t be stated enough, even if I bolded, underlined and capitalized that statement it wouldn’t be enough. Such a statement doesn’t just apply to software security, it applies to life itself. I know that it has been repeated many many times over the years but people still don’t seem to get it. Albert Einstein once said, “The definition of insanity is doing the same thing over and over and expecting different results”. History repeats itself and learning from it is essential if you are to succeed.

Open Source Communities

I enjoyed the security presentation (see above) immensely, it was one of my favorite this year but I have to say that the last presentation that I attended, Community Building and the Architecture of Participation was absolutely brilliant. Greg DeKoenigsberg was captivating and expressive. He put forward an important question that I think many of us at FSOSS take for granted. He asked whether free software was better? This got me thinking. We all assume that it is better just because it’s free and open. The presenter said that he thought the answer to his question was yes, it is better. But I don’t know if he was clear enough with his question. What does he mean by “better”? Better in price? Better in quality? Better in service? In many of these categories free software can sometimes leave the user disappointed. Nonetheless, I think the presenter is right that free software is better, definitely from a philosophical and ethical standpoint at the very least. Now, the goal of every free software practitioner and supporter should be to make it better in every other way possible, but that is easier said then done. Another interesting point that came up during the talk was the way to get new contributors involved in your open source project. The presenter expressed that fact that it is important that one must allow people to start from the “edges” and work towards the “core”. Now what does this exactly mean? Well, many times open source software can be very challenging to get involved in. Many communities fail to provide a starting point that pushes new users in the right directions and allows them to learn more about the project. Basically, when a project fails to provide areas that beginners can easily work on the barrier of entry is increased exponentially and the likelihood of that project getting new users that eventually contribute significant knowledge decreases drastically. I wholeheartedly agree with Greg on this issue. It is a major problem with some of the open source projects out there today. How can this be fixed? How can projects make it easier for beginners to get involved? Couple of things that will definitely help are clear and concise documentation and helpful community members. It is important to let new users know that they are desired and that the work they are doing is important (even if it might not be). Attracting fresh recruits is a constant challenge for any organization or project and it is important to handle it correctly to secure the future of the project.

In the End…

Overall, FSOSS was an incredible learning experience for me, as always. The speakers mostly did a great job (excluding one talk, which I will not name) and I enjoyed myself throughtout the conference. I was glad to see more Seneca students come out to than conference than I have in the past two years so that is always good. I wish more students would take advantage of this great opportunity of having such a reputable conference right in our back yard. I wish I could have attended all the talks but alas that was impossible. I look forward to when the videos for each talk are released. The only thing missing from this years conference was a key note speaker. Nonetheless it was a great conference and I would like to thank all the volunteers and faculty that made it possible. I keenly look forward to FSOSS 2009.

Posted in Open Source | Tagged , , , | Leave a comment

Pushloghtml Should Show More Than 10 Entries, Working on v0.1 Release

Posted on October 18, 2008 by Sid

Well as I stated in a previous blog post, one of my tasks for v0.1 release would be to make pushloghtml show more than 10 entries. Jorendorff, recommended that it show all the entries at once. Now, I took that to mean all the entries should be loaded onto the page on pageLoad because I assumed that the total push log entries couldn’t be more than a couple of hundred. Well, it turns out the number can be in the thousands and trying to load all of that on pageLoad completely kills the load time. I realized right there and then that this method would not work. Something else needed to be done.

I talked to jorendorff and ted about this issue and they said that the pushlog shouldn’t load all the changesets at once. They want more entries to be loaded on scroll down. Seems like there was some what of a miscommunication that took place, but no worries this sort of stuff can happen.

Now, I have never implemented this kind of feature so I had to figure out how to do this. It was recommended that I use jQuery, which allows one to measure the height of the page and the window. Thus, by the doing the following one can know when the user reaches the bottom of the page…

1
2
3
4
5

$(window).scroll(function() {
  if($(window).scrollTop() == $(document).height() - $(window).height()) {
    renderMorePushLogResults();
  }
});

The function renderMorePushLogResults() should display the next set of changesets on the page. Now, what does this function do to retrieve the correct data? Well, currently I am using a python script that returns data in JSON format. To access the data and show it on the page I can make an XMLHttpRequest to the JSON script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

function renderMorePushLogResults() {
  var pushCheckins = new XMLHttpRequest();
  pushCheckins.open('GET', '/json-pushes?startID=0&amp;endID=20', true);
  pushCheckins.onreadystatechange = function() {
  if(pushCheckins.readyState == 4)  {
    if(pushCheckins.status != 404) {
      var pushData = new Function("return " + pushCheckins.responseText) ();
      var divScroll = document.createElement("div");
      for(var i = 1; i &lt;= 20; i++) {
        divScroll.innerHTML += pushData[i].user + "n";
        document.getElementById("titlePush").appendChild(divScroll);
      }
    }
   }
  }
  pushCheckins.send(null);
}

The code above gets the data from the script, creates a div, loops through all the user names provided by the the python script and puts in the div. Then the div is appended to the table named “titlePush”. You can see the result that the above 2 functions give http://sidkalra.com/files/mercurial/xmlhttprequest_example.png

Now, obviously this is not what I want. There are some issues that need to resolved…

  • The results are not formatted properly.
  • The python script returns two more types of data, the date and the changesets which I have chosen not to show on the page for now. However, it doesn’t return the patch author name and commit message.
  • The new changesets need to be printed on the page in the right sequence (by date) and the script doesn’t seem to return data sequentially.

As I make progress regarding the 3 issues I have pointed out above I will continually update this post so stay tuned!

Edit: Well I’ve been working on this the whole day. I tried using another script that ted recommended (http://hg.mozilla.org/mozilla-central/pushlog?fromchange=aa4b3566d739) since I was having issues with the first one (see above). The following is the code…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

function getMoreChangesets() {
  try {
    var pushCheckins = new XMLHttpRequest();
  } catch(e) {
      alert("error with XMLHttpRequest object");
    }
  pushCheckins.onreadystatechange = function() {
    if(pushCheckins.readyState == 4)  {
      if(pushCheckins.status == 200) {
        var pushData = pushCheckins.responseXML;
 
        var element = "";
        for(var i = 1; i &lt;= 10; i++) {
          element += pushData.getElementsByTagName('tr')[i].getElementsByTagName('i')[0].firstChild.nodeValue; //Read the first element
          element += ' | ' + pushData.getElementsByTagName('tr')[i].childNodes[1].firstChild.nodeValue;
          element += ' | ' + pushData.getElementsByTagName('tr')[i].getElementsByTagName('cite')[0].firstChild.nodeValue;
          element += ' | ' + pushData.getElementsByTagName('tr')[i].childNodes[2].firstChild.lastChild.nodeValue;
        }
 
        var divScroll = document.createElement("div");
        divScroll.innerHTML += element;
        document.getElementById("titlePush").appendChild(divScroll);
      } else {
          alert("Error with xml" + pushCheckins.status);
        }
    }
  };
  pushCheckins.open('GET', '/pushlog?fromchange=aa4b3566d739', true);
  pushCheckins.overrideMimeType('text/xml');
  pushCheckins.send(null);
}

The above code gives the following result on scroll down…

http://sidkalra.com/files/mercurial/xmlScript.png

Since this new script returns XML I am using pushCheckins.responseXML to retrieve the XML data. Then I can access all the data I want and push it on the screen (see line 14-18). Now, I still get the same issue of the formatting not working properly but at least this script provides with me with more data that I need, although it doesn’t give me everything like the email address of the author.

Edit 2: I’ve continued to work on this issue and I’ve come up with a new iteration of code that provides some better formatting for the output i.e. everything is not a blob of unreadable text like my above example. But the new changesets still don’t match the styles and formatting of the original ones. The following is the code…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

function getMoreChangesets() {
  try {
    var pushCheckins = new XMLHttpRequest();
  } catch(e) {
      alert("error with XMLHttpRequest object");
    }
  pushCheckins.onreadystatechange = function() {
    if(pushCheckins.readyState == 4)  {
      if(pushCheckins.status == 200) {
        var pushData = pushCheckins.responseXML;
 
        var element = "";
        var divScroll = document.createElement("table");
        var test = pushData.getElementsByTagName('tr');
        for(var i = 1; i <= 10; i++) {  
          element += '<tr><td>' + pushData.getElementsByTagName('tr')[i].getElementsByTagName('i')[0].firstChild.nodeValue + '</td>'; //Read the first element
          element += '<td>' + pushData.getElementsByTagName('tr')[i].childNodes[1].firstChild.nodeValue + '</td>';
          element += '<td>' + pushData.getElementsByTagName('tr')[i].getElementsByTagName('cite')[0].firstChild.nodeValue + '</td>';
          element += '<td>' + pushData.getElementsByTagName('tr')[i].childNodes[2].firstChild.lastChild.nodeValue + '</td>';
          element += '</tr>';
 
          divScroll.innerHTML += element;
        }
        divScroll.innerHTML += "</table>";
        document.getElementById("newItems").appendChild(divScroll);
      } else {
          alert("Error with xml" + pushCheckins.status);
        }
    }
  };
  pushCheckins.open('GET', '/pushlog?fromchange=aa4b3566d739', true);
  pushCheckins.overrideMimeType('text/xml');
  pushCheckins.send(null);
}

You can see the result of the above code at http://sidkalra.com/files/mercurial/formattedXmlScript.png

As you can see from the example the formatting and readability has become much better but I am getting some repetition. I don’t quite know why that is happening. The problem is obviously between lines 16 and 20. Keep watching for more updates as I figure this out.

Posted in Mercurial Project, Open Source | Tagged , , , , | Leave a comment

Applying a patch to a Repo using Mercurial

Posted on October 14, 2008 by Sid

As I mentioned in a previous blog post, one of the features for my v0.1 release will be implementing a expand/collapse feature for pushloghtml. Well, turns out that ted has already started working on that. He’s already filed a bug and released a patch for it. According to him its not quite done and a couple of things need to be changed.

So, my first job was to download the patch and apply it to my repo. So I headed over to Bugzilla and looked up the bug. There were 2 patches that I needed to download…

  • One for pushlog-feed.py
  • Another for gitweb_mozilla

Before you read the following take into account that I’ve never applied any kind of a patch before so this is purely from the point of a beginner. So I was looking for a way to download the patch so that I could apply it. But apparently there is no download link available on BugZIlla (it took me almost 20 mins to realize that). They should really look to put in some sort of download link/button. In the end I just ended up using the crude method of “Save Page As” to download the patch file.

Now onto applying the patch. I had no idea how to do that so I tried using tortoiseMerge since I already had that installed. It is apparently capable of applying a patch. I added the path of the diff file and the directory where I wanted it to apply the patch in the two text boxes below and then clicked the OK button. Nothing happened at all. Nothing got loaded, no error message, no nothing. This had me a bit bewildered and frustrated. tortoiseMerge was basically useless.

Now onto the next option which was hg itself. I knew that there was a way since I had read about it in some documentation when I was getting know hg at the start of my project. So I headed over to google to perform a search. I tried various things such as “apply a patch hg”, “patch hg”, “install patch hg” and “mercurial apply patch”. The last one gave me a couple of good results. One was at hgbook and another was documentation about hg queues but neither of them was a straight up guide of how to install patches. I tried many different google searches but I just couldn’t find any simple and easy guide explaining how one would install a patch using hg. Well, here I am to the rescue. The following is a simple guide that gives instructions on installing a simple patch using hg…

  1. Download the .patch or .diff file onto your machine
  2. Navigate to the repo that you want to install the patch to i.e. testRepo (I will using repo name for this example, substitute it with your repo name when you are trying to apply a patch yourself)
  3. Open testRepo.hghgrc (the config file)
  4. Add the following to hgrc and save (This enables queues for hg so that we can use commands that we need, qimport and qpush)[extensions]
    hgext.mq =
  5. Now open up a command prompt and navigate to your repo directory
  6. Type in hg qimport <full-path-of-patch-file> and press enter
  7. The above command should create a patch folder within testRepo.hgpatches
  8. Next navigate to  testRepo.hgpatches
  9. Type in hg qpush <patch-file-name>
  10. Done! The patch should have been applied (if you get an error such as “abort: local changes found, refresh first” then that means you have made changes to the original files and the patch won’t work)

In my trials and tribulations while trying to apply a patch I had to first figure out what hg queues did and how to use them. I tried to just type in hg qimport but I just got a “command does not exist” error. I didn’t realize that I had to import something until ted gave me a push in the right direction. A simple guide would have really helped.

Posted in Mercurial Project, Open Source | Tagged , , , | 13 Comments