Archive for December, 2008
Some Reflections on The Open Source Course
Posted by Sid in Mercurial Project, Open Source on December 7th, 2008
Regular classes for this semester have finally come to an end which means that the open source course for this semester has also reached the finish line. This is the first time I’ve taken such a course. I think it is very unique. Seneca, or for that matter other Colleges or Universities don’t offer a course such as this one. I’ve definitely enjoyed myself while working on this, which is not to say that I didn’t go through frustrating times. Throughout this course I have been guided by jorendorff, ted, djc and bsmedberg. Thank you to all of them, without their help I wouldn’t have been able to get very far. I really appreciate the help.
I think that this course has pushed me to the limits of my ability and beyond. Before this semester I didn’t know anything about Python, jquery, Mercurial or for that matter JavaScript (at least not in the manner that Mozilla uses it). I’ve had to learn all these technologies in one semester and apply them into something that worked and would be accepted by Mozilla. It has been amazingly difficult and challenging. Out of all my subjects I have definitely committed the most amount of hours to this course, by far. Almost every other day I was working on the something related to this course. Reflecting back, over the semester I have written 40+ blog posts and release 10+ patches, not to mention the labs and contributions that I did.
I think the whole blogging idea is ingenious. This blog has become a way for me to let the community know what I am doing. This is the true essence of open source as nothing is hidden from the community. Also, I think this blog has become a great asset for me. I will be referencing this blog in my resume so that I can show prospective employers what I have done with Mozilla and what kind of skills and experiences I have garnered. This process definitely makes everything transparent.
Overall, I think this course is extremely helpful and I would recommend it to every student in the CPA and BSD programs. It really separates the men from the boys. However, I think there are ways in which the course can be improved. I believe the biggest problem is that there is not boundary line available. Throughout the course I had to ask myself, is this enough? Do I have enough contributions? Did I blog enough? Did I provide enough information? There is no right answer for these questions as Dave doesn’t want to answer them. I can understand why as these questions made me push myself to do everything possible in order to make sure that I had reached that invincible safety area. Not having the knowledge that you have done enough to get a good mark makes you do even more until there is no time left, at least that is true in my case. Even though this is a good thing it really affects the other courses that one maybe taking. This course will take up as much as time as you are willing to give to it. There is no boundary. I think there should be some sort of boundary to give you that sense of reassurance that you are in the safe zone so that you don’t dedicate all your time to this course while potentially neglecting your other courses. I don’t know if others feel the same way but I got this feeling throughout the course.
I’m relatively certain that I will be taking the next open source course. I had a conversation with jorendorff and ted and it seems that there is plenty more things that I could work on next semester related to hgweb. Although, I’m not 100% certain yet if I’m I’m going to take the next iteration of the course. I need to check my schedule and look at the other courses I will be taking and then I will make a final decision. For now all my attention will turn towards my exams, which are ominously lingering over the horizon.
Overall a wonderful and challenging course that taught many new things. I want to thank Dave Humphrey for providing the building blocks and continously pushing us to limit throughout the semester. It was fun!
Have a happy holidays everyone!
v0.3 Release Complete!
Posted by Sid in Fitness Journal, Mercurial Project on December 7th, 2008
My v0.3 release has finally come to an end along with the course. Throughout this release I have put out a total of 7 patches spread over 3 bugs and wrote 13 blog posts (not including this one). I have put in hours and hours of work and communicated with various Mozilla community members linked to my project. It has been grueling and fun at the same time. Sometimes I struggled while other times I triumphed. Some of the better moments in the past couple of weeks was the news that my patch for bug 448707 was approved on my very first try and my patch for bug 445560 was also approved after a few adjustments. I felt extremely happy to see something that I had worked on go live. Everyday Mozilla users will utilize these features and that gives me a sense of pleasure. This sort of feeling is the reason I put in all those hours of blood and tears these past few weeks and throughout the length of this project.
For this release I think I have fulfilled the goals I set out to complete. I blogged regularly, kept my project page up to date and put out various patches for all my bugs that required them. I went back and fixed problems with the patches for my v0.2 release. I was also able to implement a new feature for hgweb, the graph view, which the community wanted to see. It was the type of thing that was very different from the other bugs that I worked on. This one required knowledge of drawing graphics with JavaScript, something I hadn’t even thought about doing before I took this on. I think I definitely challenged myself with this release as the means to this feature were entirely alien to me beforehand. To view all the features implemented/fixed for with my v0.3 visit my project page.
I’ve released my final version for this course and now everything is in the hands of the Mozilla elders. They will have a look at my patches and decide if they are worthy. Only time will tell if they are…
v0.3 Release - Improved bug link for bug 459727
Posted by Sid in Mercurial Project, Open Source on December 4th, 2008
The pushlog has a buglink feature that changes a string that contains reference to a bug number into a link to that bug on BugZilla. This is done on the server side by hgpoller/buglink.py file. However, when loading more entries on scroll the bug links do not work as buglink.py is not run on them. I have to make the links on the client side. I’ve streamlined by bug link implementation by using a regex.
Previous Implementation
The following is the bug link code used previously:
//Create buglink var bugInDesc = (pushData[i].desc).toUpperCase().indexOf("bug"); if(bugInDesc != -1) { var bugLinkName = (pushData[i].desc).substring(bugInDesc, bugInDesc + 10); var bugNumber = bugLinkName.substring(4, 10); var bugLink = (pushData[i].desc).substring(0, bugInDesc) + '<a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=' + bugNumber + '\">' + bugLinkName + '</a>' + (pushData[i].desc).substring(bugInDesc + 10, (pushData[i].desc).length); } else { //No bug provided var bugLink = pushData[i].desc; }
With this implementation an indexOf() is performed to find the string, “bug” in the description string. If the string, “bug” is found then it is turned into a bug link. However, the problem is things like “debugger” also contain the string, “bug” are changed into a bug link as well which is obviously wrong. We want to avoid this.
New Implementation
The following is the new bug link code:
//Create buglink re = new RegExp("[bB][uU][gG]\\s\\d\{6\}"); var bugInDesc = (pushData[i].desc).search(re); if(bugInDesc != -1) { var bugLinkName = (pushData[i].desc).substring(bugInDesc, bugInDesc + 10); var bugNumber = bugLinkName.substring(4, 10); var bugLink = (pushData[i].desc).substring(0, bugInDesc) + '<a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=' + bugNumber + '\">' + bugLinkName + '</a>' + (pushData[i].desc).substring(bugInDesc + 10, (pushData[i].desc).length); } else { //No bug provided var bugLink = pushData[i].desc; }
Now with this regex strings like “debugger” are not turned into a bug link. Only valid strings like “bug 123456″ which match the regex pattern are turned into a link.
v0.3 Release - Further Problems with Loading More Items, bug 459727
Posted by Sid in Mercurial Project, Open Source on December 3rd, 2008
I put up a couple of patches today to fix problems with an earlier patch I had submitted for bug 459727. It seems that my fix for the problem of the initial results not loading enough entries to fill up the window does not work. Justin Wood pointed that out on the bug page, comment 15. He is right it seems.
I’ve been trying to figure out ways to make this work and the following is what I have come up with.
The Fix
var start; function setup() { if ('getElementsByClassName' in document) { var l = document.getElementById("localize"); l.style.display = ""; l.onclick = localizeDates; start = 0; getMaxEntries(); var docHeight = document.body.clientHeight; while(docHeight <= window.innerHeight) { if(start > 0) { renderMorePushLogResults(); docHeight += 500; } } } }
The setup() function is called once the inital entries have loaded. Then we measure the document height and set it to the variable, docHeight. A while loop is run until docHeight is greater than the window height. Each time you enter the while loop a new set of entries are loaded and the docHeight’s value is increased by 500 pixels (the approximate height of the new entries). This should load new entries according to the size of every users individual window height.
The Problem
Now, with this new code I am getting the following script error when I load the page:
I think this may be because of the while loop. Firefox maybe waiting for the while loop to finish before it does anything else and then this error occurs. I’m not exactly sure, this is just a poke in the dark.
It is important to note that if you click continue it works and more entries are loaded properly to fill the rest of the page.
This new code for the pushlog.tmpl can be found here
I don’t know if this is a major problem or not but I can’t see a way around this for now.
v0.3 Release - Fixing the Problems with my Patch for bug 459727
Posted by Sid in Mercurial Project, Open Source on December 3rd, 2008
I’ve been busy trying to fix problems that had arised regarding the patch I submitted for bug 459727. All that is being done right now is fixing various small problems but the overall functionality remains the same. So the following are the solutions to the problems that jorendorff identified:
To calculate the max SELECT COUNT(*) FROM … should be used
Changed my implementation to use gettotalpushlogentries(conn) which performs a select statement. I’ve gotten rid of getMaxEntries() since it is no longer required.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | def pushes_worker(repo, startID=0, endID=None): stmt = 'SELECT id, user, date, rev, node from pushlog INNER JOIN changesets ON id = pushid WHERE id > ? %s ORDER BY id ASC, rev ASC' args = (startID,) if endID is not None: stmt = stmt % 'and id <= ?' args = (startID, endID) else: stmt = stmt % "" if os.path.basename(repo.path) != '.hg': repo.path = os.path.join(repo.path, '.hg') conn = sqlite.connect(os.path.join(repo.path, 'pushlog2.db')) pushes = {} for id, user, date, rev, node in conn.execute(stmt, args): mergeData = [] ctx = repo.changectx(node) if len(ctx.parents()) > 1: for cs in ctx.parents(): mergeData.append(hex(cs.node()) + '|-|' + clean(person(cs.user())) + '|-|' + clean(cs.description())) if id in pushes: pushes[id]['changesets'].append(node) pushes[id]['mergeData'].append(mergeData) else: pushes[id] = {'user': user, 'date': date, 'changesets': [node], 'formattedDate': util.datestr(localdate(date)), 'individualChangeset': hex(ctx.node()), 'author': clean(person(ctx.user())), 'desc': clean(ctx.description()), 'mergeData': mergeData, 'max': gettotalpushlogentries(conn) } return pushes |
`new Function(”return ” + entries.responseText) ()` is saying
`eval(entries.responseText)`, perhaps use `JSON.parse (entries.responseText)`
I haven’t implemented this yet because I want more information on why the way I have done is not the appropriate way. I’ve asked on the bug page (comment 12)but haven’t heard back yet
The page loads more results when the user scrolls. What if the initial results don’t fill up the window?
I added the following CSS to always show a scrollbar no matter what. This should allow the user to scroll even if the initial entries don’t fill up the page.
html { overflow: scroll; overflow-x: auto; }
getMaxEntriesis called, then ’start’ is used once the user scrolls. But the user might scroll before the first result comes back, while start is still zero — a race condition
I changed the implementation to check the value of the start variable before more entries are allowed to load onScroll. If the value of start is still 0 then no entries are allowed to load. The following is the code:
$(window).scroll(function() { if(start > 0) { if($(window).scrollTop() == $(document).height() - $(window).height()) { renderMorePushLogResults(); } } });
Better to move CSS and formatting to stylesheets where possible
I removed explicitly setting the styles of the rows. Now I am just setting the class name of the row so that it uses the preexisting styles for that class name. View lines 37-43 for the code
The JS code isn’t consistently indented
It should be correctly formatted now. View here
This patch seems to contain some non-ASCII characters. Bugzilla doesn’t render it properly
Used &mdash in the places where non-ASCII characters were being used. View lines 73 and 121
In a few places the b tag is used where the server uses the strong tag
Changed the b tags to use strong. View lines 73 and 121
I will be putting up a new patch very soon…
EDIT:
The new patches have been posted here
v0.3 Release - Problems with my Patch for bug 459727
Posted by Sid in Mercurial Project, Open Source on December 2nd, 2008
Lately there has been some activity on bug 459727 concerning the patch I had submitted to implement an OnScroll feature for the pushlog. View comments 8 and 9 to see the problems that jorendorff as identified. So far there hasn’t been an official review but it is inevitable that I will have to make changes to the patch I had submitted.
In comments 10 and 12 I underlined what steps I will be taking to fix the problems that jorendorff has identified so far. They don’t seem to be major changes although I am worried about using JSON.parse(). I don’t know if that will work or not. I guess I will have to try it out and see what I can do.
Also I don’t really understand what is wrong with using `new Function(”return ” + entries.responseText) ()` as jorendorff mentioned in comment 9. In comment 11, Justin mentioned that it may cause vulnerabilities to XSS attacks but he couldn’t explain why that was. This has me interested and I would love it if someone could tell me why.
This is the biggest patch I submitted for my v0.2 so I’m expecting to go through several iterations before the patch is accepted. Look out for a new patch soon…
v0.3 Release - 1st Iteration of the Graph View
Posted by Sid in Mercurial Project, Open Source on December 2nd, 2008
I’ve been working hard trying to get the graph view working for the gitweb_mozilla template. Earlier, I documented the problems I was having. That is all resolved now and I’m moving on to bigger and better things.
So my goal was to get the graph to the stage of the coal template, which I am using as a reference and then make further improvements from there to make it look similar to the rest of the gitweb_mozilla pages.
To do this I created a new template called graph.tmpl which can be viewed by typing {url}/graph in the address bar. This is where the graph will be drawn. You can view the code it has here. However, with just this code my graph wasn’t rendering properly. The following is what it looked like: http://sidkalra.com/files/mercurial/graphView1.png. I couldn’t quite figure out why this was happening. It seemed like the graph was drawing properly and the data was appearing properly too and yet they weren’t coming together. Something was missing from the puzzle…
I showed my problem to a friend, Tiago Moreira who happened to be sitting right next to me as the time. He was able to point out that it might be a CSS problem, something I failed to consider. This was definately a lead and I had a look at the coal template’s css file and the answer hit me in the face straight away. The canvas tag was assigned a z-index of 5.
canvas { position: absolute; z-index: 5; top: -0.7em; margin: 0; }
This means that anything in the canvas tag will have a higher priority over everything else and it will also allow the data in other tags (in this case the two ul tags with the ids graphnodes and nodebgs) to slip in underneath the canvas giving the following result: http://sidkalra.com/files/mercurial/graphView2.png
Now my graph view looks similar to the one that the coal template has. My next objectives are to improve it greatly by making it look similar to the other pages in the gitweb_mozilla theme as I mentioned above and adding more changeset information to the each row. Basically the end goal is for it to look similar to the pushlog but with the graph drawn in as well.
v0.3 Release - A Problem with Getting data for the Graph View
Posted by Sid in Mercurial Project, Open Source on December 1st, 2008
hgweb has a graph function found in mercurial/hgweb/webcommands.py which returns a template. It also returns a variable called jsdata which contains all information that I want to display on the page along with the graph. Now, my problem was that I wasn’t quite understanding how I can could call graph() so that I could pass the data it returns to the client side.Should I have a new template or can I pass the data to an existing template.
For the longest time I was stumped on this issue. I didn’t understand whether I should be calling this function in my python code or could I call it on the client side using JavaScript? Did I have to play around with the template’s map file? I guess my lack of experience working with web applications that use templates and themes was rearing its ugly head. I was stumped!
I tried various methods via trial and error to see if I could figure it out but all in vain. Eventually I decided it was time for some help. I asked djc who told me that I would have to create a new template file which gets its data from the graph function in webcommands.py . But I still didn’t get how to actually pass the data to the client side.
So, I decided to ask for help in freenode’s mercurial channel. There, Brendan was able to point out that all I needed to do was add the following to hg_templates/gitweb_mozilla/map…
graph = graph.tmpl
The answer was staring me right in the face all along. I had glanced over this line many times while trying to figure out how this works but I guess I just didn’t realize that this is how you create a new tmpl file. Now I can access jsdata by adding {jsdata|json} in my template file.
Now I can actually start working on rendering the graph on the client side.
My patch for Bug 448707 Approved!
Posted by Sid in Mercurial Project, Open Source on December 1st, 2008
Some good news to report today. My patch for bug 448707 has been approved by jorendorff. He did find some problems but they were minor enough that they could be ignored for now.
Getting an r+ on the first go is pleasently surprising. Hopefully I can keep moving along and get a patch up for the graph view very soon.