Posts Tagged jQuery
v0.7Release - Retrieving the Total Number of Entries
Posted by Sid in DPS911, Mercurial Project, Open Source on March 12th, 2009
Earlier today I outlined the goals of my current release. One of the problems I was looking to fix was with the function, getMaxEntries(). The job of this function was to create an xmlHttpRequest() in order to retrieve the maximum number of push entries in the repository. The pushlog displays data in reverse chronological order and thus I need to know the maximum number of entries so that I know which entries to display first to maintain the same order.
For example in my test repository I have 2613 total entries. The first 10 entries are displayed by default so when the user scrolls down to load more data the very first entry shown would be #2603, then 2602 and so on. Previously, to retrieve the maximum number of entries I was using the following function:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | function getMaxEntries() { var entries = new XMLHttpRequest(); var max = 0; entries.open('GET', '/json-pushes?startID=0&endID=1', true); entries.onreadystatechange = function() { if(entries.readyState == 4) { if(entries.status != 404) { var entryData = JSON.parse(entries.responseText); max = entryData[1].max; start = max - 10; } } else return 0; } entries.send(null); } |
I didn’t really like this method of using an xmlHttpRequest() just to retrieve one value. I had to call this function onPageLoad() to calculate the maximum number of entries. This had to be done before my OnScroll function was called or various horrible errors would occur.
I’ve noticed something very unique about JavaScript. It doesn’t wait for one function to finish executing before going on to the next function. If your first function is taking too long, JavaScript will move on to your next function. Now, this feature was causing me quite a few headaches. As I said before, getMaxEntries() had to finish executing completely before the OnScroll function was called, otherwise things would go horribly wrong. getMaxEntries() sets the value of start, which is then used by loadEntries() to retrieve data (called OnScroll). Since getMaxEntries() was taking too long to execute, JavaScript was moving on to loadEntries() without setting the value of start, completely wrecking my logic.
Also, getMaxEntries() was causing Firefox to freeze for 10-15secs for reasons unknown to me. This way of calculating the max number of entries wasn’t very elegant and totally unacceptable. I needed to come up with a better solution.
The Solution
hgweb uses a template system called genshi. Basically, the maximum number of entries is calculated via a database query on the server side. In order to pass this server side variable to the client side I had to add the line 4 to pushloghtml() in hgpoller\pushlog-feed.py:
1 2 3 4 5 6 7 8 9 10 | return tmpl('pushlog', changenav=changenav(), rev=0, max=query.totalentries, entries=lambda **x: changelist(limit=0,**x), latestentry=lambda **x: changelist(limit=1,**x), startdate='startdate' in req.form and req.form['startdate'][0] or '1 week ago', enddate='enddate' in req.form and req.form['enddate'][0] or 'now', querydescription=query.description(), archives=web.archivelist("tip")) |
Now, this change ^ allowed me access to this variable on the client side. All I had to do was to use this format: {<var_name>} or #<var_name>#. However, one draw back as far as I know is that I can only access this varialble in HTML, not in JavaScript, which is a significant drawback.
So in order to retrieve this data I added an id attribute the following div tag in hg_templates\gitweb_mozilla\pushlog.tmpl:
1 | <div id="#max#" class="page_header"> |
Now, I could set the value of start to the maximum number of entries:
1 | start = $("div").attr("id"); |
Thus, I can easily pass the correct value of start to loadEntries(), which makes an xmlHttpRequest() to retrieve the correctly ordered data when the user scrolls down.
Further Progress with pushloghtml’s OnScroll Feature
Posted by Sid in Mercurial Project, Open Source on November 5th, 2008
I’ve been working hard on bug 459727 to get it working. Right now it’s starting to look like it should but there are still some problems that need to be fixed.
var start = -19; var end = start + 19; function renderMorePushLogResults() { $('test').html('<img src=\"D:\CRULSHORUKH_Mobile\SENECA\SEMESTER_7\DPS909\Mercurial\ajax-loader.gif\">'); start += 20; end = start + 19; var pushCheckins = new XMLHttpRequest(); pushCheckins.open('GET', '/json-pushes?startID=' + (start - 1) + '&endID=' + end, true); pushCheckins.onreadystatechange = function() { if(pushCheckins.readyState == 4) { if(pushCheckins.status != 404) { var pushData = new Function("return " + pushCheckins.responseText) (); var counter = 0; for(var i = end; i >= start; i--) { var trScroll = document.createElement("tr"); if(counter == 0) { counter = 1; } else { trScroll.style.backgroundColor = "#f6f6f0"; counter = 0; } var tdScrollUser = document.createElement("td"); tdScrollUser.width = "184px"; tdScrollUser.innerHTML += '<i>' + pushData[i].user + '<br />' + pushData[i].formattedDate + '</i>'; var tdScrollChangeset = document.createElement("td"); tdScrollChangeset.innerHTML += '<a href=\"/rev/' + pushData[i].individualChangeset.substring(0, 12) + '\">' + pushData[i].individualChangeset.substring(0, 12) + '</a>'; //Create buglink var bugInDesc = (pushData[i].desc).indexOf("Bug"); if(bugInDesc == -1) bugInDesc = (pushData[i].desc).indexOf("bug"); if(bugInDesc != -1) { var bugLinkName = (pushData[i].desc).substring(bugInDesc, bugInDesc + 10); var bugNumber = bugLinkName.substring(4, 10); var bugLink = (pushData[i].desc).substring(0, bugInDesc) + '<a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=' + bugNumber + '\">' + bugLinkName + '</a>' + (pushData[i].desc).substring(bugInDesc + 10, (pushData[i].desc).length); } else { //No bug provided var bugLink = pushData[i].desc; } var tdScrollAuthorDesc = document.createElement("td"); tdScrollAuthorDesc.innerHTML += '<b>' + pushData[i].author + ' - ' + bugLink + '</b>'; trScroll.appendChild(tdScrollUser); trScroll.appendChild(tdScrollChangeset); trScroll.appendChild(tdScrollAuthorDesc); document.getElementById("titlePush").appendChild(trScroll); } } } } pushCheckins.send(null); }
The above code gives the following result. This is a marked improvement from my previous results wouldn’t you say? Now how did I work all this out? If you read my previous blog post you would know how many problems I was having with the formatting and getting the right data. Let me explain how I have rectified the situation:
Formatting
Currently all the formatting is done by the template files map and style-gitweb.css, which get data from python-feed.py and format the data. Now when I load items when the browser registers the OnScroll event there is no way to render the page again so that map and style-gitweb.css can format this new data (as far as I could tell at least).
I have decided to do all the formatting on the JavaScript side for all the new entries. It is a bit tedious and clumsy I suppose but it gets the job done. Take for example the bugzilla links. For the normal entries that appear onPageLoad the bugzilla links are setup by a server side service called buglink.py but again the new entries don’t have access to this service so I have to do it manually. Basically:
- I take the description string
- Get the position of “bug” or “Bug” in the description string by using indexOf()
- Setup the link which will always be “https://bugzilla.mozilla.org/show_bug.cgi?id=” + the bugId
- Reassemble the description string but this time with the link of the bug
Acquiring the Right Data
As I mentioned in a previous post I didn’t have all the data I needed. To acquire the needed data I had to change the functionality of the server side script I was using, json-pushes. The following are the changes I made:
def pushes_worker(repo, startID=0, endID=None): #pdb.set_trace() stmt = 'SELECT id, user, date, rev, node from pushlog INNER JOIN changesets ON id = pushid WHERE id > ? %s ORDER BY id ASC, rev ASC' args = (startID,) if endID is not None: stmt = stmt % 'and id <= ?' args = (startID, endID) else: stmt = stmt % "" if os.path.basename(repo.path) != '.hg': repo.path = os.path.join(repo.path, '.hg') conn = sqlite.connect(os.path.join(repo.path, 'pushlog2.db')) pushes = {} mergeChangesets = {} for id, user, date, rev, node in conn.execute(stmt, args): ctx = repo.changectx(node) if len(ctx.parents()) == 2: #code to deal with merge changesets, u can access stuff by going ctx.parents()[0].description() or ctx.parents()[0].user() par = ctx.parents() if id in pushes: pushes[id]['changesets'].append(node) else: pushes[id] = {'user': user, 'date': date, 'formattedDate': util.datestr(localdate(date)), 'changesets': [node], 'individualChangeset': hex(ctx.node()), 'author': ctx.user(), 'desc': ctx.description(), 'isMerge': len(ctx.parents()) } return pushes
I had to alter pushes_worker() so that it recieved web.repo so that I could perform ctx.web.repo.changectx(node), which gives me the information I need like ctx.user() and ctx.description(). To clarify in the above code the passed in variable repo = web.repo
I was also able to format date in the appropriate way by calling util.datestr(localdate(date)). This required me to get the util module.
Further improvements
I’m not done yet, there are still things that need to be done:
- Chronological Order: This has me a bit confused. Right now I have it set that for the very first time the user scrolls to the bottom of the page json-pushes loads with startid=0&endid=20. The next time it loads startid=20&endid=40 and so on. This doesn’t have the same effect of going from pushloghtml to pushloghtml/2 to pushloghtml/3 and so on, which is what I want to happen, right?
- Merge Changesets: This functionality doesn’t work right now. My problem is how do I know whether something is a merge changeset or not? Well, I’ve answered that question now thanks to djc. I need to use len(ctx.parents()) which returns 2 if a node is merge changeset. Then I can access the description and user data by going ctx.parents()[0].description() or ctx.parents()[0].user()
- loader-gif: I want to add a loader gif animation so that users know that the entries are loading, which is quite a standard feature for AJAX applications. But my problem is that I don’t know where in my file structure I should be placing the gif file. I’ve tried something like
$('test').html('<img src=\"{url}ajax-loader.gif\">');
and placed my gif file everywhere possible but nothing has worked yet
- Keeping expand/collapse functionality: I want to make sure that my new entries keep the expand/collapse functionality for merge changesets.


