The Unheralded Perspective

Getting pushloghtml Up and Running Part Two

Posted on October 12, 2008 by Sid

Carrying on from where I left off , my problems with python modules continue. After some conversations with djc, bsmedberg, ted and jorendorff it seems that Mercurial is using an entirely different python interpreter and that is why even though my 2 modules are installed properly hg serve is still throwing me an error.

I installed my Mercurial using a windows binary and it seems that it is based on py2exe, which zips up its own libraries and installs them separately along with a separate Python interpreter. Now, this is quite a pain since now I have to install Mercurial from source so that it installs in my pythonscripts folder. Why doesn’t the windows binary just install Mercurial directly to the Python folder on my machine? If py2exe is trying to do its own thing and using a separate Python interpret then maybe it shouldn’t be used because what if you want to add more modules that you need (which is exact my problem)? Then you have to bite the bullet and build from source. I’m sorry to say but that’s not very developer friendly. I mean there should be an easier way to add more modules to Mercurial

Building Mercurial from Source

I used the python setup.py method to build Mercurial. I followed the instructions here. So I downloaded the source and installed Visual C++ 9 (python extensions require a C compiler). Then I ran python setup.py build from the top level directory of the Mercurial Source. Text rushed through my console and BAAM! An error!

Turns out you need to use the same C compiler to build python extensions as the one you used to build python itself. Awesome, well my particular python build used Visual Studio 2003 (I just installed it from an exe, I didn’t build it myself). I can’t use that so I need to find another way. Well the documentation that I was using provided some help as to how I could use another C compiler.

After spending hours trying to get Mercurial to build with Visual C++, cygwin and then with mingw I somehow finally got it to work with mingw with the help of my friend, Tiago Moreira. This was a very tedious and unpleasant experience. My problems with Mingw were happening because I wasn’t installing it in my C: drive (since I have no available space) but it turns out that Mingw doesn’t like being installed in any drive that isn’t C: for whatever reason. Now after I miraclously cleared up space on my C: drive and snuck Mingw in there and then set the path in the windows environment variables my installation still didn’t work. Well, it turns out the path needs to be explicitly set on the command line for the build to work.

This experience hasn’t been very pleasant and I definitely think that python needs to figure out a better way to install extensions. I mean I should be able to use any C compiler I want with relative ease. Right now one has to go through a tedious process of configuring python to use the right compiler. If anyone out there can explain to me why that is, I’m all ears. The way that the documentation is written doesn’t help either. There is definate room for improvement when it comes to the clarity and detail of the document.

I know what some of you might be thinking at this moment. Why is this guy complaining about user friendliness and ease of use when talking about developer tools? They are not meant to be user friendly. Well, I say why not? I mean I am not asking for the program to hold my hand here but I would prefer if it didn’t work against me. I know some, maybe most of these troubles have croped up because of my inexperience with these technologies but that’s exactly my point. How can you get new people involved in your open source projects if there are frustrating road blocks in the way from the very beginning?

See, what I’m wondering is whats stopping Mozilla or the developers of Mercurial from making an installation bundle, which makes it easier to get a hold of all the dependencies. I know the Microsoft SDKs that Firefox needs to perform a build are proprietary. However, that could easily be handled if the installation bundle somehow pointed you to the download page and once the setup file is downloaded the bundle could then run the installation for you. I haven’t looked into the technical details of doing this but I would think that this would be possible. Maybe the developers have thought about this but haven’t had the time or the resources to implement it, I don’t know. I think creating an installation bundle would be a good idea, especially for beginners like myself. But nonetheless, enough of my ranting. I am not one to give up once I decide to take something on. I will continue working and putting in effort into these projects but I don’t know whether others will do the same.

All the troubles aside I finally got my installation working and ready to roll. I ran hg serve from the hgpoller repo and VOILA! No more errors, pushloghtml was FINALLY up and running. Now I can actually start working on my project

hg serve, no errors!

Posted in Mercurial Project, Open Source | Tagged Mecurial, Mozilla, pushlog | Leave a comment

Bugzilla – Watch a User Feature

Posted on October 7, 2008 by Sid

As a part of an exercise humph asked us to fiddle around with bugzilla. In this case, watch another user. Now what does that mean? That means that any bugmail that the user you are tracking receives will get forwarded to you. You can set this by

going to bugzilla
logging in
going to preferences
adding a person to your watch list.

This basically gives you an idea of what kind and amount of bugmail that that person receives. It puts you in their shoes.

I decided to track bsmedberg since he seems to be a busy man. So I “watched” him for about a week and got around 70 bugmail during that time. Thats quite a lot of traffic for just one week. It definitely gives me an idea of the amount of work bsmedberg has to deal with on a weekly basis, and this is just email.

This bugzilla feature gives me an idea of how someone like bsmedberg (someone that has a more important presence in the Mozilla community than myself) deals with the various kinds of bugs that he recieves on a regular basis. I’d recommend this feature to any novice or beginner looking to get involved with tracking or fixing bugs.

Posted in Open Source | Tagged Bugzilla, Mozilla | 4 Comments

Getting pushloghtml Up and Running Part One

Posted on October 5, 2008 by Sid

The past 2 days I have been struggling to get pushloghtml working. As I had mentioned in my previous post I needed 2 modules to do this, simplejson and pysqlite2. Turns out both of them are python libraries, which bsmedberg graciously pointed out to me.

Finding and Installing simplejson and pysqlite2

First thing I had to do was to install Python on my machine. I went with version 2.6, which was just released on Oct 1st (eventually I uninstalled it and used v2.5.2 since nothing was compatible with 2.6 yet). Then I hunted down and found simplejson’s and pysqylite2′s source. Now, I didn’t know what to do from here. How do I install them? I didn’t know. I laboured through various google searches but no avail. But then I remembered what bsmedberg had told me, use ez_install.py to install these libs. So I hunted down and found the file on this blog (there doesn’t seem to be an official place where this is housed). I downloaded and installed it on my machine.

One can give Easy_install python eggs to install packages/modules. Now I needed to find simplejson and pysqlite2 in egg format, which I found at the CheeseShop (or PyPi). I ran both the files with easy_install but all it did was put the respective egg files in my Python252Libsite-packages directory. Does that mean it installed properly? I don’t know. Ohhh, the challenges of being a novice.

I tried installing the exe for pysqlite2 to see if it does the same thing as easy_install (put an egg file in my python directory). It didn’t do that. It added

D:CRULSHORUKH_MobileAppsPython252Libsite-packagespysqlite2

I guess that this means easy_install didn’t install simplejson correctly. I was sure now that pysqlite2 had been installed properly so I tried running hg serve to check whether the “no module found” error was gone for pysqlite2 at least. No luck, the error was still there.

What is the best way to know if a package installed correctly? Well, just use it! How simple eh? So I decided to create the following hello world program

import pysqlite2, simplejson
 
print "Hello World"

The result? No error! The program printed Hello World onto the screen, which means that the packages did install correctly! Now, this got me really confused. I have the packages that I need on my computer but why am I still getting the “no module found” errors?

Deciding on v0.1 Release

Posted on October 2, 2008 by Sid

ETime is expeditiously approaching for me to decide on what I will be doing for my v0.1 release. The deadline is Oct 13th. As I mentioned on a previous post jorendorff had made a great blogpost on my project asking for people to brainstorm ideas to improve hgweb.They have been summarized on the project page. I think most of the ideas on there are not ideal for a v0.1 release. It would be wiser to do something simpler to start off and then go from there.

Before I think about starting on 0.1 I need to setup a development environment for myself. This involves getting gitweb_mozilla, buglink.py and pushloghtml all installed and working. I have the first two up and running now. As far as pushloghtml goes it is a database. To be frank, I don’t know much about it and I’ll have to query one of the big guys (jorendorff, ted, bsmedberg or djc) for some help. I have gotten a hold of it thanks to ted and aravind. Ted pointed me towards a README file in hgpoller that shows how to get the pushlog working. Basically I have to add a couple of lines to the the .hghgrc file (as I suspected). I tried that but I’m getting the following errors…

hgwebjson.py: No module named simplejson pushlog-feed.py: No module named pysqlite2

In my email conversations with jorendorff I had asked him for some suggestions as to what I could do for 0.1 and the following is what he recommended…

Make pushloghtml show more than 10 entries (try to make it scroll forever)
Give large pushes in pushloghtml an expand/collapse button (Pushes can contain many changesets)
Make pushloghtml show which files each changeset touched

Now I am at liberty to do anything I want so I had a look at the ideas that jorendorff’s blog generated and the following caught my eye (keep in mind that I don’t know if any of these are too much or too less for a v0.1 release)…

Show more info about each changeset in an unobtrusive way (what information? I don’t know)
Be able to view more than 10 entries at a time in the changelog

I think out of all the the ideas I have listed I will most probably will be taking on the task of making pushloghtml show more than 10 entries. It seems quite straightforward although keep in mind this is coming from someone who isn’t quite familiar with the source code yet. Basically my tasks are

to get pushloghtml working on my machine
to locate the source files that dictate the amount of pushlog entries that get displayed and then alter it to display more entries.
to release v0.1 on time

EDIT: I asked humph to read this post and ok my v0.1 release agenda. He recommended that I take on both the tasks of allowing more than 10 entries for pushloghtml and adding an expand/collapse feature since they seem to go together. So there we go my list is final for v0.1 release…

to get pushloghtml working on my machine (some progress made on this part)
to locate the source files that dictate the amount of pushlog entries that get displayed and then alter it to display more entries
to add code that allows expand/collapse functionality (apparently all I need is a js lib according to humph)
to release v0.1 on time

Lots of work ahead!

Posted in Mercurial Project, Open Source | Tagged hgweb, Mercurial, Mozilla | Leave a comment

Playing with Mercurial styles

Posted on September 29, 2008 by Sid

As a sort of “Hello World” type of exercise jorendorff recommended that I play around with the styles in hgweb. I had 2 goals to achieve

My mission is to get the gitweb_mozilla style working on my computer
Then edit the templates to make the whole page orange

I downloaded a repository provided by bsmeberg called hgpoller. This would be my development playground. Now, I thought that my next step would be to make changes to hgpoller.hghgrc to change the style. I tried that by adding

[web]
style=gitweb_mozilla

This didn’t work as planned. The style reverted to the default. What I didn’t realize is that Mercurial gets its style specifications from the Mercurial.ini file (in Windows). The style must be adjusted from there. When I added style=gitweb_mozilla to that file the change worked and I got the appropriate result. To make Mercurial take its style information from the hgpoller.hghgrc file I added the following line with some help from jorendorff

[web]
templates=<template-dir>

Now, I can easily make changes to the hgrc file which will be reflected on hgpoller. You may be wondering how do I see the repo in a browser? To do that one needs to use the hg serve command which will start a hgweb server on port 8000 on your machine. Then one can just visit http://localhost:8000/

My objective was to get the gitweb_mozilla style working on my machine but I couldn’t find that style listed in my templates directory. The reason being that I needed to clone that style from hg.mozilla.org (thanks for pointing that out djc).

After getting the style operating the next step involved getting the bugzilla links working. The key to that is the hgpollerbuglink.py file. It has to be enabled by entering the following in hgpoller.hghgrc

[extensions]
buglink=<path-to-buglink.py>

After doing all that I was finally able to achieve my first goal which gave me this result http://sidkalra.com/files/mercurial/my_gitweb_mozilla.png

The next step was to alter the style, in this case change the whole page to orange. To do this one has to alter the <your-template-folde>staticstyles-gitweb.css file. I added the background-color attribute to the body area and set it to orange like so…

body { font-family: sans-serif; font-size: 12px; margin:0px; border:solid #d9d8d1; border-width:1px; margin:10px; background-color:Orange; }

which gave this result http://sidkalra.com/files/mercurial/my_gitweb_mozilla_orange.png

Mission Accomplished!

Posted in Mercurial Project, Open Source | Tagged css, hgweb, Mercurial, Mozilla | Leave a comment

Mercurial – Installation and beyond

Posted on September 24, 2008 by Sid

I had a chat with the jorendorff today about what I need to do to get my Mercurial (aka hg) project rolling. He outlined the steps we should look to take…

We have to make sure you understand hg
We have to decide on some very small thing that you can get running very soon
We should talk it over and make sure it’s clear how to do it
We should plan to meet up later and brainstorm some, once you’ve had a chance to play around with hg

I can add another point to this. I need to decide what exactly I will be doing for my 0.1 version release. Right now I don’t have a clue. But first things first, I need to install and play around with hg. Here goes…

Installation

I will be installing on Vista (yeah, I know what you’re thinking but seriously, it works fine) so I need to find a windows installation. The installing Mercurial guide recommends just installing MozillaBuild since hg comes packaged with it. But I rather not do that at the moment because I don’t need the rest of the tools from MozillaBuild. Well, I was in luck since there are binary packages for Windows here and even Mac. So I downloaded the binary and went through the installation.

Now to check if hg installed properly just head to the dir where you installed it (using cmd) and run hg.exe. Then type hg version to get the version info. You should get something like this…

Version Check

Now, I needed to make some changes to a config file. All the guides I read were telling me to make changes to a hgrc file but I couldn’t really find any config file with that name. It turns out you need to make changes to Mercurial.ini file for windows and the hgrc file is for UNIX systems. Here you need to tell hg what merge program to use. I decided to go with TortoiseMerge since I am already familiar with TortoiseSVN.

Downloading Mozilla Source

Now to get to the real part. All the preliminary work leads up to the moment of truth! Downloading the Mozilla trunk source! Now to do this you need to use the hg clone command and pull from http://hg.mozilla.org/mozilla-central/

One thing that hg desperately needs is a way to know your download progress. If you haven’t ever downloaded the Mozilla source then let me just tell you that it is HUMONGOUS! I’m not kidding. It took forever to download it. Ah, command line tools, you win some and you loose some.

Beyond

What’s next after downloading the source? Well you can do many things i.e. pull a file and make changes to it then push it back in and much much more. Since I will be working with everything related to history I decided to try out the hg log feature. This command, by default prints each change recorded to the project. Each of these recorded events are known as changesets because they can contain records of changes to several files.

I will continue to play around with hg and hopefully make progress. Meanwhile jorendorff has a post on his blog about hgweb in which he’s calling for ideas to improve it (this will be my job) so go over there and contribute!

Posted in Mercurial Project, Open Source | Tagged Mercurial, Mozilla, Open Source, TortoiseSVN | Leave a comment

Mozilla Developer Days Toronto

Posted on September 22, 2008 by Sid

Last week Mozilla held a Developer Day conference at Seneca College. This brought various Mozilla developers from its Toronto and California offices to Seneca to talk about all things Mozilla. The conference lasted for the entirety of two days with various people presenting different types of Mozilla technologies.

I got to sit in during one of ctalbert’s presentations. To be frank a lot of what he said went straight over my head. I wish I had better understanding of what he was talking about but that would have required me to do some preparatory work. I even got to talk to ctalbert for a bit about the Private Browsing Tests project during the BBQ lunch because at the time I was thinking about choosing that as my project. In the end I picked the Mercurial project but still it was interesting to learn a bit more about this upcoming Firefox feature.

The presentations were divided into two tracks, the desktop and web track. The presenters talked about XUL, XPCShell testing, Mochitest testing, memory leak testing and much more. Unfortunately I couldn’t attend many of the presentations because of class. I wish the conference could have been held on a weekend but I’m sure many people would object to that, which I totally understand. Surely many other people that wanted to attend couldn’t do so for various reasons but not to worry since videos of all the presenations will be coming very soon. Keep checking the wiki page for further details.

Posted in Open Source | Tagged Mercurial, Mozilla, Mozilla Developer Day | Leave a comment

Mercurial, A Distributed Version Control System

Posted on September 22, 2008 by Sid

Well, I’ve finally decided on my project for this open source course. I am happy to report that I will be working on Mercurial. For anyone that doesn’t know it is a Distributed Version Control Sytem (DVCS). Now if you are like me you’re probably wondering what in the world that is? Is it like subversion or CVS, if not then whats the difference? Let me explain…

Centralized Version Control System (CVCS) vs Distributed Version Control System (DVCS)

CVCS focuses on synchronizing, tracking and backing up files
DVCS is for sharing changes. Every change has a GUID
Downloading or Recording and applying changes are all separate steps while in a CVCS the happen together
DVCS would be suitable for larger projects with many developers

Mercurial is now used to house all Mozilla source. One of the advantages of a DVCS like Mercurial is that it frequently has branches and merges. The downside of this is that the history can become complicated and confusing very fast. For the project my job will be create a hopefully awesome browser-based UI for navigating Mercurial repository history. But before we can get to that it is important to understand how Mercurial iteself works. One can find a great getting started guide to Mercurial here. Have a read if you are interested.

Now my next task is to quickly get in touch with people who can guide me in the right direction and get me started on this project. But first I’m going to try to download Mercurial and pull the Firefox source with my own two hands. I hope to have my experiences on here in the coming week…

Posted in Open Source | Tagged CVS, Mercurial, Open Source, SVN | 2 Comments

Picking an Open Source Project

Posted on September 13, 2008 by Sid

Well the time to choose my project for the open source class is approaching very quickly. I’ve narrowed my choice down to 3 now. But I’m still in a dilemma. Which one should I go with? Well, lets look at my choices first…

I find all three to be interesting and would love to do all of them in fact. However, I need to choose one (Must follow the rules, am I right?). Lets have a closer look at each of them…

Contribute to Private Browsing Tests

Firefox is implementing a new private browsing feature that will allow users to conduct their activities on the Internet without leaving any traces behind. Chrome and IE 8 Beta have already implemented this feature and Firefox doesn’t want to be left behind. I would love to be part of this effort. Privacy is something that every user craves on the Internet and at the moment there is very little of it. Implementing this feature would definitely put Firefox in the same league as its competitors in terms of user privacy. Contributing to such a pivotal feature definitely interests me.

Create Self Serve Symbol Upload System

Mozilla has a symbol server that developers use to debug issues in release builds that don’t have debug information. Many crashes are related to third party plugins or add-ons. This project would involve setting up some way by which plugin or add-on developers could upload their symbols. It apparently involves web development, which I am very fond of. I don’t know much about symbols so that has me a bit perturbed. Nonetheless I wouldn’t mind in the least bit to work on this project.

Help Calendar Integration on Trunk

To be honest I don’t know much about this project. I definitely prefer the other 2 projects above. This is more of a backup project for me. As far as I know trunk is the main Mozilla source tree and this project mandates that a calendar be setup to integrate with trunk. I’m not exactly sure how this would be useful or how it would be implemented but I’m willing to find out.

To be perfectly honest right now I’m leaning heavily towards the private browising tests project. I’ve contacted the main resources, Ehsan and Marcia and they’ve given me further details about the project. I’m hoping to meet couple of the QA guys coming to the Mozilla Developer Conference here at Seneca to have a chat so that I can get more details about the project. Hopefully by Monday I will have decided which project I will persue so be on the look out.

Posted in Open Source | Tagged Firfox Privacy Feature, Mozilla, Open Source, QA | Leave a comment

Ubiquity, a Firefox Extension

Posted on September 11, 2008 by Sid

In our last open source class Dave asked us to work through a lab where we worked with Ubiquity, a Firefox extension. So, I had a go at it. I installed it and read the articles that introduce the add-on to a new user. I have to say that it it quite useful and intuitive. Although the extension is not yet at version 1.0 I think I will still use it. I have to say I’m a bit eccentric about extensions. I love using them and playing around with them. I can’t live without them, to be honest. Ubiquity is just another addition to my ever growing add-on list.

Ubiquity adds great convenience to your browser. It’s basically a command line tool for performing browser tasks. Anybody that likes working with an easily accessible command line will greatly enjoy this extension. All one has to do is press ctrl-space (on Windows) and ubiquity will appear in the top left corner of your browser, ready to use. Now, all you have to do is enter the right command and it will do your bidding.

One feature that I’m particularly impressed with is the ability to email your selection. This can include links, pictures and anything else. To do this all one has to do is select whatever you desire, press ctrl-space and then type “email this to [recipient's email]” or “email selection to [recipient's email]” or “email it to [recipient's email]“. Very useful if you ask me.

Ubiquity has many built-in commands that come with the add-on but it also allows users to put on their programming hats and do a bit of coding themselves to create their own commands. This feature is great since the users are no longer limited by what the developers put into ubiquity. If you feel that there should be some command in ubiquity that isn’t there currently, then you can code it yourself. This is a great example of how open source works. Users can very easily become developers and hats off to the Ubiquity development team for making this transition as seamless as possible.

I tried my hand at making my own commands. I began by coding a simple user search for our zenit/wiki.

CmdUtils.CreateCommand({
name: "zenit/wiki user search:",
takes: {"search zenit/wiki user": noun_arb_text},
preview: function( pblock, wikiquery ) {
pblock.innerHTML = "Searching For: " + wikiquery.text;
},
execute: function( wikiquery ) {
var url = "http://zenit.senecac.on.ca/wiki/index.php/Special:Search?ns2=1&amp;search={QUERY}&amp;searchx=Search";
var query = wikiquery.text;
var urlString = url.replace("{QUERY}", query);
Utils.openUrlInBrowser(urlString);
}
})

The Ubiquity development team has made it quite easy for the users to create new commands. All we have to do is follow the command structure. So as you can see here for example in the name section you type the name of the command, in the execute section you code what the command is exactly suppose to do. So in this case the function receives the name of the user and performs some URL manipulation and opens the new URL in a new tab. Very simple stuff. Other students also created their on commands. You can have a peak at them here.

I also tried to create another command for jslint. Basically the idea was to select any text (hopefully JavaScript code) and then type jslint into the Ubiquity command line. This should open up jslint in a new tab with the selected text being displayed in the main textarea of jslint (where one is suppose to paste the JavaScript code). Unfortunately, after a few hours of struggle I couldn’t get it to work. One of things that make it quite hard is that jslint is coded in Ajax and it’s URL doesn’t change, which means that I cannot perform any URL manipulation. I attempted to use html.replace and even looked into using jQuery‘s append function but all in vain. Eventually I gave up. I’m sure I could have figured it out given more time. Just a shout out to Tiago for spending the time with me to try to figure this out, he was very helpful.

Below is the code we came up with, which obviously does not work but if anybody wants to take this as a starting point and go from there please be my guest. Just make sure to post your results on here good or bad. I would love to know.

CmdUtils.CreateCommand({
 name: "jslint",
 takes: {"word": noun_arb_text},
 execute: function(directObj) {
  var selectedCode = directObj.text;
  var urlString = "http://www.jslint.com";
  Utils.openUrlInBrowser(urlString);
  var params = {
  };      
 
  jQuery.post( urlString, params, function( html ) {
   var old = "<textarea id="input"></textarea>";
   var newHtml = "<textarea id="input">" + $selectedCode + "</textarea>";
   html = html.replace( old, newHtml );
 });
 },
})

CmdUtils.CreateCommand({
 name: "jslint",
 takes: {"word": noun_arb_text},
 execute: function(directObj) {
  var selectedCode = directObj.text;
  var urlString = "http://www.jslint.com";
  Utils.openUrlInBrowser(urlString);
  var params = {
  };      
 
  jQuery.post( urlString, params, function( html ) {
   ("textarea").append(selectedCode);
 });
 },
})

Posted in Open Source | Tagged Add-on, Development, Extension, Firefox, Mozilla, Ubiquity | 1 Comment

Archives

Tags