Category Archives: Code and Stuff

Milestone 0.4: Release wha? Just fixing broken things!

For the last milestone, I’ve been focusing a lot on our implementation of Mouse Lock, and thanks to the great work and effort put in by our class and professor, Dave Humphrey, we got our implementation up for review last week Friday. Incidentally we got a review back, which wasn’t quite as bad as I had expected. A lot of things still left to do, but the first round of feedback was definitely less scathing that I was led to believe. In terms of actual implementation I haven’t really contributed as much as I would like, my major one was getting the mouse to disappear upon the lock method being called, other than that most other things have been small potatoes.

I also served as a sync point for the class’ tests. Originally I had thought this task would be fairly simple. I’ve been doing this sorta thing for a while now, keeping track of Jesse, Diogo, and Dave’s changes. Unfortunately it didn’t scale up very well. When nearly the whole class started to get involved in the test creation, handling the pull requests became quite a mess. Especially when during the merges, one or two pull requests would cause an unexpected failure on the code. In such cases, I had to put aside what I was doing to fix the repository as soon as possible in order to prevent other classmates’ repositories from going kaputz. During this time I was also able to push out a test of my own. My test file_MouseEvents.html deals with whether or not our implemention properly handles all the possible mouse events. For example, events that require the presence of a mouse cursor (i.e., mouseover, mouseout) shouldn’t exist when the mouse is locked. Also user generated events (i.e., mousemove, button presses) should only affect the element that has been locked. It doesn’t matter if event happens to occur right over another element, only the one that the mouse is locked to should get the event. My test ended up having something like 42 tests. It’s really only this big because all the tests need to be done twice. Once when mouse lock is active, and once when it’s not. The reason why I test when it’s not active is because I don’t want future changes to break how the mouse works normally. Its initial completion coincided with Friday, the day we would submit our work for review.

Unfortunately, it would seem that most of the tests did not use asynchronous calls correctly or used some sort of timeout method. This causes random failures and is overall, a hallmark of a poorly done test. Of course, my test was part of that group. orz … While fixing my test (I feel like I made it a lot more complex than it needs to be), I’m trying to also tackle a case where 2 tests would fail for mac users. For some reason, mouseout and mouseleave events were not firing under normal circumstances. I’m not really sure what to make of this, and I’ve made a slight change to my code (instead of going to pixel 100 which is the width of my box, it is now 101) and would love to see if this was the reason why it didn’t work before. Fixing the test itself (remove timing issues by utilizing the success callback) proved quite challenging. Aside from maybe a week in Java, my exposure to threads and thread programming has only been in theory. So when it came down to doing it for real, things didn’t turn out quite so smoothly. That said, it finally works without any wonky behaviours.

I’ve taken up another test, file_limitlessScroll.html which has its fair share of problems. I’m actually going to do a full rewrite. There are a lot of errors in there, but there’s one very big logic failure. All the synthesize mouse calls tell it to go to the same place 3 times. It needs to use ScreenX and Y in its calculations to truly be able to emulate movement. The aim is to have this done by Thursday night and have someone peer review this, because in its current form, limitlessScroll is causing mochitest to hang and fail.

Test wise, things are starting to look up, I have peer reviewed 2 tests, both written/rewritten by northWind, and aside from a few lines that were longer than the 80 char limit, it was good. So that’s 2 tests down. =)


Road to Mouse Lock: Part 8

It’s been almost a week and a ton of things have happened since the last time I posted. The biggest change is that our tests were more or less assigned to students, which was fine because most if not all the implementation has been done. Incidentally, I was asked to be in charge of consolidating the mochitests. The tests are stored in my github branch mouselock-tests. I was pleasantly surprised by the number of people who managed to complete their tests by Saturday night.

That said, I’ve been only doing quick looks at the code and accepting them, going with the honour system until now. I’m aiming to run through all the tests as well as looking at the code to remove the use of setTimeouts and tests that aren’t quite working.Of course, I’ve had my own tests to write, number 17 and 18.

The Problem:

Writing a mochitest that checked for various mouse events like mousedown, mouseup, mouseover, mouseenter, mouseleave, mouseout, and click. For mouseover, mouseenter, mouseleave, and mouseout, we need to make sure that these events are no longer triggered when we are in mouselock. For mouseup, mousedown, and click, we want to make sure that only the element that is locked can receive these events.

What I’ve Tried:

So for the mouse position related events, I created a div within a div. I full screened and locked the mouse to the larger container div and moved the mouse around the contained div to check the values.

For the mouse click events, again I created a div within a div, but this time the inside div is places right in the middle of the div that is mouse locked. The idea is that when I do any mousedown, mouseup, mouseclick event, only the div that we locked our mouse to catches the event. The inner div that is placed in the middle shouldn’t.

As originally mentioned I tried the createEvent javascript method. However, doing this skips over the event system totally which means that it won’t serve our purposes. So a quick search via mxr yielded a method called synthesizeMouse that served our purpose. While working on this Saturday night, Tentacle and I  (reads so wrong…), found that trying to use synthesizeMouse with the click mouse event would cause an error. The error being that the DOMWindow’s sendmousevent did not support click. The work-around I found was to call sendMouseEvent to handle the click. After that was basically figuring out the workflow of the test (where to add what event listener ) to get the test running.

The Solution:

It works! Compared to many of the tests so far that’s just 1 or 2 tests per file, mine is something like 24 tests. It seems so deceptively simple when you look at the original description but every mousedown and mouseup require a check for left, middle, and right mouse clicks, and having to keep track of the numbers can get really messy.

What’s Next:

The final stretch is here now. The first implementation (where mouselock requires full screen) is almost complete, just one or two dangling things. I plan to look at how mozilla’s full screen api handles focus lost and seeing how we can implement that for mouse lock. As well, there are still some tests remaining that need to be done and I might help do some of those as well.

Road to Mouse Lock: Part 7

So it’s been a while since my last post, work hasn’t stopped, just slowed. My other courses have large assignments that are due coming up so I’ve been focusing getting those done ahead of doing things to implement mouse lock. During the hour of spare time in between my study for a BTP500 test, I whipped up a really quick demo of how mouse lock would work on a canvas. Unfortunately there’s a stretching issue that I haven’t resolved yet, but the basic idea is there. Luckily not long after, during the recent weekend, Jesse made an incredible demo (he added in mouse lock to an existing fps browser demo). I ran it with a debug build, but I don’t think that’s the cause of my problems, but clipping seems to not work as it should at times and the mouse tracking seems to occur even when mouse lock is off, but those are minor details considering the first actual prototype with mouse lock has been implemented. Aside from that I have 2 other things I’m going to talk about…View post

The Problem:

  1. When working on a dual screen monitor, I get odd behaviours from both mouse lock and full screen. When I’m in full screen and mouse locked mode, I’m able to (if I’m fast enough) take focus away from the browser by clicking (usually the right mouse button) on my second monitor and get results that look something like the following:

    Currently, my mouse is on my second monitor, unfortunately the results I get vary a lot when I try to replicate this behaviour. Sometimes everything will work exactly as it should, other times mouse lock and full screen stays turned on, other times it’s just full screen that sticks. I suppose part of the problem with mouse lock staying on despite focus being taken off is partly due to us piggy-backing a lot of our mouse lock exit strategies on full screen. I haven’t really debugged the issue to find out what’s been happening, so the focus of this post is with problem 2 below.
  2. The other problem I’ve been working on is hiding certain mouse events like mouseover, mouseexit, and other non-user generated events. Eventually I’m assuming that suppressing the context menu for the right mouse click will be required as well, but for now the obvious non-user generated mouse events are my target. The code to do this is actually just a simple if check to see if mouse lock is on. The problem is writing the tests for it.

What I’ve tried:

I was able to do a simple in-person test to see if the events were properly hidden, and for the most part, I was successful (I need to track down other non-user generated mouse events).The problem is when I needed to write code that would used for the mochitest framework. Originally I had thought of using the JavaScript createEvent method on the document. However, when I actually tested this out all the events went through even though they should’ve been blocked. At first I thought it was because I didn’t insert my if checks at the right level and that the JS events were somehow being fired further in. So I spent 3 hours looking through nsEventStateManager.cpp looking for where I was going wrong. It seems that wasn’t actually the problem. Had I looked at the JavaScript implementation more closely, I would’ve noticed that the event is attached to the element and the element dispatches the event by itself. This to me, seems like it circumvents the whole event chain that is what normally occurs when you interact with the mouse (and probably why when I created a context menu event, a context menu didn’t pop up).

The Solution:

It would seem I’ll have to look elsewhere for my solution to this problem. I haven’t fully explored what other events I can create, perhaps there is one that exists that would allow me to do this, otherwise, maybe a content script will be necessary (I assume it will work since content scripts are powerful enough for me to check the current cursor’s image). This will going to require a bit more tinkering…

Road to Mouse Lock: Part 5

So I read Diogo’s post regarding the mouselocklost event as well as a bunch of tests he wrote. I thought it would be prudent to add his stuff into my branch and made sure my stuff worked since the stuff he’s working on will impact my code. For the most part there weren’t too many conflicts, just one dealing with the nsDOMMouseLockable.cpp where there was a conflict with the lock method. After a bit of copy and pasting I was able to get everything to build and work properly, the tests had the same results as Diogo’s earlier attempt, which is fine. As long as I don’t have fewer tests passed, I’m happy. I was going to move on but on a whim I created a new page and then called the unlock method without calling the lock method first. This caused my browser to just vanish. I opened up Visual Studio and started to poke around at what was going wrong with the unlock method. The problem was that because we didn’t have a check to see if we even needed to do the unlock method, we tried to QI a nsIDOMElement that doesn’t exist which caused the crash.

Adding an if check prevented this behaviour and while I was there, I added the steps necessary to bring the mouse back when unlock was called. I’ve also merged this back with my main mouselock branch so if anyone wants to see the code, the commit is here and my branch is now here. I’ve “decommissioned” my mouselock-MouseBeGone branch since unless there needs to be more work done on making the mouse disappear and reappear that’s drastically different, there’s no need to work on it anymore.

UPDATE: I’ve also looked at Anurag’s post regarding ESC key handling. It’s similar to how a mouselocklost event will call the unlock method. Actually, I was thinking that since the ESC key caused full screen mode to exit, which in turn will fire an event, to have that trigger mouselocklost which would call unlock. The question is how do I go about doing it.

UPDATE #2:  So there’s still more to come with regards to doing the lock method and getting the mouse to disappear. It was brought up by Anurag’s post noted above. Currently the lock method does not distinguish between full screen achieved from the menu/F11 versus full screen of an element. To remedy this, I was thinking of comparing the full screen element against the element that’s passed into the lock element. I’ve made a push to the mouselock branch with 2 lines of change that allow us to get at the element based on the document supplied during init. The problem is that the addresses are different, but the nsGenericHTMLElement is the same. I just need to figure out how to compare those two things.

On a side note, it seems the majority of my “breakthroughs” happen really late at night. It’s killing my sleeping hours.

Road to Mouse Lock: Part 3

As mentioned in the previous post, we were not able to get a nsIWidget reference which made it impossible to use the widget’s setcursor method. After discussing this with Diogo in person and using him as a sound board I decided to go back to my earlier abandoned project, using nsEventStateManager to call SetCursor. I can honestly say I made zero head way in the three or so hours I spent looking for a way. It wasn’t until Jesse started analyzing our earlier problem (missing widget) and mentioned nsGlobalWindow on IRC that I had my eureka moment. It was a moment of pure luck, at the time I was looking into methods that allowed me to gain access to the EventStateManager and was in the nsGlobalWindow.cpp. Everything just clicked after that.

Using this as reference, after a little copy and pasting I got a small prototype of what I had hoped to work. My main aim was not so much to get the mouse to disappear, this would come later because I had to pass a proper widget… which I’m not too sure how to get. The aim was to make sure I can actually call SetCursor. The end product for the lock method looks something like this:

    bool isFullScreen;
        mIsLocked = PR_TRUE;

        nsCOMPtr domWindow( do_QueryInterface( mWindow ) );
        if (!domWindow)
            return NULL;

        nsRefPtr presContext;
        presContext->EventStateManager()->SetCursor(NS_STYLE_CURSOR_NONE, nsnull, false, 0.0f, 0.0f, nsnull, true);
    return NS_OK;

For some reason when I compiled this code, the moz build looked like it did a full rebuild. It took over 30 minutes to build this time, which was significantly higher than normal. I think I might have messed up my .mozconfig at some point or something (my .mozconfig disappears when I hop branches on git). I was actually REALLY worried whether or not this code would work. When I first executed this code in VS, I didn’t even get to bring out the Web Console. VS actually crashed my whole computer to the point where I had to do a hard reset. After rebooting and testing, the code does exactly what I had hoped. It does go into SetCursor, and will use the mLockCursor which should keep the cursor at none until we unset it. I guess this is another case of the dangers of doing development on Windows. -_-

Upon looking at the code I copied, it seems that the code is used for resizing the chrome window (i.e., when we hold our mouse over the edges, get the resize cursor and perform the drag). This will serve our purpose well I think. The next thing will be to find a proper widget to put in, because right now I’ve sent in a null value. For those interested, the branch I’ve created is here on my github. I know I have way too many libraries and a bunch of unnecessary commented out code at the bottom of the lock method. The mess is the result of a bunch of things I’ve tried and have forgotten to fully clean up. Too bad programmers don’t have automatic garbage collection in real life. They’ll be cleaned up for sure, but I just wanted to put this up there and get some sleep. Yesterday’s excitement on IRC had me stay up until 3am and I had to wake up not long after.

The Road to Mouse Lock — Part 2: Lessons Learned

In class we were able to proceed forward and nail down some more things that we should get started working on which helped provide direction to our class. For me, I had hoped to be able to implement the part of mouse lock where once mouse lock has been initiated, the mouse cursor will disappear until the mouse is unlocked either by a method call or some other approach. My initial thought was to look at how FF implements their CSS cursor style since there already is a CSS style. Unfortunately being as inexperienced as I am with using dxr and mxr I spent large amounts of time looking here and there.

It’s hard to gauge whether or not I’m doing these searches right. My thoughts are usually that I’m doing it wrong because until recently, searching for answers never took very long. I got used to the belief that somewhere out there on the internet, someone will have had the same problem as you and will have posted the answer somewhere. Of course one of the unique things about this course is that many of the things being done now are not easily accessible on the web and perhaps does not exist at all. Keeping that in mind, it’s easier to be less judgmental, but my inefficiency still bugs me. Anyway, after ditching the CSS path, I decided that I would just debug my way to the core event so that I can gain access and change things as directly as possible. That, too, was a new lesson for me. Until now debuggers were the best tools ever, in many ways they still are, just not as important in my eyes anymore. They go to the problem I have right away, it stops where I need it to, and I can inspect every little thing until I felt ready to move on. In this case, it did allow me to go to the problem areas right away, stop where I needed it to stop, but the problem was inspecting the variables until I understood what was going on. I’ve spent so much time looking at variable names, not understanding many of the types and then searching them via mxr that I developed something like tunnel vision. I was so obsessed with the SetCursor located in nsEventStateManager and getting to know it fully that I didn’t bother looking for other options. I feel that I’ll need some time to get really used to dxr and mxr and learn how to use them effectively.

Moving on to things that I actually did, I’ve made a new branch for this particular issue and here’s the commit to that. On the push, I said it was an error with the SetCursor.

This is in fact NOT the case. The issue is actually in the line before that.


For the most part, these lines of code were taken from another file although the specific section of the code was posted up on the FAQ. The problem with this line (I found this out after I pushed it to my github) is that widget ends up getting a null pointer, which then screws up when it tries to dereference it. Jesse has also taken an interest with this, and I’m pretty sure between the two of us (and all the other Windows FF hackers in our class who just haven’t really said anything or come onto IRC) will be able to solve this problem fairly quickly.

I should also mention that I’ve gotten another warning while working with this build:

I haven’t really analyzed this one yet, but it doesn’t really occur on my normal test page. This will be something I’ll look into later, but my priority I think will be getting the mouse cursor to go away (and with luck this one will go too).


I mentioned in my previous post about my change in attitude with regards to IRC. After the talk Dave gave in class, I’ve noticed a significant increase in the amount of talk in #seneca. I’m really happy with the change and it’s really encouraging especially when you get stuck that you know you can ask your peers and get feedback. I hope more students will do the same. There’s definitely synergy between the people who talk on IRC, and in a way, talking on IRC is infectious.

Hopefully by tomorrow, the cause will be narrowed down and solved, if not by me, by someone else.


UPDATE: So taking a further look into why we’re getting a null for the widget, I found this while debugging:

It seems that when we pass in the nsIDOMWindow *, the main widget isn’t set. I’ll probably traverse the call stack in hopes of finding out why this has happened, that or ask why this was happening on IRC.

More attempts at building Firefox and applying patches

At the time of this post I will have successfully built Firefox 12 times… Many of these builds are very minor build changes, I’ve only had to fully rebuild Firefox 5 times, 3 of which were done today (on a side note, I came across the term clobber today on IRC which is what I did 3 times today).

First off, we were told that we should put some printfs in the code to just get it to work. While visiting Ars Technica, I noticed that every time their rotating article displayer thingie rotated, a warning would show up on my debug screen.

Since I knew how to cause the warning to show up, I thought this would be a good place to try to insert my printf. Unfortunately, it would seem I placed my printfs haphazardly because when I went to build it, caused the build to fail. Fail #1. Luckily being on git, I only really needed to do a checkout on the file to get it back to working condition. I rectified the problem (basically I loaded the file with a bunch of printfs and such inside functions, so I removed the majority of them), built FF again and lo and behold:

So every time I got the article displayer to switch articles, it would cause the warning… but it wasn’t a 1 to 1 call. There would be times where the warning doesn’t occur but my printf would. I have to admit, it wasn’t terribly exciting, so I thought I would continue on my bug finding adventure regarding the game pad business (Joystick API). My main concern was to find a way to get a working version of FF that supported joystick API, so off to IRC I went. Dave (humph) and Jon (jbuck) answered my questions and told me that I could get it working with a Debug version of FF if I applied a patch and rebuilt FF. So the first problem was to figure out how to apply a patch. We talked about what a patch really was in class, it was really just a diff. The question was how do I apply this patch… there has to be a quick and painless way of making the computer do it, I just needed to figure out how. Thankfully Dave and Jon told me how and off I went to build a debug version of FF with Joystick API enabled, simple stuff really…. right?

WRONG! I kind of expected it from the start, but I wasn’t really expecting this many problems. So after getting the patch and putting it into my repo, I used Jon’s suggestion of letting git handle the work by using git apply [patch file] . I got a few errors and warnings, in hindsight, I should’ve taken screenshots of those but at the time I was hell bent on figuring out why I was getting these errors without troubling the folks on IRC. Also because I copied and pasted the patch from browser to console and created the file there, I wasn’t sure if I did it wrong or not. In any case, I remembered earlier on IRC, Dave mentioned that he recently updated his repo meaning I should’ve done the same. Fail #2. After fetching and merging the updates, tried to patch again. This time I got a lot fewer errors, but again I still had 4 errors sprinkled around 3 files. In hindsight, I should’ve taken a screenshot because it might be beneficial  to keep track of this . Fail #3. For the longest time I didn’t know why it was wrong, a lot of the lines looked the same so the patch SHOULD work. It took me a few looks to figure out that in fact the line numbers didn’t totally match up for one of the files. In fact it was one line off and that was throwing the patch off for that particular part. My first attempt was to change the patch (even at that point in time it felt like I was messing with something that shouldn’t be messed with)… Git told me that I made my patch corrupt… Fail #4. The issue was I didn’t actually catch the ONLY difference between the two files. In fact not only were the line numbers not the same, but in the newer file, there was actually 1 extra line in it. By then I was wondering if I could just manually add the change to the file itself and remove it from the patch. After all, the patch is really just a list of diffs right? If there’s an error, it’s because there’s some code that may be in mine but not there when Ted made his patch. So I proceeded to do that for all of the files that had errors. Note: the reason why they failed was because there were extra lines of code in the newer files compared to the one in the patch. So I made the changes directly to the files, removed them from the patch, and tried to get git to apply the patch… SUCCESS! Git successfully applied the patch and I went to build FF. Midway through the build messed up (again in hindsight I wish I took a screenshot) and I got a bunch of pldhash.h failures. The problem was when I checked the file it was perfectly fine… Fail #5. I ended up clobbering the whole thing and waited almost 2.5 hours for the build to successfully complete (I thought the pldhash.h error would show up again… but it didn’t!) Eager to test my success, I went to the demo page for Joystick API and tested out my controller. Judging from what I’ve written so far, I’m sure you can tell… Fail #6

As you can see, it clearly does not work. My first thought was that I did the patch wrong and now I’d have to start again. Fortunately Jon actually had a repo branch that had the patch already applied, so I fetched that and built that in hopes that I’ll have a working prototype (I put this into another folder because I didn’t want to overwrite my older stuff). Alas, that was not to be either… Fail #7. So in the end I must’ve missed a step somewhere because the Debug builds of FF didn’t work with the joystick API. Also, I found a weird bug:

Looking at the scroll bar and list box, you can see that I have more add-ons that can be shown. The problem is, I can’t scroll down there. Whether I use the scroll itself, press the down arrow, or use the mouse wheel, every time I move down, it just pops back up to where it is now. It’s a really odd behaviour. I’ve learned a lot in the past 24 hours and I’ve probably done something programmers would consider heinous as well, haha. Hopefully though, I’ll have a proper version up and running.

Tales of building Firefox

I’ve done several different versions of .mozconfig and compiled them while in class, and for the most part I ignored a line that allowed for make to do multiple concurrent things. So I thought of trying it out, and lo and behold it worked for my laptop. Fast forward 4 or so hours back at home, I wanted to do this for my desktop as well (there’s nothing wrong with it per se , it’s just in a really long ugly directory name). Since I had a little time to waste I thought I’d quickly edit my mozconfig file and throw it up to make and I’ll get a finished build in no time. Instead I get ***INTERNAL: readdir: Bad file number and the occasional rm: cannot lstat `conftest.exe': Permission denied. It seems that when I remove mk_add_options MOZ_MAKE_FLAG=-j4 from my .mozconfig it will run fine.

Earlier today in class, Dave said that we should be happy with walls of text, I can definitely see why that’s a bonus. If you don’t see anything, you can’t possibly know where to start. However I’m feeling more than overwhelmed when I’m confronted with a wall of text that’s ALL warnings. It almost feels like every little thing I do, kills Firefox a little on the inside.

Another thing I’ve kinda bumped into is that for my debug mode for Firefox has some issue with Adblock Plus. The error I get:

Before I learned of the handy-dandy XPCOM_DEBUG_BREAK system variable, FF would just stop running and close. When I ran this via command prompt (I have a habit of double clicking to open) I get something like this when FF stops working: (It’s actually a lot less in terms of the number being leaked, this happened to be the first time it happened when I ran it via command prompt so I took a screen shot then)

Even weirder is when I build FF normally and test out Adblock Plus… it works perfectly fine. Although I assume it’s because normally FF would just suppress the assertion error.


On to 0.3 Milestone…

I’m going to continue to work on the video player. I think I’ll be able to have the basic code done by the weekend. While I don’t think I’ll be done (I’ll need to do revisions I’m sure), I’ll be picking up another bug. After chatting with Dave on irc, I was given the A-Okay to look into why Firefox might be misreading my game pads. The issue is that for two of my three game pads, the right stick’s Y-axis is mapped to another button or the Dpad. The first step would be to find out where I should be looking… and on that front, I was told to talk to ted.


A side note: while looking through the numerous documents of debugging on the MDN, I found links to posts describing how to do the editing and debugging on Visual Studio IDE as well as a way to make a prettier console than the ugly one I have now.