Unit Testing in phalanx

In a previous post, I stated that Bugdar2 is powered by a brand-new PHP framework called phalanx that I had written from scratch. In this post, I’m going to talk a little about what’s been going on with Bugdar2. The past month or so have been very busy, but I’ve managed to start writing unit tests and do some work on MacGDBp 1.4.

Phalanx is not a typical MVC framework. While it aims to keep that style of separation, it does so in a very different way than most web frameworks. Zend Framework, for instance, uses a Controller class that has various action methods. In phalanx, the controller unit is replaced by an Event object.

What is an event? An event is essentially a three-stage function with explicit input and output values. When defining an event, clients must explicitly list the input values the event requires and the output values it will produce. This is important because it makes unit testing extremely easy: if you know what should happen, it is not difficult to test it. The three stages an Event goes through are WillFire(), Fire(), and Cleanup(). The only required method is Fire(), which is where the event does the work of transforming the inputs to the outputs. WillFire() provides an opportunity to cancel an event if preconditions are not met, and Cleanup() is called regardless of if an event was successfully fired or not.

Here is an example. This is the event that creates new bugs. It takes in a title and the body of the first comment (description). On return, it provides the bug and comment IDs. From here, it’s easy to verify the results: create an input dictionary, create the event, and post it to the pump to run it.

In the code, references are made to the EventPump. This is the class that is responsible for firing events. The following is the description of control flow in a web app: In the index.php page, a HTTPDispatcher is created; this class parses HTTP requests and synthesizes the Event object for that request. The resulting event is then passed to the EventPump via PostEvent(). Since that is the first event in the pump, it begins processing immediately. If another event is sent to the pump via PostEvent() or RaiseEvent() [preempting the current event], it will be processed, too. Once all events finish processing, the OutputHandler is notified to take the list of events and figure out which one(s) it needs to display output from. The ViewOutputHandler simply takes the last successfully completed event, gathers the output from the event, and passes that information to the template. When unit testing, we can do away with the Dispatcher and OutputHandler. Just synthesize events, send them to the pump, and test the outputs.

Currently, work on Bugdar2 consists mostly of writing these unit test cases. Currently I’m blocked by engineering the API for bug attributes. Because attributes are at the heart of Bugdar2, I want to make sure I get this part right. Attributes and usergroups are the last two features before Alpha 1 is complete.

Downloads Work Again

Due to a server misconfiguration issue, the site couldn’t connect to its database. As a result, downloads stopped working since yesterday’s rollout. This has now been fixed. Sorry for any inconvenience.

Welcome to the new Blue

After about two months of work, I’ve finally completely redesigned the website. I think it sucks a lot less than the previous version, which is always a good thing. It’s written in HTML5 using CSS3. The best part is that I have no idea whether or not it works in Internet Explorer. My traffic is always less than 10% IE, so I’m not going to bend over backwards to support it. Anywhoo, check it out! You’ll also notice that the entire site is now being served over HTTPS.

MacGDBp 1.4: Becoming Asynchronous

In between working on Bugdar 2, Phalanx, and Chromium, I’ve also been working on MacGDBp 1.4.

With the 1.3.0 release, I tried to address a couple of issues with the underlying network communication layer (called SocketWrapper, which was an Obj-C bridge to a BSD socket). The primary issue I tried to address was with Unicode characters; MacGDBp really only knows how to deal with ASCII text, which has proven problematic for international users. I tried to address this with the 1.3 branch by switching how data was stored and processed. But the 1.3.0 release was an utter failure from this perspective. It crashed extremely often, which lead to the 1.3.1 release that reverted the new network-layer changes. (That said, the new features introduced by 1.3 have been well-received.)

But the issues with the network layer ran far deeper than just dealing with Unicode text. MacGDBp uses synchronous communication with the debugger engine. When you issue a command (step in/out/over, get source, get properties, etc.), the command would be sent and then the thread would block until it received a response. This is bad, especially because MacGDBp does not use background threads; all the communication happens on the main UI thread. This can lead to beach balling and a bad user experience. So, why did I do this? Because synchronous communication makes for a dead-simple API. Asynchronous communication requires a lot more bookkeeping and works through callbacks.

So, for 1.4.0, I decided it was time to revisit the debugger backend. Rather than just trying to adjust how data is transmitted (ASCII vs. Unicode), I decided to rewrite from scratch the entire Xdebug communication layer. Rather than using raw BSD sockets, the 1.4 branch uses the CFNetwork API. This yields two huge benefits. The first is that all network activity is asynchronous because socket stream events are scheduled on the run loop. When data is available, a callback is executed and the UI is updated (as opposed to blocking the thread while waiting for a response). This also makes the application more robust, because if the response never comes, it will not lock up the entire UI. Secondly, by using a Foundation-level framework, UTF8 support comes pre-baked via the toll-free bridge to NSString.

I’ve been rewriting the back end in my spare time and it’s finally starting to stabilize. This release is still fairly far out, though. This core network change is also leading to a large refactoring of the entire application to make it more easily unit testable in the future. One of the results of this is creating a new LoggingController to record all network activity. The goal with this “hidden feature” is to make it easier to track down causes of bug reports. The log looks like this; it’s not pretty, but it should get the job done:

I haven’t yet selected the set of feature enhancements that are scheduled for this milestone. Before any new feature work can be started, the 1.4 trunk needs to become as stable as the 1.3 branch. If you have any feature requests, please search for or file new issues and vote them up in the bug tracker.

Tracking Info:
Current SHA1: a7725f8
Version: 1.4.0.40 β
Last Release Build: 2010-03-29 17:51:39

Public Repos and Bug Attributes

Bugdar 2 is now sufficiently far along that I’m comfortable posting the git repository. The Phalanx repository has been public from the beginning (albeit unnanounced) at Github: http://github.com/rsesek/phalanx. Bugdar 2’s repository is now also available there: http://github.com/rsesek/Bugdar2. At the same time, they will be availiable at their canonical home in https://bluestatic.org/git/. The Github mirrors are just easier for most people.

Development of Bugdar 2 is still moving fast. In the past few days I’ve laid down the core of the search system and added support for resubmitting a POST request after login (don’t lose your work if you’re not logged in). There’s two more major milestones before I consider us past the ‘first alpha’ stage: usergroup permissions and bug attributes. What are bug attributes?

In Bugdar 1, there were a set of fields that Bugdar compelled you to use: product, version, status, resolution, priority, severity, and assignment. If you didn’t want to use one of these fields, you couldn’t remove it — you just had to ignore it. There were also custom fields, but the architecture under them was always shaky. By having all these required fields, we were defining a bug as so:

CREATE TABLE bug (
	bugid int unsigned NOT NULL AUTO_INCREMENT
	userid int unsigned NOT NULL
	dateline bigint unsigned NOT NULL
	product int unsigned NOT NULL
	component int unsigned NOT NULL
	version int unsigned NOT NULL
	summary varchar(255) NOT NULL
	priority int unsigned NOT NULL
	severity int unsigned NOT NULL
	status int unsigned NOT NULL
	resolution int unsigned NOT NULL
	assignedto int unsigned NOT NULL
	duplicateof int unsigned NOT NULL
	dependency text NOT NULL
	hidden smallint unsigned NOT NULL
	initialreport int unsigned NOT NULL
	-- [A lot of other cache information here]
);

That’s a lot of data! If a bug isn’t going to use one of those fields, it’s still going to carry around all that weight. In Bugdar 2, this is how we define a bug:

CREATE TABLE bugs (
	bug_id int NOT NULL PRIMARY KEY AUTO_INCREMENT,
	title varchar(250) NOT NULL DEFAULT '',
	reporting_user_id int NOT NULL DEFAULT 0,
	reporting_date int NOT NULL DEFAULT 0,
	hidden boolean NOT NULL DEFAULT 0,
	first_comment_id int NOT NULL DEFAULT 0
);

That’s a huge difference. You’ll notice all the attributes — the metadata — are not stored on the bug itself anymore. Instead, that’s done through a tuple:

CREATE TABLE bug_attributes (
	bug_id int NOT NULL DEFAULT 0,
	attribute_title varchar(50) NOT NULL DEFAULT '',
	value varchar(250),
	PRIMARY KEY (bug_id, attribute_title)
);

The implications of this are that an single (unique), arbitrary attribute can be attached to a bug at the user’s will. The requirement of data is now imposed by the administrator rather than the developer. It also helps to normalize the database schema. So how do attributes work?

Attributes can be defined formally in an administrative panel, or they can be defined ad-hoc while editing bugs. Attributes can be titled to create a field, or they can just be values, which will be represented as tags. The goal with this system is to create a highly flexible data model in which all the stock Bugdar 1 fields could be represented as attributes.

Formally defined attributes can be one of five types: text, boolean, list, date, or user. Text fields are self explanatory. Booleans are interesting because they don’t necessarily have binary value; they have ternary value: true, false, and unset — but that’s more of a semantic detail. Lists are a set of selectable, pre-defined options. The last two types are new since Bugdar 1. Date fields will have a calendar selection widget. Finally, user fields allow the bug to be attached to some user in Bugdar’s system; this is how assignments work.

This is a rendering of how attributes will be arranged on the bug display:

That same information takes up more than half the page in Bugdar 1. That’s all for now.