?

Log in

No account? Create an account

Next 10

Jun. 17th, 2009

Evolution Will Soon Bid Farewell to Bonobo

This is a combination announcement, history lesson, status report and call for help.

For nearly a year I've been laboring over an Evolution branch named kill-bonobo, whose goal is exactly that: eradicate Bonobo from Evolution once and for all. According to the GNOME release schedule, I'm due for a status report. (Note, this only pertains to the application itself. The data server is already being ported from Bonobo to D-Bus by Ross Burton. The two projects are independent.)

The kill-bonobo branch is strictly an internal cleanup effort, albeit a massive one. It's not about adding features or radically changing the end-user experience. It's about making Evolution easier to maintain and enhance as we enter into the GNOME 3 era. So if I've done a good job, users will hardly notice any difference when the branch is finally merged.

Those who are interested in testing the branch can skip to the "How to Help" section.

Why We Need It

I was not present for the early evolution of Evolution, so this account is based on historical research and discussions with some of the old-timers. Hopefully they won't clobber me for getting the details wrong, or for calling them old-timers.

Evolution's original design consisted of a simple skeletal "shell" which served as the framework for out-of-process Bonobo components. Each component managed different types of information: one for email, one for calendar events, one for contacts, etc. The components talked amongst themselves via CORBA, which handled all the IPC. Ettore Perazzoli, in his paper for the 2001 Ottawa Linux Symposium, wrote:
It is interesting to note that the various pieces of Evolution are currently out-of-process components, and that CORBA deals with the inter-process communication nicely and transparently; it would be possible to turn these components into shared libraries without any substantial changes to the code.
And that's exactly what happened. The components were turned into shared libraries sometime later to help address some design issues and to make the application more debuggable. Evolution then became — and has remained — one monolithic process. Most of its functionality is still supplied by in-process Bonobo components loaded at run-time. Bonobo was kept around in part for the menu merging capabilities of libbonoboui, but also because it was already very deeply ingrained in the application by then. To this day, Bonobo still handles all the inter-component communication, even though it's all one process.

Fast forward to present day. Bonobo has fallen out of favor with the rest of the GNOME community and its libraries are now deprecated (or planned to be deprecated — the distinction is unclear). GTK+ has gained a menu merging capability in GtkUIManager, GObject has gained a module loading system in GTypeModule, and developers are scrambling to migrate code away from deprecated libraries and API in preparation for GNOME 3.

Moreover, the inability of Evolution components to interact directly with the central shell continues to be a major design impediment. Certain bugs cannot be fixed and enhancements cannot be implemented with Bonobo in the way. Even something as simple as setting the proper relationship between a dialog window and the main application window can't be done under the current design without breaking the component/shell abstraction.

In short, Bonobo's time has passed and it needs to get gone.

What's Finished

A New Shell

I have rewritten Evolution's shell from scratch. While still adhering to the original design principles, it now loads components (I'm calling them "shell backends" just to get away from the CORBA nomenclature) at startup via GTypeModule. Each main shell window provides a central GtkUIManager instance for shell backends to merge and un-merge their menus and tool bars. The new shell provides a bunch of other convenient services that it didn't, or couldn't, before. The API is now mostly stable, and it's documented! (A trend I hope to continue.)

The shell itself is a subclass of UniqueApp from libunique, so Evolution is still a single instance application. In fact, that feature has improved. Starting a second Evolution instance with no command-line options will now raise and focus the current window instead of opening a new one; a behavior greatly preferred by users. Also, Evolution will now terminate when it's finished handling a command-line URI and there are no other Evolution windows running.

Infrastructure

Beyond the shell, it's mostly been a matter of slogging through the rest of the code and adapting it to the new shell API and the more modern GTK+ APIs. Turns out, Evolution has a lot of code, and a lot of infrastructure that piggybacks on Bonobo. Needless to say, that's why it's taking so long.

All of the menus and tool bars, and many of the stand-alone buttons and combo boxes, are now proxy widgets for GtkActions. EMenu and EPopup — the mechanisms that allow plugins to extend menus — are being phased out in favor of EPluginUI, which works with GtkUIManager. In fact, many of the plugins themselves are being phased out — their features being properly integrated into the application.

Asynchronous activity tracking — that's those percent-complete and error messages in the status bar — is now more object-oriented. The shell even lends a hand in routing and tracking these activities. I have a number of ideas for leveraging this new framework. Eventually I'd like to eliminate pop-up error dialogs altogether in favor of something less obnoxious, such as "inline" alert messages similar to those in Firefox, gedit, Sound Juicer and Evince. Also, better shutdown management — where Evolution tells you what network activities are keeping it from shutting down and allows you to cancel them.

Just getting the thing to build was another challenge. What I assume was once a nice layered design with clean separation of concerns has grown into a tangled mess of circular dependencies. I've managed to resolve most of the linking issues by shuffling source code around and routing application-wide events through the shell. But there's still the issue of library modules linking to library modules (plugins linking to shell backends and shell backends linking to one another), which is not portable and in fact prevents Evolution from building on Mac OS X.

Shell Backends

The contact, memo and task backends are done, and I'm currently wrapping up some loose ends on the mailer. All are usable and ready for testing. I personally have been using the kill-bonobo branch for daily email and task management since February.

Calendars, however, are another story. See below.

What's Unfinished

The branch is about 75% complete. It's usable, but there's still significant work to be done.

The Calendar

The calendar is half-finished and is not yet usable. This is the last major piece left.

I was attempting to split up the massive GnomeCalendar class into smaller, more manageable pieces. But I got burnt out and had to set it aside for awhile, and then never got back to it. At this point I'm in favor of just getting it working as quickly as possible. I can take another shot at refactoring it later when I'm feeling masochistic again.

Plugins

Many of the plugins still have to be adapted to EPluginUI and the new shell. This is a highly parallelizable area where I could use some help from volunteers. The working and non-working plugins are listed in "configure.ac" (search for PLUGINS NOT BUILDING YET).

Evolution Exchange

Evolution Exchange (formerly the "Ximian Connector for Microsoft Exchange Server") presents an interesting problem. The evolution-exchange-storage process is the last out-of-process Bonobo component that talks to the shell. Ximian originally released this software under a non-free license, and used the split process design to bypass the linking restrictions imposed by the GPL. Later, after Ximian was acquired by Novell, it was released under an open-source license. But the design remained unchanged, even after Evolution was collapsed into a single process application.

I'm uncertain of what to do with this. To me, the best solution would be to convert the storage process to shared library modules for the Evolution and Evolution Data Server processes. But that could take some doing, and time is short. Plus, Evolution Exchange is being phased out by Evolution MAPI anyway. Another option is to hack together a quick and dirty D-Bus API for the shell, which the storage process can use. But I'm hesitant to expose a poorly thought out D-Bus API just for a corner case.

Third-Party Extensions

Third-party extensions such as evolution-rss, evolution-brutus, and evolution-jescs will likely require some re-design. I will lend a hand here as much as possible.

How to Help

The best way to help right now is to test drive the kill-bonobo branch and file Evolution bug reports at http://bugzilla.gnome.org/. Since this is an unofficial branch I have my own little system for tracking reports in Bugzilla. But really, if you do file a bug just mention that you're running the kill-bonobo branch and I'll take care of the rest.
  • Existing bugs that the branch may be able to address are tagged with evolution[kill-bonobo] in the Status Whiteboard field. (Bug List)

  • Existing bugs that the branch has successfully addressed are tagged with evolution[kill-bonobo] in the Status Whiteboard field and the Summary field prefixed with [KB-Fixed]. (Bug List)

  • Regressions that the branch has introduced get tagged with evolution[kill-bonobo] in the Status Whiteboard field and the Summary field prefixed with [regression]. These are what I'm looking for. (Bug List)

Evolution developers and contributors looking to hack on stuff could start by picking a non-working plugin and getting it back on its feet (see the previous section). That would probably acquaint you with the new shell API, and possibly expose bugs or shortcomings in the design. Catch me on GimpNet (channel #evolution) if you're interested in this.

For Fedora users, I will attempt to publish a kill-bonobo RPM repository for Fedora 12 / Rawhide in the near future. Stay tuned.

Credits

Finally, I have to give props to my employer for humoring me through this ordeal, especially since I've consistently underestimated the workload. While the work has been largely exploratory and at times overwhelming, in the end it's been highly educational and satisfying.

Apr. 15th, 2009

Evolution Attachment Rewrite

Today I finished a rewrite of Evolution's attachment handling code. What was supposed to be a short detour from a much larger ongoing project (more on that later) ballooned into a month-long overhaul. I had already sketched out plans for a rewrite a year ago. This was just preparation meeting opportunity.

The goals of the rewrite were primarily internal code cleanup, but there's some UI bling too:

  • Attachment handling code is currently scattered throughout Evolution, with lots of duplication. The rewrite collects everything in one place.

  • Migrate from GnomeIconList (deprecated) to GtkIconView, with a custom GtkTreeModel specifically for attachments. And throw in a GtkTreeView option just for kicks.

  • Better utilize GIO for file operations, thumbnails, emblems, etc. Abstract away the distinction between local and remote files. Stop using threads, and make sure all blocking calls are asynchronous.

  • Migrate the attachment popup menus to GtkUIManager.

  • Kill off a couple frivolous attachment-related plugins and support the functionality directly. We ship way too many plugins as it is and I'm gradually killing off the silly ones.

  • Make the whole thing extensible, to help resolve some cyclic dependencies within the application. The mailer now handles the mail-related parts, the calendar handles the calendar-related parts, and so on.


Here's a few screenshots. The first two show off the new icon and list views when composing email, and demonstrate simultaneously loading from or saving to remote stores (I used some big audio files here for better illustration of the progress bars). Both file operations are cancellable. The third screenshot shows off the attachment UI for received emails.


Icon View


List View


Received Attachments

Feb. 26th, 2009

GNOME Video Arcade has moved to GitHub

To better acquaint myself with the way of the future, I've moved my GNOME Video Arcade project from SourceForge to GitHub, and also set up a horribly amateurish website for it.

GNOME Video Arcade is a simple MAME frontend written to suit my own aesthetic taste better than other UNIX-based frontends, most of which I find too clunky and cluttered. And it's something to hack on when I'm burnt out from my real job.

Nov. 17th, 2008

ExoBinding and Settings Management

Over the weekend I learned about Xfce's excellent ExoBinding extension for GObject properties, a feature I've been wishing for for quite a long time. It allows you to bind two object properties together, with optional transformation functions, so their values automatically stay in sync.

A simple use case is desensitizing a portion of a dialog when a check button is inactive. With ExoBinding, you can do this by simply binding the GtkCheckButton's "active" property to a GtkContainer's "sensitive" property, both of which are boolean values. Done. No callbacks, no manually setting widget sensitivity, just one call to exo_binding_new(). I like to think of it as wiring objects together.

So I had an idea.

Last year I learned of and raved about a little utility library named GConfBridge, which binds GObject properties to GConf keys and automatically keeps them in sync. Combining these features seemed like a pretty powerful way to implement a user preferences dialog, but then there's the problem of providing access to these settings to the rest of the application. In Evolution, at least, the settings values are kind of scattered all over the place. Centralization seems like the right way to go.

Again looking for existing solutions, I took a closer look at GtkSettings which, if I understand it correctly (the documentation is a bit sparse), allows widgets to install their own configuration settings that desktop themes can tweak. GtkSettings centralizes these settings and exposes them as GObject properties.

Perfect. Now we're on to something.

I wrote a simplified version of GtkSettings which dumps the parser but keeps the property registration, and added it to an Evolution branch I'm working on. This singleton object serves as a repository for application settings, accessible via GObject properties, complete with "notify" signal emissions when a settings value changes.

Then, for each application setting, I register a new property with the GtkSettings-like singleton, bind that property to the appropriate GConf key using GConfBridge, bind it again to the appropriate widgets in the preferences window using ExoBinding, and (ideally) bind it yet again to some object property deep in the application where the setting will take effect.

I prototyped all this last night with some of Evolution's settings and, I'll be darned, it seems like a viable approach. Just a bit of wiring during initialization and that's that.

All this magic mimics much of what dconf and GSettings promise to make easier, and taking advantage of GConfBridge and ExoBinding now might help set the stage for a smoother transition to GSettings when the time comes.

In the meantime, I'm rooting for this bug to get some attention.

Nov. 9th, 2008

GObject-based Camel

Over the weekend I hit a milestone with a science experiment I started earlier in the year to turn Camel into a GObject-based library with minimal API breakage. The short-term goal of this effort is to make it easier to generate language and D-Bus bindings for Camel.

I now have running code which I've posted to a new evolution-data-server branch called "camel-gobject". The branch contains a couple very small patches (evolution.patch and evolution-exchange.patch) that must be applied before building Evolution against the GObject-based Camel library.

Evolution seems to be working just fine with my standard setup, but I haven't tested all the built-in Camel providers and various third-party extensions. If others want to chip in with the testing I'd be grateful.


Techie Details

My approach was basically to pull off the same stunt that was done to GtkObject when GObject was introduced: turn the base CamelObject class into a GObject subclass, and convert the CamelObject API into a set of thin wrappers for equivalent operations in GObject. For example:
  • camel_object_ref() now calls g_object_ref()
  • camel_object_unref() now calls g_object_unref()
  • camel_type_register() now calls g_type_register_static()
...etc...

Obviously this breaks Camel's ABI. There's no way around that.

The (intentional) API breakage is very minimal:

  • CamelType is now an alias for GType. It is no longer a pointer to the class structure. That was supposed to be an implementation detail anyway, but Camel (and Evo, and Evo-Exchange) exploits that in a number of places. The class structure can be properly accessed using camel_type_get_global_classfuncs(), which is now an alias for g_type_class_peek().


  • The corollary to that is when to fetch the parent class structure. Most classes define a static "parent_class" pointer for chaining up in class methods. The "parent_class" pointer should be set in the class initialization function, NOT before the type registration in get_type(). Just like you would in normal GObject code. There were LOTS of places in Camel were that needed fixing.


  • Camel interfaces are dead, but they were never really used anyway.


The other half of the problem was CamelObjectBag. It formerly shared a mutex with CamelObject's reference counting, I think because CamelObject ref's and unref's were non-atomic. GObject's reference counting is atomic, so with that issue out of the way I gave each CamelObjectBag its own mutex, cleaned up the code a bit, and split it off into a separate source file (camel-object-bag.c). After an evening's worth of debugging it seems to be working fine now.


Now What?

So with the "interesting" part now done, next comes the long and tedious part: rewriting all the internal CamelObject boilerplate into GObject boilerplate and making sure we're not using any newly deprecated API. But that can be done gradually and with help from others.

In time I'd also like to explore taking greater advantage of GObject signals and properties in Camel, and also deprecating CamelArgV and CamelArgGetV.

I haven't discussed this with the other Evolution/Camel developers yet, so there are currently no plans to include it in 2.26. That may change, but for now it remains purely experimental.

Nov. 6th, 2007

GNOME Video Arcade 0.5.2

Has this happened to you? You upgrade to the latest and greatest MAME release and suddenly a bunch of your favorite games stop working. Now I do understand that the MAME project is primarily concerned with documenting the ins and outs of old video game hardware, and the fact that you can actually play the games is a mere side-effect of the process. But you also can't ignore the fact that a rather large community has grown up around this "side-effect" over the past decade. Yet almost every MAME release continues to break ROM sets and (especially) front-ends without warning. I think the MAME developers could do a better job of alerting the community about changes that break backward compatibility.  Expecting users to sift through change logs and individual developer blogs to learn what broke just doesn't cut it.

Until now GNOME Video Arcade has also been guilty of not alerting users about games that have broken due to a MAME upgrade.  But as of version 0.5.2, this is no longer true.

GNOME Video Arcade Screenshot

This dialog now appears when GNOME Video Arcade detects errors during its analysis of your ROM files.  The Save As button allows you to save the analysis results to a text file for later viewing.

Sep. 16th, 2007

GNOME Video Arcade 0.4.4

Released a GNOME Video Arcade update that finally adds basic search functionality to the application. I scaled back my grand plans for an advanced search window and instead implemented a simple Google-style entry box. With the game database and automatic conversion of query results to a GtkTreeModel already in-place and working, and for as long as the "Search Results" button has been non-functional, I'm a bit embarassed to admit that it only took about an hour to glue the components together to get search working.

Sep. 1st, 2007

GNOME Video Arcade 0.4.3

I released a minor update to GNOME Video Arcade today with a few bug fixes and user interface tweaks, among which is the main window size being preserved across sessions, courtesy of libgconf-bridge. It also adds all available game data from MAME to the game database now, so the database construction phase takes longer once again. But this is necessary for implementing the advanced search functionality I have planned for 0.5.

Aug. 30th, 2007

Blogging again

I'm trying out the blogging thing again, hoping I can stick with it this time. Now that I have a job that allows me to talk about the interesting things I work on, perhaps I'll fare better at it. Prior to my current job, I spent eight years working on various military projects at Boeing such as the ill-fated X-32 Joint Strike Fighter.

These days I work at Red Hat maintaining Evolution, an open source mail and calendaring application for the GNOME Desktop Environment.

Outside of work I like to hack on a pet project of mine called GNOME Video Arcade, a simple xmame and sdlmame front-end, because I still haven't outgrown those old golden-era arcade games that I spent all my allowance money on growing up. And they make for good five minute mental breaks while hacking.

Next 10