Firefox 8: Faster, by delaying tab loading

Short version: Preferences -> General -> "Don't load tabs until selected", now you can have hundreds of tabs "open" without slowing down your computer.

I like to keep many tabs "open" in Firefox.  I use a tree-based tab manager (Tree Style Tab view) that lets me build collapsible trees of tabs, built around a common theme.  Yay.  But having a tab open takes up memory and sometimes CPU (with Javascript in the background).  Bookmarks don't work for me, because they're an extra layer of management and "out of sight-out of mind" applies.  I just bookmark and forget forever, so instead I leave the tabs open until I have time to deal with each one.  Some are open forever, for good reason.



I used to use an extension called BarTab, which would allow me to delay the loading of a tab until I actually clicked it to visit it.  So, when I'd start my browser and it would restore all my tabs from my last sessions, they wouldn't load and thus save me a lot of memory.  Once I clicked on a tab, the page would actually load.  This was wonderful, but BarTab hasn't officially supported a version of Firefox since 3.x, and today we're on 8.  (I've been editing the extension to alter the version it supports, but that's fragile and not a good long term solution.)



Good news though!  Firefox 8 now supports on-demand tab loading!  I would post screenshots, but my interface is in German, so just follow these instructions if you need detail.  Elsewise, just go to Preferences, General, and with "restore tabs from last session" selected as your default home page, check the "Don't load tabs until selected" box.  Yay!

Fedora 16: Nice new features

Deliciously in Deutsch

Tablet support

One of my favourite new features in Fedora 16 and GNOME 3.2 is improved tablet support, complete with some programmability, under the System Settings menu.  There are more features to support, but at least the basic ones don't require text file editing anymore. :)



On-screen keyboard

There is even a nice on-screen keyboard under the accessibility menu, which is great if you have touch on your tablet PC.  :)  It even hides neatly in the notification tray.



Online accounts and privacy concerns

Something I'm wary of is the new Online Accounts manager, found in the user menu (top right).  Basically, you can log in once with GNOME to a web service provider like Google and it will share access to applications that request it.  A bit simpler than giving your login credentials to a dozen different programmes.  A bit safer in that respect.



But then there doesn't seem to be a facility to control which applications have access to that. I'd like to log in through GNOME, and then give permission to Evolution to just access my E-mail and Calendar, and give permission to Empathy to just access my contacts.  Perhaps this is restricted, and I have to find out before I use it, because I don't want just any application to go accessing my private data.

Fedora 16: "Oh no!" failure at log in (solution within)

I've just upgraded to Fedora 16, and it was an almost perfect upgrade except for one ugly problem.


Unlike my upgrade to Fedora 16 Beta, this upgrade of packages went well.  I used the preupgrade tool to download all the packages and let them update after a reboot: almost no input required.  However, when I went to log in to my system, I saw a useless error message:

"Oh no!  Something has gone wrong."

It then asks me to log out.  I quickly realised that nothing really seemed wrong.  I clicked on the full-screen error message and pressed alt-F4 to close it and continue working.  I didn't want to deal with this every time, so I figured out the source (method documented below):



The culprit is installed colour profiles from Fedora 15 causing a key component to crash invisibly in the background due to security policy issues.  Ugh.



Solution

On a terminal,

restorecon -r ~/.local/share/icc 

An idea situation would be to have random ICC colour profiles not cause this.



Problem Solving Method

• test a fresh account: does it have the problem?  No, so it must be my user's configuration.


• back up my configuration and then try resetting various parts to default until I can find the source of the problem.   


• With .config and .gnome2 reset, the problem persisted, so it must be something else


• Try resetting all configuration directories (/home/user/.somecfgdirname)




• alright, that worked, so it is in one of the . directories.




• I have 189 of them, so bisect my original configuration to narrow down its location




• Alright, a-m reset resolves the problem, so it's in there




• A quick scan and I see .local which seems relevant, as lots of applications store information in there.




• Testing my original configuration without my .local resolves the problem. 


• I don't want to get rid of everything in my .local




• Try removing .local/lib: problem still there


• Try removing .local/share: problem fixed


• Alright, which application is storing problematic files in .local/share?




• bisect again, it's in a-m


• bisect again, it's in h-m


• take a look, and icc, the colour profile configuration seems like a possible candidate




• Reset icc and bingo!  

Then feel stupid because I actually read about this very problem in Fedora 16's common bugs page (it's a wiki that has things added with work arounds until they are resolved).   However, I didn't think it was my problem because there it says that GNOME Shell fails, and I unfortunately knew that my GNOME Shell was still running, but just giving me an error message inexplicably. 



From the Fedora Project's common bugs page:

Starting GNOME Shell fails after upgrade from Fedora 15 with color profile installed

link to this item - Bugzilla: #741549


If you used the gnome-color-manager
tool to install a color profile for any of your hardware in Fedora 15,
then after upgrading to Fedora 16, you may not be able to log in to
GNOME Shell with SELinux enabled. Login will fail with the "Oh no!
Something has gone wrong" error screen that GNOME pops up if a component
is crashing repeatedly.



The issue is caused by gnome-settings-daemon
crashing when it encounters a color profile with an incorrect SELinux
context: the correct context for color profiles changed between Fedora
15 and Fedora 16, but the upgrade process does not re-label existing
profiles.




To resolve the issue, boot to a different desktop or to a console and run the command restorecon -r ~/.local/share/icc. After doing this, GNOME login should work correctly.

Problem solving



I like hunting and solving problems.   I like narrowing down causes, and combining bisection with intuition and existing knowledge to understand the source of such things.



Sadly, many users don't have the same knowledge.  They might not know where to start.  My Fedora 16 Beta installation problem, where it trashed my system partition, was potentially catastrophic but ultimately not a big deal because I could handle myself.  This ICC colour profile permissions problem is relatively simple, but thanks to an oblique error message, what could a "normal" user do?  It's stuff like this that prevents me from recommending an otherwise wonderful system to my friends.  I fear the problem is systemic.

Error: Protected multilib versions

As noted in a previous post, I had to restore the root portion of my system after a failed attempt to upgrade to F16 Beta.  I'm now reinstalling some software I use, and was trying to install gtk3-devel so I can continue working on GXml this weekend.

I encountered this error message:

"Error: Protected multilib versions: libXi-1.4.3-2.fc15.i686 != libXi-1.4.3-3.fc15.x86_64"





One of the first Google search results is on Fedora Forum, titled "[SOLVED] Issues installing wine (FC15) - FedoraForum.org".  Sadly, the solutions that worked for others didn't apply to me.  Instead, my problem results because, for some reason, after installing from the live USB key, the two software repositories that Fedora was looking for software in were "fedora" and "updates-testing" rather than "fedora" and "updates".  I switched it to the latter two, but now I have some packages installed from "updates-testing" that are too new. 





I realised this when I re-read the message and realised that they were different version numbers, and that the latest one wasn't in "updates".  So, I downgraded it with





# yum downgrade libXi





Yay!

Catastrophic failures while upgrading to beta releases

GNOME 3.2 was released last month, for which I attended a release party and met cool people.  I've wanted to upgrade to it from GNOME 3.0 for a little while, but I use Fedora and Fedora's next release including GNOME 3.2 comes out on November 8th.  Obviously, I'm not that patient.



One way I could start using GNOME 3.2 would be to build it locally using jhbuild.  That might be fun, but I'd like to ensure that the surrounding distribution played well with it, so my intent had been to upgrade to a Fedora 16 Beta.  A beta release: what could go wrong?  I mean, it's not an alpha!  For the record, I usually wait until one of the release candidates comes out before prematurely upgrading to a new Fedora.  On the weekend, I used preupgrade with a Fedora 15 installation on a tablet to successfully upgrade to Fedora 16 Beta.  (Yay for an on-screen keyboard, though I have to file bugs for when using it eventually prevents mouse clicks somehow?)  So, I thought upgrading on my main laptop should go just as smoothly!

Basically says there was a catastrophic failure and installation can't continue

So yah, one of the core packages, util-linux, failed to upgrade, and the installer could not roll back, and my hybrid Fedora installation could not boot afterwards, stuck in its half-upgraded, inconsistent state.  OH NO.



How I've avoided disaster

1. backed up: All my personal data is backed up regularly, even in another city.  Two back-ups, different locations, and backed up before trying the upgrade.  Phew.


2. separate system and data partitions: My personal data is on a separate partition.  On a 500GB HD, I had half for the system (of which only 12GB were actually used) and half for my personal data (of which about 100GB is in use).  This would let me re-install my system without reformatting everything and losing my personal data. 


3. recovery live USB key: I have a nifty Live USB key with Fedora 15 on it that my laptop can boot from, so I can still access my files and use my computer.


4. creating redundant system partitions:
   
   
   
   • I don't actually want to get rid of my old system set-up: if possible, I want to figure out what went wrong, and I want to have access to the list of packages I previously had installed so I can let them reinstall overnight, so I can use resize2fs and lvreduce to first resize the system's file system down to something petite (25GB), resize the partition (logical volume in my case) down to the size of the file system it holds, and then create a new system partition with the left over space (actually, I just used another 25GB for that).
   
   
   • Now I have one large partition with personal data (which is safe), and
     two smaller partitions for system installations: I can actually install
     two systems and switch between them if I like (dual-boot between two
     Fedora Linux installations? :D).  I'll keep this layout for a while, so
     if things go wrong in future upgrades, I'll always have a safe
     installation to boot from without needing a Live USB key.
   
   
   
   

If you have any advice or tips of your own for recovering from system upgrades gone awry, please share :D

#Development and GXml

(Wow, I really do like the new Blogger interface!)



I'm creating another category on this blog via labels, and it's #Development.  This will include a stream of minor thoughts and problems.  I don't want to put them in #General anymore, as I want that to feature posts that aren't technical, and I can't #GNOME minor things, or I'd be spamming the planet.



With GXml, I'm now getting the libgdata picasaweb test suite to pass.  Hooray!   A lot of my issues were in how I chose to replicate libxml2's old behaviour, and there weren't many bugs in GXml itself at this point!  Hooray!



Next, to add introspection support to GXml.

GXml, libgdata, and introspection

libgdata



During the summer, to exercise the GXml API, I wrote a series of patches for libgdata, which uses libxml2 nodes a lot.  I ran into an issue where libgdata's test suite requires Internet access to communicate with Google's servers, and while in Germany, I didn't have reliable Internet access, making testing my patches difficult.  However, now that the start of this school semester is behind me and things are settling into a routine, I've finally been able to investigate issues I had with my patches.  Here are some related thoughts.

• memory management changes: libxml2 often returned allocated strings that it wanted the user to free.  GXml often returns allocated strings that it does not want the user to free, as the relevant object (e.g. an Attr) owns it (e.g. the Attr's value).  After directly substituting APIs, this was causing random memory corruption, where I was freeing data prematurely or twice.  I still need to go through and identify all the free statements to be removed, but for now, I'm using g_strdup () to wastefully create a copy that the pre-existing code can act upon as it used to.


• longer function names: because GXml is written in Vala and uses namespaces and less cryptic naming in general, the function names are longer than in libxml2, like gxml_dom_element_get_attribute as opposed to xmlGetProp.  So, clearer functions, but more typing.


• NULL return values: libxml2 often returned NULL, like when an attribute wasn't set, but the DOM Level 1 Core spec wants an empty string "" to be returned instead.  Currently, I have a couple libgdata wrapper functions for situations like this that return NULL if they can determine that an attribute wasn't set in GXml, rather than "" itself, but taking another look at how libgdata handles unset attributes and such will be worth while.

The work so far (from the summer, and resuming from this past week) is now available in the libgdata repository in a branch "gxml".  There shouldn't be any real need to use that right now, and there's actually a small increase in the total lines of code after porting, due to trying to preserve assumptions from libxml2 to minimise changes to behaviour for now. 



Once libgdata is fine, I'll be releasing a porting guide with advice for those interested in using GXml in the future





introspection and javascript



I've received some questions about using GXml with JavaScript already, so I'm going to work on introspection this week before I'm done with libgdata.  Then I can provide some examples.



new release?



The last official release has no name space support (git does) and is lacking recent bug fixes.  I'll try to push a new release out this weekend for people wanting to play with it now.  If that goes well, I might do a small release every other week.

GXml and the End of Summer

Silly mistakes in namespacing



Stupid mistakes!  I made some bad assumptions about how libxml2 handled namespaces (perhaps I could patch its documentation to be clearer?) that led me to mis-implement namespace support in GXml.  Of course, this was discovered while trying to understand the results I was getting from libgdata's test suites.  Hooray for testing the library by porting a real user of XML!



Sometimes I used namespace definitions instead of the namespace on an element itself, and didn't have a way to separately access them.  At first I thought I had just failed to assign something somewhere, and I spent too much time with gdb and print statements before I realised my mistake was more fundamental.  Oops.



I committed and pushed the changes and updated my test case last night.  There are also more internal classes of NodeList, mwahaha.  Now that the namespace support should be right, I can should finally be able to submit my patchset for libgdata.  After more testing today.  Mwahaha.



Google Summer of Code



The end is nigh!  By the 22nd, there'll be a new release you can play with.  API stability won't be quite guaranteed until I've ported a few more users, though.  GXml development is randomly projected to continue until 2018.  The format of my blogging about it will change, but you'll still be able to enjoy interesting progress (instead of weekly reports) here.  Bring popcorn.

GXml and the Amazing Desktop Summit!

(whoops, this was posted last week, but I failed to tag it correctly to appear on Planet GNOME.)  



The week previous was divided between a school assignment and finalising arrangements to fly from Guelph, Canada to Berlin, Germany for the 2011 Desktop Summit.  Since arriving for the summit last Thursday, I've been able to meet my awesome mentor, Alberto Ruiz, in person, and sort out the remaining two weeks of GSoC



Desktop Summit 2011



This is an amazing experience.  I'm still incredibly grateful for the GNOME Foundation's sponsorship.

I will write more nearer the end of the month about it, but for now I'll just say that it is amazing to meet developers I've followed on Planet GNOME for so long.  I can now testify that GNOME developers (including Google Summer of Code students) are a special and amazing breed of people.  I am also glad to say that about KDE developers, and hope that joint Desktop Summits continue in the future.  There were quite a few excellent talks, and the collaborative and productive atmosphere is invigorating.



Also, hackergotchies are not the best way to identify people in person. :)



Remaining plans for GXml

• namespace support: it's going to be read-only for users of GXml right now.  There are API complications when you allow people to create nodes and attributes with and without namespaces in the same document, and it's unclear how we should handle that.  However, if you parse an existing document, we support accessing the namespace information that the underlying libxml2 parser parses.  Our original goal was to implement DOM Level 1 Core's API, and namespace support isn't part of it, but is necessary for many GNOME users of XML.  Oh wait, I just pushed this last night! :D  It is now in git.  It was blocked a little by a bug involving syncing GXml Attrs and
  libxml2 attrs (the bug being that I had forgot to finish implementing that earlier
  :O), but that has been solved inelegantly for now.


• libgdata: with the advent of read-only namespace support (as of last night!), I can now finish the patch for libgdata to GXml.  I will be reviewing my patch and splitting it up for Philip Withnall's sanity.  Ha!


• API examples: I'm going to provide a few example files on how to use GXml from C and JavaScript. 


• bindings: my mentor has indicated that the build system we're using (WAF, not autotools!) has imminent support for handling my binding generation, so hopefully that will become available before the end of my GSoC.  It may delay examples for JavaScript, but C is working well already (see libgdata support :D)


• gtk-doc: we already provide valadoc documentation, so I'll worry about gtk-doc until shortly after the summer.


• live.gnome.org: I want to update thi-oh, there, I just did it.


• NamedNodeMap: right now we return a HashTable of attributes to users, but this is annoying because we can't see what changes they make to the attributes unless we keep a ref to the table and check it before we need to resynchronise the HashTable we gave a user and the local list of attributes for a given element. This means I will probably implement NamedNodeMap from the spec by wrapping a HashTable (or, if we end up using libgee, extend its HashMap) to update our libxml2 list of attributes on an element as the NamedNodeMap gets altered (attributes added, removed from an element).


• everything: so, after talking to a variety of developers including my mentor Alberto, Michael Meeks, Shaun McCance, Philip Withnall, and many others, GXml will attempt to add everything ever, and it will be awesome!  We'll probably start with more complete namespace support, then SAX parsing, then XPath, and then I'll implement the much desired and maligned XPath2 and XQuery, and then we'll optimise GXml to make it superefficient, and then implement a tea brewing API.  Mwahaha.  I'm very grateful for all the good feedback and interest I've gotten at the summit, and encourage anyone with an opinion on the future of GXml and XML in GNOME in general, to please let me know! (aquarichy AT gmail DOT com, or in the comments!)  Or, if you're at the summit and you see me walking around, poke me!

 Cheerio!

GXml and the end of July: an update

This past week has been spent doing

• Desktop Summit preparations!  This actually consumed most of my Friday, even, and while it's not strictly part of GSoC, I am hoping attending the Desktop Summit will benefit GXml. :)


• Polishing code, including the Document class and DocumentFragment.  DocumentFragment through libxml2 didn't behave exactly as I had hoped, so I had to add more local logic in GXml.  I've tried to keep the amount of local logic to a minimum, but that has sometimes involved rewriting chunks of code or redesigning a few things, which can take more time than just writing the DOM Level 1Core's functionality myself.


• NodeList support.  NodeList is iterable and supports methods from GLib.List, but does not subclass GLib.List, since depending on the usage, it is sometimes backed by libxml2's nodes' linked list.  One of my favourite features that I worked on this past week which took a little bit of planning and prototyping was having a live NodeList for get_elements_by_tag_name ().  By "live", we mean that if you obtain a NodeList of descendant elements from element A with tag name B, if you later add a new descendant to A with the tag name B, it should be present in the earlier-obtained NodeList.  This now involves each element maintaining a list of such NodeLists that have been requested at that element, and addition and removal of nodes to an element now notify its ancestors which check whether they're watching a NodeList for the new/old element's tag name.  If so, they update the NodeList.  Despite being happy to have implemented it, as its an API I used in other DOM implementations, I'm considering disabling this functionality by default, because I can imagine it being expensive when you're not actually using get_elements_by_tag_name () but have many insertions or deletions to do via the DOM, and letting users enable it at the Document level.  Or enabled by default with an option to disable.  We'll see.


• Namespace support investigation.  Support for namespaces exists in DOM Level 2 Core, but I believe it would require too much work beyond the scope of this GSoC right now to implement all the namespace-related changes from DOM Level 2 Core.  I looked at libgdata to try to understand just where and for what it required namespace support, so perhaps I can implement a subset now at the Document level, and worry about it at the Element and Attr level later.  That's going to be a focus of this week, as well as polishing the libgdata patch so I can give it to Philip Withnall for comments.

I'll be making my way from Guelph, Ontario to Berlin, Germany over part of  Wednesday and Thursday this week, so my next update might be longer or shorter depending on how well I can work on planes and trains!

GXml: test port of libgdata and documentation

This week has proven interminably long.  Work has surrounded two main tasks

• libgdata port: I'm porting libgdata as a test.  The patch is almost complete, but not quite.  It replaces almost all the libxml2 usage and compiles.  It's volatility means that I'm linking a diff patch rather than pushing it to a repo, as up until Thursday, I was still figuring out how to handle specific cases, or changing my mind on how I was already handling them.  




• memory: I don't have a great understanding of how to handle memory for APIs that were generated through Vala, and I need to review that.  I've learned a lot, but some of the Vala documentation is kind of out-dated, like that on transferring ownership.  Perhaps when GSoC is done I can use what I've learned and update the documentation :D


• namespace support: It isn't part of DOM Level 1 Core that we're implementing, but libgdata uses namespaces, so right now those sections are commented out or ignored.  I will try to implement something in this next week.


• verbosity: While some things are smaller/more concise using GXml, other stuff is more verbose, because libxml2 already has a compact naming "convention".   After writing "gxml_dom_..." a hundred times, I'm considering removing the "_dom_" namespace section, so GXml.Element.get_attribute () will be gxml_element_get_attribute () rather than gxml_dom_element_get_attribute ().  The Dom namespace is there to help separate future work from SAX and XPath, but enough of its classes will be used in other namespaces to justify omitting Dom to me.


• porting guide: I have a text file that outlines common mappings for things, like "how to get a property in GXml" and compares against libxml2.  I'm going to move it onto live.gnome.org when I have time to format it. 




• documentation: with a little bit of help, and a new conceptual understanding of what valadoc's options do, I can now generate documentation.  Each class and method has something written about it now, though some of it is vague, some is inaccurate (!), and some of it could do with examples.  I'll note that I'm annoyed that it complains when I use a double-space after a period. 


• sed: it is my friend.  After hand-porting lots of code for libgdata, I eventually got comfortable enough with the process to start automating it by creating a bunch of sed rules.  That sped things up greatly this past week, though it still left me to manually review and fix the things it missed, which sometimes took extra time because it became hard to see what was originally intended.  Hehe.


• Desktop Summit:









• Hooray!
  
  While I'm only a GSoC student, living in Canada, I wouldn't have been able to afford this myself, so I am incredibly grateful to the GNOME Foundation, since I am
  
  I'll actually be in Germany for about 3 weeks, as my parents are both from Germany and I have many relatives there, and have never been before. 




• I am going to sleep now.  Tomorrow is a big, terrifying day.

GXml, fewer features, more stuff

A summary in bullet points for busy days!

• git.gnome.org: GXml's development is now hosted on git.gnome.org under gxml.


• libxml2: Patching libxml2's VAPI file.  This is hand written rather than generated normally, so I've been adding bindings for functionality I want exposed for gxml.  I should clean it up and post the patch to bugzilla soon, to make sure that I'm making changes appropriately :)


• I/O: Thanks to the above VAPI work, we now have Stream and File support for I/O operations.  For future I/O work, I might add asynchronous support.


• configuration: Spent time figuring out pkg-config and how to generate a .pc file for GXml.  Turns out I just had to hand-write it.  Struggled with passing variables from WAF unless I started guessing.  Hooray!


• documentation: The amount of todos and personal notes in the source code have grown enough and most code is there, such that I started to prioritise documenting stuff this week.  I want to use valadoc, but honestly, I haven't been able to correctly generate the documentation using it yet.  I'm going to start asking for help next week :)  This has had the added bonus of revealing deviations from the spec, which have been being fixed.


• porting libgdata: I'm grateful for all the projects that volunteered their code.  Based on scope of usage, and a bit of nepotism, I'm writing patches for libgdata, a GNOME library primarily used to communicate with Google services.  I contributed to libgdata over a few months in 2009, working on PicasaWeb support, and I still use the library myself.  I asked Philip Withnall if I could just make a branch in the official repository for it, and he said sure.  I spent time this week figuring out how to include libraries compiled from Vala directly into C code (hence the pkg-config work) and have been reviewing its libxml2 usage and playing with changes to figure out what I need to do.  There are features they use that we don't support yet since they're not strictly part of the spec, like getting a string representation of an XML node structure.  I'm starting to realise that I should provide porting guidance once I'm done with this for other interested projects.


• intimidation and quality control: To be honest, having work hosted at gnome.org is pretty intimidating.  It took me a little while at first to get comfortable with committing imperfect changes frequently to my gitorious repository, but my good mentor Alberto Ruiz encouraged me to, to at least demonstrate progress as it was made.  However, previous efforts to submit patches to GNOME projects have generally gone through careful review and revision before being committed, so I've been reluctant this week to commit and push changes like documentation and libgdata toying until I was happy with it.  This won't work, though, as the week is over and because the documentation work still doesn't generate correctly and because I'm not sure whether libgdata changes I'm toying with are what I'll want to keep, code hasn't been pushed for a few days.  I do care about the quality of GNOME's code, and once GXml is sufficiently mature, I'll be much more conservative with the changes I push, but for now for the sake of transparency, I'm going to have to push things even when unready.  Sorry!


• GSoC midterm evaluations: Despite having personal issues this past month and losing productivity for a couple weeks, I've passed my midterm evaluation, indicating support and confidence from my mentor, for which I'm glad.  While life still isn't smooth yet, it is great to be trusted and allowed to continue this awesome project.   Hooray!

This post brought a day late thanks to J. K. Rowling. 

GXml 0.0.1 and Interruptions

GXml (formerly GDom) development was sidetracked for almost two weeks due to personal matters that required my attention and energy, but after discussions with my mentor, things are progressing again.



Since the last blog update, a lot has happened, some of which was reported on gnome-soc-list, and some of which in the last week.

• GXml: We decided to rebrand our effort as GXml, though we're still just implementing the DOM Level 1 Core API.  In the future, GObject SAX and XPath may be provided.


• I/O: We can read files from GInputStreams (given a patch for libxml2's bindings).  We can also save documents to paths, but not yet GOutputStreams, which will require more enhancements to the libxml2 Vala bindings.  (Patches to come.)


• website: https://live.gnome.org/XML was updated, but is now out of date again.  I will update it again this weekend including scheduling information for the rest of the summer.


• bugzilla: we have a 'gxml' bugzilla product now, which I've started to use to manage features to come.  I still need to enter more.


• building: we now use WAF as our build system.  This proved trickier than I expected, but I like how clean the result is.  


• extended coverage: more classes work more fully now, including CharacterData, Text, NodeList, and a few others, and come with fun tests.  Some classes in the API do not seem to have counterparts to wrap in libxml2 (like ProcessingInstruction and EntityReference) and we might have to forego supporting them.  


• first release: while the source code has been available in a gitorious repo since the beginning, I have wrapped up a tarball and uploaded it to my server for today: http://kosmokaryote.org/files/gxml/gxml-0.0.1.tar.bz2.  I've done this entirely so we can migrate to git.gnome.org, which wants at least one release for a new project.  I don't really recommend using it until libxml2 binding patches are available, though, so GIO can be used rather than file system paths.


• project port: my request for volunteer projects from yesterday got a good response and I'm going to look at them this weekend and then decide by Monday which I'd like to use.  Thank you for your helpful responses!


• libxml2 bindings: I'll need to submit patches for the libxml2 bindings so GXml can implement some of the features we want.  For example, I've been intent on using xmlSaveToIO to get an xmlSaveCtxtPtr, since it aligns well with GOutputStream's methods, but those two libxml2 items aren't bound yet.

Thanks for reading and have a good weekend.



Update:

I linked the wrong tarball.  It's the same contents, but the tarball should be gxml, not gdom, and be generated by distcheck, not by hand.  The link has been updated. 

Wanted: software to port to GXml

Hello GNOMEys.



GXml is pretty functional now, and it's time to start exercising it in a real-world situation.  I'm looking for a GNOMEy project which makes liberal use of an XML DOM via libxml2, and might be interested in seeing how GXml's GObject API could make life easier.



If you'd like to suggest your project, you're not under an obligation to accept the resulting patches if they don't meet your needs (though they will be irresistable), you don't need to do much work (though I might ask a few questions), you help ensure the usefulness of this Google Summer of Code project and help improve XML programming in GNOME.



Preferably, your project will be mainly in Vala or C.  GXml itself is being written in Vala.  XML use should be important to your project.



A first release, 0.0.1, will be out this week.  I had a Life Situation slow progress for two of the past weeks, but things are sailing smooth once more.  A status update comes tomorrow.



Cheerio!

GDom: three weeks in!

Hello again.



On the planet, you may have just seen me publish last week's blog post an hour ago.  These blog posts take a bit of care for me, since I don't want to rush garbage out to so many readers.  I send a simpler point-form report out to the gsoc mailing list (which did go out last week) and then usually write the nicer blog post the next day.  Waiting for some editing, I actually forgot to put the last one up after getting busy.  Sorry!  Here is the current progress, though.



Past week



Scheduling and keeping up-to-date

 

I'm getting better at keeping the git repository for GDom up to date, though still not quite as regularly as my dutiful mentor Alberto Ruiz has requested.  I do have some school work this summer as part of my Masters that takes up part of my earlier week, so I'm going to shift my GSoC work earlier to the start of the week to ensure that I can have everything ready for Friday evening, rather than sending out my mailing list reports late on Friday night and following up with the blog post later. 



Document, Element, and Node



The support for these three interfaces is almost complete now.  Document will need some novel work on loading and saving from GIO streams, but the interfaces pass the unit tests.  Speaking of which, these three interfaces all have beautiful unit tests now.   There's probably still a little room for improvement, but I'm moving on to focus on other classes now.



Attr



I actually intended on finishing implementing this and creating tests for it this past week.  The implementation was working mostly but I ran out of time for its test suite, which I'm working on today.



Refactoring



This probably isn't too interesting, but my initial plans for the class hierarchy have had to be refactored a couple times (didn't take too long), because not all interfaces described in the spec correspond perfectly to libxml2's DOM support.   We now have a base DomNode that describes the DOM spec's interface, and a BackedNode that does a general implementation of it using libxml2's Xml.Node.  Then, other interfaces that use libxml2's Xml.Node are based on BackedNode, while interfaces that don't just base off of DomNode, and sometimes require some local data.  This might get messy when saving files back, though.  Yikes!



Upcoming



Build System



When I talk to my mentor Alberto Ruiz next, I hope to discuss autotools versus WAF.  I keep meaning to investigate it more myself, but ended up with Internet disruptions last week (went to Ottawa for academic reasons and my accommodation's Internet was out) and was generally busy, so this is now an item for this week. 



GIO integration



Need to figure out the best way to use GIO to load and save files by stream for our underlying libxml2 usage.  Currently, I just have Document reading from a Unix file path.  Hehe. 



Attr test case



Finish this.  My general approach has been to at least stub out all the properties and methods and try obvious implementations for them first (usually doing something with an underlying libxml2 structure), and then going through the spec to write tests for the class.  The tests usually reveal all the tricky points in my handling of libxml2 data, or my class hierarchy, and I make another pass to get the class compliant.  It's FUN!



live.gnome.org/XML updates



I have a list of things to go up there, now that I'm into the project, including

• list of similar projects and how we differ


• performance test cases and results (haven't started measuring yet, though)


• list of features and suggestions to be considered in the future, after GSoC

I think I'd also like to put a simple class diagram up.



Good day



Thanks for reading this far.  Also, as noted in my previous post, if you have specific test cases (e.g. large or complex files) that you'd like to see supported, please send them to me: aquarichy at gmail dot com.



 Cheerio!

GDom, after two weeks

This was actually supposed to go up last Monday, but got missed in the busy-ness. I did send my weekly report to the GSoC mailing list, though those reports are less verbose :)




GDom is now two weeks old.  Here's what we have so far:

• Most of the DOM spec's functionality for Node and Document is working


• Lots of other classes work partially


• Beginnings of test suites for Document, Node, and Element

Enough of the base should be laid now that I'll be able to make better commits against git.  I've often found myself in states of not-compiling as I tried to flesh out large chunks of API at once for expediency.



Past week



I managed to do more research, more fully reading the DOM Level 1 Core spec (rather than my quicker review of it when I started) and it answered many
of the questions I encountered after writing code the previous week.



Performance



Many people have offered a lot of suggestions regarding the direction of
GDom so far and I got to discuss them with my mentor this morning [last Monday]. 
To ensure something useful gets created and implemented well within the
GSoC time frame, GDom will remain limited to essentially wrapping
libxml2 and complying with the DOM Level 1 Core spec.



In the comments from last week [two weeks ago], there was notable concern about performance of libxml2 and GObject.  GDom will not be directly addressing those.  It will simply be providing a much nicer API for GNOME developers to work with in most cases.



That said, I am quite interested in the resulting performance and would like to start testing performance as I go along, once most of the base has been laid.  I'll probably simply measure the memory and time performance differences between libxml2 and GDom and identify some choke points.  I don't expect GDom to be faster, but I hope for it to not be significantly worse.   After the Summer of Code is over and GDom can provide a pretty, standard, GNOME-y API, it will be worth considering a faster, more efficient implementation of the interface.  This is something I'm interested in pursuing, and would be something I would have started with this summer except for the strict time frame and progress requirements.



Request for you, XML-using reader!



If you have any specific test cases (e.g. work loads you'd like me to consider like super huge files you deal with :D), please send or explain them to me.  I will test with those and see what I can do to ensure GDom isn't useless to you :D



My e-mail: aquarichy at gmail dot com



Features



Some people have named some features they'd like to see on top of GDom that go beyond the spec.  I'm going to track these and keep them as a list at live.gnome.org/XML to keep them in mind and consider pursuing them after the summer.



Similar Projects



So far, I've encountered a few similar projects and had one noted to me.  They're all similar in that they handle XML documents and they're written in Vala.  They also no longer seem active and/or have had a goal of implementing functionality rather than reusing libxml2.



They were

• dominar


• libgdom3


• xnoise-utils (still active, part of xnoise media player)

 If you know of another project that I should peek at and consider, let me know.  :D



Tests



I've started implementing GTests to guide the implementation of details for the classes.  Hopefully this will increase API compliance and correctness, so anyone already familiar with the DOM API can use the code without surprises.



Next Week [plans for this past week, actually]

• Talk to my mentor Alberto Ruiz about the build system.  I don't know autotools too well, and WAF was brought up as a potential solution.  


• More implementation and testing for Document, Element, Node, and Attr.


• Ensure I commit and push files more regularly.

GDom Progress

Good day.



I'm grateful for the ideas provided in the comments on my last post.  I didn't have enough time to thoroughly consider them all by today, so I'm just going to discuss where GDom is at now.



Learning



I've used the Vala tutorial to acquaint myself with some of its finer points.  I welcome recommendations for other best practises when writing code in Vala, though.



I've also played with the older libgeexml recently to observe an example of libxml2 getting wrapped and abstracted in Vala.  This entailed a lot of changes to get it to compile again (a lot in Vala has changed over a few years), which are currently available at a gitorious repo.  I'll check if there's still interest in the project before polishing a patch set and sending it to git.gnome.org.



Prototyping



After this past week, we can now load an XML file and traverse its node structure.  I went about creating a bunch of stub classes, and trying to implement the Document and Node DOM interfaces to start.  Again, since GDom is initially relying on libxml2 (concerns about its memory consumption have been noted) for its back-end functionality, a lot of the work involved simply transforming data into user-friendly GObjects. 



The code right now is available at the GDom gitorious repository.  The initial commits have been large lumps, and the code base still requires a lot of massaging and stub-filling.



Issues



I encountered some issues having GDom's Attributes implement the same Node interface as all the other nodes, like the DOM API wants.  Mostly because libxml2 treats its attributes as slightly distinct from its nodes.   I also had a headache in trying to keep the attributes synchronised for a node between the GDom and the libxml2 representations, since the DOM API wants us to provide something like a HashTable (a NamedNodeMap) to users, but I want to keep that and the backing libxml2 structure in sync.  I had originally hoped there would be easy signals to catch, but now I just have to be careful to sync them at the right time.  I also have to take some care to avoid creating multiple distinct GDom Nodes for the same libxml2 node.  I'd like a 1-1 relationship here.



The wonderful Alberto Ruiz has provided good guidance on all my questions so far, though, helping ensure GDom and I can continue to move forward :)





Next Week



Try to get Document and Node much more complete.  This involves implementing more of other classes, too.  The Document interface defines a bunch of methods for creating other elements, for example, so even if they don't function perfectly, I still want to have a dumb version for Document to handle. 



Also, quite importantly, I'm going to create tests for Document and Node to start.  Right now, I have a simple main () function that uses them to load an XML file and navigate it.  I would like to have a more complete test package available to guide GDom's progress, though.



Also, finish considering and addressing comments left on my last post.  I've read them all, and some of them warrant replies.  Thank you for the feed back already.  (I'll note that I'm busy this weekend, and won't get back to it until Monday, though.)



Also, I need to fill out more of http://live.gnome.org/XML.  It's still a little sparse.



Have a good week.

Hello GNOME. Meet GDOM.

GDOM

Good day. GNOME has many beautiful APIs. For manipulating XML, many projects use the powerful libxml2. However, its API is not consistent with GNOME conventions. It uses different data types, and does not benefit from good integration with GLib, missing out on libraries like GIO.

GDOM is an effort to implement the DOM Level 1 Core API for GLib.

GObject API and GLib integration

All objects, such as Nodes, Documents, and Attributes, will be GObjects. They'll use GErrors for exception handling, handle GIO's GInputStreams, etc.

Functionality via libxml2

We are not implementing the API from scratch. In fact, almost all its functionality is coming from libxml2, which is already robust and has good performance.

Vala

GDOM will be implemented in Vala which should help minimise the work necessary to complete the API.

I really want to hear your thoughts on XML and GNOME. I am at a disadvantage of never having had to write anything too large or intensive that used XML, just parsing and building simple files for my own projects. If you're involved with a project that already uses libxml2 (or another XML API), let me know. I'd like to see how you use it and how GDOM might make life better for you.

GSoC and Me

This is a Google Summer of Code 2011 project, and my mentor is Alberto Ruiz. Me? I'm a Masters student at the University of Guelph. I fell in love with GNOME about 7 years ago, but never got around to writing much code until 2009. My Masters has kept me busy recently, but thanks to Google, I can justify focusing on GNOME for a while :)

Greetings

Hello GNOME. I'll introduce myself soon :)

[Technology] Connecting to uog-wifi-secure from Linux

So, at the University of Guelph, apparently there are current, active security concerns about using the regular, unencrypted uog-wifi wireless network.



They provide uog-wifi-secure, but it's not entirely obvious how to connect to it. CCS provides this helpful page: http://www.uoguelph.ca/ccs/node/858 that gives instructions on how to set up access to it for Windows, and provides some general parametres for other systems.



I'm connecting from Fedora 14 Linux, using GNOME 2.30, using Network Manager. I wasn't sure what to do with some of those parametres, because it turns out I don't need them. Here are the parametres I ended up using, except I filled in my username and my real Central ID password:

That alone did not work. The authentication servers may not have your current Central ID password saved. As their website suggested, I had to go to https://www.uoguelph.ca/ccs/apps/password/change/ and re-set it so that my Central ID password could work with uog-wifi-secure.



If you're wondering where I got the certificate file for Thawte Premium Server CA, I exported it from Firefox. I went to Edit -> Preferences -> Advanced -> Show Certificates, scrolled down until I got to Thawte Premium Server CA, clicked it, and clicked "Export...". I added the .pem file extension to the proposed name and saved it to my file system. Then, in Network Manager's connect window, I browsed my file system and selected the newly saved copy of the certificate.



Voila! I also did a similar treatment to my Nexus One Android phone. My browsing sessions should remain relatively more private, for now.