[Technology] Testing for memory leakage with Vala and GLib

For my thesis, I implemented my algorithm in Vala, in part because I want practise, as well as because the candy of a high level language can theoretically expedite implementation, while the nature of Vala being compiled down into C helps avoid the penalties of running code in a run time.

Vala manages memory for you mostly using reference counting.  This is mostly invisible.  Running my experiment the other week revealed a large amount of memory leaking.  I shuddered, and realised I'd have to analyse it and create tests for this.

I used the GTest framework and  malloc.h's struct mallinfo and mallinfo () to calculate memory usage before and after various blocks of logic.  (I mostly just encapsulated calls to functions + setup.)  I also occasionally used valgrind and gdb.  My methodology was to start at the highest level functions and measure memory leakage.  Then, if one was detected, I'd create new tests narrowing down the source.  This was sometimes arduous as I started learning how GType and GSlice worked.

First, the first time you use a type in GLib, there's often a bunch of memory allocated that will (as of late apparently) never be freed.  That's fine, because it's a one-time penalty and it's necessary.  To avoid having these allocations cause a test to unduly fail, I ended up with "priming" code before a lot of tests ran, that would simply invoke types for their first time, so nothing new on that account would have to be allocated during the time.

Second, after reading about how to use valgrind on GLib applications, I started to learn about GSlice.  GLib encourages the use of the GSlice memory allocator instead of g_malloc () or malloc ().  The idea seems to be that it will reserve a large contiguous block of memory for you to play with, that won't be released back to the system any time soon.  That way, if you would normally allocate and deallocate space many times quickly, you save on the cost of actually reallocating it, since you never really let it go.  That sounds great; but for the purpose of testing, I really want to measure only the memory the system is actually using, not how much I am prudently reserving.  To work around that, I set an environment variable G_SLICE=always-malloc.

Third, I encountered what may be a bug in Vala's code generation.  Bug 684182.  When using asynchronous methods, the generated C code that calls g_simple_async_result_new () passes a brand new GObject to it a a source without ever unref'ing it, so every time you run an async method, you lose 32 bytes.  Also, there is occasionally an additional 16 bytes lost in relation to running them, but I haven't narrowed down where that came from.  The 32 bytes are relatively negligible for my purposes, but it was still resulting in failed tests.  Consequently, I now define a margin that memory leakage can fall within for a test that will only result in a warning and no failure.  My margin is 32 bytes * # of async method calls + a possible 16 bytes.

Fourth, I discovered an actual issue with some of my code.  I ran into a reference cycle in my code.  Referring back to the ever-useful Vala referencing handling page helped me understand what I needed to do for that.  I made sure that the appropriate references had the "weak" modifier attached, and all was well there.  For a given use case, this reduced memory consumption from 681kB to 546kB. (Both with the TreeSet fix below.)

Fifth, the most significant issue I encountered wasn't a problem with my own code but a memory leak in libgee before version 0.8.5.  The current stable libgee in Fedora 18 is only 0.8.1.  It has a memory leak in TreeSet where the free method for the TreeSet's members is never called so all elements added to a TreeSet are leaked.  That is incredibly expensive as I use TreeSets a great deal.  Upgrading to 0.8.5 resolved this, and reduced memory consumption from 26,402kB to 546kB!  (Both have the "weak" fix above.)

So, now I have a nice set of reproducible test cases for memory usage to help prevent memory leaks in the future.  Vala does a really nice job of managing memory via references, so it's almost no problem at all.  While the biggest win comes from dealing with a memory leak in code that isn't even mine, the tests helped me locate it, and if the leak had not been fixed in the last couple of months, I would have been able to fix it myself, and now I'll know if any new leaks are created.  I'll probably trust the GSlice allocator in actually running my experiment, and be grateful for its purported benefits (rather than using always-malloc all the time), since there is no actual memory leaked.  Finally, it's nice to have a better understanding of memory usage in Vala and GLib.  I feel like it's generally important to be well-informed about how memory is being handled, and that's something I haven't had the time to educate myself on for Vala until now.

Keine Kommentare:

Kommentar veröffentlichen

Dieses Blog durchsuchen


#Technology #GNOME gnome gxml fedora bugs linux vala google #General firefox security gsoc GUADEC android bug xml fedora 18 javascript libxml2 programming web blogger encryption fedora 17 gdom git emacs libgdata memory mozilla open source serialisation upgrade web development API Spain containers design evolution fedora 16 fedora 20 fedora 22 fedup file systems friends future glib gnome shell internet luks music performance phone photos php podman preupgrade tablet testing typescript yum #Microblog Network Manager adb apache art automation bash brno catastrophe css data loss debian debugging deja-dup disaster docker emusic errors ext4 facebook fedora 19 gee gir gitlab gitorious gmail gobject google talk google+ gtk html libxml mail microsoft mtp mysql namespaces nautilus nextcloud owncloud picasaweb pitivi ptp python raspberry pi resizing rpm school selinux signal sms speech dispatcher systemd technology texting time management uoguelph usability video web design youtube #Tech Air Canada C Electron Element Empathy Europe GError GNOME 3 GNOME Files Go Google Play Music Grimes IRC Mac OS X Mario Kart Memento Nintendo Nintendo Switch PEAP Selenium Splatoon UI VPN Xiki accessibility advertising ai albums anaconda anonymity apple ask asus eee top automake autonomous automobiles b43 backup battery berlin bit rot broadcom browsers browsing canada canadian english cars chrome clarity comments communication compiler complaints computer computers configuration console constructive criticism cron cropping customisation dataloss dconf debug symbols design patterns desktop summit development discoverability distribution diy dnf documentation drm duplicity e-mail efficiency email english environment estate experimenting ext3 fedora 11 festival file formats firejail flac flatpak forgottotagit freedom friendship fuse galaxy nexus galton gay rights gdb german germany gimp gio gjs gnome software gnome-control-center google assistant google calendar google chrome google hangouts google reader gqe graphviz growth gtest gtg gvfs gvfs metadata hard drive hard drives hardware help hp humour ide identity instagram installation instant messaging integration intel interactivity introspection jabber java java 13 jobs kernel keyboard language language servers languages law learning lenovo letsencrypt libreoffice librpm life livecd liveusb login lsp macbook maintainership mariadb mario matrix memory leaks messaging mounting mouse netflix new zealand node nodelist numix obama oci ogg oggenc oh the humanity open open standards openoffice optimisation org-mode organisation package management packagekit paint shedding parallelism pdo perl pipelight privacy productivity progress progressive web apps pumpkin pwa pyright quality recursion redhat refactoring repairs report rhythmbox rust sandboxes scheduling screenshots self-navigating car shell sleep smartphones software software engineering speed sql ssd synergy tabs test tests themes thesis tracker travel triumf turtles tv tweak twist typing university update usb user experience valadoc video editing volunteering vpnc waf warm wayland weather web apps website wifi wiki wireless wishes work xinput xmpp xorg xpath
Powered by Blogger.