Hacktivate

It has been an interesting week of hacking.

These days it is rare to even have to think about efficient code. Memory
and processing power are seemingly limitless, except when they are not.
And this week they are not.

First off, I have been writing perl code to match similarly but not
exactly named items in a database. The database is not small, with
around 100,000 rows in the various tables. Essentially I have written a
search engine in perl which ranks words based on uniqueness within the
set and uses other little hacks to try and make the best match. But with
tens of thousands of SQL lookups it takes about a second per match, so
my laptop has been running very hard and very hot for hours. It has been
fun to find a little optimization here or there and watch things double
in speed, but it isn't the sort of instantaneous computing we have all
become accustomed to.

Secondly, and more interestingly is the touchscreen data collection for
the HIV treatment center. The touchscreen machines are based on the
i-opener, which is a relic from the dot-com days, but now available
secondhand for pretty cheap. It was designed as a "web appliance" so you
can check email and surf the web on your kitchen counter. It is a nice
compact machine, with a decent LCD screen, about 200Mhz of pentium power
and thanks to the director's innovation, it has been upgraded to 32mb of
memory, can now handle USB devices as well as external flash cards, and
perhaps most importantly is fitted with a touchscreen device.

So that is the hardware. Previous versions of the Baobab software have
all been based on either Visual Basic or Real basic. So the i-opener
boots into windows 98 lite, and runs the basic application in full
screen mode. I have proposed and worked up a proof of concept that will
use Firefox running in full screen mode to do all of the same functions.
The prototype even uses Ajax to perform various database lookups. Click
"w" then "a" on the onscreen keyboard and an AJAX request will be sent
to a bit of PHP that returns all the warts you can imagine without
needing a refresh the page. Simple web 2.0 stuff, and it works great on
my machine. Unfortunately, when it gets to the i-opener, what was an
instant wart retrieval system is now 5 seconds of anticipation. All of
this web 2.0 goodness requires extra power. I am not sure if it is the
Javascript that is slow, or if Firefox is taking a long time to
negotiate the TCP/IP connections. Most likely it is a combination of both.

So the challenge is for us to figure out how to make this happen faster.
We have tried K-Meleon a lightweight version of Firefox, but it didn't
make much of a difference. Optimizing Firebox (using about:config),
optimizing the javascript, and investigating connection bottlenecks is
our short term approach to solving this. In the long run though, we
would like to have a 100% open source solution, which means getting
Linux running on the i-opener. I am positive that a Linux installation
with everything optimally compiled for the i-opener will result in
everything being much much faster. But creating a custom Linux
distribution requires more knowledge than simply installing Debian on a
laptop. There are some Linux distributions already created for the
i-opener like Midori Linux and Jailbait Linux. But neither of these work
out of the box (or out of sourceforge actually) and the developers seem
to have disappeared. So we will hack on, but if anybody has any
suggestions please get in touch - children with HIV need our help!

(Isn't blogging great? After writing the above, I realized I don't need
to do tens of thousands of SQL searches. Instead I stuffed all the
relevant entries (most of the database) into a gigantic strategically
structured hash table. It takes longer to startup and uses 70MB of
memory to run, but it is an order of magnitude faster - yippee! Now
about those i-openers...)