Monday, August 11, 2008

jdresolve on GitHub

I'm trying to force myself to switch to Git from Subversion for my current projects. The safest way to do it seems to be using an older project as a guinea pig :) So I just pushed jdresolve over to GitHub. I'll scavenge for the uncommitted patches and the unanswered emails regarding jdresolve and will commit to the Git repository soon (enough).

Sunday, July 27, 2008

Google geocoding charset encoding currently broken

Update (2008-09-09): Google seems to have finally fixed this.

About 2 weeks ago I started seeing weird characters when geocoding addresses via Google using the YM4R gem. The addresses are outside the US and so contain plenty of accented characters that used to be properly encoded in UTF8. Although Google's XML claims to return UTF8, it currently doesn't, sending what looks like ISO-8859-1 encoded characters in some fields instead. This is more than likely a problem with their outsourced partners not having properly setup UTF8 environments and updating GIS information using local encodings.

I search for the issue and found if mentioned in ep's blog. He ended up seeing the same error message:

#<REXML::ParseException: Missing end tag for 'DependentLocalityName' (got "DependentLocality")

That'll usually result in a nice 500 error in your Rails app (if that's what you use) because it raises an exception.

His solution (forcing further charset translations to occur) worked quite well, except that I had to mention the origin charset in my case. So instead of calling to_utf8 plainly, I pass it the charset with to_utf8('iso-8859-1'). This is a very ugly hack in all so I hope that Google fixes the issue soon. I personally didn't report the bug 'cause I never got any feedback from any information I ever sent their way or any requests that I've made in the past.

Wednesday, May 21, 2008

Fixing mint tables after a 32bit to 64bit migration

I just migrated a web site from a regular 32bit processor (Intel(R) Xeon(TM) CPU 2.40GHz) to a 64bit slice (Dual Core AMD Opteron(tm) Processor 265). This made mint v2.14 (or actually the underlying MySQL database) start showing the visitor IPs as mostly 127.255.255.255. A little searching got me to here and here and here. I don't run PhpMyAdmin so here are the raw SQL command for you to run on your console:

alter table mint_visit modify ip_long int(10) unsigned;
alter table mint_visit modify referer_checksum int(10) unsigned;
alter table mint_visit modify domain_checksum int(10) unsigned;
alter table mint_visit modify resource_checksum int(10) unsigned;
alter table mint_visit modify session_checksum int(10) unsigned;


After running these a show columns from mint_visit should return the following:

+--------------------+---------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+---------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| dt | int(10) unsigned | NO | MUL | 0 | |
| referer | varchar(255) | NO | | NULL | |
| referer_checksum | int(10) unsigned | YES | MUL | NULL | |
| domain_checksum | int(10) unsigned | YES | MUL | NULL | |
| referer_is_local | tinyint(1) | NO | MUL | -1 | |
| resource | varchar(255) | NO | | NULL | |
| resource_checksum | int(10) unsigned | YES | MUL | NULL | |
| resource_title | varchar(255) | NO | | NULL | |
| search_terms | varchar(255) | NO | | NULL | |
| img_search_found | tinyint(1) unsigned | NO | MUL | 0 | |
| browser_family | varchar(255) | NO | | NULL | |
| browser_version | varchar(15) | NO | | NULL | |
| platform | varchar(255) | NO | | NULL | |
| resolution | varchar(13) | NO | | NULL | |
| flash_version | tinyint(2) | NO | MUL | NULL | |
| local_search_terms | varchar(255) | NO | | NULL | |
| local_search_found | tinyint(1) unsigned | NO | MUL | 0 | |
| window_width | smallint(5) | NO | MUL | -1 | |
| window_height | smallint(5) | NO | MUL | -1 | |
| ip_long | int(10) unsigned | YES | MUL | NULL | |
| session_checksum | int(10) unsigned | YES | MUL | NULL | |
| visitor_name | varchar(255) | NO | | NULL | |
+--------------------+---------------------+------+-----+---------+----------------+
23 rows in set (0.00 sec)


The sooner you fix your database the less info you'll lose. Took me a couple of hours to realize this was happening...

Wednesday, March 5, 2008

Keeping VNC clipboards in sync

My typical "desktop" consists of two environments: the local machine (usually my notebook) and a VNC session to some fancier hardware. The logic behind this is that I can play around with my local apps and even reboot if I need to without losing any real work, which is usually done on the remote machine.

This works out quite well except for sharing information between environments. Quite often I'm browsing a web site inside the VNC session and want to open it up in my local environment, to have access to multimedia features or simply not to bloat my pristine remote session. Trouble is that X11 has a concept called the "cutbuffer" and another for "selections", which depend on which apps you're running. And, you've guessed it, VNC syncronizes the wrong one (namely the cutbuffer). So to simply copy a URL between sessions I had to select the URL using the mouse, paste it (middle-click) into an xterm window, select it again (!), switch to the other environment, paste it (middle-click again) into another xterm, then re-select it, and finally paste it into my other browser. This took care of syncing all of the related buffers manually.

Today I finally got fed up with it after doing this 3 times within 10 minutes. I found a project called autocutsel that does this automatically for me. Now all I have to do is keep autocutsel running in the background, and every time I copy something inside the local browser (using Ctrl-C) it is instantly available for use inside the VNC session (and vice-versa if I run autocutsel both in my local and my remote VNC environments).

Saturday, February 23, 2008

Setting WEP keys in Kamikaze

I finally had a pressing need to OpenWRT-ize a Linksys WRT54GS v4 I bought over a year ago. The idea is to use the unit for field testing, so I want to be able to constantly change settings without rebooting or running complex scripts. Basically I want to SSH into the WRT and change settings at will as easily as possible.

The version of OpenWRT that I flashed was Kamikaze 7.09 and it has been working beautifully. Very fast boot times, organized filesystem and configuration structure, and plenty of RAM and Flash to spare. The radio in this box is a Broadcom, which I'm not very used to. I have a BCM4318 in my budget Acer Aspire 3002 laptop, and it works quite well, but I basically use it for connecting to a single access point using WPA encryption.

On the WRT box (which I fondly hostnamed 'wart') I wanted to try out WEP (yes, the old, insecure, useless WEP), mostly because a lot of access points in my area use it. Typically one could just use something like iwconfig wl0 enc <WEP key> to accomplish that. Sadly this doesn't work, although no error message is returned. Simply nothing happens, and the status of the interface ends up like:

wl0       IEEE 802.11-DS  ESSID:"MUSIK"
Mode:Managed Frequency:2.462 GHz Access Point: 00:02:2D:0D:1B:39
Tx-Power:19 dBm
RTS thr:2347 B Fragment thr:2346 B
Encryption key:<too big>
Link Signal level:-42 dBm Noise level:-96 dBm
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:516 Invalid misc:0 Missed beacon:0


Notice the Encryption key:<too big> line. It seems that the wireless tools don't play too well with the Broadcom's "advanced" settings. Even after I got this to work I still got the same output from iwconfig.

After some searching I found out that the utility that controls most of the internals of the Broadcom hardware is called wlc. Sure enough it was a "wepkey" option that should do the trick. The syntax seemed to be wlc wepkey <WEP key>, only that always returned:

Command 'set wepkey' failed: -1


I couldn't find a docs for wlc by searching online, so I fetched the full Kamikaze source tree to check its source code (hail to Open Source!). It turns out that the syntax expects you to inform the key slot to store to (1 through 4), and to use "=" to assign the "PRIMARY KEY", which is the slot that will be actually used. Thus, the command that worked for me was wlc wepkey =1,<WEP key>. Notice that I'm using an hex WEP key. If you use an ASCII WEP key, prefix it with "s:".

And that did it for me. Now with a simple call to wlc I can change WEP keys on the fly and get instant results.