Python script to update hosts file for ad blocking

There are many ways to do Ad Blocking. Probably the most popular one being an ad blocking add-on or extension to your browser. These are available for at least Chrome/Chromium and Firefox.

The trouble with adding stuff to the browser is that it also makes the browser more resource heavy. In the world of mobile computing, especially real mobile computers that can multi-task and do stuff that "proper computers" can do, every resource counts. One of these computers is the Nokia N900 running Maemo Linux.

This is where hosts files come in. The default Maemo browser (and the Firefox Mobile that is also available) do have AdBlock+ extensions available. I however like to keep the browser as light and responsive as possible.

Excerpt from Wikipedia about hosts files:
This method exploits the fact that most operating systems store a file with IP address, domain name pairs which is consulted by most browsers before using a DNS server to look up a domain name. By assigning the local 127.0.0.1 IP number to known ad servers the user directs traffic intended to reach those ad servers to the local machine. Running a suitable web server locally the ad content can be replaced with anything the user wishes. For instance if the web server sends a blank html page for any request the ads completely disappear from the pages they were originally intended to appear on.

There are lots of different ready compiled hosts files for offer that contain the majority of major advertisement and banner servers, effectively removing them from web pages. One of these is updated at mvps.org. To automate the process of regular updates of this file, I've made a small Python script to retrieve the newest version and add the content to your /etc/hosts file. This script can be run manually or scheduled to run in Alarmed or Cron like I have done.

The script is designed for Linux distributions that have their hosts file in /etc/hosts, for example Maemo, Ubuntu, Fedora, etc. Just save the script anywhere on your file structure (I prefer to keep all mine in /opt/scripts) and as root type "python [path to script]/gethosts.py" to update your hosts file. If not as root, add "sudo" in the front to run the script with root permissions. Only root has editing rights to /etc/hosts by default so the script will give an error if it cannot write to the hosts file.

Download the script here.

FootieBot ownz your World Cup

The World Cup. It's been on (almost) everyones mind for the last month or so. A lot of people are also guessing out the scores for fun or profit. There are also leagues set up to compete on guessing out the scores. I participated in one of these set up by a colleague, fun and play, no money involved.

Let's make one thing clear. Sports doesn't interest me. At all. I have never been all that interested in football, or any other sports for that matter. But I do like to compete and when Janne, our CEO made a script to calculate scores for our previous betting league (Ice Hockey world championships?), it was like throwing down the gauntlet. On our company forum I swore I would create a bot to participate in the Football World Cup.

And so I did, and he's called FootieBot. Bolted together with Python and equipped with web page analyzing components and Twitter for communications - FootieBot bravely set out to go for win. And well, failed, and is now coming in last :)

Even though FootieBot didn't get the win, it was a fun little project and gave me a good reason to try the power of Python in automated web page handling. As said, FootieBot is completely automated, and scheduled on my phone to run every 5 minutes. The program flow in short;

• Log in to Veikkaajat.net and get the next matches
• Store matches in an SQLite database
• For any matches within the nex 24 hours, get hits from Twitter for the last 5 minutes. Twitter is searched with the official name of the country from the pages of FIFA.
• Store amount of hits in the db
• Calculate a likely result and submit to Veikkaajat.net
• If there is only 20 minutes to kick off, post result on Twitter and mark match as submitted to ensure post is only done once

Requirements;

• Python 2.5 or higher 2.x version is needed.
• simplejson (needed for Python 2.5 only, used by Twython)
• Twython (Twitter API)
• Mechanize (web page automation)
• BeautifulSoup (excellent HTML parser)
• SQLite3 for Python

FootieBot should run on any platform with Python. I've been running it on my Nokia N900 phone, scheduled to run every 5 minutes via Alarmed. How many phones can you schedule Python scripts on? Maemo rocks.

For anyone interested, the code is posted below. It requires the Veikkaajat.net game ID and login details, and Twitter login details. Also some small changes have been made to it over time so it is possible the original World Cup winner and top player -parts are not bug free. Oh and Veikkaajat.net is a Finnish language website for organizing betting leagues. With small modifications the script could be used for any similar website.

Download the code here!

MeeGo 1.0 on my netbook and running Dropbox

Finally installed MeeGo 1.0 on my Acer Aspire One netbook a few days ago and I must say it really is a promising little Linux distro. It carries a lot on from Moblin 2.1, but is even more snappier even on a really low power netbook like the AA1 original "half a gig ram/8 gig flash drive" -netbook.

MeeGo is the only Linux distro to come with Chrome or Chromium as the default browser, which I like a lot of course. There aren't a lot of applications yet in the main repos but it is Linux so in reality there's a huge amount of apps to choose from.

Here are some instructions on how to install Dropbox for MeeGo (courtesy of MakeUseOf.com):

• Go to the Dropbox Linux download page and grab the Fedora x86 package
• Open up the Terminal, change to the folder containing the Dropbox rpm and do the following:
  sudo yum --nogpgcheck localinstall [name of rpm file]
• The Dropbox app can then be opened from the Applications menu
• Remove the automatically added Dropbox repository by opening the terminal and typing:
  sudo rm /etc/yum.repos.d/dropbox.repo
  Otherwise Yum might not work

Happy syncing!

Get MeeGo for your netbook here.
MeeGo 1.0 netbook experience review by Ars Technica

How to generate an easy to remember unique password for each site?

Here is a tip I've wanted to share for a while now. A tip that has made my online life a whole lot easier.

A question: How many passwords do you use on all the websites that you visit? Two, three, four? Or just one for all? With tens or even hundreds of websites where one needs to register as an active net user it becomes impossible to remember a distinct password for all the sites, especially if you only visit some sites maybe once or twice a year, if even that. But that registration form needs a password, so you have to type one in. A common huge security risk is to use a collection of passwords on all sites. This way you will always manage to log in by trying your most usual passwords. I have been using this tactic for ages, having maybe 5 different passwords I have been using.

Then it happens, one of your passwords is exposed in plain text for example by a security exploit to anyone for grabs. Passwords are also compromised by badly designed sites that send them to you upon registration or a "forgot my password" link. Whatever happens, all the sites you use that password for are open for someone else to login with your account. And what if you don't even remember what sites are using a particular password? All of them need to be changed, which is difficult if you don't even know all the sites you have used the password on.

Solutions are numerous, password managers being a popular one. However the problem with these is that they are restricted to either using a particular browser, service or local database (like KeePass, for example). While handy tools, they are slow to use for sites that are not visited often. So here is a tip I found came across some time ago that made all this simple and effective. I couldn't find the original source of the tip, but I'm quite sure it was from Lifehacker.

1. Generate a password you can remember that has at least 5 letters. Apply the general rules, ie include some capital letters and numbers and make sure it is NOT a dictionary word.


2. Create a rule to generate 3 letters from the URL of the website. Do not make it too simple, don't for example use the first 3 or last 3 etc. Something like the first, the last and the second of the second (if any) word.


3. For extra security and variety add to the end the content of another rule, for example a number containing the length of the longest word.


4. Memorize the generated password and the rules. Save them in a password manager if you are the type of person that saves passwords for all sites, this way if you forget either part of the rule you can check them later from there.

When registering to a site you can now generate yourself a decent security, but easily remembered distinct password for all sites. All sites have an unique URL and this makes each password unique. And unless your rules are something too simple like the last 3 letters of the URL, an exposed password is unlikely to give out our rule (unless someone knows you are using this method like anyone will now know of me).

Of course these are just examples of what kind of rules could be created to create distinct passwords for web sites. Sure made my struggle with all the password managing a lot easier.

Add a reboot button to the N900 power menu

Nice N900 usability tip from Mobile Livingroom. Not often will you need to completely shutdown your device, but sometimes a reboot is needed. I usually have a terminal window open so I just type sudo reboot. But the N900 actually has a reboot button implemented - it is just hidden for some absurd reason. All you need to do is edit the system UI configuration file and enable the reboot button. Here is how, instructions courtesy of Mobile Livingroom.

1. Open up a terminal and gain root:
   # sudo gainroot


2. Change to the folder with the System UI config:
   # cd /etc/systemui


3. Backup the old config:
   # cp systemui.xml systemui.xml.bak


4. Open up the config file with your favorite editor or with the following command:
   # vi systemui.xml


5. Go to the line where you see
   Delete <!-- before it and insert <!-- after </menuitem>
   
   Your code should be looking like this:
   
   <menuitem priority="20" name="Reboot">
<return>6</return>
<po>osso-powerup-shutdown</po>
</menuitem>
<!--
<menuitem priority="10" name="Soft poweroff">


6. Exit the editor with :wq (if using vi editor). You will see the Reboot-button active in the menu after a reboot.

If you feel like some more tweaking, don't forget to check out the Maemo.org wiki for some more customization tips, some easy and some only for hard core modders.

[thanks Mobile Livingroom]

Microsoft announces future advertisement driven Windows licence (APRIL FOOLS)

Has the inevitable happened? The rise of free operating systems (Linux, and a future Chrome OS to name a few), and an increasingly free (or ad driven) internet with services from email to online photo editors seems to have changed the view in the Microsoft camp also.

Today an announcement from Redmond tells us of an advertisement driven future Windows 7 licence available at end of May. This will be a free download of "Windows 7 Free Edition", which will have the same features as the Home Edition. Users will be greeted by advertisements during logon (which on Windows is long enough to allow showing advertisements). Launching any program will also launch a pop up advertisement which can only be closed after a 5 second wait.

Is this the first step in Windows making the transition to a free operating system? Would you be interested in this kind of Windows edition, and will it danger the growth of Linux?

Time will tell how successful this edition of Windows will be.

EDIT 2nd April: This of course is an april fools story, there is no Windows Free Edition. Although on the other hand I wouldn't be surprised if Microsoft will need to release some kind of Free Edition at some point in the future. The free OS alternatives (mainly Linux) are getting in to a stage where Microsoft surely needs to take some kind of defensive measures to keep its market share.

Python classes are objects

Python just keeps on surprising me with it's versatility and simple, but powerful, architecture. I just learnt yesterday that Python classes can be used as objects as soon as they are defined. You don't actually need to create an instance of it.

So, consider the following simple class;

class MyClass:
    x = 1

Now you can use the variable x in the class even before any instances have been created from the class. So the following works;

print MyClass.x

See the official Python documentation for classes here ("As in Smalltalk, classes themselves are objects.").

Rocking with the Lucid Lynx beta 1

A few days ago the Ubuntu team released the first beta of the next Ubuntu LTS (Long Term Support) version 10.04, codenamed "Lucid Lynx". Now that the distribution is out of alpha, I thought I would go for it and upgrade. I also took Karmic into use when it hit beta and had little problems.

All you need to do if you're running a desktop version is to fire up the terminal and type:

update-manager -d

Update manager should tell you that a new distribution upgrade is available. There are a lot of packages to update (as you can see from the screenshots). It says "close all programs during upgrade", which I didn't do. It might be a good idea since I did run into a few problems which of course could be related to other issues with the beta version also.

Late in the package installation the installer stopped due to issues with gufw which is a Firewall GUI manager. This was easily solved by removing gufw (sudo apt-get remove gufw).

Restarting update manager informed me of a "partial upgrade". This actually happened a few times and for the same packages. Clearly some packages were not going through correctly and this required a bit of cleaning after which update manager didn't want to do the partial upgrade anymore.

After installation problems;

• Nautilus will not open sftp:// locations. This is possibly related to the partial upgrade, so until it is fixed I am using Dolphin to access my sftp locations (like the N900).


• The new MeMenu is nowhere to be found. Probably something to do with the partial upgrade. Gwibber, the new social client, is cool though and a nice addition to the default applications.


• My external USB drives did not auto-mount as before after the upgrade, they didn't even show up in Nautilus. After manually mounting them with mount they became visible in Nautilus after the next boot. No idea why they went missing but have had no problems with them since.

Also a note to Dropbox users. The upgrade removes the package nautilus-dropbox. It also will not install through the application manager after the upgrade. Installing the Karmic deb file from the Dropbox site directly re-installs the needed packages and Dropbox is back in business, no need to redefine your login details or local Dropbox folder. I suspect nautilus-dropbox will be updated for Lucid before the full release.

So, was upgrading worth it? Not if you don't feel adventurous and are not willing to fix a few problems (although there were actually less than I had anticipated). If you do however, go ahead. The problems faced are no showstoppers for home usage.

What has changed? Well the most noticiable change is that boot time has got even faster. Even as it is a beta version! This is due to HAL being removed from the boot process. Also a more controversial change is that the GNOME window manager places the "maximize minimize close" buttons to the left, instead of to the right as before. This takes some time getting used to and after a few hours I decided to use Ubuntu Tweak to move the buttons back to the right. Windows are easier to resize than before. A lot of small visual changes and Nautilus is definately better (with multi-pane support built in). There are more not instantly noticiable changes, check them out from the official Lucid Lynx beta 1 page. Even though it is beta I've not had a single application crash yet.

Apart from the few small problems, which I'm confident will soon be fixed, Lucid Lynx is running great, fast and looking good.

Fill Me Up 0.8 alpha file shuffler

Here we go. My first ever publicly released software project out live to the world to see. It's a small software project, and I suspect you are already asking "What the hell is a file shuffler?". Well I'll tell ya.

Fill Me Up is an utility that copies random files. Sounds funny but it's something that I actually needed myself. I have a large music collection and I hate choosing what music to take with me when I head out. Since I can't take everything, I had to find a way to shuffle the music beforehand. So came to light Fill Me Up, which is also my first PyQt project. To make a third first, it's also my first multi-platform compile as there is a version for Linux and Windows (both from the same source code).

And yes, I know there are music managers out there that shuffle music for you. Unfortunately Rhythmbox, which I use and love, doesn't have that functionality, and I don't want to change my music player. But that's the great thing about writing code - if something is missing some functionality, just code your own.

The other reason I wanted to "go all the way" and distribute Fill Me Up was that I've never distributed anything before, and I wanted to see what kind of work it is to release something for several different platforms. The most time consuming part in my opinion was the testing, ie. setting up clean virtual machines to try the application on and to optimize the shipped dependencies. It was all more work than I originally thought. A lot of lessons learnt all in all.

So if you feel like you might need a program that copies random files to a specified target, read more about Fill Me Up and feel free to download and try! It's open source and all so the source is there for grabs as well. Feel free to comment on that too if you're an experienced PyQt GUI designer  and looking at the code makes you weep ;)

Maemo Mapper for N900

Huge news! Maemo Mapper gets promoted to Extras, which means it's considered stable enough for general public use. Maemo Mapper is basically an open source navigation and mapping tool with support for things like real time over the air map downloading, navigation and POI's.

In addition to the basic navigation features offered by commercial competition like Ovi Maps and Google Maps (the latter which there is no Maemo version yet and we are also still waiting for the new Ovi Maps for Maemo also), Mapper also has route tracking and a whole lot of customization that isn't possible on the commercial clients.

Have not tested it in the car yet but so far it's looking quite stable and responsive. For an open source project to compete on the navigator front it's certainly huge!

If you're rockin' the N900, give it a try by installing it from the application manager or directly from this link.