Search Image Online

Image search is evolving quickly. Today you don’t even need to use words to search images. Now you can use colors!. That’s right you can search images by color.

Let’s search the web for an autumn-like palette (yellow, red, brown) using the following 3 services. Let’s see how the search results are like.

1. PicItUp

PicItUp is a visual image search engine that combines word-based with color based search.

Test it: search for [autumn] and choose any color from the left column (I chose yellow). Now let’s look at the results:

Impressive, isn’t it?

What the tool lacks: you can’t combine several colors to refine your search (for example you can’t search for a yellow-red palette) - you are only able to search for one color.

2. Multicolr

Unlike the above one, with the Flickr-based Multicolr (already mentioned at Make Use Of) service you can combine as many colors as you want.

Test it: choose several colors and watch how the image list changes with each color added (I combined red and yellow):

What the tool lacks:you can’t combine word-based and color-based searches with this service. You are limited with playing with colors only.

3. Etsy

Etsy, an online shop of handmade products, has developed color search for their own product database.

Test it: It’s real fun to play with. Just browse your mouse across the screen and watch various colors and same-palette products pop up.

What the tool lacks: here you are unable to either set the color option or the search term. Nonetheless the tool is such fun to use that you definitely need to try it out.

Now, what do you think could be possible practical applications for these tools?

Next Movie ?

If you frequently find yourself cluelessly searching for some good movie to watch, this article is for you.

It will give you a quick overview of two little-known movie recommendation sites, that are easy to use, easy to get started and indeed help you find the movies you’re likely to enjoy.

Tastekid - Quick Movie Recommender

Tastekid gives you a basic search field where you can enter a movie title of your choice and get back a list of similar movies. No need to signup or rate anything, just enter the movie you like and discover similar movies. Quick and easy.

When it comes to movie recommendations it might need some improvements though. I’d say only about half of the movies it returns are actually similar to the one you entered. However it still works great for me.

Its killer feature is the ability to get a quick preview window for any listed movie without leaving the page. Simply point your mouse cursor over the ‘?’ mark next to each title and you’ll get a small pop-up window with short movie description and trailer. Very useful.

MovieLens - Personalized Movie Recommendations

As stated above, MovieLens is a personalized movie recommendation engine. What this means is that your recommendations are tailored to your taste and based on your ratings of other movies.

The obvious advantage of it is that you get better recommendations. The disadvantage is that it takes time to setup and get it working. First of all, you have to signup and rate at least 15 titles before it can recommend any movie to you.

Now, you should also keep in mind that the more you tell Movielens about the movies you love and hate, the better your recommendations become. So just keep on rating as you browse the site.

Once you have rated your first 15 movies, you will be able to access your account area. Below is a screenshot of my account page (click to enlarge).

Although there are plenty of features here, you will rely upon only 3 things.

1) Predictions

Based on your movie ratings, MovieLens generates a so called prediction ratings for movies you haven’t seen yet.



The higher predictions are the more likely you will enjoy the movie. Moreover, your search results on Movielens are also sorted based on the prediction ratings.

2) Search

The Search feature is what you’re probably going to use the most. MovieLens lets you search for recommended movies by genre, year of release, title and more. Now, let’s say you want to find some good crime movie that you haven’t seen earlier. For that, simpy leave the title field blank, select the crime genre, the year of release and click search. Movielens will then get you all the crime movies matching your criteria, sorted based on prediction ratings.



There is also an Advanced Search mode where apart from searching for all the “best movies” within a certain domain, you have an option to exclude certain genres. That is to say, you can see what the “best movies” are that are in the crime category but not in Drama. Sweet.



3) New movies

Another neat feature, is a quick list of newly released movies along with a prediction for each of them. See a sample below (based on your ratings, your list might be different)

That’s about it. There are some other things you can do at Movielens but I don’t think any of them are really unique enough to be mentioned here.

To sum up, I think both websites offer something useful. Tastekid is more suitable for quick lookups and trailer peaks, movielens on the other hand for more personalized recommendations and for discovering great movies you never knew about. We have also reviewed a service called The Filter, though my personal choice stays with the above ones.

10 Apps !!!

Ubuntu comes pre-installed with a wealth of apps - covering almost every task you need to do, while still managing to fit on a CD. Are you ready to discover some pretty cool Linux apps that don’t get the attention they deserve? Let’s check them out.

You can easily install any of these apps using the ‘Add/Remove Applications’ from the upper panel.

1. Midori

Midori is a web browser that uses the well-known WebKit rendering engine, providing quite a speed bump from Firefox and standards complying page rendering. Midori is currently packaged into Xubuntu and features include tabs, bookmarks and customizable search engine box. The browser is extensible via Javascript.

2. gDesklets

gDesklets is a platform for widgets, tiny applets sitting on your desktop in a symbiotic relationship of eye candy and usefulness. You can populate your desktop with status meters, icon bars, weather sensors and news tickers.

I managed to configure gDesklets to emulate the OS X Dashboard feature in addition to the standard placement on the desktop.

3. AcidRip

AcidRip is a GTK application for ripping and encoding DVD’s. It neatly wraps MPlayer and MEncoder, which I think is pretty handy, seeing as MPlayer is by far the best bit of video playing kit around for Linux.

As well as creating a simple Graphical Interface for those nervous of the MEncoders command line interface, it also automates the entire process.

4. Cheese Webcam Booth

Cheese is a Photobooth-inspired GNOME application for taking pictures and videos from a webcam. It also includes fancy graphical effects based on the gstreamer-backend.

5. Gmount-iso

Gmount-iso is a small tool written using PyGTK and Glade. It allows you to easily mount your CD images.

6. KGRUBEditor

A tool to view and edit the menu.lst file of the GRUB boot manager. It offers many features and it is the perfect solution for those who want to change the way GRUB works, without messing with the menu.lst file.

7. Flickr Uploader

Postr is a small app that lets you upload photographs to Flickr, with tight integration into the GNOME desktop.

8. DOS Emulator

DOSEMU stands for DOS Emulation, and allows you to run DOS and many DOS programs, including many DPMI applications such as DOOM and Windows 3.1, under Linux.

Features include:

• word size and addressing modes of the iAPX86 processor family’s “real mode,” while still running within the full protected mode environment
• simulate a hardware environment over which DOS programs are accustomed to having control.
• provides DOS services through native Linux services; dosemu can provide a virtual hard disk drive which is actually a Linux directory hierarchy.

9. E-book reader

FBReader is an e-book reader with a lot of great features:

• supports several open e-book formats: fb2, html, chm, plucker, palmdoc, ztxt, tcr (psion text), rtf, oeb, openreader, non-DRM’ed mobipocket, plain text, epub
• reads directly from tar, zip, gzip, bzip2 archives (you can have several books in one archive)
• supports a structured view of your e-book collection
• automatically determines encodings
• automatically generates a table of contents
• keeps the last open book and the last read positions for all open books between runs
• automatic hyphenation (patterns for several languages are included)

10. HomeBank

HomeBank is a fast, simple and easy to use program to manage your bank accounts. It differs from gnucash for the better look and feel, and for the faster start-up speed. It has a lot of features such as easy analysis with graphical charts (statistics, budget, overdrawn, car cost), multi-accounts support, budget management, reminders, import from OFX/QFX-CSV files, visual status of operations. It is based on GTK2.

Shell Commands LINUX

Hollywood movies often have a tech geek entering commands and doing amazing things. While it may not be that easy to hack into public transport systems or or control the world like Eagle Eye, the command line is certainly a geeks playground.

Want to show off your command line prowess to your geeky friends but don’t know any cool tricks? Well you are in luck. The following websites will give you plenty of tricks and tips to satiate your hunger:

shell-fu

“Fu” is defined as possessing superior skills related to an art. Aptly named “Shell fu” houses user-submitted command line goodies, tips and tricks. You can vote entries up if you like them and vote them down if you find they are harmful or do not work as advertised.

If you have some tricks of your own you can even submit them and they will be added to the site after the moderators have a look at them.

command-line-fu

Shell fu and Command-line fu are almost identical in concept. Command-line fu allows users to share their CLI wisdom with others and also vote up and down on submitted snippets.

Both shell-fu and command-line fu allow you to follow snippets through RSS and Twitter although I did find command-line fu offers a bit of extra control on RSS feeds.

snipt

Snipt is the “long term memory for coders”. Snipt is a collection of frequently used commands and code snippets. You can share these snippets with your co-workers or make them public. Public snippets are viewable to everyone and for our specific case, we are interested in the bash section. However feel free to browse other sections as well.

good coders code, great reuse

While all the other resources mentioned utilize the power of the crowds to build up content, this one is different. It is Peteris Krumins blog about programming, software and hacking.

What’s so special about it? The famous sed, awk one liners, vim tips, plugins and cheat sheets that Peteris has fabricated over time. The occasional musical geek friday is fun as well.

While it may not be updated as often as the others, but more often than not the content is worth the wait.

Pick up the basics

While the places mentioned above have plenty of content to keep you busy learning, they are not the best places to start your journey. If you want to start with basics have a look at Introduction to the Linux Command Line published on MakeUseOf earlier. You may also find command line basics here and here. For advanced topics you can read the Advanced Shell Scripting Guide.

That’s plenty of information to keep you involved and build up your Linux-fu! Do you know of any good sites to learn Linux commands? Or have you referred to a Linux command line tutorial that you really like? Share with the world in the comments!

10 Programmers Interest Sites

Most programmers are not just desktop programmers, web programmers or scripting gurus - they are often all of the above. Over time, most software developers find that it’s much easier to re-use code components than it is to recreate the wheel every time you write an application. Over time, these developers typically archive a library of these reusable modules in order to save time the next time they need to do the same task.

Now, thanks to the power of the Internet, there is a proliferation of websites where professional programmers offer these modules to the general public for absolutely free. For other application developers, these libraries are a goldmine. Whether you write in VB, C, C++, C#, Perl, PHP, Ruby, or any other programming language, there’s a website out there where you can find such reusable sample code.

Choosing the Top Websites for Sample Code

Every developer has a list of their favorite websites where they not only search for free sample code, but they also take part in the developer community that exists at these sites. Programmers often develop a very passionate attachment to some of these websites.

The following is, by no means, an exhaustive list of the best developer websites online. Such a list would be difficult, since there are thousands of excellent resources throughout the net. The following is simply a list of ten of the best places to start if you’re looking for high quality sample code. So please, folks, no food fights if your favorite isn’t mentioned. The following websites are listed in no particular order.

#1 - Programmer’s Heaven

Programmer’s Heaven is a huge online resource that is very much a community of many different programmers. The site includes information, tips, and forums on languages like Basic, C++, Delphi, Java, Perl, PHP, Python, Ruby, Visual Basic and more. You can also search the site by application or platform.

Programmer’s Heaven has forums, blogs, articles, files, links and news for every single programming language listed on the site. The size of this website is impressive, and the community is growing quickly into one of the largest developer communities on the net.

If you need sample code that’s not already on the site, just ask for help in the forums.

#2 - SourceForge.net

It goes without saying that SourceForge is one of the most significant open source communities on the Internet. When you need to review sample code to learn how to do something, this is a great place to search.

From the main page, just click “Find Software,” and then “Development.” The next page will astound you. You’ll discover an entire world of open source projects.

You read that right, that’s over 54,000 listings under “Software Development.” If you find a software project that sparks your interest, and you can see how the software could be greatly improved, you might consider joining up with one of the many open source projects.

See the SourceForge overview for how this works. Open source is a way of life, and these coders take it very seriously.

#3 - CodeGuru

CodeGuru is also a cool site for developers. It mostly covers just Visual C++/C++, .Net/C# and Visual Basic. While this isn’t a huge selection of programming languages, the site is filled with articles that are far more comprehensive than your standard short forum posts that you find at other websites. CodeGuru offers useful explanations of how to accomplish specific tasks.

The example above is an article on how to load images from or save images to a database using Visual Basic. The article has quality screenshots and well written sample code.

#4 - The Code Project

CodeProject is a booming developer community that’s organized very much like a free article directory website, except in this case the content producers are actually programmers who are trying to provide the best programming articles available. Most of these come complete with well written explanations along with the sample code.

The Code Project isn’t your run-of-the-mill resource where you’ll sift through the site for the code you need and then run. You’re likely to get sucked into the many prize competitions, surveys, articles, message boards, or even the job board. This community is a lot of fun and a good way to connect with other programmers with common coding interests. Before you know it, you may be tempted to write an article yourself!

#5 - DevX

No list of developer resources would be complete without a mention of DevX. This site is a massive programmer’s paradise - a huge portal to other huge sites covering the most popular programming topics such as Java, C++, Database programming, Visual Basic, Mobile programming and much more.

Running your mouse over the left menu bar, as shown above, will display the sub-communities that exist under the DevX umbrella portal. When I had a significant Visual Basic project to complete in 2001, this website quickly became my home away from home.

#6 - Planet Source Code

Planet-Source-Code is one of those websites that you hate to like, but you need it anyway. It’s horribly designed, with a weird tiny-font menu listing at the top and ads splattered throughout the page, but if you know where to click, you’ll find a huge library of very useful sample code. At the top right where it reads “To start, just choose:” select the programming language of your choice. The site covers most of the major languages. On the next page you’ll see many categories of sample code to choose from.

Most of the languages have search filter options, and as you can see here, each category has tens, and even hundreds of thousands of lines of sample code available.

#7 - GNU and The Free Software Directory

If your thing is Open Source, then you can’t miss out on the Free Software Directory that’s linked directly from the GNU home page. This directory is just an impressive list of free software within categories like audio and video, database, email, games, graphics, Internet apps, communications and much more.

#8 - Google Code

Google has also joined the open source crowd with Google Code, a site that offers mostly web based sample code within categories like browser apps, website ads, product APIs, and social networking apps.

Google Code received a honorary mention in the MakeUseOf article “10 Google Services that Get No Love,” by Mark.

As always, Google uses its standard format - a very clean and easy to use website where you’ll have no problem quickly finding exactly the open source code that you need.

#9 - DevelopersDex

DevelopersDex is another developers community that has rewarded some of the highest quality contributors with “guru” status. The site is essentially a giant developer’s forum, currently approaching just under 50 thousand members. The site covers ASP, C#, SQL, VB, and XML.

Each programming category features complete subcategories such as FAQs, job directories, news, and of course forums and newsgroups.

If you’re a developer using one of these languages and you’re looking for a massive web portal to browse through, this is a good one.

#10 - DevArticles

Saving one of my favorites for last, DevArticles offers one of the most comprehensive directories of articles offering sample codes on the Internet. The list of available software categories is astonishing, including like Flash, embedded tools, and even mobile Linux.

Clicking on any one of those categories turns up a list of useful articles complete with sample code and full descriptions.

Making Bootable Linux CDs

Introduction

These are some discussion notes about my endeavours in creating a bootable CD. The main outcome of all this is my 'Pauls Boot CD'. There is an atypical page describing it here.

Why put linux on a bootable CD? It means being able to take a useable linux system and start it on any PC you like without the pain of installing it all. You could use it as an install disk for a linux distribution or perhaps some other operating system. Or you could make some sort of 'appliance' out of it:

• MP3 player. You could burn all your MP3s plus a small bootable linux to turn your PC into a jukebox.
• You could make your own 'Network Appliance' that has the operating system and server software on CD, and use a hard disk for data storage only. It would mean you'd never have to install an OS at all.
• You could use it as a quick *nix toolkit. Just put enough on the CD to get networking and X windows going, start up Opera or netscape and you have a minimal graphical interface on a network. Or put pppd on the CD and you can dial up an ISP ala an internet appliance.

For other uses of bootable CDs have a look at ThinkNIC . Its a CD based Linux like thin client. DemoLinux is a distribution that only works on a CD.

This article outlines building a very general purpose 'base' for building a bootable linux system ... something that you can add to to create your own personal setup.

By the way, this is a major restructuring of an earlier document I wrote. You can find the old one here.

Research

Unfortunately, when I first started looking for information on the net, there didn't seem to be too much info about how to make bootable Linux CDs. After a bit more searching I've found the following good references:

• RescueCD by Martin L. Purschke
• Linux Appliance Construction Set by Michael W. Shaffer

The standard for booting CDs is called 'El Torito'. The concept is that your BIOS treats your CD like a floppy initially and expects to find the 'image' of a bootable floppy on the CD. Your imaginary boot floppy is meant to have CDROM drivers on it which can somehow access the CD in full. This seemed overly complicated until I came across SysLinux. Its a boot loader that is part of Debian I think. It primarily is a floppy bootloader (like LOADLIN), but it now has an extra bit called ISOLINUX, that simplifies booting off a CD. The key benefit is that you no longer need a special floppy image. ISOLINUX will load the kernel and an initrd for you and away you go.

The boot process

The usual Linux boot process is something like:

• The PC starts up and runs LILO or some other bootloader
• LILO knows where the kernel image is and starts to load it
• The kernel runs. When its finished doing all its checks, it attempts to mount the root filesystem. The major and minor numbers for this device are usually encoded in the kernel itself or passed to it as arguments from LILO
• Once the file system is mounted, /sbin/init is executed and your system starts up as per your inittab ... and your /etc/rc*.d scripts

The CD boot process is somewhat different. Again we need a boot loader, but we don't necessarily know what device our CDROM is. It could be /dev/hdb, /dev/hdc, /dev/hdd. Even if we told the boot loader where the kernel is, we would still need to tell the kernel where its root filesystem is. ISOLINUX helps us get around this, by working out where the CD is. This allows us to boot the kernel, but it doesn't really help us to load the initial root filesystem. Many boot disks use a thing called an initrd (Initial ram disk) to get around this. initrd is an initial root filing system running in RAM. It loads prior to when the kernel attempts to mount the 'real' root file system. Yes this is odd. The idea is is that your initrd starts up, loads some critical modules, then mounts your real root file system.

Now have a look at my CD startup process.

• The CD is installed with ISOLINUX, so it boots first.
• ISOLINUX loads the kernel from the /isolinux directory on the CD.
• ISOLINUX now loads the initrd.gz compressed ext2 file system. Its important to note that ISOLINUX loads this, and not the kernel. The kernel will grab it later.
• The kernel starts up and eventually decompresses the initrd.gz to ram (in /dev/ram0 actually) and mounts it as root. You have to enable INITRD and Ram disk support in the kernel to have this happen.
• The kernel tries to execute the /linuxrc file in the new root filesystem (because its an 'initrd').
• The linuxrc program tries to mount the CDROM (It has to make a few guesses to work out where it is), then copies a compressed 'real' root filesystem from the CD into /dev/ram1. We mount the new filesystem just so we can add a softlink in its /dev directory for the CD (so we don't have to work out where the CD is again).
• When the linuxrc script finishes, control returns to the kernel and it attempts to mount its configured root filesystem. In this case, I've rdev'd the kernel to make it use /dev/ram1 as the root filesystem. This will mount our newly created ram based root file system. In this file system I have an /sbin/init and the system starts up running entirely in RAM.

So why does it have to be a two phase process? Linux doesn't really know how to boot off a CD yet. The beauty of the initrd phase is that the initrd filesystem is loaded by the bootloader (ie. not the kernel). It means that we can effectively boot off any device, so long as the bootloader is able to read from it.

Basic Requirements for building Bootable CDs

First, you're going to need a machine pre-installed with a Linux distribution to create the CD on. I'm using Slackware 7.1 on a machine with a HP9100i CDwriter. I installed the cdrecord, mkisofs etc tools. I've had to compile into the kernel all the necessary SCSI options to use the IDE CD-Writer (See the CD Writer HOWTO), plus I have loop file support compiled in. I'd suggest you buy yourself a couple of blank CD-RWs to play with as you're going to be formatting, burning, trying it out ... repeatedly.

You'll also need Syslinux. You need Nasm to build it. However, if you're too lazy to do this, all you need is the ISOLINUX.BIN file that is created.

Create a kernel for the CD

Now you need to create a kernel that can load the initrd. I'll assume you know how to build a kernel. All the experimentation here was done with a 2.2.18 kernel. You must compile in initrd support and RAM disk support. I am using the default 4096K RAM disk size. Other things that you'll need are ISO9660 filesystem support, ext2 filesystem support. Once you have this kernel, you need to set the root device on it:

   eg. Say you've just done a make bzImage

    rdev /usr/src/linux/arch/i386/boot/bzImage /dev/ram1

   NB: If you don't have a /dev/ram1 on your system, create one with:

    mknod -m 640 /dev/ram1 b 1 1

Create a directory tree

You need a bit of template structure to work with. The layout I have is below:

The key directories are:

cdimage/	This is what mkisofs will use to write an image to the CD
initrd/	Holds the tree of our initrd filesystem
root/	Holds the tree of our real (final) root filesystem

If you download my bootkit, it lays everything out in this structure.

Setting up initrd

The INITRD phase has a simple goal ; Get a real root filesystem into /dev/ram1. I've actually tried a lot of different alternative ways to do this. In order they are:

1. Use a little shell script that uses the ash shell to mount the CD, gunzip a rootfs.gz file directly into /dev/ram1 and exit. This requires libc, so the initrd.gz image is about 600k. This is the most straightforward approach ... but the most wateful.
2. Use an i386 assembly language program as 'linuxrc' to perform the same task except that I used an alternative compression/decompression algorithm as gunzip was too complex to implement in assembly. linuxrc ends up being about 1k!! and requires no external libraries. This ends up with an initrd.gz of about 10k.
3. Use a statically linked C program for linuxrc that uses the gunzip.c source from Busybox as a basis, so that we effectively perform the same task as the shell script. By statically linking with dietlibc the linuxrc actually ends up being smaller than a dynamically linked one against libc 2.1.3!. The total size of linuxrc is about 20k, and the initrd.gz is about 15k.

The C based linuxrc is really the best option. Its still quite small and uses a 'standard' for compressed files. Read about the asm one here, and the shell one here.

The basic sequence of events in the C program are as follows:

• Mount /proc
• Open the /proc/ide/ide0/hda/media file. If it says 'cdrom' inside it then we've found our boot CD, otherwise try /proc/ide/ide0/hdb/media, then /proc.../ide1/hdc/... , then /proc.../ide1/hdd/... until we find the first CDROM drive. This undoubtedly limits us to bootable IDE CDROMs on the two standard internal chains.
• Change directory to /dev and create a symlink for 'cdrom' to the newly found CDROM device (eg. cdrom --> hdc).
• Mount /dev/cdrom on /cdrom
• Open /cdrom/rootfs.gz and extract it into /dev/ram1. It must be a gzip of a 4mb ext2 filesystem.
• Mount /dev/ram1 on /ram
• Change directory into /ram/dev and create the same symlink for the CDROM again.
• Unmount /ram, /cdrom and /proc

The last step is simply to exit. The kernel should now take over, and mount /dev/ram1 as /, and try to run /sbin/init.

The linuxrc.c source is here . To compile it you should get dietlibc . Version 0.9 has a 'diet' command that simplifies compiling programs using the new library. You should be able to compile linuxrc.c using: diet gcc -o linuxrc linuxrc.c

The 'Real' root filesystem

You can't do too much with an initrd only system. You really need to create a useable root filesystem. Again, I'm using a 4mb RAM disk, so its a bit of a squeeze (why didn't I go for a bigger RAM disk? I just hate wasting space, and I wanted this to be useable on machines with 16mb of ram or more). I chose Busybox to provide most of my /bin tools. In 0.51 of Busybox, you even get things like vi and wget. Its getting fatter at 250Kb or so these days with every tool compiled in (NB: You can reduce its code size by editing Config.h and commenting out the defines for the tools you don't want), but 250K is fine for our 4mb filesystem. I've put some libraries in too:

   ld-2.1.3.so
   ld-linux.so.2 -> ld-2.1.3.so
   libbz2.so.1.0 -> libbz2.so.1.0.0
   libbz2.so.1.0.0
   libc-2.1.3.so
   libc.so.6 -> libc-2.1.3.so
   libcom_err.so.2 -> libcom_err.so.2.0
   libcom_err.so.2.0
   libdl-2.1.3.so
   libdl.so.2 -> libdl-2.1.3.so
   libe2p.so.2 -> libe2p.so.2.3
   libe2p.so.2.3
   libext2fs.so.2 -> libext2fs.so.2.4
   libext2fs.so.2.4
   libm-2.1.3.so
   libm.so.6 -> libm-2.1.3.so
   libncurses.so.5 -> libncurses.so.5.0
   libncurses.so.5.0
   libtermcap.so.2 -> libtermcap.so.2.0.8
   libtermcap.so.2.0.8
   libuuid.so.1 -> libuuid.so.1.2
   libuuid.so.1.2

Not all of these are really required to get the system up and running. I think some are left over from when I had mke2fs. /bin is just busybox and a sh*#load of soft links. I have an /sbin/init (not linked to busybox, its a real sysvinit). In /etc I have an inittab, termcap and an rc.d directory. Currently I just have an rc.S startup script that is run during the sysinit phase of the init startup. The key parts of my inittab are:

   id:1:initdefault:

   si::sysinit:/etc/rc.d/rc.S
   c1:1235:respawn:/bin/sulogin /dev/tty1

By using sulogin on the console, I force the user to enter a password (just a space), with the benefit that root's shell starts up in a normal login environment (ie. /etc/profile is run followed by ~/.profile)

/etc/fstab looks like:

   /dev/ram1       /        ext2        defaults   1   1
   none             /dev/pts  devpts     gid=5,mode=620  0   0
   none             /proc    proc        defaults   0   0

I also have an empty /initrd directory. This means that once that the initrd phase has finished, and the real root filesystem on /dev/ram1 is mounted, the initrd filesystem (on /dev/ram0) is moved under /initrd. This means we can unmount it and free the ramdisk space.

My /etc/rc.d/rc.S script looks like:

#!/bin/sh

PATH=/bin;export PATH
echo "System init"
mount -t proc none /proc
mount -o remount,rw /
echo "Find the extras on the CD and mount it"
if [ -r /dev/cdrom ] ; then
   mount -t iso9660 /dev/cdrom /cdrom
   mount /dev/hda9 /a
   fi
fi
cd /
umount /initrd
freeramdisk /dev/ram0

First we mount /proc, and then remount / (remember we have a valid entry for / in /etc/fstab now) as read/write. Now we check if the /dev/cdrom device is there (it should have been created in the initrd phase), then mount the cd. I have a symlink from /usr (in the root filesystem) to /cdrom/usr. This means that you can add your own extra binaries and extra libraries on the CDROM under its own /cdrom directory. For additional testing, I also mount an ext2 partition on my harddisk as /a.

Burning a CD with a kernel , initrd and a rootfs

Now that we have a kernel, initrd structure and rootfilesystem structure, how do we get them onto the CD. Lets look at the basic structure of the CD (how it would look if you just mounted it as an iso9660 CD):

 /rootfs.gz

 /isolinux/
           isolinux.cfg   - the isolinux config file
           isolinux.bin   - the isolinux boot program
           vmlinuz        - our kernel with initrd support
           initrd.gz      - our initial ram disk

The isolinux.cfg file contains:

   label linux
    kernel vmlinuz
    append initrd=initrd.gz

Its similar to a LILO config. The main things are the kernel name and the append string which includes the name of our initrd. The fact that its gzipped doesn't matter. the kernel will automatically decompress it later.

What about the initrd and rootfs? These are both 4mb files that have been compressed with gzip. The contents of the 4mb file is a valid ext2 filesystem. Here's how I create them, given that I have a directory on my hard disk with the structure of the initrd or rootfs:

   BASE=/src/iso
   SRC=$BASE/initrd
   DEST=$BASE/cdimage/isolinux/initrd
   dd if=/dev/zero of=$DEST bs=1k count=4096
   losetup /dev/loop1 $DEST
   mkfs -t ext2 -m 0 /dev/loop1
   mount /dev/loop1 /mnt
   cd $SRC
   tar cf - . | (cd /mnt ; tar xf - )
   umount /mnt
   losetup -d /dev/loop1
   gzip -f $DEST

To burn the CD, I have the cd tree shown above under my /src/iso/cdimage directory (.ie I have a /src/iso/cdimage/isolinux directory). I run mkisofs as per the isolinux.doc that comes with Syslinux:

        mkisofs -o /iso.img -b isolinux/isolinux.bin -c isolinux/boot.cat \
                -no-emul-boot -boot-load-size 4 -boot-info-table -l \
                -R -r /src/iso/cdimage

The boot.cat file is created by the mkisofs command. The -l , -R and -r options are essentially for RockRidge extensions which allow us to have softlinks on the CD and mixed case filenames. We should end up with a couple of meg iso.img file. You can now burn this to the CD using cdrecord (I'll leave this up to you as your speed and dev settings are undoubtedly different to me).

Reboot your system, enter the BIOS setup screens to check that your system will boot off a CD and load the CD and see if it boots. Hopefully, you should see 'Loading vmlinuz', then 'Loading initrd.gz', then the kernel should do its stuff. You should end up with a 'please enter the root password' message (just enter SPACE the ENTER). You should be able to cd and ls and so forth.

A basic image to try out

I've uploaded a very basic ISO that you can burn to a CD and have a play with. It doesn't do too much as yet, but it has the full set of busybox 0.51 commands available and I've added some scripts to simplify the setting up of your lan card and IP address details. To allow different LAN cards to be configured, I've included all the modules for different LAN cards in the /modules directory of the CD.

First, download the iso.img.gz file. Decompress it with gunzip and burn it to a CD (I'd suggest a CDRW) using cdrecord or if you want to do it from Windows, you probably just need to rename it to blah.iso and Easy CD Creator or whathaveyou should be able to burn it.

The CD contains my own initrd.gz and rootfs.gz. You can have a look at these by decompressing them and mounting them as ext2 filesystems on a loopback mount.

Reboot with the CD in your CD drive and you should get some ISOLINUX message and away it goes. I've enabled framebuffer support, but you can select whether you want plain text or one of the graphical modes.

You'll get an sulogin prompt (please enter the root password or press ctrl-D), so enter the root password (a single space) and you should be at a bash prompt.

To configure your LAN card enter cfgcard and follow the prompts. The list of cards that is shown is simply the modules directory on the CD. Cards like 3c509 are obvious, but others are not. I have an rtl8139 based card which is the most common el-cheapo 10/100Mbps card you can buy.

Now do a cfglan which allows you to set the IP address, netmask and default gateway etc. It also asks you about nameservers and such as it creates a /etc/resolv.conf so name resolution will work.

Note: to keep the size of the ISO quite small, I have not included any X related stuff.... for now.

Framework for extensibility

OR ... how to add your own extra bits. As I said previously the /usr directory on the CD is symlinked in to become the actual /usr. Create a /usr/bin on the CD and add in all the tools that Busybox doesn't include. Add some libraries into /usr/lib ... and what I've been working on is the basic requirements to get X up and running. One thing to remember is that you really do have to run mkisofs with RockRidge extensions on in order for softlinks and mixed case filenames to be created properly on the CD.

Framebuffer

One of my experimental aims is to have an X windows environment on a boot CD. To achieve the widest possible compatibility, I've chosen to enable the Framebuffer console mode and to use the XF86_FBDev X server (its just the one from Slackware 7.1 at the moment). Note: Even though I am aiming for wide compatibility just so I can run X, Framebuffer mode doesn't work with pre VESA 2.0 video cards which means you may not want to add in Framebuffer if all you really need is a console prompt. To activate Framebuffer console mode you need to make sure some things are compiled into the kernel, typically this includes:

   [*] VGA text console
   [*] Video mode selection support
   [*] Support for frame buffer devices (EXPERIMENTAL)
   [*] VESA VGA graphics console
   [*] Advanced low level driver options
   <*> 8 bpp packed pixels support
   <*> 16 bpp packed pixels support
   <*> 24 bpp packed pixels support
   <*> 32 bpp packed pixels support
   <*> VGA characters/attributes support
   [*] Select compiled-in fonts
   [*]   VGA 8x8 font
   [*]   VGA 8x16 font

The other thing that I didn't realise until later is you have to make sure you set a graphical mode for the console when it boots in order to use the X server in default mode. This means putting a specific vga= setting appended to the kernel at boot time. Specifically, you need to change the /isolinux/isolinux.cfg file on the CD so it looks something like:

label linux
 kernel vmlinuz
 append initrd=initrd.gz vga=791

The '791' means to start up in 1024x768x16bit colour mode. Hard coding the display resolution is fine if you know for certain that your video card/monitor can handle it, but what I've done is to let the user choose a display option at boot time. My isolinux.cfg looks like this:

timeout 30
prompt 1
display menu.txt
default 1
label 1
 kernel vmlinuz
 append initrd=initrd.gz

label 2
 kernel vmlinuz
 append initrd=initrd.gz vga=788

label 3
 kernel vmlinuz
 append initrd=initrd.gz vga=791

menu.txt is a simple text file that looks like:

   1) Text Mode
   2) 800x600 x 16bit colour
   3) 1024x768 x 16bit colour

The user just enters '1' if they want text mode, 2 for 800x600 and so on.

Notes

• The isolinux.cfg file seems rather dodgy if I put something like:

  label linux
     kernel vmlinuz
     append initrd=initrd.gz root=/dev/ram

but seems to work OK if I take out the root=/dev/ram and simply use rdev to set the root fs in the kernel to /dev/ram1.
• You can't make /linuxrc start init properly. It just keeps spitting out a 'Usage' string. I think its because init is unable to start as pid 1.
• mkisofs will only write 8.3 style uppercase only filenames by default. If you want filenames up to 31 chars long, then specify the -l option. If you want mixed case filenames you need to enable the Rock Ridge extensions with the -R option, and may want to use the -r option as well which assigns sensible user/group ownership to all files. You will definately need the -R option if you want to copy whole unix directory trees to the CD.

With all the risks involved in using file sharing networks, browsing unprotected website directories is probably a lot easier and safer. Only when you start doing that do you realise half the stuff that people keep in their website folders (Sports Illustrated magazines!)

Then you start to realise how silly they are for not password protecting those folders and leaving them wide open for the whole world to walk in and take a look!

This might be really old news for a lot of people but I thought I would just quickly jot down the search parameters for finding files in unprotected website directories.

In case you don’t know, an unprotected website directory is a website that doesn’t have an “index” file created for it - index.htm, index.html, index.php. So if you try to access a website directory which doesn’t have a password controlling it or which doesn’t have an index page, you will be able to see a list of all the files and folders that are inside that directory. If you can see that, you can then click on the files and both download them and open them.

Here’s what a typical unprotected directory looks like :

Directories like these will have all kinds of files. Things like pictures, music, video files, documents, you name it.

Now you can do a general search and go through literally hundreds of thousands of these directories. But to do that sort of search is both time consuming and a bit mind numbing. But if you want to do it, just put into the search box (be it Google, Yahoo, whatever) the following search string :

-inurl(html|htm|php) intitle:”index of” +”last modified” +”parent directory” +description +size

This will bring up EVERYTHING and you can go hunting for whatever you can find. Good luck.

But don’t you want to be selective? Don’t you want to look for something in particular? Well, if so, you can change the search string to look for ONLY pictures or ONLY music or ONLY video. So….

-inurl:(htm|html|php) intitle:”index of” +”last modified” +”parent directory” +description +size +(wmv|avi)

This will only look for wmv and avi video files. You can easily alter it if you don’t want “wmv” or “avi” or if you want “mpg” instead. You get the idea.

-inurl:(htm|html|php) intitle:”index of” +”last modified” +”parent directory” +description +size +(jpg|gif)

This will only look for jpg and gif files. Again, you can alter the file formats to suit yourself.

-inurl:(htm|html|php) intitle:”index of” +”last modified” +”parent directory” +description +size +(wma|mp3)

This will only look for wma and mp3 music files. Again you can easily change the file formats to suit yourself.

Just put the search string you need into the search engine box. Then hit the ‘enter’ button and your results will come up. I guarantee you’ll be hooked for ages trying to see what you can find! You can also put a certain search term after your file format so :

[-inurl:(htm|html|php) intitle:”index of” +”last modified” +”parent directory” +description +size +(jpg|gif) "britney spears"]

Obviously you are not going to get perfect results. You are going to have to wade through a lot of irrelevant and useless stuff some of the time but quite often you do find a lot of good stuff too. It’s quite fun peeking into people’s unprotected folders seeing what they have stashed away. Stuff like embarrassing photos, drunken videos, “provocative” material, and much more.

Some people have embedded these search algorithims into software which makes it easier to search for files. One of them is Clickster which I reviewed last May. It searches for MP3’s in peoples unprotected directories and it has a very nice simple GUI.

Some of you might say what right do we have to go browsing through people’s website folders? But look at it this way - these people posted this stuff online - in an unprotected unsecure website folder. It’s as if they are asking for it to be found. They are making no effort to keep it hidden or secure and putting it out on the World Wide Web is the most stupidest thing in the world to do if you want to keep something private and hidden.

Ever worked late into the night on your computer and found yourself blinded by the light from the monitor? Yes, all of us have experienced it at some point or another.

So what do you do then? Get down and dirty and adjust the screen’s brightness to be a little easy on the eyes, only to find it ineffective and then find yourself adjusting it again the next morning? There’s definitely got to be a better way to automate this. And surely there is - “F.lux“.

A no fuss application that doesn’t bother you too much and sits quietly in the system tray adjusting the screen for the better. F.lux uses your location and the time of the day to find out what would be the lighting conditions around you. It is an known fact that there is considerable difference in lighting conditions when the sun is out and glowing and when it is set. The natural lighting (due to the sun) has a cooler feel to it than artificial lighting.

F.lux takes all of this into account and with the knowledge of your local sunset and sunrise timings it adjusts the color of your monitor display to suit the time of the day. The display gets warmer during night time and cooler during day time. Odd you would think? Shouldn’t it be the other way round? Actually no because surprisingly less color temperature means more blue whereas more color temperature means more red.

F.lux does an amazing job in detecting your location and current state of lighting. I just had to install and run it and instantly it started adjusting the display and did it right. Running it the first time you might feel a bit odd because the first adjustment would be big and you would be able to notice the color changing. After that the changes are subtle and you won’t even notice them. You can click on the main interface and see a 24 hr preview of the changes that would occur.

If for some reason F.lux is not able to set up your location correctly or if you want to take matters into your own hands you can always summon the F.lux GUI from the system tray. Click on ’settings’ and you will be able to see the current location F.lux is using. If you want to change it and you know the latitude and longitude values for your city/town you can do so here (check out Google Maps or just Google the city name if you have trouble finding the correct values).

F.lux also lets you customize the lighting values. Adjust the sliders to choose the maximum and minimum values within which the display color will oscillate. If you use fluorescent lighting you can adjust the night slider so that colors don’t become too warm for your lighting conditions. You can also change the transition speed if you have issues with it. Fast seemed to work just fine for me.

If you are doing photo editing or some other color sensitive work you can choose to disable F.lux for one hour.

F.lux is available for Windows, Mac and Linux. Note that F.lux is not the same as the ambient light sensors that come with laptops these days and adjust the brightness of the screen according to the lighting conditions. You don’t require any sensors for F.lux to work and moreover it adjusts the color as opposed to the brightness.

It’s amazing how such a simple idea can become part and parcel of your work in no time. I wish they had a portable version so one could take it along in a USB drive, run it and feel at home.

Is your ISP throttling your Bittorrent Downloads

So the cat’s been out of the bag for a while now that ISP’s are throttling or in some cases even blocking your BitTorrent downloads or seeding.

Comcast came out and admitted to it’s practices in the United States last year. You can read about it here on wired. Since then there has been lot’s of speculation on if other ISP’s are doing the same thing…

What do you think?

Yeah we do also!

Now there is a free tool that will simulate BitTorrent activities and let you know if your speed or activities have been effected and to what extent. This program (aka bittorrent throttling test) and group have picked up momentum since 2008 and now have the backing of Google and other big names.

Are you curious as to if your ISP is throttling your torrent downloads and doing you dirty?

Well so are we.

Run the tool below on any web browser on any operating system and post back what your results are in the comments. It is strictly web based and nothing gets installed. So you can try it pretty much anywhere.

I tried this in the office on my Cogent circuit. It is a 100mb Ethernet handoff so I was not really expecting any throttling. All you need to run the test is browse on over to:

a title=”http://broadband.mpi-sws.org/transparency/bttest.php” href=”http://broadband.mpi-sws.org/transparency/bttest.php”>http://broadband.mpi-sws.org/transparency/bttest.php.

You will be greeted by a web page that looks like this:

Simply scroll down on the page until you see the Start Testing button. You can run the detailed or the simple test. The only difference is the intensity of the test. Some ISP’s are believed to only throttle torrents after a certain period of time (greater than 5 minutes) so this will have you covered.

After you hit the Start testing button your test will begin and you will see a count down in a window that looks something like what you see below.

When the test is complete you will get your results. They will also look something like mine which you can see below. My ISP is NOT throttling my BT downloads or seeding activity and they go into detail on each point they checked as well as the speeds they were able to achieve with the transfers.

I love geeky statistics. Here are my results: