Saturday, 31 May 2008

Defragmenters for Linux

There were a lot of words were told about defragmentation of Linux filesystems. Summing them up, one can say: if fsck tells that fragmentation is below 10% then defrag is not necessary. But a few defragmentators for Linux were written. So let's discuss them.

Search results
A brief look at defragmentation tools for Linux is enough to find two utilites, a defrag and a shake. The defrag utility by Cone Colivas seems to be outdated. But there is a shake that can defragment a selected directory in a file system. There is also a useful
script frag.perl located here that can measure fragmentation level of directories:

sudo ./ /root
3.2258064516129% non contiguous files, 1.04838709677419 average fragments.
Using these tools, shake and frag.perl, we are going to fight with filesystem's fragmentation in Linux.

Setting it up
The setup of shake is slightly complicated process. The sources are located here, and if you are using Ubuntu you can take the binary file here. And here I had experienced some difficulties...

Firstly, latest versions of shake depends from fresh version of cdbs; that is why it is better to use older version. For the shake to be built, following packages are reqiured:
dpkg-checkbuilddeps: Unmet build dependencies: cdbs (>= 0.4.50) cmake (>= 2.4) help2man (>= 1.26) libattr1-dev (>= 2.0.0)
The program had compiled as well as her auxiliary tool unattr. For using shake more efficiently you might to mount a filesystem with user_xattr. To do such you need to edit /etc/fstab such like:
/dev/sda2 /mnt/flash/exthdd ext3 rw,user_xattr,user,auto 0 0
Then rmount: mount -o remount /dev/sda2
This trick allows the shake to works faster. And the last but not least, both shake и frag.perl are required root privileges so use sudo.

An example
I had tested shake on Hardy Heron repository that I downloaded. Checking fragmentation level gives:
./frag.perl /home/beast/flash/exthdd/linux/i386/ubuntu/pool/main 0.795800880460549% non contiguous files, 1.00998984083982 average fragments.
That is not a Windows situationan apocalypse. Then I had started shake:
./shake /home/beast/flash/exthdd/linux/i386/ubuntu/pool/main
The defragmentation took about 15-20 and all MD5SUMs are correct. So we can use shake for more extended experiments.

The extended experiment
For such experiment I had used to defragment Firefox's profile on my laptop: Firefox, ReiserFS, Asus M5200AE (P-M 1.75, 512Mb RAM).

There are a lot of files, and even worse: in Firefox profile there is a huge collection of webpages captured by Scrapbook. Estimated fragmentation is:
./frag.perl /home/beast/.mozilla/
3.24491439557299% non contiguous files, 1.12239224753839 average fragments.
I should say that my Firefox is covered with extensions like a Christmas tree, so it starts very sloooooooowly:
time /opt/firefox2/firefox
real 0m40.467s
user 0m5.389s
sys 0m0.407s

time /opt/firefox2/firefox
real 0m41.383s
user 0m5.453s
sys 0m0.403s
Firefox had been started two times with laptop clod-start each time.

Then defragmentation had been performed by shake. It took about 9 minutes. After that I started Firefox again:
time /opt/firefox2/firefox
real 0m39.522s
user 0m5.398s
sys 0m0.374s

time /opt/firefox2/firefox
real 0m39.214s
user 0m5.279s
sys 0m0.370s
There are no miracles, and 39 seconds are not fantastical results.

Sure, I can defrag something else but it is unlikely that results will be different. The analogous results are
here and there.

Monday, 31 March 2008

Magic SysRq Key

Nothing is perfect in this world, and even Linux sometimes crashes. But even then there is a connection with kernel - Magic SysRq Key.

What is this?

If you look to the keyboard then you see a strange key PrtSc / SysRq. Most people thinks that such key is for taking screenshots, but SysRq appeared on keyboards many years before Redmond behemoth.

Using SysRq key it is possible to force a Linux system to sync disks, remount disks into "read-only" mode, and then reboot computer safely. The key combination is intercepted directly by kernel, hence it is guaranty that Alt+SysRq is workable almost in any situations.

How we can use it?
First, we need to ensure that Magic SysRq key is enabled in kernel config: CONFIG_MAGIC_SYSRQ value must be Y (in most distributions it is so). To use such key, press and hold Alt, then press SysRq, release it and press another key - meanings of most useful keys are listed below

Simultaneously pressed Alt + SysRq +

'H'elp - prints to output all key combinations;

loglevel'0'-'8' - can change verbosity of the output from 0 (only critical messages) to 8 (most verbose mode);

re'B'oot - immediately reboot, like RESET button (no sync, no unmount filesystems);

'C'rashdump - runs kexec to reboot for obtaining errors info;

hol'D's - shows all blockers that blocks devices and files;

power'O'ff - correctly shuts down the system (if tweaked);

'S'ync - tries to sync all mounted filesystems and remount them in ReadOnly mode; you must see "Emergency Remount R/O" in console, and "Emergency Remount Complete" when it is over;

'U'nmount - tries to remount all mounted filesystems to ReadOnly mode;

show'T'asks - shows all tasks that are running now;

sho'W'-blocked-tasks - shows all uninterruptable tasks that are blocked or waiting I/O;

'F'ull - running oom_kill (out-of-memory kill) to kill applications that are eating all of memory;

secure access 'K'ey - to kill all tasks on the current console (don't do it on a console where X server running; in this case you can not see anything);

un'R'aw - takes away mouse and keyboard from X server; it it useful if X is trapped so you can change console and kill unwanted tasks;

t'E'rm - send SIGTERM signal to all processes except init.

K'I'LL - send SIGKILL signal (immediate termination) to all processes except init.

Thus for emergency sync you should press ALT+SysRq + s then ALT+SysRq + u and after that you can reset you computer or press ALT+SysRq+b

Except kernel documentation you may be interested in reading this and this links.

Saturday, 23 February 2008

ACPI and WiFi card Intel PROWireless on Asus M5200ae M5A

In this post tweaking wireless card Intel PROWireless 2200 is described. It is a little bit tricky to get such card to work on Asus M5200AE notebook. But using latest extras from acpi4asus project, my wifi card works perfectly.

Intel PRO/Wireless 2200
Because this adapter is made by Intel, no problems should appear during installation of the drivers. It is required to download the firmware from here, unpack it and copy to /usr/share/firmware

Also you must install the packages for wireless networking:

aptitude install wireless-tools, kwifimanager, kwirelessmonitor-net kwirelessmonitor kwrapper
And also install packages for ACPI:
aptitude install acpi-support acpid laptop-mode-tools
If all steps were performed correctly then during boot process should be such strings:
ieee80211_crypt: registered algorithm 'NULL'
ieee80211: 802.11 data/management/control stack, 1.1.14
ieee80211: Copyright (C) 2004-2005 Intel Corporation
ipw2200: Intel(R) PRO/Wireless 2200/2915 Network Driver, 1.1.3dmprq
ipw2200: Copyright(c) 2003-2006 Intel Corporation
ACPI: PCI Interrupt 0000:01:05.0[A] -> GSI 18 (level, low) -> IRQ 17
ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
ipw2200: Radio Frequency Kill Switch is On:
Kill switch must be turned off for wireless networking to work.
ipw2200: Detected geography ZZM (11 802.11bg channels, 0 802.11a channels)
It seems to work, but there are suspicious string emphasized by red bold font. What is a "Kill Switch" and why you can not connect to wireless networks when it activated?

Radio frequency killer
There is a combination of keys that turns on and off a radio transmitter of the WIFI chip. Such combination is often mentioned as "Kill Switch", thus is you did not deactivated such kill switch then yours WiFi ciph is turned off and nothing can help you.

To test Kill Switch is on or off, you can type such command:
# cat /sys/bus/pci/drivers/ipw2200/0000\:01\:05.0/rf_kill
According to the documentation we have:
read -
0 = RF kill not enabled (radio on)
1 = SW based RF kill active (radio off)
2 = HW based RF kill active (radio off)
3 = Both HW and SW RF kill active (radio off)

End of story: Kill Switch is on, Wifi's radio transmitter is turned off. It also can be checked by iwconfig:

eth3 radio off ESSID:"virens"
Mode:Ad-Hoc Frequency:2.412 GHz Cell: Not-Associated
Bit Rate:0 kb/s Tx-Power=off Sensitivity=8/0
Retry limit:7 RTS thr:off Fragment thr:off
Encryption key:off
Power Management:off
Link Quality:0 Signal level:0 Noise level:0
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:0 Invalid misc:0 Missed beacon:0

As usual, Kill Switch key combination on notebooks is Fn+F2 but on your notebook it may be different.

For the key combination to turn on you need to activate ACPI extras for your notebook. For Asus laptops it is Asus Extras. You need to obtain latest version of the asus_extras driver from SVN of the Acpi4Asus project.
For those who compiled the kernel by themselves, here are some guidelines:

Power management options (ACPI, APM) --->
[*] Power Management support ---->
[*] ACPI Support:
<М> AC Adapter
<М> Battery
<М> Button
<М> Video
<М> Fan
<М>Thermal Zone
< > ASUS/Medion Laptop Extras
Next, turning on the support of a Radio Frequency Kill Switch

Networking --->
[*] Networking support -->
Improved wireless configuration API
--- Wireless extensions
<М> Generic IEEE 802.11 Networking Stack (mac80211)
[ ] Enable debugging output
<М>Generic IEEE 802.11 Networking Stack
[ ] Enable full debugging output
--- IEEE 802.11 WEP encryption (802.1x)
<М> IEEE 802.11i CCMP support
<М> IEEE 802.11i TKIP encryption
<М> Software MAC add-on to the IEEE 802.11 networking stack
[ ] Enable full debugging output

In the Networking section turn on:
<*> RF switch subsystem support --->
<*> Input layer to RF switch connector
It is certainly that you need the NAT for providing Internet connections from you laptop, thus you need to activate NAT and Masquerading:

Networking --->
[*] Networking support -->
Networking options --->
[*] Network packet filtering framework (Netfilter) --->
Core Netfilter Configuration --->

<*> Netfilter netlink interface
<*> Netfilter connection tracking support

IP: Netfilter Configuration --->
<*> IPv4 connection tracking support (required for NAT)
[*] proc/sysctl compatibility with old connection tracking
<*> IP tables support (required for filtering/masq/NAT)
<*> IP range match support
<*> TOS match support
<*> recent match support
<*> ECN match support
<*> AH match support
<*> TTL match support
<*> Owner match support
<*> address type match support
<*> Packet filtering
<*> REJECT target support
<*> LOG target support
<*> ULOG target support
<*> Full NAT
<*> MASQUERADE target support
<*> REDIRECT target support
<*> NETMAP target support
<*> SAME target support
<*> Packet mangling
<*> TOS target support
<*> ECN target support
<*> TTL target support

Next thing to do is activation of the wireless card.

Device Drivers ---->
Network device support --->
[*] Network device support
Wireless LAN --->

[*] Wireless LAN (IEEE 802.11)
<М> Intel PRO/Wireless 2200BG and 2915ABG Network Connection
<------- this is our driver, let's compile it as a module
[*] Enable promiscuous mode
--- Enable radiotap format 802.11 raw packet support
[*] Enable creation of a RF radiotap promiscuous interface
[*] Enable QoS support
[*] Enable full debugging output in IPW2200 module.

Enabling firmware load:
Device Drivers ---->
Generic Driver Options ---->
Hotplug firmware loading support

For the connection to be encrypted you need to turn on encryption support:

Cryptographic options ->
<*>ARC4 cipher algorithm
<*>Michael MIC keyed digest algorithm
<*>AES cipher algorithms (i586)

And also this one:
Library routines ->
<*>CRC32 functions

After all you need to turn on the LED's support for indication of a wireless status. In the Device Drivers section:

Device Drivers ---->
LED devices --->
[*] LED Support
<*> LED Class Support
--- LED drivers
--- LED Triggers

That's all; compile the kernel and reboot with it.

Let's fire up!
It is practically all that needed for your wireless happiness. During the boot process our kernel must detect the wireless card and turn it on. Now press Fn+F2 and the WiFi LED must turn on. If it works then all is done correctly and WiFi now can be used. Let's check the state of Kill Switch:
# cat /sys/bus/pci/drivers/ipw2200/0000\:01\:05.0/rf_kill
That's it, the radio is turned ON. Now type:
# iwconfig
lo no wireless extensions.

eth1 no wireless extensions.

Warning: Driver for device eth3 has been compiled with version 22
of Wireless Extension, while this program supports up to version 20.
Some things may be broken...

eth3 unassociated ESSID:"virens"
Mode:Ad-Hoc Frequency=2.412 GHz Cell: Not-Associated
Bit Rate:0 kb/s Tx-Power=20 dBm Sensitivity=8/0
Retry limit:7 RTS thr:off Fragment thr:off
Encryption key:off
Power Management:off
Link Quality:0 Signal level:0 Noise level:0
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:0 Invalid misc:0 Missed beacon:0
It works! The rest of the job is simple: just configure an ad-hoc network. For the network to be configured you need to edit /etc/network/interfaces like this:

auto eth3
iface eth3 inet static
wireless-mode ad-hoc
wireless-channel 1
wireless-rate auto
wireless-essid virens

Now let's tweak network card:
iwconfig eth3 mode Ad-Hoc channel 1 essid virens
That's all and you got simple ad-hoc network with identifier virens .

Sunday, 10 February 2008

SSH for mere mortals

There are a lot of documents about the SSH, but in this post I want to tell about simple things. Let me explain some simple and obvious SSH tricks...

How to install SSH in Debian

From the Etch's release, packages for the SSH client and server are separated. Hence let's install SSH using command:

# aptitude install ssh
aptitude install openssh-server openssh-client
and wait a few moments for configuration of the packages.

On a client side
Now we must edit the configuration files located in the /etc/ssh directory. A client-side config file is ssh-config and a server-side config file is sshd-config. On a client-side let's allow to receive X11Forward; or such purposes, changing:
ForwardX11 yes
ForwardX11Trusted yes
Now the client machine can start the graphical applications on a server. Now let's go to the admin of the far-far-away kingdom server...

On a server-side
We need to edit config file (you have root password from the server, aren't you? :-)) in  /etc/ssh/sshd-config and changing the keys to such ones:
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes
Here we allowed to start graphical applications remotely and redirect them in the client machine. Restart the SSH daemon:
sudo /etc/init.d/ssh restart
That's practically all.

If there is necessity to restrict access to the machine, one need to change config file /etc/ssh/sshd_config in such way:
AllowUsers hacker@*
AllowUsers *@192.168.1.*

SSH in work
It's all done, now open console and type:
$ ssh remoteusername@ip_address_remotemachine
For example, in my case I write: ssh beast@

After that, the SSH daemon asks us: this IP address isn't recognized yet, can I trust them? Sure! :-) Next, type the password of remote system, and if the password is correct then you log in into shell of the remote machine. During password typing you see nothing; you have 3 attempts or connection lost.
So, the SSH system will greet us like this:

penta4@penta4rce:~$ ssh beast@
Linux notebeast #3 PREEMPT Sun Jul 2 12:51:07 MSD 2006 i686 GNU/Linux

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

Last login: Tue Oct 10 19:23:57 2006 from

Now we control remote computer: don't entangle machines or you shut down the wrong one! :-)

Remote using of the graphical applications
When you logged in, just type the name of the program with ampersand at the end of command:
$ gimp&

This command starts the GIMP on the remote machine and returns command prompt to you. If you do not enter ampersand then command prompt of the current SSH shell locks and returns only after application's over.

Thus, started remotely application is redirected to your machine and works the same as it started on your machine. Of course, the remotely started application will use files on the remote filesystem.

It is clear that SSH can be useful to mere mortals and not only for gurus. More about SSH one can  read in Goooogle or in this  this or this inks.


Saturday, 26 January 2008

Debian printing via CUPS

Task: to print documents via local printers in Linux.

Solve: it is required to install CUPS printing system and add the printer, which can take less than minute.

Let's firing up the CUPS
The CUPS is Common UNIX Printing System. Before you install CUPS, you ought to look if you printer is supported, starting from here. If you printer is supported, install the next packages:

aptitude install cupsys gs-esp foomatic-bin foo2zjs
It is significant to install not only CUPS itself, but a programs used by him.

It is really common mistake when people installing only cupsys,package and forgetting to install foo2zjs when using Hewlett Packard printers. As a result, during the print process, errors like listed below appearing:
I [23/Aug/2005:14:41:46 +0400] Adding end banner page "none" to job 7.
I [23/Aug/2005:14:41:46 +0400] Job 7 queued on 'HPDeskJet930C' by 'penta4'.
E [23/Aug/2005:14:41:46 +0400] Unable to convert file 0 to printable format for job 7!
I [23/Aug/2005:14:41:46 +0400] Hint: Do you have ESP Ghostscript installed?
I [23/Aug/2005:14:41:46 +0400] Hint: Try setting the LogLevel to "debug".

When all packages will be installed, you must add printers and start to print.

In press!

For new printer to add or to change settings of already installed printer, launching any browser and go to the address:


After some time, settings page should appeare:

The CUPS page may loading in a long time if there is misconfigured DNS server in the local machine or in network. It is preferable to deactivate unnecessary network interface
ifconfig eth0 down
Pushing Printers menu item - if there are no printers, CUPS must confirm that. For new printer to add, a root privileges are required:

Next, writing printer's name with Latin font and without spaces. After that considering a printer's connection type (in my case, it's a USB printer):

Thus, choosing a printer from a list:

A few clicks after and all work is done. It is a good idea to enter in Printers menu and check if it appearing in list of ready-to-print devices.

It can be easily configurable resolution, paper size and so on.

Printing from the GiMP

To print from GIMP, several additional packages are required:

aptitude install cupsys-driver-gimpprint gimp-print

Now open the GIMP, any picture and select File - Print. After that, a big dialog box should appear, where all printer properties can be tweaked.

Here are some obstacles: gimp-print by default assuming you printer definitely postscript-compatible, what is sometimes wrong. To prevent incorrect printer output, printer must be selected directly:

That means to push Configure printer, selecting a model, approving and saving tweaks. Finally, all should work perfectly.


Wednesday, 23 January 2008

Howto simply configure network card in Debian

Besides presence of eye-candy graphical configurators in Linux, in many situations it is more straightforward to change config files. It is really simple, and below it is shown how to do it.

How to see current network card settings
For brief review of network settings, it is enough to type in command prompt:

# ifconfig
sudo ifconfig
And something like this should appear:

lo        Link encap:Local Loopback
          inet addr:  Mask:
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:104 errors:0 dropped:0 overruns:0 frame:0
          TX packets:104 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:9387 (9.1 KiB)  TX bytes:9387 (9.1 KiB)
eth0      Link encap:Ethernet  HWaddr 00:0A:E4:53:AA:2D
          inet addr:  Bcast:  Mask:
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:21 Base address:0x2c00

It is evident, that there are two interfaces had been configured: loopback named lo, and conventional Ethernet card eth0.

How to configure network connection in Linux
To do so, you need root privileges and _your_favorite_text_editor, as well as knowledge of which IP address you need to enter.

To find out which network interface need to be configured, type:
dmesg | grep -i Eth
and next strings should appear:
8139too Fast Ethernet driver 0.9.28
eth0: RealTek RTL8139 at 0xdf822c00, 00:15:f2:51:ad:da, IRQ 21
eth0: Identified 8139 chip type 'RTL-8101'
It looks like it is eth0 (because ethernet, 0 - zero device, pretty logical). Here and below it is assumed that interface is eth0

Configuring Linux network with static IP
Just edit the file:
# nano /etc/network/interfaces
sudo nano /etc/network/interfaces
For your local network static IPs are surely enough. In particular, for static IP networking you need to enter: IP-address, netmask and gateway. Change  /etc/network/interfaces to something like this:
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
In this example, IP-address is set.

Configuring Linux network with dynamic IP addresses
Continue to edit network config:
# nano /etc/network/interfaces
sudo nano /etc/network/interfaces
It is simpler here:
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

Similarly, network interface can be configured via command line:
#dhcpcd eth0
For this, you should install in Debian aptitude install dhcp3-client.

Configuring DNS nameserver in Linux
If you don't configure DNS, you cannot connect to any Internet resource by it's name. But it's very simple to tweak it: just edit or create file /etc/resolv.conf
# nano /etc/resolv.conf
sudo nano
And type addresses like this:
as much as it is need. The word nameserver is required. It is curious, but in fresh Debian installation there is no resolv.conf...

For changes to take place immediately...
... one can reboot the system, or type:
# /etc/init.d/networking restart 
sudo /etc/init.d/networking restart 
That's all, changes will be applied for all network interfaces.

Saturday, 12 January 2008

Disk burning errors and some workarounds

Sometimes in a process of disk recording errors are happening. Such discs are often being threw away, but there are some tricks to finish the recording process. After that, disc is passing MD5 check and reads normally. Below are mentioned several tricks to do so.

Error during start or in the middle of recording
It is rarely happens, but at he beginning or in the middle of recording process growisofs suddenly broking precess and message appears:

Executing 'builtin_dd if=1.iso of=/dev/cdrom obs=32k seek=0'
/dev/cdrom: "Current Write Speed" is 16.4x1352KBps.
          0/4393730048 ( 0.0%) @0x, remaining ??:?? RBU 100.0% UBU   0.0%
          0/4393730048 ( 0.0%) @0x, remaining ??:?? RBU 100.0% UBU   0.0%
          0/4393730048 ( 0.0%) @0x, remaining ??:?? RBU 100.0% UBU   0.0%
          0/4393730048 ( 0.0%) @0x, remaining ??:?? RBU 100.0% UBU   0.0%
          0/4393730048 ( 0.0%) @0x, remaining ??:?? RBU 100.0% UBU   0.0%
    1671168/4393730048 ( 0.0%) @0.4x, remaining 1007:27 RBU 100.0% UBU   1.9%
:-[ WRITE@LBA=330h failed with SK=3h/ASC=0Ch/ACQ=00h]: Input/output error
:-( write failed: Input/output error
/dev/cdrom: flushing cache
/dev/cdrom: closing track
:-[ CLOSE TRACK failed with SK=3h/ASC=0Ch/ACQ=00h]: Input/output error
/dev/cdrom: closing session
/dev/cdrom: reloading tray

All that you need is to start burning again: growisofs restarts the process of recording from the last informational sector and will have record disc completely. After that, disc can be read without any problems and it passes MD5 check.

Close session error
Disk had been fully recorded, but I/O error appears while closing the session:

2144544*2KB out @ average 9.9x1352KBps
/dev/cdrom: flushing cache
/dev/cdrom: closing track
/dev/cdrom: closing session
:-[ CLOSE SESSION failed with SK=5h/ASC=72h/ACQ=03h]: Input/output error

It is pity, that disc had been burned completely but session isn't closed. Such disc is unreadable. But that can be corrected using simple trick from authors of dvd+rw-tools: here is a little program close, and here is the source code. Downloading it from here, installing sources of  dvd+rw-tools and copying close.cpp in the directory with dvd+rw-tools sources. After that, let's compile close.cpp :

g++ -o close close.cpp
Don't forget to give it appropriate permissions, and insert disc with unclosed session in the drive. Close session by command:
close /dev/hda
If you DVD-recorder have different device file than /dev/hda please correct it.