Nye fagre uddannelseskort

Som studerende på en videregående uddannelse er jeg pr. juli måned dette herrens år tvunget til at udskifte mit gode gamle blå SU-pendlerkort fra DSB med et fimset grønt plastikskolekort.

Nå, det æstetiske præg er jo en smagssag – det nødtvungne NemID et senere debatemne.

Mit første forsøg på at logge ind dirigerede mig videre til borger.dk hvor jeg skulle vælge kommune hvilket skulle vise sige fuldstændig overflødigt.

Jeg har jo tidligere benyttet mig af uddannelseskort.dk i forbindelse med at jeg gik på adgangskursus i Lyngby. Derfor har de åbenbart stadig min gamle adresse i deres system.
Nå ja, selvom jeg er logget på med min NemID SSO-adgang-til-alle-mine-oplysninger, kan den jo smutte – selvom den burde opdatere selv.

Jeg må jeg så opdatere min adresse manuelt. Jeg trykker på det øverste punkt der hedder “Ret din profil” hvor jeg ganske rigtigt kan se min adresse, men ikke ændre den.

Jeg fortsætter min søgen efter stedet hvor jeg kan ændre min adresse..

Der er et punkt der hedder “Du flytter”. Mjae, men det er nu et stykke tid siden.

Her kan jeg igen se min adresse i et indtastningsfelt jeg ikke kan rette i. Der er dog et lille link ude i siden af feltet med teksten “Ret”. Bingo!
Jeg trykker triumferende… Og … Intet sker. Jeg trykker igen på bedste “den har nok glemt jeg har tykket”-manér, men heller ikke dette tryk registreres. Jeg sætter mig slukøret tilbage i stolen og mærker nederlagets bitre smag på mine læber.

Bedst som jeg sidder der og har ondt af mig selv dukker en lille dialogboks op hvor jeg nu kan indtaste flyttedato. Så er vi tilbage på sporet! Der er endvidere en knap med skriften “Jeg kan ikke finde min adresse” desværre lader den ikke til at have nogen effekt overhovedet.

Jeg forsøger at ringe til uddannelsesstyrelsen, men får et “Alle vores linjer er optaget, ring igen senere” Hvorefter der meget uhøfligt bliver lagt på uden et “Ha en go’ dag”, eller “Det var hyggeligt (ikke) at snakke med dig”. Nå, men jeg er da så ikke den eneste der har problemer.

Jeg får ideen at prøve med anden browser (Iceweasel aka. firefox), og det går straks bedre. Nu kommer der en lille roterende vifte når jeg trykker på et link der ikke umiddelbart gør noget med det samme. Der er dog stadig ~4s forsinkelse på alle navigationstryk.

Det lykkedes mig (vistnok) til sidst efter en længere kamp at få bestilt et uddannelseskort og “meldt flytning”.

Ideen med at skifte alle over på uddannelseskortet er rigtig god, og da jeg i sin til oprettede det erindrer jeg ikke at have haft de store problemer. Dengang loggede man på med en selvoprettet adgangskode og sit CPR nummer. Man kan i øvrigt stadig finde den gamle logind side – det var tredie hit på google for mit vedkommende.
Det lugter lidt af at en politisk beslutning har forsaget at systemet skulle løfte nogen helt andre krav end de oprindelige, og skalerer ikke til kravene. Jeg henviser selvfølgelig til de sløve svartider.

Til sammenligning tog det mig 5 minutter at få fornyet mit gamle pendlerkort i forbindelse med jeg alligevel var på stationen. Det tog mig omkring en time at bestille det nye kort.

Min akilleshæl har i denne forbindelse været to ting: Jeg har benyttet mig af Chromium (12) browseren, som ikke helt har været understøttet. Og at jeg allerede havde en konto oprettet under det gamle system.

Hvis Danmark skal fremgå som et IT-fremgangsland syntes jeg det er for sløvt at man har en så træg arbejdsgang når der skal udføres rimelig simple tast-selv opgaver. Jeg er klar over at det hele skal gå op i en større enhed rent dataintegrationsmæssigt – men det er til grin at man skal taste tingene ind manuelt når vi nu har en SSO løsning. Kom ind i kampen.

Fornuftig usability og sparet tid er vejen til at få samfundet digitaliseret – ikke tvang og frustration.

Til slut lidt tø-hø’er af småfejl jeg mødte:

uddannelseskort billede

HD time-lapse movies with Motion and Linux

NSLU2 with webcam
The system

Background

I have previously experimented with time lapse videos, but wanted a more dedicated platform which could be set up, and run pretty much anywhere.

This is the first iteration, where the the purpose is to get the system up and running with headless operation.

Components

The original idea was to use a PC Engines alix1d system board in a box1c enclosure, but unfortunately the board I had was running very unstable – so I brought in an old friend of mine:

The Linksys NSLU2 aka. “slug”. The one I had ran Debian 5.0 Lenny, but had to be upgraded in order to get the webcam to work.

I recklessly tried doing a dist-upgrade, but ended up with bricked slug. Guess a fresh installation was the right answer indeed.

Debian Squeeze on a NSLU2

Due to a required proprietary firmware, the official Debian 6.0 installer does not ship with support for the on board Ethernet controller – which is bad because this is the only way of communicating the the device. Well, technically you can use the serial pin header or an USB Ethernet device, but I think I have burned the circuit for the serial port in a previous modding attempt :-\

There is a few guides that  give you directions on how to add the proprietary firmware to the installer image, and after about 5 reflashes I finally had one that worked.

Before starting the installation, I checked around for known installation errors. The installation takes about 5 hours, so you really want to get i right the first time.

I learned that others had experienced out of memory errors during the installation. Though luck.

To the rescue came Martin Michlmayr. He has the answer to all my quarrels; a compiled guide, with a complete Debian 6 userspace and kernel. This saved me a lot of time.

 Install and configure Motion

You can install motion by

apt-get install motion

as root or via sudo.

On Debian (Squeeze in my case), Motion is disabled by default – as many other services. Enable it, as mentioned in the notice:

Not starting motion daemon, disabled via /etc/default/motion ... (warning).

Setting the value start_motion_daemon to yes in /etc/default/motion as such:

start_motion_daemon=yes

 

The trick to disable motion detection in Motion, is to set the threshold to 0 in the config file:

threshold 0

Enabling time-lapse by setting the following in /etc/motion/motion.conf:

# Use ffmpeg to encode a timelapse movie
# Default value 0 = off - else save frame every Nth second
ffmpeg_timelapse 10

In this case, I take a pictures every ten seconds.

You should also adjust the width and height parameters, and the target_dir.

You can also get a copy of my preconfigured motion.conf by running the following set of commands

/etc/init.d/motion stop
mv /etc/motion/motion.conf /etc/motion/motion.conf.orig
wget http://retrospekt.dk/files/motion.conf -O /etc/motion/motion.conf
mkdir /home/motion
chown motion:motion /home/motion
chown root:motion /etc/motion/motion.conf
chmod g+r /etc/motion/motion.conf
/etc/init.d/motion start

An example can be seen here: http://retrospekt.dk/files/timelapse.mpg

Budget-friendly FreeNAS raid-z

When I wrote my previous post, I did not want to too much into detail about my NAS setup.  But, I still had an urge to tell about the splendid configuration.

My motivation for setting up my own freenas server, was my very positive previous experience with the software. And, by having my own configuration, I would be better able to provide both usability and technical troubleshooting.

These sort of posts are usually only of interest of potential buyers googling a specific product – and likewise software product.

But, without further adieu here is:

Yet another hardware configuration blog post

FreeNAS logo

The NAS consists of the following components

  • Jetway NC9C-550-LF motherboard
  • 2GB DDR3 1333 SODIMM I bought along with the motherboard
  • Jetway 4x SATA daughterboard
  • 4x WD20EARS harddisks
  • Lian Li 6070 Scandinavian edition chassis
  • An old usb key (2Gb .. I think)
  • An old pci ethernet adaptor

Lian li has apparently taken the chassis off their site, but Anandtech still has a nice review of the case.

The main reason I used this chassis is because I had it laying around, so to speak. The same goes for the motherboard, as it was a surplus from my previous NAS building experience.

The motivation for building the was the horrible near-datadeath experience I recently  had.So, I thought the time was ripe for a fault-tolerant storage medium.

As I had previously had a positive experience with both FreeNAS and zfs, the choice naturally fell on these. The installation is so very very easy: Download the the embedded gzipped image, put in an empty (or with non-precious content) usb key, and run the following (on a Linux box):

gunzip -c <path>/FreeNAS-amd64-embedded-xxx.img | dd of=/dev/sdx

Replacing the x’s and <path> with the relevant parameters.

Getting the RTL8111E to work

Note: This only applies to FreeNAS 7. The interface is supported in the FreeBSD 8.0 branch, and hereby FreeNAS 8.

The two onboard Realtek interfaces is not supported by the FreeBSD 7.3-RELEASE kernel.  This is also where the old pci network adapter comes in play. I used an older 3com 10/100 card, these are well supported.

However, you can get the onboard NIC’s up and running by downloading and installing the appropriate driver.

You can download the driver here: Realtek RTL8111E FreeBSD 7.3 64-bit driver

Remount /cf as read-write

# mount -o rw /cf

And place the driver in /cf/boot/kernel/

Lastly, you need to update /cf/boot/loader.conf to include the line if_rl_load=”YES” – and your’re done.

Now you can reboot, or remount /cf as read-only and use kldload to load the new driver.

You can follow a related discussion about the driver here.

The filesystem for the disks is zfs, and the raid-z pool is created manually as described in this previous post.

To finish it off: here are some photo’s of the current setup:

The 120mm fan is almost as large as the mainboard
A mini-ITX board looks kind of lonely in an ATX case 🙂

Some words about performance

The embedded Atom CPU is definitely not a speed demon in any way. SSH transfer speeds peaks at 5 MB/s but are usually around 3-4 MB/s.
Sequential FTP uploads are about 25 MB/s. CPU usage is about 70% across all “cores”

As far as i can remember, the whole system uses about 50W.

The price of the system is not really represented here – as I have used spare parts, but similar components can be bought for about 450€

All in all a good, stable and robust system.

ZFS drive replacement

This is a post about the robustness of zfs, and can serve as a mini how-to for people who wants to replace disks and do not have hot spare in the system.

Background

Last Monday, our local area was hit by a tremendous rainfall which caused our basement to be flooded. You can see the pictures of the flood  here. Sorry about the quality. The primary objective was to savage various floating hardware :-\

Wet hardware is also the reason fort this post. Upon entering the basement I remembered my fileserver that was standing on the floor and quickly (and heroically) dashed to its rescue.

Unfortunately the server had already taken in quite a lot of water and three of its four raid-z (raid5) disks were already ankle deep in water.

I did not manage to take any pictures at the time, but took some today in order to illustrate where the waterline was.

 

This is the inside of the case side. If you look carefully, you can see the traces after the water.

My crude drawing skills was put to the test in order to create this.

An approximation of the waterlevel

Needless to say, I was quite worried about the state of my data. I quickly removed the power plug and rushed the computer off to dry land (the living room) where a brave team consisting of my girlfriend and son; started drying the disk components after I had disassembled them – well, removed the circuit board at least.

After each disk had been dried, I carefully put them back together and tried to power them on – one by one.
Surprisingly, they all spun up, meaning that the motors were okay – yay!

Next step was to put them back into the fileserver and hope for the best.

And, to my relief, It booted! And the zpool came online! That was amazing! Apparently, nothing was lost. But just to be sure i ran a scrub on the pool.

This is the result:

  pool: pool1p0
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub scrub completed after 5h0m with 0 errors on Tue Aug  2 03:20:10 2011
config:

	NAME        STATE     READ WRITE CKSUM
	pool1p0     ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    ad4     ONLINE       0     0     0
	    ad6     ONLINE       0     0     0
	    ad10    ONLINE      51     0     0  1.50M repaired
	    ad12    ONLINE       0     0     0

errors: No known data errors

I consider myself a very lucky man. Only 1.5M of corruption? 3 of 4 disks partially submerged in water. Wow!

Anyway. I rushed to buy three new disks, and replaced one of them (ad10) as soon as it arrived I started replacing them, one by one.

I of course did a full rsync of the date in the storage pool to a another computer.

Replacing the disks

Upon replacing the first diske, (I chose ad10 as this was the one that was marked as bad) I got this error:

nas1:~# zpool status
state: DEGRADED
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
   see: http://www.sun.com/msg/ZFS-8000-8A
 scrub resilver in progress for 6h22m, 86.62% done, 0h59m to go
config:

	NAME                       STATE     READ WRITE CKSUM
	pool1p0                    DEGRADED     0     0    10
	  raidz1                   DEGRADED     0     0    60
	    ad4                    ONLINE       0     0     0  194M resilvered
	    ad6                    ONLINE       0     0     0  194M resilvered
	    replacing              DEGRADED     0     0     0
	      6658299902220606505  REMOVED      0     0     0  was /dev/ad10/old
	      ad10                 ONLINE       0     0     0  353G resilvered
	    ad12                   ONLINE       0     0     0  161M resilvered

errors: Permanent errors have been detected in the following files:

        <metadata>:<0x32>

The zfs administrators guide explains that the corruption is located in the meta-object set (MOS), but does not give any hint on how to remove or replace the set. Admitted, I have not looked thoroughly into what the MOS actually is.

I put the original (faulted) ad10 disk back in, and the error went away (after a reboot).

Then I decided to try again. This time with ad4. Physical replacing the disk on the sata channel revealed this:

nas1:~# zpool status
  pool: pool1p0
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
	invalid.  Sufficient replicas exist for the pool to continue
	functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-4J
 scrub: none requested
config:

	NAME                     STATE     READ WRITE CKSUM
	pool1p0                  DEGRADED     0     0     0
	  raidz1                 DEGRADED     0     0     0
	    2439714831674233987  UNAVAIL      0    32     0  was /dev/ad4
	    ad6                  ONLINE       0     0     0
	    ad10                 ONLINE       0     0     0
	    ad12                 ONLINE       0     0     0

errors: No known data errors

Okay, then the replacement.

nas1:~# zpool replace pool1p0 2439714831674233987 /dev/ad4

… And the resilvering started. The eta eventually settled at ~5:00 but took about 7,5 hours – which was probably caused by the relative slow Atom processor, being the bottleneck.

nas1:~# zpool status
  pool: pool1p0
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
 scrub: resilver in progress for 0h0m, 0.00% done, 708h0m to go
config:

	NAME                       STATE     READ WRITE CKSUM
	pool1p0                    DEGRADED     0     0     0
	  raidz1                   DEGRADED     0     0     0
	    replacing              DEGRADED     0     0     0
	      2439714831674233987  REMOVED      0     0     0  was /dev/ad4/old
	      ad4                  ONLINE       0     0     0  2.30M resilvered
	    ad6                    ONLINE       0     0     0  1.53M resilvered
	    ad10                   ONLINE       0     0     0  1.52M resilvered
	    ad12                   ONLINE       0     0     0  1.38M resilvered

errors: No known data errors

The resilvering revealed a total of 4 corrupted files, which I could replace from backup.

However, this lead me to the next challenge:

Clearing errors, and merging replacement disks

I could get rid of the errors, effectively leaving the zpool in a permanent degraded state. Every document I could dust up lead me to conclusion that I should remove the files – which I did, and then run zfs clean on the pool to clear the errors.

The solution was to reboot after I had removed the files, and let it resilver again. This worked and let me to believe that I could have simply done a clean and thena scrub to verify the consistency of the data.

After this, I could repeat the somewhat lengthy process for the next disk.

Summery

In total I have had ~10 minutes downtime, caused by replacing the disks.
Plus of course a couple of hours downtime while the server dried. This is, in my opinion, very impressive. Another vote for zfs, or +1 on google+ 🙂

I have actually found this zfs recovery exercise very enlightening. It is something you usually do not get to do under such “relaxed” circumstances as I had been privileged with.

Update: The new disks does not support temperature polling, apparently Western Digital has removed the feature.

screenshot
Only the remaining "old" disk now support temperature monitoring

MBUS and Ada

I recently got my hand on a OM13006 Power Plug meter development kit from NXP Semiconductors. The entire process of actually getting the kit was a quite cumbersome one, and tale left for another time. Though I would like to thank the people at Future Electronics for their invaluable help.

So, now I have a kit which i have spent €200+ on, and no idea how to program it or use it. The smart metering website at NXP has an SDK zip file which you can download. It contains some diagrams and a lot of C/C# source code respectively. More importantly, it contains the (windows) software also depicted on their website.

The software is pretty functional, but lacks an automated log function (besides raw bus data), so you can grab usages and use them in a software model.

A challenge, then!

As the kit consisted of the a meter to plug between your outlet and device, and a USB dongle, the first step was to figure out how to communicate with the device.

After some spent digging through the supplied source code, I realized that the device was interfacing as a serial device. A quick check with minicom confirmed this.

By then, it was just a matter of parsing the line (ASCII encoded) based protocol.

The sources can be found here, or by going to the projects page.

To build the test application, you need to use gnatmake which is part of GNAT. It is part of any respectable Linux distributions repository. When it is installed, just run

gnatmake -P mbus_logger

To build the sample application

The curse of being a geek

I am currently in Spain working on a EU funded project called Developing Open Source Systems Expertise in Europe – or just DOSSEE (the certificate is invalid).

The first day after I arrived, I saw a bus with the following written on it:
20110318_002.jpg

I think I will leave the punch line to be deducted by the real Linux nerds.

I remembered that I was very please to see a graffiti tag back in Denmark that apparently promoted the use of the Gnu Compiler Collection 😉

A lesson in recovery techniques

I recently got this message from fsck.jfs:

Unrecoverable error writing M to /dev/sdb3. CANNOT CONTINUE.

Okay, so this is an error that can be ignored – right? I can just force mount the partition and extract the data with superblock marked as dirty … right?!

krc@X61s % mount -o ro -f /dev/sdb3 /mnt/rec_mount
krc@X61s % ls /mnt/rec_mount
krc@X61s %

Damn it! This was an 1,4 TB parition with 900GB of data including home videos and .mkv rips of my dvd’s. Most of data could be restored, but a lot work would be lost.

I am running JFS on all my storage drives, as I have found this a good all-round file system especially in smaller devices with limited resources. Unfortunately this a kind of niche file system that does not have a broad variety of recovery tools.
I found jfsrec as the only (non commercial) tool. Unfortunately this tool was unable to read from the partition directly and stopped with an early EOF marker error.

Jfsrec pointed me in the direction of the dd_rhelp tool. This tool turned out to be a life saver. There was just one thing. I needed a disk big enough to hold a complete dump of the partition.

A few days later, armed with a new disk, I was able to continue. I used this guide at debianadmin.com to get started. The command could not be simpler to use:

krc@X61s % dd_rhelp /dev/sdb3 /mnt/rec_target/bad_disk.img

And it started copying data! Yay!
After some time, it settled on a transfer rate of 2500 … KBps! … Wow… This is rather slow…
Quick calculation: (((1400000000)/2500)/3600)/24 = 6.48 days.

One week later:

krc@X61s % ssh atom1
ssh: connect to host atom1 port 22: No route to host

Hmm… I had done this periodically over the last week

krc@X61s % ping atom1
PING atom1 (172.16.0.122) 56(84) bytes of data.
From atom1 (172.16.0.122) icmp_seq=1 Destination Host Unreachable
From atom1 (172.16.0.122) icmp_seq=2 Destination Host Unreachable
From atom1 (172.16.0.122) icmp_seq=3 Destination Host Unreachable
From atom1 (172.16.0.122) icmp_seq=4 Destination Host Unreachable
^C
--- atom1 ping statistics ---
6 packets transmitted, 0 received, +4 errors, 100% packet loss, time 5059ms

Hmm… Thats odd. I didn’t remember putting a ; halt -p after the dd_rhelp command.

A few pings and some reflections later I acutally got up and checked the room where the recovery setup is located.

This was what I found:

20110218-154606_redone.jpg

To quote Freddie Frinton;

I’ll kill that cat!

20110218-154628_redone.jpg
Notice the dangling sata power cables in the top of the photo… I have always found Linux a stable operating system, but a system disk physically disappearing is valid excuse for a crash!

Fortunately, dd_rhelp got to finish the disk dump – which was very lucky because after the fall, the damaged disk is now officially dead. It no longer spins up, and is not recognised by bios.

I tried running a fsck.jfs directly on the disk image, and it managed to fix the errors in the partition. Now i could mount the disk image like so:

krc@X61s % sudo mount -o loop /mnt/rec_target/bad_disk.img /mnt/rec_target

And copy the files from /mnt/rec_target.

Whew!