Wednesday, July 7, 2010

data integrity check from CSV

To test the data integrity in your CSV file and locate any lines that contain the delimiter in the data field so that it can be removed manually.

Regular Expression:
/(?:.*?\|){4,}.*/

Will select lines that contain 4 or more pipes per line.
(?: ) is a non-capturing group. Similar to ( ), except that it doesn't store the result for later reference.

A more greedy approach is to write something like this:
/^(?:[^|]*\|){4}[^|]*$/

This will match lines that contain only four pipes

Test it:
http://rubular.com/r/f7Vd9O1c4k

Or using ruby one-liners:

#print only lines that match a regular expression (emulates 'grep')
$ ruby -pe 'next unless $_ =~ /regexp/' < file.txt

#print only lines that DO NOT match a regular expression (emulates 'grep')
$ ruby -pe 'next if $_ =~ /regexp/' < file.txt

Saturday, May 24, 2008

How to setup local Yum repository in Fedora 8

The advantage of running a local repository is to save bandwidth. With this method you can distribute RPMs across your network from a central server and only worry about keeping your server in sync with the other Fedora mirrors.

1) Install Apache web server: yum install httpd and start it: /etc/init.d/httpd start
2) Install a tool to index your repo: yum install createrepo
3) Now is the time to download the Fedora repository. If you have the install DVD you can copy the contents from there, however it only contains a limited set of packages. If you want everything then download it from a good mirror server.

In this case I used rsync but you can also use http or ftp protocol and download using wget command.

rsync -avrt rsync://rsync.muug.mb.ca/fedora-enchilada/linux/releases/8/Everything/i386/os/Packages/ /var/www/html/yum/base/8/i386

The RPMs will be located in your apache web root folder /var/www/html/yum/base/8/i386

4) Index the repo: createrepo /var/www/html/yum/base/8/i386
5) Now we need to tell yum to use the local repo. Edit the yum.conf file: nano /etc/yum.conf and append this information:

[base-local]
name=Fedora 8 - i386
failovermethod=priority
baseurl=http:///yum/base/$releasever/$basearch
enabled=1
gpgcheck=0

6) To confirm that we have everything setup run these commands:

yum repolist
output:

repo id repo name status
base-local Fedora 8 - i386 enabled
fedora Fedora 8 - i386 enabled
livna Livna for Fedora Core 8 - i386 - Base enabled
updates Fedora 8 - i386 - Updates enabled


base-local is enabled and we can now start using it.

Lists all imported packages on the local repo.
yum list all --disablerepo=fedora,updates,livna

Installs liferea from the local repo. Note that if you don't disable the internet repo and yum finds a later version of liferea online it will try to download that instead.

yum install liferea --disablerepo=fedora,updates,livna

Sunday, November 4, 2007

Beware of GParted

I was in a hurry to create a new partition with Gparted v0.3.3.2 based
on libparted v1.7.1 which appears to be the latest in the Gutsy
repository as of now. There appears to be a strange bug that will do
nasty things to your partition table after you apply your changes and
close the program before it had a chance to repopulate the list of
partitions. I was left with an empty drive, and to be sure I ran "fdisk
/dev/sda" from a live CD which seems to prove that.
Doing a quick Google search I discovered TestDisk and attempted to
recover my lost data. The tool did an excellent job and recovered
everything, it also sets up the MBR from the first root partition, this
was not my original setting as I had it setup from another Linux root
partition.

Saturday, October 13, 2007

Ontario Linux Fest

Arrived at 8:15, registered and headed to the first presentation called "Howto write a linux kernel module". Never really fiddled with kernel level coding before but I was surprised to learn that it was much easier then I previously thought. The presenter Robert Day from crashcourse was good at walking though the code and showing us simple "hello world" like examples of kernel modules and how to load them.

"Linux forecast through 2020" was another interesting presentation by Dr. John Nash who predicted among other things the fall of Microsoft.

I was pleased to hear Louis Suarez-Potts from OOo mentioning Seneca College. He noted the success of Mozilla work that we have done and how similar student contribution can help OOo in its development.

The morning key note was held by Theodore Ts'o. Here he is finishing off with a userfriendly.



The other presentations I attended were Gnome, Eclipse IDE, performance tuning for High-traffic web sites.

In the exhibition hall I bumped into Cesar and Lukas who did a good job at promoting the next FSOSS.

Finally maddog closed the fest with a ending keynote on LTSP.

Monday, October 1, 2007

UbuntuZilla

Found a cool script to install the latest version of Firefox, Seamonkey and Thunderbird on Ubuntu.

http://ubuntuzilla.wiki.sourceforge.net/