Mysql Backup all databases

0 0 * * * for i in $(mysql -N -e 'show databases' -uroot -p'secret' ); do mysqldump -uroot -p'secret' $i | gzip -9c > /root/MYSQL_DB_BACKUP/$i\_$(date +\%Y-\%m-\%d_\%Hh\%M).sql.gz; done

This cronjob will create backups of all the databases on the server, compress and put them in /root/MYSQL_DB_BACKUP/name_of_database_datestamp.sql.gz

How to install JavaScript runtime in Rails

*** Exception PhusionPassenger::UnknownError in PhusionPassenger::Rack::ApplicationSpawner (Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes. (ExecJS::RuntimeUnavailable))

Solution: Install Node.js
install on RHEL:

wget http://nodejs.tchol.org/repocfg/el/nodejs-stable-release.noarch.rpm
yum localinstall --nogpgcheck nodejs-stable-release.noarch.rpm
yum install nodejs-compat-symlinks npm

Its also possible to add gem 'therubyracer' to the projects Gemfile and bundle install it.

Tomcat: check if process is running

Put this in a cronjob and check every 5 min:

#!/bin/sh

if [ `ps x | grep "java.*" | grep -v grep | grep -vi screen | wc -l` = "0" ]
then
cd /root/my_app
./start > /dev/null &
fi

Run Linux in RAM

vmlinuz initrd=initrd.img ro quiet
root=CDLABEL=Fedora7-Live-i386 rootfstype=iso9660
liveimg live_ram

RAM <- /lib, /usr/lib (0.5G)
You can also make a folder in /usr called preload (or whatever) and mount that in RAM, link things back to /usr/lib and the symlinks that rely on (..) to point to /usr will still work. Lets you pick and choose whats loaded and could be automated with back script!

Install VMware tools on Linux

1. Install your Red Hat or CentOS virtual machine (VM) using whichever mechanism works best for your environment. I use a Preboot eXecution Environment, or PXE, bootserver for such things.
2. Configure the Red Hat or CentOS VM using your standard procedures.

Then, perform the following additional configuration steps:

1. rpm --import http://packages.vmware.com/tools/VMWARE-PACKAGING-GPG-KEY.pub
2. Use the following script to create a repository file useable by the yum command.

   Note that you need to substitute with 3.5u2, 3.5u3, or 3.5u4. You also need to substitute with either i686 or x86_64 depending on whether your VM is a 32-or 64-bit VM, respectively.

cat > > /etc/yum.repos.d/vmware-tools.repo < < EOF [vmware-tools]
name=VMware Tools for Red Hat Enterprise Linux $releasever - $basearch
baseurl=http://packages.vmware.com/tools/esx//rhel5/

enabled=1
gpgcheck=1
gpgkey= http://packages.vmware.com/tools/VMWARE-PACKAGING-GPG-KEY.pub

EOF

For example, within a 32-bit VM where the ESX version is v3.5 U3 I used the following:

cat > > /etc/yum.repos.d/vmware-tools.repo < < EOF
[vmware-tools]
name=VMware Tools for Red Hat Enterprise Linux $releasever - $basearch
baseurl=http://packages.vmware.com/tools/esx/3.5u3/rhel5/i686
enabled=1
gpgcheck=1
gpgkey= http://packages.vmware.com/tools/VMWARE-PACKAGING-GPG-KEY.pub
EOF

To remove the tools, make the following commands:

1. yum remove vmware-tools-* open-vm-tools-*
2. yum -y install xorg-x11-drv-vmware xorg-x11-drv-vmmouse
3. yum -y install vmware-tools open-vm-tools-xorg-drv-display open-vm-tools-xorg-drv-mouse

Attempting this method with RHEL 5 Update 3 versions of VMware Tools, however, will result in failure to install VMware Tools as the GPG key is handled incorrectly. The only solution is to disable GPG checking. To do so, change the line within /etc/yum.repos.d/vmware-tools.repo from

gpgkey=1

to

gpgkey=0

Once completed, you can safely update and install VMware Tools without GPG errors.

Upgrading VMware Tools
It's easy to upgrade VMware Tools when you update your Red Hat distribution by using the following command:

yum -y update

If, however, your kernel is too new, you can't update from the VMware Tools OS-specific package repository and will have to build drivers locally. To do so, execute the following commands. Of importance: will be 3.5u2, 3.5u3 or 3.5u4. Also, you'll most likely want to run these commands on a development machine, as you need to install compilers and other build tools. The result will be a redistributable VMware Tools kernel module image that you can install in other VMs.

1. yum list < /tmp/t

2. vname=`grep vmware-tools /tmp/t|awk '{print $2}'`
3. version=`basename $vname .el`
4. wget http://packages.vmware.com/tools/esx//rhel5/SRPMS/open-vm-tools-kmod-$version.src.rpm
5. wget http://packages.vmware.com/tools/esx//rhel5/SRPMS/vmware-tools-kmod-$version.src.rpm
6. yum -y install yum-utils rpm-build

   You may need to double-check the version of kernel-devel that is installed by the next command so that it matches your running kernel. In some cases, it will not be the latest version of the kernel.

7. yum-builddep -y open-vm-tools-kmod-$version.src.rpm vmware-tools-kmod-$version.src.rpm
8. rpmbuild --rebuild open-vm-tools-kmod-$version.src.rpm
9. rpmbuild --rebuild vmware-tools-kmod-$version.src.rpm
10. rpm -ivh /usr/src/redhat/RPMS/i386/open-vm-tools-kmod -`uname -r`-$version.i386.rpm /usr/src/redhat/RPMS/i386/vmware-tools-kmod-`uname -r`-$version.i386.rpm

-Edward L. Haletky

sha2 encryption in ruby

require 'digest/sha2'; hash = Digest::SHA2.new; hash << "text to encrypt"; hash.to_s

<< basically adds the parameter to the object. 'f' << 'oo' # -> 'foo' and [1] << 2 # -> [1,2]

but it might be a left shift also ;) 2 << 3 # -> 16

it shifts the bit representation of a number. 0b001 << 1 # -> 0b010

with 0b001 == 1 and 0b010 == 2

Web Developer Tools

General debugging tools:
Firebug - Popular Firefox extension for web developers.
Web Developer - Another Firefox extension containing good set of tools.
JS Deminifier - Good for expending minified JS.
Firecookie - Cookie manager for Firebug.
JS beautifier - Reformats JavaScript source code to make it more readable.
JS beautifier2 - Reformats JavaScript source code.

Web Application Testing Frameworks:
Watir - Ruby based framework for browser automation. examples
Webdriver - Browser automation framework.

Create Rails project with any version

gem list
rails (3.0.5,2.3.11)

rails -v
Rails 3.0.5

#Create a rails project with version 2.3.11
rails _2.3.11_ MyProject

MySql dump with Cronjob

0 0 * * 0 /usr/bin/mysqldump -uroot -p'PASS' DBName > /path/DB_backup/$(date +\%Y-\%m-\%d_\%Hh\%M).sql

example output file:
2011-02-06_00h00.sql

Sym Link to multiple CIFS mounts

A simple script that creates sym links to multiple cifs mounts and dumps them into a master folder.


#!/bin/bash

DIR="/media/MDrive/"
DIR2="/media/ZDrive/"
DIR3="$DIR $DIR2"
DIR4="masterDownloads"
#echo $DIR3

IFS=$'\n'

#for i in $(find $DIR -type f -printf "%f\n")
for i in $(find $DIR -type f)
do
echo "Processing... $i"
mkdir -p ./$DIR4/$(dirname $i)
ln -s $i ./$DIR4$i
clear
done



Quick Image resize


for a in $(find . -iname \*.jpg); do convert $a -resize 640x480 small/$a; done


data integrity check from CSV

To test the data integrity in your CSV file and locate any lines that contain the delimiter in the data field so that it can be removed manually.

Regular Expression:
/(?:.*?\|){4,}.*/

Will select lines that contain 4 or more pipes per line.
(?: ) is a non-capturing group. Similar to ( ), except that it doesn't store the result for later reference.

A more greedy approach is to write something like this:
/^(?:[^|]*\|){4}[^|]*$/

This will match lines that contain only four pipes

Test it:
http://rubular.com/r/f7Vd9O1c4k

Or using ruby one-liners:

#print only lines that match a regular expression (emulates 'grep')
$ ruby -pe 'next unless $_ =~ /regexp/' < file.txt

#print only lines that DO NOT match a regular expression (emulates 'grep')
$ ruby -pe 'next if $_ =~ /regexp/' < file.txt

How to setup local Yum repository in Fedora 8

The advantage of running a local repository is to save bandwidth. With this method you can distribute RPMs across your network from a central server and only worry about keeping your server in sync with the other Fedora mirrors.

1) Install Apache web server: yum install httpd and start it: /etc/init.d/httpd start
2) Install a tool to index your repo: yum install createrepo
3) Now is the time to download the Fedora repository. If you have the install DVD you can copy the contents from there, however it only contains a limited set of packages. If you want everything then download it from a good mirror server.

In this case I used rsync but you can also use http or ftp protocol and download using wget command.

rsync -avrt rsync://rsync.muug.mb.ca/fedora-enchilada/linux/releases/8/Everything/i386/os/Packages/ /var/www/html/yum/base/8/i386

The RPMs will be located in your apache web root folder /var/www/html/yum/base/8/i386

4) Index the repo: createrepo /var/www/html/yum/base/8/i386
5) Now we need to tell yum to use the local repo. Edit the yum.conf file: nano /etc/yum.conf and append this information:

[base-local]
name=Fedora 8 - i386
failovermethod=priority
baseurl=http:///yum/base/$releasever/$basearch
enabled=1
gpgcheck=0

6) To confirm that we have everything setup run these commands:

yum repolist
output:

repo id repo name status
base-local Fedora 8 - i386 enabled
fedora Fedora 8 - i386 enabled
livna Livna for Fedora Core 8 - i386 - Base enabled
updates Fedora 8 - i386 - Updates enabled


base-local is enabled and we can now start using it.

Lists all imported packages on the local repo.
yum list all --disablerepo=fedora,updates,livna

Installs liferea from the local repo. Note that if you don't disable the internet repo and yum finds a later version of liferea online it will try to download that instead.

yum install liferea --disablerepo=fedora,updates,livna

Beware of GParted

I was in a hurry to create a new partition with Gparted v0.3.3.2 based
on libparted v1.7.1 which appears to be the latest in the Gutsy
repository as of now. There appears to be a strange bug that will do
nasty things to your partition table after you apply your changes and
close the program before it had a chance to repopulate the list of
partitions. I was left with an empty drive, and to be sure I ran "fdisk
/dev/sda" from a live CD which seems to prove that.
Doing a quick Google search I discovered TestDisk and attempted to
recover my lost data. The tool did an excellent job and recovered
everything, it also sets up the MBR from the first root partition, this
was not my original setting as I had it setup from another Linux root
partition.

Ontario Linux Fest

Arrived at 8:15, registered and headed to the first presentation called "Howto write a linux kernel module". Never really fiddled with kernel level coding before but I was surprised to learn that it was much easier then I previously thought. The presenter Robert Day from crashcourse was good at walking though the code and showing us simple "hello world" like examples of kernel modules and how to load them.

"Linux forecast through 2020" was another interesting presentation by Dr. John Nash who predicted among other things the fall of Microsoft.

I was pleased to hear Louis Suarez-Potts from OOo mentioning Seneca College. He noted the success of Mozilla work that we have done and how similar student contribution can help OOo in its development.

The morning key note was held by Theodore Ts'o. Here he is finishing off with a userfriendly.



The other presentations I attended were Gnome, Eclipse IDE, performance tuning for High-traffic web sites.

In the exhibition hall I bumped into Cesar and Lukas who did a good job at promoting the next FSOSS.

Finally maddog closed the fest with a ending keynote on LTSP.

UbuntuZilla

Found a cool script to install the latest version of Firefox, Seamonkey and Thunderbird on Ubuntu.

http://ubuntuzilla.wiki.sourceforge.net/

MediaWiki+Ldap

Recently I started implementing a wiki for my company, specifically for the HR department. They addressed the need to have a system that would allow them to edit pages seamlessly, upload pdf's which contain sensitive information only visible to certain groups on an internal server and authenticating against MS Active Directory.

I installed the latest mediawiki release and selected Paul Gu's mediawiki skin which is rewrite of Monobook default wiki skin. Thanks Paul.

Next I installed the LDAP extension for mediawiki. It was a bit of a pain to configure the first time, but after reading their wiki a few times and trying out different settings, the debugging option will also help in determining the problem.

All the settings are applied in LocalSettings.php which makes it convenient to edit.

#Beginning of LDAP settings======================================
require_once( "includes/LdapAuthentication.php" );
$wgAuth = new LdapAuthenticationPlugin();
$wgLDAPDomainNames = array( "exampleDomain" );
$wgLDAPServerNames = array( "exampleDomain"=>"IPofADserver" );
$wgLDAPSearchStrings = array("exampleDomain"=>"exampleDomain\\USER-NAME" );
$wgLDAPEncryptionType = array( "exampleDomain"=>"false" );
#$wgLDAPUseLocal = true; //allow use of local user DB $wgMinimalPasswordLength = 1;
$wgLDAPRetrievePrefs = array( "exampleDomain"=>"true" );
#$wgLDAPUpdateLDAP = array( "exampleDomain"=>"false" ); //disables mediawiki from updating LDAP
#$wgLDAPAddLDAPUsers = array("exampleDomain"=>"false");

$wgLDAPDebug=3;

#DNs in $wgLDAPRequiredGroups must be lowercase, as search result attribute values are...
$wgLDAPBaseDNs = array( "exampleDomain"=>"dc=otn,dc=local" );
$wgLDAPSearchAttributes = array( "exampleDomain"=>"sAMAccountName" );

#Allo two groups to log in tech team and HR
$wgLDAPRequiredGroups = array(
"OTN"=>array("cn=technical_team,ou=global security groups,ou=otn,dc=otn,dc=local",
cn=hr_team,ou=global security groups,ou=otn,dc=otn,dc=local"
) );

$wgLDAPGroupUseFullDN = array( "exampleDomain"=>true );
$wgLDAPGroupObjectclass = array( "exampleDomain"=>"group" );
$wgLDAPGroupAttribute = array( "exampleDomain"=>"member" );
$wgLDAPGroupSearchNestedGroups = array( "exampleDomain"=>true );
//Pull LDAP groups a user is in, and update local wiki security group.
$wgLDAPUseLDAPGroups = array( "exampleDomain"=>"true");
$wgLDAPGroupNameAttribute = array( "exampleDomain"=>"cn" );

$wgShowExceptionDetails = true;

After that grant specific access to certain groups:

#Restrict access to non logged in users========================
#Restrict edit to logged in users
$wgGroupPermissions['*']['edit']=false;

$wgGroupPermissions['user']['read'] = false;
$wgGroupPermissions['user']['edit'] = false;
$wgGroupPermissions['user']['createpage'] = false;
$wgGroupPermissions['user']['createtalk'] = false;
$wgGroupPermissions['user']['upload'] = false;
$wgGroupPermissions['user']['userrights'] = false;

$wgGroupPermissions['technical_team']['read'] = true;
$wgGroupPermissions['technical_team']['edit'] = true;
$wgGroupPermissions['technical_team']['createpage'] = true;
$wgGroupPermissions['technical_team']['createtalk'] = true;
$wgGroupPermissions['technical_team']['upload'] = true;
$wgGroupPermissions['technical_team']['userrights'] = true;

$wgGroupPermissions['hr_team']['read'] = true;
$wgGroupPermissions['hr_team']['edit'] = true;
$wgGroupPermissions['hr_team']['createpage'] = true;
$wgGroupPermissions['hr_team']['createtalk'] = true;
$wgGroupPermissions['hr_team']['upload'] = true;

#Prevent new registrations from anonymous users(Sysops can still create accounts
$wgGroupPermissions['*']['createaccount'] = false;

#Define the pages un-authenticate users can see. This is crucial. Otherwise, there's
#no way for people to login
$wgWhitelistRead = array( "Main Page", "Special:Userlogout", "Special:Userlogin", "-", "MediaWiki:Monobook.css" );
$wgGroupPermissions['*']['read']= false;


Many companies are running their intranet on wiki technology and it gives non-technical people a short learning curve on how to create and edit pages.
For the new group of BSD students taking DPS909 you will learn a lot about wikis and its a valuable resource to have.

Chat program in C

Here is a proof of concept chat program written in C.
Beeing written in C, the code brings a higher level of understanding
to the programmer about the underlying functionality of network programming.

Download the code here

GitHub repo has been created to host this project: http://github.com/dtolj/simple-chat-client-server

This is the server code:

Server accepts parameters "chat" or "chat -p [PORTNUM]"
In case of only one parameter we will use MYPORT which is 7400

/*Server==================================================*/

   if(argc==1 || argc == 3){

     if(argc==3){

       if(!strcmp("-p",argv[1])){

         sscanf(argv[2],"%i",&port);

       }else{

         printf("Invalid parameter.\nUsage: chat [-p PORT] HOSTNAME\n");

         exit(0);

       }

     }else port=MYPORT;

     

     printf("\n*** Server program starting (enter \"quit\" to stop): \n");

     fflush(stdout);

Here we are creating a socket for the server which is defined by this function: int socket(int domain, int type, int protocol);
Next we populate the server_address instance of sockadd_in structure
Finally bind the port to the servers file descriptor.
A note about file descriptors, fd0=stdin, fd1=stdout, fd2=stderr. These are preserved and therefore the next available fd is 3 for listening socket and the first client connected to the server will receive a socket fd 4.

     /* Create and name a socket for the server */

     server_sockfd = socket(AF_INET, SOCK_STREAM, 0);

     server_address.sin_family = AF_INET;

     server_address.sin_addr.s_addr = htonl(INADDR_ANY);

     server_address.sin_port = htons(port);

     bind(server_sockfd, (struct sockaddr *)&server_address, addresslen);

int listen(int sockfd, int backlog);
We are now listening for incoming connections on that fd.
We zero out the readfds set and assign the server_sockfd to it as well as the stdin aka fd0

     /* Create a connection queue and initialize a file descriptor set */

     listen(server_sockfd, 1);

     FD_ZERO(&readfds);

     FD_SET(server_sockfd, &readfds);

     FD_SET(0, &readfds);  /* Add keyboard to file descriptor set */

     

     //struct clientinfo c;

     

     //printf("sizeof(readfds)/(testfds): %d/%d\n",sizeof(readfds),sizeof(testfds));

     //printf("Number of bits in readfds/testfds: %d/%d\n", sizeof(readfds)*8,sizeof(testfds)*8);

Assign readfds to testfds, we are now working with testfds.
select() allows for monitoring several sockets simultaniously.
The set currently contains 2 fds 0 and 3

     /*  Now wait for clients and requests */

     while (1) {

        testfds = readfds;

        select(FD_SETSIZE, &testfds, NULL, NULL, NULL);

                    

        /* If there is activity, find which descriptor it's on using FD_ISSET */

        for (fd = 0; fd < FD_SETSIZE; fd++) {

           if (FD_ISSET(fd, &testfds)) {

              

              if (fd == server_sockfd) { /* Accept a new connection request */

                 client_sockfd = accept(server_sockfd, NULL, NULL);

                 /*printf("client_sockfd: %d\n",client_sockfd);*/

                

                                

                 if (num_clients < MAX_CLIENTS) {

                    FD_SET(client_sockfd, &readfds);

                    fd_array[num_clients]=client_sockfd;

                    /*Client ID*/

                    printf("Client %d joined\n",num_clients++);

                    fflush(stdout);

                    

                    sprintf(msg,"M%2d",client_sockfd);

                    /*write 2 byte clientID */

                    send(client_sockfd,msg,strlen(msg),0);

                 }

                 else {

                    sprintf(msg, "XSorry, too many clients.  Try again later.\n");

                    write(client_sockfd, msg, strlen(msg));

                    close(client_sockfd);

                 }

              }

              else if (fd == 0)  {  /* Process keyboard activity */                 

                 fgets(kb_msg, MSG_SIZE + 1, stdin);

                 //printf("%s\n",kb_msg);

                 if (strcmp(kb_msg, "quit\n")==0) {

                    sprintf(msg, "XServer is shutting down.\n");

                    for (i = 0; i < num_clients ; i++) {

                       write(fd_array[i], msg, strlen(msg));

                       close(fd_array[i]);

                    }

                    close(server_sockfd);

                    exit(0);

                 }

                 else {

                    //printf("server - send\n");

                    sprintf(msg, "M%s", kb_msg);

                    for (i = 0; i < num_clients ; i++)

                       write(fd_array[i], msg, strlen(msg));

                 }

              }

              else if(fd) {  /*Process Client specific activity*/

                 //printf("server - read\n");

                 //read data from open socket

                 result = read(fd, msg, MSG_SIZE);

                 

                 if(result==-1) perror("read()");

                 else if(result>0){

                    /*read 2 bytes client id*/

                    sprintf(kb_msg,"M%2d",fd);

                    msg[result]='\0';

                    

                    /*concatinate the client id with the client's message*/

                    strcat(kb_msg,msg+1);                                        

                    

                    /*print to other clients*/

                    for(i=0;i < num_clients;i++){

                       if (fd_array[i] != fd)  /*dont write msg to same client*/

                          write(fd_array[i],kb_msg,strlen(kb_msg));

                    }

                    /*print to server*/

                    printf("%s",kb_msg+1);

                    

                     /*Exit Client*/

                    if(msg[0] == 'X'){

                       exitClient(fd,&readfds, fd_array,&num_clients);

                    }   

                 }                                   

              }                  

              else {  /* A client is leaving */

                 exitClient(fd,&readfds, fd_array,&num_clients);

              }//if

           }//if

        }//for

     }//while

  }//end Server code

This is client code:

Client accepts parameters "chat [hostname]" or "chat -p [portnum] [hostname]"
In case of only two parameter we will use MYPORT which is 7400

 /*Client variables=======================*/

   int sockfd;

   int result;

   char hostname[MSG_SIZE];

   struct hostent *hostinfo;

   struct sockaddr_in address;

   char alias[MSG_SIZE];

   int clientid;

/*Client==================================================*/

   if(argc==2 || argc==4){

     if(!strcmp("-p",argv[1])){

       if(argc==2){

         printf("Invalid parameters.\nUsage: chat [-p PORT] HOSTNAME\n");

         exit(0);

       }else{

         sscanf(argv[2],"%i",&port);

         strcpy(hostname,argv[3]);

       }

     }else{

       port=MYPORT;

       strcpy(hostname,argv[1]);

     }

     printf("\n*** Client program starting (enter \"quit\" to stop): \n");

     fflush(stdout);



     /* Create a socket for the client */

     sockfd = socket(AF_INET, SOCK_STREAM, 0);



     /* Name the socket, as agreed with the server */

     hostinfo = gethostbyname(hostname);  /* look for host's name */

     address.sin_addr = *(struct in_addr *)*hostinfo -> h_addr_list;

     address.sin_family = AF_INET;

     address.sin_port = htons(port);



     /* Connect the socket to the server's socket */

     if(connect(sockfd, (struct sockaddr *)&address, sizeof(address)) < 0){

       perror("connecting");

       exit(1);

     }

     

     fflush(stdout);

     

     FD_ZERO(&clientfds);

     FD_SET(sockfd,&clientfds);

     FD_SET(0,&clientfds);//stdin

     

     /*  Now wait for messages from the server */

     while (1) {

       testfds=clientfds;

       select(FD_SETSIZE,&testfds,NULL,NULL,NULL);

       

       for(fd=0;fd          if(FD_ISSET(fd,&testfds)){

             if(fd==sockfd){   /*Accept data from open socket */

                //printf("client - read\n");

                

                //read data from open socket

                result = read(sockfd, msg, MSG_SIZE);

                msg[result] = '\0';  /* Terminate string with null */

                printf("%s", msg +1);

                

                if (msg[0] == 'X') {                   

                    close(sockfd);

                    exit(0);

                }                             

             }

             else if(fd == 0){ /*process keyboard activiy*/

               // printf("client - send\n");

                

                fgets(kb_msg, MSG_SIZE+1, stdin);

                //printf("%s\n",kb_msg);

                if (strcmp(kb_msg, "quit\n")==0) {

                    sprintf(msg, "XClient is shutting down.\n");

                    write(sockfd, msg, strlen(msg));

                    close(sockfd); //close the current client

                    exit(0); //end program

                }

                else {

                   /* sprintf(kb_msg,"%s",alias);

                    msg[result]='\0';

                    strcat(kb_msg,msg+1);*/



                    sprintf(msg, "M%s", kb_msg);

                    write(sockfd, msg, strlen(msg));

                }                                                 

             }          

          }

       }      

     }

   }// end client code

Sort Digital Photos

Over the years I have collected thousands of digital photos from birthday parties, events etc. Having only a 64MB in my Sony Memory Stick it gets filled up very fast even at 600x480 resolution which is the lowest setting. Its become a habit dumping the pictures onto a new folder on my hard drive every now and then. After trying to sort the mess manually to try to organize the photos chronologically I managed to lose the one most important piece of information, the datestamp when the image was originally taken. Apparently when you do a copy in bash it does not preserve the datestamp instead it overwrites it with the one at which time the file was copied. Had I used the -a switch with copy I wouldn't be in this mess.
Fortunately there was still hope. I discovered that all jpegs included header information which contains the original datestamp as well as some other things that I wasn't interested in.

This lead me to write a simple bash script to extract the datestamp and sort the pictures chronologically in my album.

Here is the pseudo code:
*accept minimum 2 arguments the path_to_images path_to_write_location recursive_flag
*parse the date string from the jpeg
*loop thru all the images
*if write location does not exist create it
*construct a directory structure to place the images in the format Year/Month
*if format directory structure does not exists create it
*otherwise check if the file name exists in that directory so that we dont overwrite it
*if it does rename it before we copy it to the destination folder

Here is the entire code:

#!/bin/bash
#filename: chronJpg.sh
#Date: Feb 17 2007
#Author: Dejan Tolj 
#License: GPL
#
#Bash script to arrange jpeg images chronologically
#
#Usefull when original picture taken date has not been preserved by
#accidentally coping without the -a option.

which identify &>/dev/null
if [ $? != "0" ];
then
 echo -e "identify not found, please install ImageMagick. http://imagemagick.com";
 exit 0
fi

restoreDateStamp(){
#  echo "Date stamp =$1"
#  echo $1 | tr -d [:space:][:punct:] |touch -t /{} $2
 date=`echo $1|cut -d: -f 1-4|tr -d [:space:][:punct:]` #2005:05:18 13:50:66

 touch -t $date $2
#  echo "Touch $date ->$2"
}

Month[1]="Jan"
Month[2]="Feb"
Month[3]="Mar"
Month[4]="Apr"
Month[5]="May"
Month[6]="Jun"
Month[7]="Jul"
Month[8]="Aug"
Month[9]="Sep"
Month[10]="Oct"
Month[11]="Nov"
Month[12]="Dec"

#parameters: jpg_path, write_location, recursive copy(optional)
PARAMSIZE=2

if [ $# -lt "$PARAMSIZE" ];
then
 echo "Enter the path to your jpeg files and the write location"
 echo `basename $0` "[path] [write_location] -r"
else
 #get first parameter
 jpg_path=$1
 #  echo $jpg_path
 #get second parameter
 write_location=$2
 #echo $write_location
 #get third parameter if present else search in current folder only
 if [ "$3" = "-r" ]; then
   echo "$3"
   recursive=""
 else
   recursive="-maxdepth 1"
   echo "$recursive"
 fi

 #Loop thru all jpegs and find the original date
 for i in $( find $jpg_path $recursive -iname \*.jpg );
 do
   original_date=`identify -verbose $i |grep "Date Time Original"|awk '{print $4, $5}'`
   #echo "OD=$original_date"

   #Get Original year
   original_year=`echo $original_date|cut -d: -f1`
   #echo "OY=$original_year"

   #Get Original month
   original_month=`echo $original_date|cut -d: -f2`
   #optionaly change date stamp
 
   if [ ! -d "$write_location" ];
   then
     mkdir "$write_location"
   fi

   mydir="$write_location/$original_year/${Month[original_month]}"
   filename=`basename $i` #strip the name of the image


   if [ ! -d $mydir ];
   then
     #   echo "Original month =$original_month"

     mkdir -p $mydir
     cp $i $mydir
     #Optionally change the date stamp
     #restoreDateStamp "$original_date" "$mydir/$filename"
   else
     #check name
     if [ -f "$mydir/$filename" ];
     then
       echo "Renaming duplicate file"
       if [ -f "$mydir/$filename"_$count ];
       then
         count=`ls $mydir/$filename_* |tail -1|cut -d_ -f2|tr -d [:alpha:][:punct:]`
         count=$[$count+1]
       else
         count=0
       fi
       #rename
       cp $i "$mydir/$filename"_$count
     else
       cp $i $mydir
     fi
     #restoreDateStamp "$original_date" "$mydir/$filename"
   fi

   echo "$i -> $mydir";
 done
fi


exit 0

Building Firefox

As a challenge I decided to build Firefox from source. The first thing I noticed is that unlike other Open Source software programs Firefox is not built using the standard configure, make, make install methods. They have a slightly different build script which I will explain in a moment.

My build process was completed on Ubuntu Dapper Drake, Pentium 4 2Ghz and 512MB RAM.

Here is the list of needed packages that you can download from Synaptic/apt-get in Ubuntu.

• build-essential
• libIDL-dev0.8
• cvs
• g++4.0
• gcc4.0
• make8.0
• libgtk2.0-dev

Now that you have all the tools you need for building you need to check out the Mozilla source tree.

1. This code will checkout client.mk into ./mozilla from the Bon Echo branch. Go to that directory and the last command will checkout the browser project. This step may take a while depending on your internet speed.

$ cvs -d :pserver:anonymous:anonymous@cvs-mirror.mozilla.org:/cvsroot co -r
MOZILLA_1_8_BRANCH mozilla/client.mk
$ cd mozilla
$ make -f client.mk checkout MOZ_CO_PROJECT=browser

2. Before you can build you need to have a .mozconfig file in your ./mozilla directory. I have created this one which builds firefox with shared libraries, no optimization and I also enabled debugging. I also used gtk2 because I have version 0.8 of libIDL. By default the build will use gtk1. Create a new file called .mozconfig and copy the contents below. For a full list of acceptable flags you can see it here.

. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-static
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --enable-application=browser
ac_add_options --disable-static
ac_add_options --enable-shared
ac_add_options --disable-tests
ac_add_options --disable-optimize
ac_add_options --enable-debug
ac_add_options --enable-default-toolkit=gtk2
ac_add_options --disable-installer

3. Finally I fired the build command to compile firefox.

make -f client.mk build

Finally the build was complete