Parallel re-fetcher (using wget).

This script is a parallel re-fetcher (i.e. it can restart downloads removing the old files and updating them with a new ones). the comment says it all.

#! /bin/bash
rm -Rf /tmp/fetching_links
# Takes in a file, and a number which defines how many links to be fetched in parallel. This'll also utilize the wget feature to reuse an existing TCP connection.
# e.g. ./fetch_parallel <path to list> <no. of threads (n)>
# The file taken in will be spitted into n parts and each part will be stored in /tmp/fetching_links (with text file named 0, 1, 2, 3 etc... depending on value of n).
# Each file will be fed to wget for fetching.
# third argument is optional. Third, if present will will refetch the downloads after n seconds (as specified by the third argument) and overwrite the existing files. If the argument is 'c', it'll refetch without waiting.
# Files will be stored in ./downloads
# variable to store all links
links="$(cat "$1")"
# Generate list of links (depending on no. of parallel fetches).
# variable each will contain the no. of lines to exist in each instance of wget.
declare -i each
each=$(($(echo "$links" | wc --lines)/$2))
# Helper index variable $num
declare -i num
num=1
#assign link list for each wget instance to process and split the file list in arrays of $links.
while [[ $(($num - 1)) != $2 ]]
do
    links[$num]=$(echo "$links" | head -${each})
    #trim list of files to allow second run.
    #remaining lines (not divisible with each) will be in links[0]
    links="$(echo "$links" | tail -n +$(($each+1)))"
    num=num+1
done
#num will be used later as the no. of valid indexes in links. Thus reduce it's value to the right one.
num=num-1
mkdir /tmp/fetching_links
# index variable i will be used to echo lines in array links to /tmp/fetching_links, each will be read separately by wget.
declare -i i
i=0
# if links[0] = "" or \n, it means there were no left overs after the split of the link list. I need to make a mark of this.
    if [[ ${links[0]} == "" || ${links[0]} == "
" ]]
    then
# Set variable f as "empty" if links[0] was "". For future use.
        f="empty"
    fi
# num variable contains the no. of indexes available. For future use, it's copied over to another variable max.
declare -i max
max=$num
while [[ $num != -1 ]]
do
# File names will be same as the index variable in variable links
    echo "${links[$num]}" > /tmp/fetching_links/$num
    num=num-1
    if [[ $f == "empty" ]]
    then
        if [[ $num == 0 ]]
        then
            break
        fi
    fi
    done
# function which needs to be called again and again for repetitive fetching.
unset links
# Value of max will again be used, thus storing in num again.
declare -i num
num=$max
call_wget(){
# Resetting value of max in case of a recall.
    max=$num
# calling of wget starts here. Using previous variable max and 'f'
    while [[ $max  != -1 ]]
    do
        wget -N --tries=3 --timeout=5 -U "Mozilla/5.0 (X11; U; Gentoo Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/11.0.696.71 Safari/534.16" -i /tmp/fetching_links/$max &> /dev/null &
        #debug
        max=max-1
# file[0] will be missing if f==empty
        if [[ $f == "empty" ]]
        then
            if [[ $max == 0 ]]
            then
                break
            fi
        fi
    done
}
t=$3
# If $3="", wget will just run once.
if [[ $t > 0 || $t == "c" ]]
then
    while [[ "1" != "2" ]] # An infinite loop
    do
        call_wget
        echo "Fresh call"
    # See if any wget's running in bg using jobs.
        status="running"
        while [[ "$status" == "running" ]]
        do
            jobs | grep Running &> /dev/null
            if [[ $? == 0 ]]
            then
                status="running"
                sleep 2
                echo "running"
            else
                status=""
            fi
        done
        # The sleep timer, or no wait, depending on value of $3.
        if [[ $3 != "c" ]]
        then
            echo "waiting $3 seconds"
            sleep $t
        fi
    done
else
    call_wget
    status="running"
        while [[ "$status" == "running" ]]
        do
            jobs | grep Running &> /dev/null
            if [[ $? == 0 ]]
            then
                status="running"
                sleep 2
                echo "running in background"
            else
                status=""
                echo "All done."
            fi
        done
fi
rm -Rf /tmp/fetching_links

Making openoffice/libreoffice save faster.

With large documents, openoffice/libreoffice slows down miserably saving documents; most of the problem is HDD I/O.

That can be solved in most Non-Microsoft systems by saving to ram instead of the HDD.

To start off, make a folder named X... this'll be the place where you'll copy over the document you'll be editing (COPY, NOT CUT).

mount a tmpfs filesystem (portion of RAM) to this folder (as root) -

mount -t tmpfs none 'full path of X enclosed in single quotes'

Then copy over the document to folder X, notice if you turn off the PC, you'll loose the document, thus it's critical to make backups (i'll tell you an automated method).

Notice, when saving changes to the document, it'll be faster... there's a another hack to make things yet faster, but before that let's make the automated backup. Run the following (not as root) in a terminal -

while [[ $i != '78' ]]; do sleep 60; cp 'full path to document in folder X in single quotes' 'full path to original document in single quotes'; done

This'll make backups withing a single minute of the document.
As of further making things faster, first make a dir Y in /tmp -
mkdir /tmp/Y

Then mount a tmpfs file system on it (as root) -

mount -t tmpfs none /tmp/Y

Goto openoffice, in tools > options > openoffice/libreoffice > paths

In here set the temp directory to /tmp/Y.

You're done, saves should be a lot faster now (restart all instances of openoffice). However notice, under VERY large documents and limited amount of ram (usually less than 1 GB with KDE), this may fill up your RAM very quickly.

Micromax 352G on Linux.

This's just another regular modem with a storage device as the default mod. First you need to install usb-modeswitch, then plug in the modem (or replug the modem), then run (as root) -

wvdialconf

If it says modem not found, it means the USB device has not switched it's state, thus, do so as follows -

usb_modeswitch -v 1c9e -p f000


Then again try wvdialconf, it's ought to work.


You'll now have a file name /etc/wvdial.conf, edit this as root, enter your username, password (usually anything will work), then the number to be dialed (usually *99#). Uncomment these lines also.


Next run wvdial command, and things should work fine. Notice the color coding on the modem also to see if it even catches the network.