Tuesday, March 29, 2016

Apache vs Nginx benchmark/How to make Apache faster than Nginx.

Unlike other benchmarks, both Apache and and Nginx have been tuned for best performance for the task they're doing (serving static content).
Test using apache's ab utility.

Results --

Direct hits --

















Rewrite hits --



















Test done

There are 2 sets of tests done --
  • Hitting URLs with rewrite rules
  • Hitting URLs with file paths directly.
For each of these, test where done from concurrency 1000 to 20000.
In the charts, legends which represent tests done by hitting URLs with file paths directly is suffixed with _direct, while for the rewrite rules it's suffixed with _rewrite.

Configurations --

Nginx --

user nginx nginx;
worker_processes 4;
events {
multi_accept on;
worker_connections 11000;
}


error_log /var/log/nginx16/nginx.log;


http {
server {
access_log off;
listen [::]:80 backlog=2 so_keepalive=60:60:0;
root /home/nginx;
server_name RHEL6;
sendfile on;
reset_timedout_connection on;
server_tokens off;
open_file_cache max=20 inactive=99999;
open_file_cache_min_uses 1;
open_file_cache_valid 99999;
open_file_cache_errors on;
log_not_found off;


rewrite ^/file1$ /0 last;
rewrite ^/file2$ /1 last;
rewrite ^/file3$ /2 last;
rewrite ^/file4$ /3 last;
rewrite "^/list([234]){0,1}/(.*)$" /$2 last;


location / {
deny all;
}
location ~ ^/[0-9]$ {
allow all;
}
location = /hello.php {
allow all;
}
location ~ ^/file[1234]$ {
allow all;
}
location ~ "^/list([234]){0,1}/[0-9]$" {
allow all;
}
}
}

Apache tuned --

ServerRoot /opt/rh/httpd24/root/usr/lib64/httpd/
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule mpm_event_module modules/mod_mpm_event.so
#LoadModule php5_module /usr/lib64/httpd/modules/libphp5-zts.so


Listen [::]:80 http
ListenBackLog 2
MaxConnectionsPerChild 0
MaxMemFree 0
ServerLimit 4
StartServers 4
# optimized for concurrency
#MaxRequestWorkers 10000
#ThreadLimit 2500
#ThreadsPerChild 2500
#MaxSpareThreads 10000
#MinSpareThreads 10000


# optimized for benchmark/HTTP header throughoutput
MaxRequestWorkers 100
ThreadLimit 25
ThreadsPerChild 25
MaxSpareThreads 100
MinSpareThreads 100


DocumentRoot /home/apache
ServerName RHEL6
User apache
Group apache
ErrorLog /var/log/httpd24/apache.log


LogLevel alert
AcceptPathInfo off
ContentDigest off
FileETag Inode Mtime
KeepAlive on
KeepAliveTimeout 60
MaxKeepAliveRequests 0
ServerTokens Full
TimeOut 5
EnableMMAP on
EnableSendfile on
ExtendedStatus off
LimitInternalRecursion 1
MaxRangeOverlaps none
MaxRangeReversals none
MergeTrailers off
Mutex pthread rewrite-map


RewriteEngine on
RewriteRule ^/file1 /0 [END,PT]
RewriteRule ^/file2 /1 [END,PT]
RewriteRule ^/file3 /2 [END,PT]
RewriteRule ^/file4 /3 [END,PT]
RewriteRule ^/list([234]){0,1}/(.*) /$2 [END,PT]


AllowOverride none
Options -FollowSymLinks
Require all denied
#SetHandler php5-script
Require all granted
Require all granted

Test commands --

echo -n list/1 list3/8 file1 list2/5 list2/0 file2 file4 list/7 list4/6 list3/3 | xargs -r -P 0 -n 1 -d ' ' -I {} /bin/bash -c 'ab -c -k -g /home/de//_$$ -s 1 -t 60 -n 9999999 -r http://[fc00::1:2]/{} &> /home/de//_stdout_$$'


echo -n {9..0} | xargs -r -P 0 -n 1 -d ' ' -I {} /bin/bash -c 'ab -c -k -g /home/de//_$$ -s 1 -t 60 -n 9999999 -r http://[fc00::1:2]/{} &> /home/de//_stdoout_$$'
The output of ab along with the report of each link served have been uploaded.

Test strategy --

Concurrency is the main thing we need to test.
We need to simulate situation when there are multiple low-bandwidth clients (all of them needs to be served concurrently to prevent some connections from being stalled). So we'll just increase the concurrent requests sent to the server using all the test machines's bandwidth.
Multiple TCP connection must be established in parallel; each of these connections will be reused to send multiple requests (keep alive will be turned on). Since the TCP connection has been established, we're not benchmarking the kernel. Speaking of which I could not get Nginx's keepalive to be turned off; that maybe the reason why Nginx was so fast in other benchmarks.
Since all webservers use sendfile() for delivering files, it's pointless to make the file large; we're not benchmarking the kernel. We're interested in how quickly the server creates HTTP headers.

Concurrency vs throughoutput (total requests/second).

Serving request serially, as opposed to concurrently is more efficient because of context switching and management overhead; we cant do anything about context switching, but the management overhead and the efficiency in constructing HTTP headers is what we want to benchmark.
But concurrency matters more than throughoutput.
If the server is independent, i.e. only it's CPU resources are used (network, disk I/o, a separate server like database are not the bottleneck like with these benchmarks), then increasing the webserver's concurrency will reduce the efficiency of the CPU cycles because of the context switching overhead. In these cases it's better to start serving another request when it finishes serving one request.
When there is a in-server server bottleneck like the network or the disk (for e.g. we'll take this e.g. for this para) and the requests are such that they take up quiet a lot of time reading the disk/network, it'll happen that the other requests timeout, or take too much time to respond. The longer the queue, more likely this'll happen. In these situations, increasing the concurrency will let the server serve multiple request in parallel sending progress to each user, abet slowly as compared to serving a single user at a time but without timing them out.
Another kind of bottleneck is towards the end user. A classic e.g. is downloading files where the client's network or the Internet is the bottle neck. If we do 1 download at a time, we wont be able to use all our hardware (disk, network etc..) to the fullest since the user's Internet connection is the bottleneck; to use it to the fullest we have to serve multiple clients. When it comes to these kind of situations, resource utilization IS about concurrency and concurrent efficiency becomes more important as the difference between the server's network speed and the client's network speed increases because that'll mean the server can serve more clients in parallel.
A similar bottleneck is when there is are multiple backend server (physical) which can handle, like, N queries in parallel. If there are X server, then the webserver must serve N*X requests in parallel to get the maximum utilization of the backend servers.

Cheating web servers --

Suppose we have a timeout of x seconds.
If the webserver is not serving the requests concurrently (to reduce context switching and management overhead and increase throughoutput), some of the requests will be within x seconds, while others will timeout.
A webserver which serves requests concurrently, will have all the requests timed out if the load exceeds a certain value.
A webserver which does not have a better concurrency is designed for benchmarks and will only perform good at benchmarks.

Apache vs Nginx in concurrency –

Nginx also appears to be serving the requests concurrently but with not as much concurrency as with Apache, but with Apache the responses were like within 21ms or lower, where as with nginx they were under 400ms; for Apache the distribution of the no. of requests served vs the interval under which they were served were not noted down for under 100ms, thus it may be cheating also.
Because of client machine limitation (it's a 10 years old machine), Apache maybe a lot faster than Nginx.

Verdict –

Apache is the clear winner.
If you switched to Nginx for the speed, your assumptions where false. Apache has a bigger toolkit, is faster and at the same time is security oriented. And in case you're wondering about the bigger CVE for Apache, it's because it has a bigger tookit and is older.
So it's all about for what purpose you tweak Apache.
Personally I don't understand the purpose of the Nginx project. If they want to optimized, they rather contribute to Apache or fork the project or create modules (something which Nginx doesn't even support) instead of creating a rival.

Saturday, February 27, 2016

Refilling HP 703 cartridge

Most HP cartage can be refilled from the top, I.e ink is put in via needles from the top of the cartage.

The ink is held by a sponge like substance which has to be poked in by the needle and refilled. Poke deep into the sponge, but not so deep so as to touch the base. A bit below the middle most depth will be good.

First, before you start to refill, seal off the printing heads (the steel part) with cello tape to avoid touching.

For black cartridge, open the top sticker, you'll see 5 holes. Refill using the middle most (and largest) hole. Refill till you see a little bit of liquid ink on one of the 5 holes. If you filled in excess, remove it using the suction tool. Sign of excess is that it starts dripping from the other end (nozzle). If you fill in excess, you'll waist ink and ruin the cartridge.

During refilling, do it slowest possible and wait at times between pushing the ink (do it in sessions). If you do it fast, it'll look as if the ink has been filled (the symptoms will appears), but in reality it's just a shallow fill.

Seal off the exposed holes with tape.

Wait for 24 hours before starting to print. Put the cartridge in the horizontal position (printing heads facing the floor). This's to allow the ink seep into the printing heads. If you do it earlier, you may see faded out prints; which must not be mistaken for low ink. You just need to wait in case that happens to a newly filled refill.

Create a test page to print black. If it does not, use the suction tool to open up the blocked pores. Do minimal amount of suction so as to just bring ink to the syringe.  As that happens, the pores will get opened up again.

For color cartridge, you'll also see 5 holes, the largest hole is for Magenta, left for yellow, and right for Cyan. This is when the cartridge's nozzle (or the chip) is opposite of you or the Magenta hole is on the top.

For 703 cartridge, the scheme is a little different. First, all the holes are of the same size. The top most (TH) is yellow (with the chip NOT facing against you and the middle most (yellow) hole far away from you). Left is Magenta and right is Cyan.

In the left and right, there will be a pair of holes (PH), you can use any of the 2, but it's recommended to use only 1, I'll tell you why.

When you see the holes in light, you'll see them having a sponge, and on the sponge there'll be ink stains. For PH, use the hole with the ink stains.

To do the actual refill, you need to follow similar protocols to that of the black refill, with a little bit of modification.

So you did realize this's a sponge right? You need to refill till the sponge gets filled up with ink; i.e. it gets all wet, but don't fill in excess, it must get JUST wet; the wetness must be such that the sponge starts looking almost completely black and wet. As that happens it may happen that the other hole in the PH starts getting wet too, if it does stop refilling.

For the tests, use Cyan, magenta and yellow colors separately.

Tuesday, December 15, 2015

Garbage test database for testing.

I've written a multithreaded C program (fast) to fill a SQL database with garbage text.

This was with an intention of creating a database which is heavy for the server, so you can test performance.

First, create the table --

create table complex (f1 varchar (100), f2 varchar (200), f3 varchar (300), f4 varchar (400));

Then compile this program (use gcc -lgomp -fopenmp random.c) --

#include
#include
#include
// compile using gcc -lgomp -fopenmp random_no_genrator.c
void rand_str(char *dest, size_t length) {
    char charset[] = "0123456789"
                     "abcdefghijklmnopqrstuvwxyz"
                     "ABCDEFGHIJKLMNOPQRSTUVWXYZ";

    while (length-- > 0) {
        size_t index = (double) rand() / RAND_MAX * (sizeof charset - 1);
        *dest++ = charset[index];
    }
    *dest = '\0';
}

int main () {
    #pragma omp parallel
    {
        char* random100;
        random100 = malloc (sizeof (char) * 100);
        char* random200;
        random200 = malloc (sizeof (char) * 200);
        char* random300;
        random300 = malloc (sizeof (char) * 300);
        char* random400;
        random400 = malloc (sizeof (char) * 400);
        printf ("set autocommit=0;\n");
        for (;;) {
            rand_str (random100, 100);
            rand_str (random200, 200);
            rand_str (random300, 300);
            rand_str (random400, 400);
//             change the format if needed
            printf ("insert into complex values ('%s', '%s', '%s', '%s');\n",random100, random200, random300, random400);
        }
    }
}


And write it's output to a file. This's going to run forever so you've to interrupt it to stop it. Go to the end of this file and remove any partial queries that were generated cause of the interrupt. This part is optional.

Append 'commit;' to the end of the file, otherwise you wont see any changes.

The import is pretty fast too.

Thursday, August 13, 2015

Migrating OS across hardware (including VM to bare metal and opposite).

In the OS which is to be migrated/replicated if you've not tampered with the udev configuration or have not done something which forces some modules to be loaded in the kernel which causes problems with the new hardware, you can migrate with ease.

First you need to take backup of the OS which you need to migrate. This is best done offline, however it is possible if the system is up and files/directories etc... belonging to the OS are NOT being upgraded. Reason being the files which are read to start the OS successfully will generally not be updated while the OS is operational; dynamically modified files are re-read anyway because changes are common there (like in /var).

When backing up a live system, you have to exclude  psudo (commonly /dev, /sys/, /proc and maybe run /run too (tmpfs)) filesystems. WARNING -- the /dev directory has device nodes which are in the rootFS and are used when devfs/udev is not mounted on early boot. You NEED these to boot the system especially if it does not have an initramfs (commonly Gentoo setups).  One tip is to mount --bind the RootFS; the mount is not recursive (not rbind) so you get the core rootFS exposed (which includes the contents of the /dev directory). Backup this mount --bind ed FS.

For an offline backup, I recommend systemrescuecd.


Backup can be done using squashfs or tar (use switches which preserve xattr/selinux (security.selinux namespace), permissions. With tar DO NOT use --attrs. It has bugs which I have to report (another pending bug report :( ).

Make partitions and FS to migrate the OS.

Mount the FS to the directory tree where you'll extract the backed up OS and extract it (note: tar swithces).

Fix fstab entries to reflect the changes in your new setup.

If the bootloader is not managed by another OS on the same system, then you've to update/install it too. First step is regenerating the grub.cfg (using grub2-mkconfig) or manually modify menu.lst. Then finally install the bootloader (grub2-install); you may have to repeat these steps a few time otherwise you may get a crippled boot menu.

Whenever I say 'rootfs' I mean all the FS across which OS files exist. Thus if you have a separate /usr, /boot etc... partitions

How is this supposed to work? Question is why will it not work. Udev do not store persistent information about the hardware -- it detects it every time you boot your Linux OS. So new hardware doesn't matter.

Of course if the target machine's hardware is not support or has driver issues, then the usual problems imply as with a fresh install.

This is the official installation technique of Gentoo and my preferred technique. I first install a template OS in a VM, customize it to my needs and then migrate it to the bare metal or other VMs.

Sunday, May 10, 2015

Sending mails to ip addresses (and ipv6 addresses).

This simple thing is not discussed on the Internet anywhere. If it is, the questions are still open.

Answer --

anemailaccount@[88.4.3.54]

For ipv6

testemail@[ipv6:ff::fe]

Sunday, January 18, 2015

Speeing up MTP (android) transfers.

MTP is inherently flawed, it having it's origins with the developers of Microsoft.

It provides slow linear data transfer, but when it comes to transferring individual files (the efficiency of the protocol is tested here), it's the slowest thing you've ever seen.

So if you have a number of files, just make an archive off it, transfer it to the MTP device, an extract it there.

Sunday, December 14, 2014

ZTE MF197 under Linux and review.

First -- This modem has a bug. Some wont see some (very popular) networks and the company refuses to fix or admit the problem.

This's a one of a kind modem. It has a built in NAT; so you don't have to dial it internally everything is in the web interface at 192.168.0.1.

Very good design.

For the installation, remove usb_modeswitch; otherwise this wont work.

And that's it. In generic distros, you'll get a new network interface on plugging; then operate the modem through 192.168.0.1.

I've not seen any other modem like this in the market.

Tuesday, November 11, 2014

HP K45 review.

I've to say this's the first non-mechanical keyboard which actually made it! For the first few months i.e.

Otherwise I've never encountered a keyboard which has lasted more than 2 months without a replacement.

The only complaint is that it's keys have become a little bit hard now; but that's not much of a problem (I'm a man) and despite the fact that this's not a mechanical keyboard, it doesn't have any multimedia keys.

Although HP claims this keyboard only works with Windows, I can confirm it's a standard plug and play keyboard.

Logitech MK220/M150 review

This Microsoft keyboard from Logitech is only intended for people with either 2 fingers or people who type with 2 fingers.

This 100% Microsoft, and Microsoft only compatible keyboard and mouse ensures tired arms cause --

1) It does not contain a palm rest
2) The keyboard does not have a lower edge boundary. The lowermost rows of keys make up the lower boundary of the keyboard.

Thus, if you attempt to rest your arms on the table any one of atrl, fn, alt, space, arrow keys and Logitech's favourite Windows keys will be pressed accidentally.

This's not the first time I've encountered a bad ergonomic keyboard from Logitech.

Another stupidity the company has done is that the mouse takes in 2 AA batteries making it heavy and keyboard takes in 2 AAA batteries in an attempt to make it light.

Anything more than 1M and the keyboard mouse starts giving communication issues. Maybe cause I made it work with Linux.

The DPI of the mouse is too low. So it has NO application in gaming or designing space. Buy a Microsoft keyboard/mouse for that . Logitech sells the same (it's only purpose being manufacturing HID devices to operate Windows) thing but at a higher price.

Friday, November 7, 2014

Booting GPT disks with Gigabyte's hybrid EFI.

I stated in my previous post, how horrible Gigabyte motherboards are. I continue to do the same now.

Basically Microsoft Gigabyte's hybrid EFI is nothing but a piece of Windows software which enables Windows to access 3+TB hard drive.

There's nothing EFI in the BIOS. It cannot boot GPT disks, it cannot run EFI applications and it does not provide any EFI API to the OS.

Sunday, November 2, 2014

Digital audio and sound cards interface (as provided to the OS).

PCM Or pulse code modulation is a method to digitize audio.

Just like a digital camera, the pixels are replaced with 'samples per second' and the bit depth is also in bits, which specifies the detail which each sample holds.

A sample contains the amplitude and frequency of the analog sound at a given point in time. Since analog is analog, it can have an infinite range of amplitude. But a digital sample can have only limited. Thus each sample also has a depth which's the amount of information stored in each sample (specified in bits, commonly 16, 24, 32) which then defines the range of amplitude the same can have. The depth is also called the 'sampling format' cause it defines the encoding of each sample as stored in a digital medium. There are various encoding schemes available like signed 8/16/24/32 bit, unsigned 8/16/24/32 bit, LE 8/16/24/32 bit, BE 8/16/24/32 bit etc...

To record a whole time range of an analog sound (or a complete waveform), samples are taken at regular intervals. This interval is called the sampling rate. This's analogous to the FPS in a video.

It happens that sampling rate limits the maximum frequency PCM can store (for some reason, probably cause low sampling rate has a high probability of missing out peaks). 44.1 KHZ sampling rate can store frequencies from 20 to 20,000 Hz; however higher the sampling rate, the better.

A sound card is a device which either converts analog signals to digital or opposite or both.
The sound cards do I/o with the OS in PCM format. For if you're playing an audio file which's not stored as PCM, the audio player will first have to decode it to PCM and then send it to the sound card. Same thing happens for recording. The audio sent by the sound card to the OS is almost always in PCM.

A sound card will support various sampling formats of I/O at various sampling rates.

When PCM is stored raw on a digital medium, the file usually has an extension '.wav'.
Wav can store audio in stereo. To accomplish this, the samples are placed in pairs in format such as LRLRLR; this's send to the sound card which knows this formatting. It'll send the left sample to the left speaker and right sample to the right speaker.

Now for multichannel audio. The PCM format can be encoded to store multichannel audio in similar way as stereo, where each channel will have it's individual sampling rate and L/R channel; this includes the subwoofer (LFE) channel.

For multichannel PCM to be decoded and changed to analog, the sound card needs to be told by the OS to operate in multichannel mode; in this mode the analog ports which otherwise were used for input may change their purpose for output.

Understanding ALSA device, subdevice and cards.

ALSA calls each audio controller a 'card'; each card has 'devices'; a device is something which's capable of either processing an audio stream (for playback) or sending an audio stream (capture).

ALSA gives these cards and devices numbers starting from 0.

There are also subdevices which are a part of an input or output device. There must be at least 1 subdevice. In context of output devices, a device having multiple subdevices means the hardware can do mixing, i.e. it can take multiple streams of PCM and mix them to produce a single output. This's called hardware mixing and no, multichannel is a different thing. The no. of streams a device can take depends on the no. of sub devices a device has. Using ALSA API, you can send audio to each of these subdevices simultaneously; the result will be seen in the output audio.

Subdevice in capturing means the card can take and digitize input of multiple audio streams at once.
Usually subdevice have 'modes' in which they operate. For output subdevices, it means the multichannel mode they operate in. Like 2 or 4 or 6 or 7 etc... which's activate other multichannel ports on the sound card.

For input subdevice it means from which port will the input be taken from (like line in, mic, front mic etc...)

There's a ncurses based mixer which alsa provides name alsamixer... change your volumes from there. You may use amixer which you can use to change volume levels via command.

Here you can set your base and treble settings also (it might be present depending on your sound card). Press m for mute/unmute.

In the playback section, can set the mode the soundcard is (same as subdevice mode). Options will be 2ch, 4ch, 5ch etc... Also you'll be also to select the output levels of multiple speakers in the multichannel setup. In stereo mode (2ch) the front speaker levels specify the volume of the 2 channels.

In the playback section you might also see mixers which specify input device... here it will stream the input from that device to the output device. In the same playback section and in some devices you will be able to set the mode of the recording for the input jack on your sound card (line in or mic in).

In case there's a lot of disturbance in the output audio, turn off a few reluctant channels.

Again depending on the audio device, you might be able to configure the output of the digital SPDIF interface, this interface has it's own separate PCM. S/PDIF will also contain an optional called S/PDIF Playback Source... here you gotta set from where will the audio be streamed from.

In the capture section, you'll find 'mux' which acts as a sort of amplification to the input.

You'll see a no. of 'input source'; each mark an input subdevice. You may change their value to mark an input port like mic, front mic, line in etc...

The various capture channels set the volume for the various input source.

ALSA maintains a table of possible sample rate and sampling formats the device takes. Not all possible combinations can be taken in by the card.

ALSA provides plugins via which you can access the device; a device has to be named this way –
:,,

Unlike with OSS where you can send raw PCM directly to a device which was spawned in /dev, with alsa, these devices are still there but sending streams to them is not that simple and controlling them is a different challenge. These ALSA devices are present in /dev/snd. Control* mark control of the device/subdevice and pcm* mark the device to which PCM has to be sent or PCM has to be received from. Alsa-lib is a set of libraries which make the task easy. These plugins are provided by alsa-lib.

Some sample plugins –

hw – for raw device access for both playback and recording. The stream sent to the device should be something accepted by the sound card (suitable format/sample rate).

plughw – This's transparent conversions between various wave sample format and sample rate to something which can be accepted by the sound card.

You may omit one or both of ,. ALSA will be on it's instincts to select one.
The plugin with the hardware is also called the device. Thus “:,,” will also be called a device.

Some plugins take in these devices (i.e. plugin in a plugin) e.g. –

tee:\'hw:0,0\',\'hw:0,3\',raw pro_logic_ii_the_other_side_44khz.wav

The tee plugin which'll work like tee – stream to 2 device off which one device may or may not be a file. raw pro_logic_ii_the_other_side_44khz.wav is the input file.

Similarly there's a file plugin.

Monday, August 18, 2014

Block device tester bash script.



First argument is the block device to test.

Requires the dd command and /dev/urandom to be present.


#! /bin/bash
echo -e "\033[1m\E[36;41m\E[5m\033[4m!!!!!!!!WARNING!!!!!!!!\E[0m\E[0m\033[0m\033[0m\E[36;41m\033[1m\033[4m This will erase $1 completely!! Do you want to continue?(y/n) \033[0m\033[0m\E[0m"
read decision
if [[ "$decision" != y ]]
then
 exit
fi
unset decision
# Tests block device.
# First argument -- block device.
#stores size of device in bytes.
declare -i size
declare -i progress
declare -i total_blocks
declare -i remainder_size
progress=0
size=$(blockdev --getsize64 $1)
#Data will be written in 10MB blocks 
total_blocks=$(($size/10485760))
#block count starts from 0
total_blocks=total_blocks-1
# size may not be a multiple of 10
remainder_size=$(($size%10485760))
# Create a random file for testing
dd if=/dev/urandom of=/tmp/"$0_tmp_file" bs=512 count=20480
chksum=$(sha1sum /tmp/"$0_tmp_file" | cut -d ' ' -f1)
while [[ $progress -le $total_blocks ]]
do
 dd if=/tmp/"$0_tmp_file" oflag=direct of=$1 bs=10M seek=$progress
 if [[ $(dd if=$1 iflag=direct skip=$progress bs=10M count=1 | sha1sum | cut -d ' ' -f1) = $chksum ]]
 then
  echo "$((progress*10)) to $(((progress*10)+10)) OK"
 else
  echo "Checksum failed between $((progress*10)) to $(((progress*10)+10))"
 fi
 progress=progress+1
done
# Test remaining blocks, not a multiple of  10M
if [[ $remainder_size -gt 0 ]]
then
# $progress does not needs to changed. Since progress -1 does exist, currently $progress points to a location towards the end of the device such that 10M will not be able to be accumulated from that point.
 dd if=/tmp/"$0_tmp_file" oflag=direct bs=10M seek=$progress of=$1
 if [[ $(dd if=/tmp/"$0_tmp_file" bs=$remainder_size  count=1 | sha1sum | cut -d ' ' -f1) = $(dd if=$1 iflag=direct skip=$progress bs=10M | sha1sum | cut -d ' ' -f1) ]]
 then
  echo "Last $remainder_size bytes OK"
 else
  echo "Checksum at last $remainder_size bytes failed"
 fi
fi

Friday, July 11, 2014

Airflow of computer/CPU/cabinet fans in series.

I attached 2 fans facing (the output of one fan is fed into the output of another) each other and sealed together with insulating tape. When one fans runs (call this power fan), cause of it's airflow, the blades of the other fan will also run (call this turbine fan); the other fan will also generate an EMF. I'll measure that. It's a measure of RPM, and so airflow of the turbine fan.

The power fan is runs at 24V (overvolted), it generated 1.42V in the turbine fan.

 Another identical fan's output was attached to the input of the power fan and roughly sealed. This fan was also given 24V.

The total output of the turbine fan increased to 1.97V, that's roughly 1.387323943662 times the output of a single fan. Assume it to be less than 1.5 in perfectly sealed condition.

However notice, with 2 fan do not expect the power consumption to rise to 2 times. That's cause the second power fan is running at higher RPM cause of air flow from the 1st power fan.

So unless you're really running out of area, it's not a good idea to run fans in series.

Friday, June 6, 2014

Strontium/HP/Sandisk/Kingston pendrive/thumbdrive seek times benchmark.

Here I'll benchmark the seek times of various thumbdrives. Seeks times are useful when you're syncing Bitcoin/Altcoin wallets.

I'm using the seeker tool for the purpose. All units in ms.

HP -- 0.53
Kingston -- 2.2
Sandisk cruiser blade -- 0.88
Sandisk cruser micro -- 0.73
Amkette robusto 0.93
Strontium Bold -- 0.64
Strontium TNT -- 1.06 
Transcend Jet USB 3.0 -- 0.7ms (on USB 2.0)

















As expected, Kingston is the most popular and worst brand which's expected cause it's the people's top choice, similar to Windows.

Altcoin/bitcoin/litecoin difficulty, block graph/CVS generator.

This bash script will generate CSV output for any altcoin/Bitcoin given the path of it's daemon binary. It makes RPC calls via the binary to a running altcoin/Bitcoin server to generate a CSV value to stdout. You can plot it later on in Libreoffice Calc.


#!  /bin/bash
# generates comma separated value of blockno. and  it's difficulty at custom block intervels and range 
#first_argument bin name
#second argument block interval to sample difficulty (optional)
#third argument start from block (optional)
# fourth argument last block till which to analyze
# name of binary (with full path if not in $PATH)
bin=$1
declare -i block_intervel
declare -i start_from
declare -i end_till
block_intervel=$2
start_from=$3
end_till=$4
if [[ $block_intervel = 0 ]]
then
 block_intervel=1
fi
if [[ $end_till = 0 ]]
then
 end_till=$($bin getblockcount)
fi
echo block,diff
while [[ $start_from -le $end_till ]]
do
 echo -n $start_from,
 $bin getblock $($bin getblockhash $start_from) | grep difficulty  | cut -d : -f2 | sed -e s/,// -e s/\ //
 start_from=start_from+block_intervel
done
 
Copy-paste this in a text file and give it a .sh extension. Read the comment to learn how to use it.

Tuesday, May 6, 2014

Darkcoin review.

Energy Efficiency -- Partial
Stability in prices -- Partial
51% protection -- Partial
Fast confirmation times -- No
Prevent instamine -- No
Compress protocol -- No
Transaction comments -- No
Expansion Scope -- No
Anonymous Payment -- No
No or less transaction fee -- No

Total points -- 13.33

See this for judgement criteria.

'Darksend' -- A technology used to overcome a deficiency which's not a major concern for 99% of people and can be fixed by a simple proxy since protocol analysis and IP tracking is the root of the problem -- otherwise the wallet addresses itself are completely anonymous.

If the protocols would've simply had TSL encryption support, protocol snooping whould've been killed, killing the root of the problem.

But the developer decided to implement other complicated ways.

As of KGW, DGW etc.. etc... etc... intelligently crafted (but not complicated) block re-targets and fast block times (like 1 minute or still better -- 30 seconds) like that of Quarks is enough to prevent intamine and multipool issues. These complicated algorithms increase surface of security vulnerabilities like what happened with KGW -- the same can happen with DGW.

This coins applies perfectly to Einsein's prefectly true quote --

"Everything Should Be Made as Simple as Possible, But Not Simpler"

Wednesday, February 12, 2014

Sync Bitcoin/Litecoin/Altcoin wallet fast without SSD.

This can only be achieved in Linux.

Bitcoin/Altcoin wallet syncing is hard disk i/o intensive; specifically it requires fast seek times. But hard drive are bad in exactly that. As a result, your Internet connection is not the bottleneck for syncing, but your slow HDD seek times are.

This's apparent even if you've downloaded the block chain and reindexing it.

If you don't plan to buy an SDD for syncing your wallets, it's possible to sync equally fast with a Pendrive or flash disk instead.

First format the flash disk in a flash oriented file system like Nilfs (vfat and NTFS are the slowest file systems available). Now mount it and take a note of the mount point (I call it X)

As you might have known, the altcoin/Bitcoin wallet creates a directory named .[coin name]  in your home directory where it downloads the block chain, and stores other relevant information like wallet.dat, transactions, block index, logs etc... For Bitcoin it's .bitcoin, for Litecoin it's .litecoin. It's different for every alcoin, often the coin name. In my e.g. I'll call the directory Z.

Now you got 2 options. 

First option --

Copy this directory to X. Lets call the copied over directory Y. Delete the contents of Z (which resides in the home directory).

Now you need to use the mount --bind command as such -- 

mount --bind [path to directory Y] [path to directory Z]

Restart wallet and see the speed of your sync.

Second option --

Move Z to X. Symlink Z to your home directory either using the ln command or your file manager.

And you're done! Now you can quickly sync.

I'll make an nilfs tutorial which will help you further to sync faster.

Monday, December 23, 2013

Reaper linux howto (scrypt, sha256 coins, cryptocurrencies).

This's a simple undocumented program to GPU mine scrypt, sha256 based coins. It was the 1st implementation of scrypt GPU mining.

This program will take a set of 2 configuration files – the first I call the 'master config', the second I call the 'coin config'.

All configs assume Windows endlines. Convert before use.

The config parameters are line separated and contain data in the following form –

(variable) (value)
(variable2) (value2)
etc...

First build the program. Reaper binary will be placed in the build directory.

This program takes in opencl implementation of various algorithms from files named *.cl in the same directory

As of the current time, there're 3 implementations – bitcoin (sha256), solidcoin and litecoin.

The master config will contain generic coin-unspecific option and coin config will contain coin specific options which also include the GPU parameters.

Start reaper without any arguments, and it'll assume the master config to be reaper.conf. If you've specified an argument, it'll be assume it's the name of the master configuration.

Master config options –

kernel (kernel file suffix)
mine (kernel file prefix or coin name)

These 2 specify which kernel file to use for mining with OpenCL. The *.cl files are supposed to be in the following format –

(kernel file prefix or coin name)-(kernel file suffix).cl

Thus if you want to mine litecoin, it's file name should be (kernel file suffix)-litecoin.cl

By default the (kernel file suffix) is reaper.cl. In the default directory, 3 files exists – solidcoin-reaper.cl, litecoin-reaper.cl and bitcoin-reaper.cl

To mine scrypt coin, use mine litecoin

device (opencl device no.) – Use this device. Reaper lists all devices at startup, use 1 or multiple by specifying multiple device parameters. There may be CPU devices also, and by default all devices will be used (including the CPU). So you may like to specify one of the devices.
enable_graceful_shutdown (yes/no) – Press q and enter to shutdown.

Coin config options include –

host (value)
port (value)
user (value)
pass (value)
long_polling (yes/no)
protocol (litecoin/bitcoin)
cpu_mining_threads (no.) – for CPU devices
worksize (value – good 256)
aggression (0 to 20) – How much intensely to engage the GPU in the task? Higher means more intense. To test the best value, start with the highest, and lower it 1 by 1.
threads_per_gpu (value)
lookup_gap (1 to 4) – Same as in BFG/Cgminer. 2 is for performance.
gpu_thread_concurrency (value) – How much memory to use? For 1GB card, 8192 will occupy the complete card with X running on it. It's value also depends on how much memory is avilable.
Include (file) – include these configs.
save_binaries (yes/no) – Do not delete the OpenCL binary file after compile. This'll make OpenCL reuse the file next time it starts. Ensure to delete this file for device and driver (graphics) changes.

If there values are invalid in a config file, reaper will ignore the files and say config file missing. You may have misplaced a value in the wrong config (master/coin config).

Saturday, December 7, 2013

Quark coin/cryptocurrency review

See
http://delogics.blogspot.com/2013/12/the-ultimatebest-cyrptocurrency.html for judgment criteria.


1) energy efficiency -- Almost
2) Stability in prices -- Full
3) 51% attack protection -- Partial
4) Fast confirmation times -- Full

5) Prevent intamine -- No
Factors 6 to 9 -- No
10) No or less transaction fees -- Partial

Total score -- 28 out of 69

One common question that people ask about Quarks is that almost all the coins where mined by early miners, so the distribution is bad.

There has been no premining by the devs; I will not answer why cause kids ask that question -- figure it out yourself. If cannot, you're dumb. Yes, the devs did mine, but after the coin was released to the public. They belong to the public after all.

Yes -- as compared to other cryptocurrencies which follow a Bitcoin like model, Quark does have this problem. In a Bitcoin like model, initially the profits of mining are high, but it slows down, but does not stop completely. As the coin becomes popular (over time) the mining process becomes more distributed, as a result, better distribution.

But if you believe many people, they say that it's distribution is better than Bitcoins, cause everyone has a CPU (as compared to your otherwise good-for-nothing ASIC miners) and the early miners sold them cheap (there was a phase where Quarks were VERY cheap); they back it by the volumes Quarks is traded but I'ld like to see evidence.

However, Quark's distribution model is not 'unfair'. Everyone was allowed to mine Quarks, if you could not mine it, it's your fault you were not scavenging the bitcointalk altcoin forums for new coins to mine. People who were doing it (i.e. working for it), got paid the rewards, you did not.

Your criticism is equivalent to saying a company's IPO is bad cause the shares where not distributed among the whole world's population; you're not seeing your fault -- you did not have a trading account, and/or were not aware of the IPO. You say all of this after realizing that people who bought from that IPO made huge money, now you regret and criticize the company instead. Then you start trolling that it's fraud and Quark is a pump-and-dump scam coin.

If you've an argument against these points please read this first.

QRK -- QYXvZn5aoQMup869tNV2S5CpTxj9kLawiu
BTC -- 1LEBwLeTNSh8bAejg64RkFpMdjPcnreMQr 

Thursday, December 5, 2013

The Ultimate/Best cyrptocurrency.

Structure of text --
0) feature -- Analysis/reasoning (importance score of the feature out of 10)
FAQ -
    A) Reason 1
    B) Reason 2
    C) Reason 3
    ...
    Z) ...

1) energy efficiency -- In the long run, it should not waist energy by unlimited mining. (2)
FAQ -
    A) Limited coins will not solve it.
        This's partially true. Miners will compete for TX fee.

2) Stability in prices -- Cryptocurrencies are not commodities. It's designed to be spent and a medium to receive payments from. If a cryptocurrency is not that, it's basically a pump and dump scheme.

Actually if you see for a fair cryptocurrency which is not seen as an investment, there should be a little bit inflation, cause that way people will not hold on to it like investment; instead they'll use it to buy investments like shares and gold -- which is something to be seen as an investment.

In terms of stability of prices, cryptocurrencies are better candidates cause it's something global -- no Forex is involved here. So one of the major strengths of a cryptocurrency compared to the national currency is it's stable non-volatile price. (10)
FAQ -
    A) Limited coins supply is not that good for stability, neither is long term mining of coins.
        The lost coins will increase the prices (instability in prices -- not desired; this'll happen cause Bitcoins are more rare). People will continue to loose they same way they continue to loose data.

        Coins with limited but long term supply by mining (like for 6 years or a above) will have inflation for a long time (till all coins have been mined). Will people wait for this much time for it's prices to stabilize? Or will they quit for their national currency? The prices will increase cause of it's popularity + lost coins, and decrease cause of mining activity; agreed these variables will stabilize it's prices for the decade while it's being mined, but there're just too many variables involved; it's like shares, there's a risk from what news comes about the cryptocurrency, and then there's the mining. If it's good news, mining will help stabilize prices, if it's bad then prices will crash faster than expected.
       
        I may seem like targeting Bitcoins -- but it has an advantage cause, cause it's the first cryptocurrency, people have not yet realized it's disadvantages and it may just happen that by the time it's disadvantage is realized, the inflation will be very low; but still there's the lost coin problem.
       
        The currency should try to reduce the factors it's price depends on in the technical front so as to reduce the variables on which it's price depends on.
       
        External factors will always vary the price of any cryptocurrency regardless of the technicals. External factors also includes initial demands of a new coin.
    B) Unlimited coin supply may also be bad.
        With more coins mined at a high rate, prices will decrease cause of more fresh supply.
       
    C) Proof of stake is not good.
        It encourages holding the coins as investment, avoiding it's circulation, i.e. true use as a cryptocurrency. Then worst -- PoS looks at the coin age; the longer you're holding a large amount of cryptocurrency, the higher the chance of mining a block, which further reduces chance of circulation. However if the profits proof of stake is giving you is negligible, then we may nullify this disadvantage.
       
    D) Limited inflation is ideal.
        Mining should not die; it should be done to recover from lost coins (i.e. there should be inflation). After all coins have been mined (in a relatively short period of time), the coins made per unit time should be enough to maintain the prices. Compared to the same amount of inflation provided by PoS, PoW provides higher increase in difficulty and better return rate than PoW for the reasons stated in 3) B).
       
        Even real currency have an inflation problem and it's usually quiet a lot. Actually it depends on the country. But do people have problem with that?
       
        If they did not like their current currency, they always had options. They whould've chosen fossil fuels anytime (who's value will always increase); the reason why they didn't is cause the inflation of national currency is negligible for them cause they use the national currency to spend, rather than invest and hold.
       
        So inflation rate of the crypto should be at par with inflation rate of real money.
       
        I took into considerations the following data. The average was take out for the last 20 years and excluding outliers and removing inflation about 100%
       
        http://www.coinnews.net/tools/cpi-inflation-calculator/ 3% (USD)
        http://www.measuringworth.com/ukcompare/relativevalue.php?use[]=CPI&use[]=NOMINALEARN&year_late=1994&typeamount=100&amount=100&year_source=1994&year_result=2014 4.535% (British pound)
        http://www.thisismoney.co.uk/money/bills/article-1633409/Historic-inflation-calculator-value-money-changed-1900.html 3.6805% (British pound)
        http://fxtop.com/en/inflation-calculator.php?A=100&C1=GBP&INDICE=UKCPI2005&DD1=10&MM1=06&YYYY1=1994&DD2=10&MM2=06&YYYY2=2014&btnOK=Compute+actual+value 2.65% (British pound)
       
        http://www.inflation.eu/inflation-rates/china/historic-inflation/cpi-inflation-china.aspx 4.5% (Yen)
        http://fxtop.com/en/inflation-calculator.php?A=100&C1=CNY&INDICE=ZHCPI1994&DD1=10&MM1=06&YYYY1=1994&DD2=10&MM2=06&YYYY2=2014&btnOK=Compute+actual+value 4.04% (Yen)
        http://fxtop.com/en/inflation-calculator.php?A=100&C1=CNY&INDICE=ZHCPI1994&DD1=10&MM1=06&YYYY1=1994&DD2=10&MM2=06&YYYY2=2014&btnOK=Compute+actual+value 4% (Yen)
       
        http://www.rba.gov.au/calculator/annualDecimal.html 2.7% AUD
        http://fxtop.com/en/inflation-calculator.php?A=100&C1=AUD&INDICE=AUCPI1990&DD1=10&MM1=06&YYYY1=1994&DD2=10&MM2=06&YYYY2=2014&btnOK=Compute+actual+value 3.9265
        http://meercat9.com/calc/inflation.php 2.56 AUD
       
        http://fxtop.com/en/inflation-calculator.php?A=100&C1=EUR&INDICE=EUCPI2005&DD1=11&MM1=06&YYYY1=1994&DD2=11&MM2=06&YYYY2=2014&btnOK=Compute+actual+value 2.3% (Euro)
        http://www.lawyerdb.de/Inflationrate.aspx 1.7 (euro)
        World bank 3% (Euro)
       
        http://www.inflation.eu/inflation-rates/japan/historic-inflation/cpi-inflation-japan.aspx 0.013% (Yen)
        http://www.measuringworth.com/japancompare/result.php 0% (Yen)
        http://fxtop.com/en/inflation-calculator.php?A=100&C1=JPY&INDICE=JPCPI2010&DD1=11&MM1=06&YYYY1=1994&DD2=11&MM2=06&YYYY2=2014&btnOK=Compute+actual+value 0.235% (Yen)
       
        http://www.inflation.eu/inflation-rates/brazil/historic-inflation/cpi-inflation-brazil.aspx 7.29 (Real)
        http://data.worldbank.org/indicator/NY.GDP.DEFL.KD.ZG?page=3 7.78 (real)
       
        World bank 19.75 (RUS)
        http://www.inflation.eu/inflation-rates/russia/historic-inflation/cpi-inflation-russia.aspx 17.4 (RUS)
       
        http://fxtop.com/en/inflation-calculator.php?A=100&C1=INR&INDICE=INCPI1958&DD1=11&MM1=06&YYYY1=1994&DD2=11&MM2=06&YYYY2=2014&btnOK=Compute+actual+value 15.747 (INR)
        http://www.inflation.eu/inflation-rates/india/historic-inflation/cpi-inflation-india.aspx 7.4 (INR)
        World bank 6.6
       
        World bank inflation rate world since 1992 -- 39%
        Average excluding outliers calculated 5.67
       
        So inflation between 5-6% will not have any impact on stability in prices score, above this value will give almost, if more than 7%, half, more than 9% means partial, more than 10% means no points on this front.
       
3) 51% attack protection -- There should be some means to prevent a 51% attack. (10)
FAQ --
    A) GPU mining is the answer.
        The best 51% protection is provided by a GPU coin, so only full blown computers can compute it, and one doesn't have to buy special purpose hardware. The hardware will be useless otherwise and after all coins have been mined (unless the incentive to mine is still up and maintained in some way like unlimited coins or high transaction fee), the miner will actually throw it away or sell it to someone else (who may be a cracker who's accumulating firepower for bad mining). This is not true for GPU hardware, cause they have a gaming and general computation purpose apart from mining, and probably the miners will themselves use it in gaming, so even after the mining incentive is lost, the hardware wont be sold cheap, and an attacker cannot buy them cheap.
       
        With new ASIC hardware, mining efficiency will increase; the new miners of the new hardware will start accepting transactions of lower transaction fees making the old hardware redundant cause they take more electricity. They will be sold cheap cause of this reason (lower profit margins or even losses).

        For an attacker, electricity cost is not of a concern, cause the attack time will be limited; he's going to earn money by making false transactions in blocks he mined.
       
        Yes -- the old hardware is old generation, but that doesn't mean it'll be slower.
       
        New hardware will not focus on higher hash rate, but power consumed per hash rate, thus, hash rate of new hardware may not increase exponentially making the network more vulnerable to attacks with older hardware.
       
        I'm not sure about the architecture of ASIC miners to comment on that. It's not an ARM processor which can be made faster by simply by increasing the clock speed. Modern AMD and Intel processors are not 4xPentium/athlon 64 @6GHZ. So an improvement in architecture does not necessarily mean faster speeds, it may have been done to reduce electricity consumption by half. Look at mobile processors for instance; they are more efficient than their Desktop (per Whetstone and dhrystone score) counterpart but slower.

        We just may have a similar trend for ASIC miners. Same consumption in electricity@little less hashing power.
       
        This's still worst for small alternative cryptocurrency using the same hashing algorithm. They're easy to exploit cause of low difficulty.
       
        If the algorithm is complex and unique, it'll need a LOT of R&D, the hardware will be expensive and insanely hard to make. It'll require a well funded team which only large organizations can afford and that too cost recovery will be from mining + selling of hardware not just by mining or attacks (where the price of cryptocurrency will drop resulting in a loss of the cracker cause he has to exchange a lot of coins in a small amount of time in order for it to be profitable). It'll take years to make such special purpose hardware; buying GPUs is cheaper. In contrast sha256 is way too simple and that's why making special hardware for was comparatively easy.
       
        CPU mining has a major disadvantage -- there's a lot of CPU power (not GPU power or control over special purpose hardware) in the hands of Botnets; under control of a cracker, he may have 51% hashing power for a small amount of time. But people are moving away from their virus-infested Microsoft Windows Desktop/Laptops to Android tablets and phones. So feasibility of such an attack is questionable, but still as of the current time botnets are big. Another disadvantage is that if an attacker uses CPU power of a system all the time, the owner will grow suspicious and reformat his Windows computer, but that wont be a problem for a short term to do a 51% attack; but one thing's for sure, the power of botnets to mine on a regular basis will be very limited cause most people own and work on laptops. Apart from this, if someone is wanting to do a 51% attack on a CPU only crypto, his best shot is to buy a lot of VPS for a short period of time; and then it's to be noticed that CPU is something owned by everyone, graphics card and ASICs are not. So expect the difficult to be high cause a lot of people will be mining it. People easily buy a more expensive CPU cause it's generally useful, not only for mining which's the case with ASIC and partially with Graphics chips (in case the miner is not a gamer or not using GPGPU for other purposes). So the crypto mining power will be more in the hands of the common people. It's also to be seen that the cost of VPS is mostly for the bandwidth available, not the CPU. The CPU model is not give in the most of the cases, giving a high risk factor. Also VPS cannot be sold off. So the attackers profitability relies completely by selling the coin who's price may decrease rapidly after the attack. Then check pointing makes things more difficult for him. But compared to GPU -- as of the current time, there's no one who lends GPUs. So the only way to attack a GPU coin is to buy GPUs which's the most expensive.
       
        GPU only mining will increase rating by 1. Anything below this (even CPU/GPU hybrid) will not add any points.
        B) Unlimited mining.
            Unlimited mining will sure shot prevent a 51% attack; however it's not known if mining profitability will increase or decrease in the future. This combined with CPU only mining will make an artillery against 51% attacks, however in this case (i.e in the long run), the algorithm being CPU only is highly questionable.
           
            If almost all coins have already been mined and if there's limited mining of coins forever (like with 2) D)), then the incentive to mine will neither be high or low. If the coin is CPU only (i.e. complex hashing algorithm), then there'll be less initiative to make exclusive mining hardware, cause all coins have already been mined on the CPU. This's going to prevent a 51% attack further. Then people pool mine, they'll use their free CPU cycles to earn a little bit of money driving the difficulty up, further reducing the chance of an attack. This is very likely to happen cause everyone known they got a CPU, everyone has and there's no harm in using it to earn a little bit of money for almost no electricity cost (65W). So difficulty will be up, and still there'll be no incentive to make new mining hardware. 5-6% inflation will be ideal for the purpose. Anything above this value will not improve this rating, below 5% will reduce ratings.
           
            An advantage as compared to PoS for the same inflation rate is that, cause less no. of people will be mining via PoW as compared to PoS (cause PoW requires investment in electricity cost, and hardware), the distribution of the inflation rate will be limited to less no. of people making the amount profit via the limited inflation rate more significant. This can be seen practically. For popular alts, the hashing power is more distributed cause the difficulty is low and profitability is high. Overtime, more people come up to mine the coins many of which are large professional miners and as the difficulty rises profitability falls reducing the no. of small time miners (cause profit is not significant anymore -- low profit margin), but the large professional miners persist (low profit margin, but higher hashing rate means more $$$s for a small % margin), increasing profitability for them. Of course the no. of miners also depends on the popularity of the crypto; but given the same popularity, we see this trend.
           
           
        C ) Proof of stake adds vulnerabilities.
       
            Following this article --
           
            https://bitcointalk.org/index.php?topic=604716
           
            A higher degree of PoS means more vulnerabilities. PoS coins will not get any score in 51% protection.
           
            Apart from this, you need to run a full node in order for the wallet to mine blocks based on PoS and it should be up always; how many people will do that? (especially when the interest rate is low)? When these coins will be made popular, 90% people will run light weight wallets giving power to the hands of these 10% (PoS difficulty will be low in this case cause the no. of people mining is less). Yes it is true, like with PoW the distribution of mining power will shift towards more professional miners who'll hold large amount of coins, but if the interest rate is low, they rather sell their coins to invest in real world schemes which provides more profits than this crypto, as an advantage they'll have lower risk cause the crypto market is very volatile.
           
            In case of PoW, power will may be in the hands of 5% (lower no. of people than PoS) but they'll have a lot of hashing power which if compared to the amount of coins held by these 10% is a much higher value increasing the difficulty in comparison to PoS, which results in higher comparative difficulty for PoW.  As compared to PoW, acquiring enough coins to do a 51% is easier (at these low difficulty and especially when interest rate is low.).
           
            If you try to fix 2) C), this disadvantage will be more apparent.
           
        D) Quick difficulty re targets adds protection.
       
                If a crypto has fast difficulty re-targets, it's difficulty to do a 51% attack, cause in the forked chain the difficulty will increase rapidly and will soon reach the target block times, the block time of the main chain will be the same, making a 51% attack impossible.
               
                If the main chain's difficulty was high cause of the attacker's majority hashing power, it'll drop to sustain a block interval equal to the attacker's fork chain.
               
                As of KGW, it's 'smooth' difficulty retarget means, while the attacker creates a long forked chain, the difficult will remain the same initially. But we want sudden difficulty increase to protect the network. I'm not sure about KGW, so if a coin does have KGW, it wont be taken as a plus.
               
                It's not known now DGW works either.
               
                Aggressive difficulty retarget each block will increase this rating by 1. The definition of 'aggression'' can be seen in 5) A). Apart form that the last 3 blocks or lower should be considered to get a +1 point off this rating. More no. of blocks considered means slow adoption to difficulty spikes.
               
        E) More confirmations add protection
                For a forked chain, it'll need a high hashing power for a longer amount of time to overcome the main chain, on top of that, the difficulty re-target algorithm will increase the difficulty making it yet more difficult to overcome the main chain.
               
                Since the amount of confirmation blocks depends on the receiver, this factor does not have any affect on the ratings.
               
        F) Changing algos to maintain ASIC resistance is ideal and will add 1 point.
       
4) Fast confirmation times -- like 5 minutes (50 seconds block). Less than a minute if a the coin wants to enter physical point of sales (10)
FAQ --
    A) This can kill the cryptocurrency.
        People will prefer using their cards for payment instead of waiting for hours for a confirmation. If confirmations take that much long, people will only use cyrptocurrencies for illegal payments, international payments where using other means to pay is practically impossible and transfer of funds among individuals where an hour or so is not that long. However, in these applications also, shorter confirmation times is desired. For e.g. for currency transfer among merchants, it may have to be done frequently. Even an hour is too much for that.
       
        To avoid long confirmation times, people may be willing to go through the formalities and technicals of cards.
    B) Disadvantage of short block times -- Chain will be forked way too often.
        This will turn off miners cause their confirmed mined blocks will no longer be valid. Receivers of coins who have broadcasted their transactions may have to retry attempts to broadcast.
       
        Since the difficulty will be such that a new block will be found by the network at around the same time as the block interval; a block interval large enough to broadcast a mined block throughout the whole network is desirable to avoid forking chains.
    C) Proof of stake may be problematic.
            What guarantees do you take that the person holding a large amount of coins and running a full node, keeping his wallet up and who has just generated a block has a good Internet connection so as to receive all the broadcasted transactions and include it in the block? These people are not experts, they've common stuff like firewall, anti-virus which may be up blocking the incoming transactions. This's unlike miners who usually have a good Internet connection and technical understanding.
5) Prevent intamine. (6)
    FAQ --
        A) Quick difficulty change may be the answer.
            Quick difficulty retargets ensures a power miner does not take advantage of the low difficulty coupled with laggy difficulty retargets.
           
            This accommodates for a sudden increase in network hash rate and to a provide smooth confirmation times. A sudden powerful miner may increase the difficulty to sky high; then the miner may be down skyrocketing the block times cause the difficulty is so high increasing the confirmation times to an unbearable amount.
           
            This may also serve as a protection against downtime of large networks which will lower the difficulty cause miners will be disconnected. Otherwise the confirmation times will suffer for the same reason.
           
            Aggressive difficulty retarget each block is recommended. The network should fix itself within 1 hour (and lower) and the block times should be back on track; that means a difficulty retarget should be 15 minutes; that way even if the block times reduce to 1/4th (1/4th the hashing power goes missing), the network can fix itself in an hour.
           
            As of instamine, allowing an instaminer to take away 3 blocks at a time is a reasonable. That's negligible advantage, no one would bother with that. 3 blocks means making a forked chain will be harder (since the average confirmation times of most coins is 6 blocks).
           
            So the block retargets should be 3 or lower and difficult retargets should be 15 minutes or lower for a coin to take full advantage of this rating.
            Block retarget of < 6 will give almost rating, less than <= 9 means half, <=12 means partial.
           
            KWG is characterized by smooth difficulty increase on rapid increase in network hashrate, which helps intaminers as compared to sudden difficulty retargets of a simple algo.

            Cause the difficulty increase slowly, the intaminer will take advantage by quickly generating blocks and stop mining once the difficulty is high enough.

            But when the instaminer is gone, the high difficulty is left for regular miners to take care off, and the block chain speed slows to a crawl. What makes matters worst is that the difficulty goes down slowly over time instead of suddenly going down like we have with regular algorithms. It takes the same amount of time to recover from the high difficulty set by the instaminer.

            So as compared to simple algorithms which re-target every block, KGW has a disadvantage -- the regular algo will increase the difficulty suddenly preventing an instamine, KGW does it slowly helping in the instamine.

            KGW has only one minor advantage -- it'll prevent the blockchain from halting for a longer period of time, but will not prevent the instamine.

            Digishield is worst -- it helps the instaminer further by slowly increasing the difficulty, while the difficulty goes up, the total no. of blocks mined is over the target. But Digishield brings the difficulty down aggressively which again means more blocks mined per unit time as compared to the target block interval. So basically that means in these situations the target block interval will be horribly off target and there will be over inflation.
           
            DGW is also similar. It's 'smooth' difficulty retarget helps the instaminer.
           
            So coins with KGW and Digishield will get half rating cause it does help in reducing instamine as compared to very simple slow difficulty retarget algos.
           
            Essentially, with the above algos, you're going to have similar behavior when a large no. of blocks are considered

6) Compressed protocol -- For quick setup of a full node. However this's only an advantage to miners who probably have a fast Internet connection. (2)
    FAQ
        B) how will this help?
            Notice the size of you block chain vs the amount you download while synchronizing? Thus there's a lot of scope of improvement. For starters, the older blocks can be delivered in compressed groups (of like 1000) as per the request of the full node.

7) Transaction comments -- both public (recorded in the block chain) and private (only between the sender and receiver) (4)
FAQ --
    A) What real use?
        For an online store, they may ask their customers to copy paste some one time code to prove that they have paid the bill.
       
        Then in general it helps in management in a human readable way.
    B) Size of transactions.
        This may increase by quiet a lot cause of this feature. This may be a disadvantage.

8) Expansion scope. In case small & new feature has to be added, it can be done in a backward compatible way. This requires void space allocation in the block and updating the clients to use that space. For e.g. transaction comments may be added this way to existing coins. Then one may add support for the name of the client to which transaction has been received which may unlock features exclusive to that client. (5)

9) Anonymous payment -- It should be impossible to intercept information about the owner of the address in any way. (10)
FAQ
    A) Bitcoin is not anonymous, the following vulnerabilities exist --
        A) Owner to address mapping.
            Usually a single person holds a Bitcoin receiving address.
           
            All transactions to that Bitcoin address is known, based on the value of the transactions and other evidences like third parity transaction records, receipts, Bills etc... which contain the amount of the Bitcoins sent, it can be proved that the address belongs to a particular person.
           
            Similarly along with 3rd party evidences of a rough transaction amount, and time of the transaction, the owner of an address can be traced with a certain degree of accuracy even when the sender does not corporate.
           
            Solution is to create new address for each transaction, but that inconvenient and sometimes not possible when there is no communication between the sender and the receiver.
           
            All these disadvantage can be applied to the sender also.
        B) Address balance.
            Continuing from A, after it has been proven that the address belongs to a particular person, his balance can be found out.
           
            It's to be noted that if there's 100% 3rd party evidence of the transaction, the balance the receiver holds cannot be hidden by the amount the 3rd party evidence proves. So what can be done is hiding the rest of the balance.
           
            Same thing can be said about the sender also. The balance of the sender can be determined after a transaction that has proved the owner of the address.

10) No or less transaction fees -- i.e. you may try to reduce the transaction fee by other means like PoS or inflation. Higher transaction fee may make people avoid Bitcoin transactions and instead use their Visa/Mastercard cards. Small transactions will be of no use cause the transaction fee will be too high. Transaction fee adds more complexities to the picture like adoption of the coin in poor countries and rejection of transaction based on low/no transaction fee.

No one wants to pay taxes or Bitcoin transaction fees.

Good amount of transaction fee may ensure quick confirmation (cause miners may reject based on low transaction fee); the delay will vary in multiples of the block times.

 What happens when you're using cryptocurrencies in low economy places like somewhere in Africa, South Asia & America or many of the islands. The transaction fee will simply be too much for them to pay; they'll avoid using it. There's no guarantee that people will pay transaction fee in the first place putting the miners at risk. Right now you see good transaction fee cause most of Bitcoin economy is in developed nations where they can afford to pay the fee in the first place.
 Then there's competition involved -- If the developed nations realize that their developing/under developed counterparts are not paying transaction fee, they too may stop paying. It's human behavior, it's uncertain.(10)
    FAQ
        A) Fast block times will help.
            As said before -- "the delay will vary in multiples of the block times"; faster block times may reduce this delay for the same reason.
           
        B) Inflation via mining will help.
            Certain amount of inflation per year will ensure that the transaction fee is distributed among users (cause in the long run, and without inflation the users will mine for transaction fee, but if there's inflation, they'll mine for coins removing the uncertainty of transaction fee), thus the effective transaction fee will be negligible per user cause it's distributed among the large user base and there will be no transaction fee wars.
           
            Following point 2) D), 6% or above inflation will give this full points. Below 5% will give it almost. Below 3% half, below 1% partial and less than equal 0.1% no points.
           
        C) PoS inflation will be given no weightage.
            The reason why TX fee exist is to strength the network. But PoS is full of security flaws. No matter how much inflation is provided via PoS, this point will remain 0 for a 100% PoS coin.
           
In my blog, coins will be rated based on these criteria. Each coin will the following ratings for each criteria --
No -- 0/(importance score)
Partial -- (importance score)/3
Half -- (importance score)/2
Almost -- (importance score)/1.5
Full -- (importance score)

For each coin the total will be calculated.
           
If you're making a new cryptocurrency based on this, please DO notify. Thank you!

QRK -- QMYbuBKzLTsrpDTqvoW5a6u6pN8149UMSY
BTC -- 1CrNnuoCXpggwajwasDTnvpxNpVNJPCTuc

Monday, August 26, 2013

Intel CPU only OpenCL benchmark.

I used John and ImageMagick to do the benchmark with the Intel OCL sdk installed.

The results (John) --

mscash2-opencl --
OpenCL platform 0: Intel(R) OpenCL, 1 device(s).
Using device 0:         Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz
Optimal Work Group Size:32
Kernel Execution Speed (Higher is better):0.000444
Benchmarking: M$ Cache Hash 2 (DCC2) PBKDF2-HMAC-SHA-1 [OpenCL]... DONE
Raw:    418 c/s real, 105 c/s virtual

mscash2 --
Benchmarking: M$ Cache Hash 2 (DCC2) PBKDF2-HMAC-SHA-1 [128/128 AVX intrinsics 4x]... (4xOMP) DONE
Raw:    2464 c/s real, 620 c/s virtual

wpapsk-opencl --
OpenCL platform 0: Intel(R) OpenCL, 1 device(s).
Using device 0:         Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz
Benchmarking: WPA-PSK PBKDF2-HMAC-SHA-1 [OpenCL]... (4xOMP) DONE
Raw:    790 c/s real, 199 c/s virtual

wpapsk --
Benchmarking: WPA-PSK PBKDF2-HMAC-SHA-1 [32/64]... (4xOMP) DONE
Raw:    924 c/s real, 232 c/s virtual

bf-opencl
OpenCL platform 0: Intel(R) OpenCL, 1 device(s).
Using device 0:         Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz
****Please see 'opencl_bf_std.h' for device specific optimizations****
Benchmarking: OpenBSD Blowfish (x32) [OpenCL]... DONE
Raw:    1131 c/s real, 381 c/s virtual

bf
Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... (4xOMP) DONE
Raw:    2448 c/s real, 613 c/s virtual

With ImageMagick convolve filter --
With OpenCL --

convert -convolve -1,0,-1,0,4,0,-1,0,-1 background.png test.png
real    0m5.175s
user    0m5.776s
sys     0m0.064s

Without OpenCL (OpenCL excluded during compile) --

convert -convolve -1,0,-1,0,4,0,-1,0,-1 background.png test.png
real    0m4.985s
user    0m5.560s
sys     0m0.052s


As we can see, the Intel SDK is of no good. Maybe they could've spend more time on developing their hardware and drivers instead of making shiny GUI on .NET.

Monday, May 13, 2013

Parallel/threaded rename to numbers script.

#! /bin/bash
# Rename all the files and folders in the current directory serially.
# DOES NOT PRESERVE EXTENSION.
#1st argument -- no. of processors
# 2nd argument -- no of presiding 0s
names="$(ls -1)"
# Remove current script file
names="$(echo "$names" | grep --invert-match --line-regexp "${0//.\/}")"
# No. of items -- required.
count=$(echo "$names" | wc --lines)
declare -i quot
declare -i i
declare -i from
declare -i till
from=1
i=1
# Function does the actual renaming.
# takes in arguments as the line no. of $names
# Since each line in $name is a file to be renamed randomly, an instance of the function will have to be supplied with with range of names it'll process.
# 'an instance' cause it's multithreaded
# first argument -- from which line will it start processing.
# second argument -- till which lines will it process.
rename_till () {
 declare -i c
#  from $c
 local c=$1
 declare -i entries
 local entries=$2
#  Till $entries in $names
 while [[ $c -le $entries ]]
 do
#   head -$c output's the first $c lines. Last line is of interest only, it's the cth line.
  local name="$(echo "$names" | head -$c | tail -1)"
#   File processed, proceed to next line.
  c=$c+1
  mv -i "$name" $(printf "%06d" $c) &
  echo "$c of $entries done." &
 done
}
# rename_till() calling mechanism.
# $till determines values of $2 in rename_till()
# Each thread will process $count/$1 lines
# This truncates decimals.
till=$(($count/$1))
while [[ $i -le $1 ]]
do
 rename_till $from $till &
 i=$i+1
 from=$from+$(($count/$1))
 if [[ $i -eq $1 ]]
 then
#   Remaining fines (truncated by $count/$1) added to last rename_till() call.
  till=$till+$(($count/$1))+$(($count%$1))
 else
  till=$till+$(($count/$1))
 fi
done