Apache vs Nginx benchmark/How to make Apache faster than Nginx.

Unlike other benchmarks, both Apache and and Nginx have been tuned for best performance for the task they're doing (serving static content).

Test using apache's ab utility.

Results --

Direct hits --

Rewrite hits --

Test done

There are 2 sets of tests done --

• Hitting URLs with rewrite rules
• Hitting URLs with file paths directly.

For each of these, test where done from concurrency 1000 to 20000.

In the charts, legends which represent tests done by hitting URLs with file paths directly is suffixed with _direct, while for the rewrite rules it's suffixed with _rewrite.

Configurations --

Nginx --

user nginx nginx;

worker_processes 4;

events {

multi_accept on;

worker_connections 11000;

}

error_log /var/log/nginx16/nginx.log;

http {

server {

access_log off;

listen [::]:80 backlog=2 so_keepalive=60:60:0;

root /home/nginx;

server_name RHEL6;

sendfile on;

reset_timedout_connection on;

server_tokens off;

open_file_cache max=20 inactive=99999;

open_file_cache_min_uses 1;

open_file_cache_valid 99999;

open_file_cache_errors on;

log_not_found off;

rewrite ^/file1$ /0 last;

rewrite ^/file2$ /1 last;

rewrite ^/file3$ /2 last;

rewrite ^/file4$ /3 last;

rewrite "^/list([234]){0,1}/(.*)$" /$2 last;

location / {

deny all;

}

location ~ ^/[0-9]$ {

allow all;

}

location = /hello.php {

allow all;

}

location ~ ^/file[1234]$ {

allow all;

}

location ~ "^/list([234]){0,1}/[0-9]$" {

allow all;

}

}

}

Apache tuned --

ServerRoot /opt/rh/httpd24/root/usr/lib64/httpd/

LoadModule authn_core_module modules/mod_authn_core.so

LoadModule authz_core_module modules/mod_authz_core.so

LoadModule unixd_module modules/mod_unixd.so

LoadModule rewrite_module modules/mod_rewrite.so

LoadModule mpm_event_module modules/mod_mpm_event.so

#LoadModule php5_module /usr/lib64/httpd/modules/libphp5-zts.so

Listen [::]:80 http

ListenBackLog 2

MaxConnectionsPerChild 0

MaxMemFree 0

ServerLimit 4

StartServers 4

# optimized for concurrency

#MaxRequestWorkers 10000

#ThreadLimit 2500

#ThreadsPerChild 2500

#MaxSpareThreads 10000

#MinSpareThreads 10000

# optimized for benchmark/HTTP header throughoutput

MaxRequestWorkers 100

ThreadLimit 25

ThreadsPerChild 25

MaxSpareThreads 100

MinSpareThreads 100

DocumentRoot /home/apache

ServerName RHEL6

User apache

Group apache

ErrorLog /var/log/httpd24/apache.log

LogLevel alert

AcceptPathInfo off

ContentDigest off

FileETag Inode Mtime

KeepAlive on

KeepAliveTimeout 60

MaxKeepAliveRequests 0

ServerTokens Full

TimeOut 5

EnableMMAP on

EnableSendfile on

ExtendedStatus off

LimitInternalRecursion 1

MaxRangeOverlaps none

MaxRangeReversals none

MergeTrailers off

Mutex pthread rewrite-map

RewriteEngine on

RewriteRule ^/file1 /0 [END,PT]

RewriteRule ^/file2 /1 [END,PT]

RewriteRule ^/file3 /2 [END,PT]

RewriteRule ^/file4 /3 [END,PT]

RewriteRule ^/list([234]){0,1}/(.*) /$2 [END,PT]

AllowOverride none

Options -FollowSymLinks

Require all denied

#SetHandler php5-script

Require all granted

Require all granted

Test commands --

echo -n list/1 list3/8 file1 list2/5 list2/0 file2 file4 list/7 list4/6 list3/3 | xargs -r -P 0 -n 1 -d ' ' -I {} /bin/bash -c 'ab -c -k -g /home/de//_$$ -s 1 -t 60 -n 9999999 -r http://[fc00::1:2]/{} &> /home/de//_stdout_$$'

echo -n {9..0} | xargs -r -P 0 -n 1 -d ' ' -I {} /bin/bash -c 'ab -c -k -g /home/de//_$$ -s 1 -t 60 -n 9999999 -r http://[fc00::1:2]/{} &> /home/de//_stdoout_$$'

The output of ab along with the report of each link served have been uploaded.

Test strategy --

Concurrency is the main thing we need to test.

We need to simulate situation when there are multiple low-bandwidth clients (all of them needs to be served concurrently to prevent some connections from being stalled). So we'll just increase the concurrent requests sent to the server using all the test machines's bandwidth.

Multiple TCP connection must be established in parallel; each of these connections will be reused to send multiple requests (keep alive will be turned on). Since the TCP connection has been established, we're not benchmarking the kernel. Speaking of which I could not get Nginx's keepalive to be turned off; that maybe the reason why Nginx was so fast in other benchmarks.

Since all webservers use sendfile() for delivering files, it's pointless to make the file large; we're not benchmarking the kernel. We're interested in how quickly the server creates HTTP headers.

Concurrency vs throughoutput (total requests/second).

Serving request serially, as opposed to concurrently is more efficient because of context switching and management overhead; we cant do anything about context switching, but the management overhead and the efficiency in constructing HTTP headers is what we want to benchmark.

But concurrency matters more than throughoutput.

If the server is independent, i.e. only it's CPU resources are used (network, disk I/o, a separate server like database are not the bottleneck like with these benchmarks), then increasing the webserver's concurrency will reduce the efficiency of the CPU cycles because of the context switching overhead. In these cases it's better to start serving another request when it finishes serving one request.

When there is a in-server server bottleneck like the network or the disk (for e.g. we'll take this e.g. for this para) and the requests are such that they take up quiet a lot of time reading the disk/network, it'll happen that the other requests timeout, or take too much time to respond. The longer the queue, more likely this'll happen. In these situations, increasing the concurrency will let the server serve multiple request in parallel sending progress to each user, abet slowly as compared to serving a single user at a time but without timing them out.

Another kind of bottleneck is towards the end user. A classic e.g. is downloading files where the client's network or the Internet is the bottle neck. If we do 1 download at a time, we wont be able to use all our hardware (disk, network etc..) to the fullest since the user's Internet connection is the bottleneck; to use it to the fullest we have to serve multiple clients. When it comes to these kind of situations, resource utilization IS about concurrency and concurrent efficiency becomes more important as the difference between the server's network speed and the client's network speed increases because that'll mean the server can serve more clients in parallel.

A similar bottleneck is when there is are multiple backend server (physical) which can handle, like, N queries in parallel. If there are X server, then the webserver must serve N*X requests in parallel to get the maximum utilization of the backend servers.

Cheating web servers --

Suppose we have a timeout of x seconds.

If the webserver is not serving the requests concurrently (to reduce context switching and management overhead and increase throughoutput), some of the requests will be within x seconds, while others will timeout.

A webserver which serves requests concurrently, will have all the requests timed out if the load exceeds a certain value.

A webserver which does not have a better concurrency is designed for benchmarks and will only perform good at benchmarks.

Apache vs Nginx in concurrency –

Nginx also appears to be serving the requests concurrently but with not as much concurrency as with Apache, but with Apache the responses were like within 21ms or lower, where as with nginx they were under 400ms; for Apache the distribution of the no. of requests served vs the interval under which they were served were not noted down for under 100ms, thus it may be cheating also.

Because of client machine limitation (it's a 10 years old machine), Apache maybe a lot faster than Nginx.

Verdict –

Apache is the clear winner.

If you switched to Nginx for the speed, your assumptions where false. Apache has a bigger toolkit, is faster and at the same time is security oriented. And in case you're wondering about the bigger CVE for Apache, it's because it has a bigger tookit and is older.

So it's all about for what purpose you tweak Apache.

Personally I don't understand the purpose of the Nginx project. If they want to optimized, they rather contribute to Apache or fork the project or create modules (something which Nginx doesn't even support) instead of creating a rival.

Refilling HP 703 cartridge

Most HP cartage can be refilled from the top, I.e ink is put in via needles from the top of the cartage.

The ink is held by a sponge like substance which has to be poked in by the needle and refilled. Poke deep into the sponge, but not so deep so as to touch the base. A bit below the middle most depth will be good.

First, before you start to refill, seal off the printing heads (the steel part) with cello tape to avoid touching.

For black cartridge, open the top sticker, you'll see 5 holes. Refill using the middle most (and largest) hole. Refill till you see a little bit of liquid ink on one of the 5 holes. If you filled in excess, remove it using the suction tool. Sign of excess is that it starts dripping from the other end (nozzle). If you fill in excess, you'll waist ink and ruin the cartridge.

During refilling, do it slowest possible and wait at times between pushing the ink (do it in sessions). If you do it fast, it'll look as if the ink has been filled (the symptoms will appears), but in reality it's just a shallow fill.

Seal off the exposed holes with tape.

Wait for 24 hours before starting to print. Put the cartridge in the horizontal position (printing heads facing the floor). This's to allow the ink seep into the printing heads. If you do it earlier, you may see faded out prints; which must not be mistaken for low ink. You just need to wait in case that happens to a newly filled refill.

Create a test page to print black. If it does not, use the suction tool to open up the blocked pores. Do minimal amount of suction so as to just bring ink to the syringe.  As that happens, the pores will get opened up again.

For color cartridge, you'll also see 5 holes, the largest hole is for Magenta, left for yellow, and right for Cyan. This is when the cartridge's nozzle (or the chip) is opposite of you or the Magenta hole is on the top.

For 703 cartridge, the scheme is a little different. First, all the holes are of the same size. The top most (TH) is yellow (with the chip NOT facing against you and the middle most (yellow) hole far away from you). Left is Magenta and right is Cyan.

In the left and right, there will be a pair of holes (PH), you can use any of the 2, but it's recommended to use only 1, I'll tell you why.

When you see the holes in light, you'll see them having a sponge, and on the sponge there'll be ink stains. For PH, use the hole with the ink stains.

To do the actual refill, you need to follow similar protocols to that of the black refill, with a little bit of modification.

So you did realize this's a sponge right? You need to refill till the sponge gets filled up with ink; i.e. it gets all wet, but don't fill in excess, it must get JUST wet; the wetness must be such that the sponge starts looking almost completely black and wet. As that happens it may happen that the other hole in the PH starts getting wet too, if it does stop refilling.

For the tests, use Cyan, magenta and yellow colors separately.

Garbage test database for testing.

I've written a multithreaded C program (fast) to fill a SQL database with garbage text.

This was with an intention of creating a database which is heavy for the server, so you can test performance.

First, create the table --

create table complex (f1 varchar (100), f2 varchar (200), f3 varchar (300), f4 varchar (400));

Then compile this program (use gcc -lgomp -fopenmp random.c) --

#include
#include
#include
// compile using gcc -lgomp -fopenmp random_no_genrator.c
void rand_str(char *dest, size_t length) {
    char charset[] = "0123456789"
                     "abcdefghijklmnopqrstuvwxyz"
                     "ABCDEFGHIJKLMNOPQRSTUVWXYZ";

    while (length-- > 0) {
        size_t index = (double) rand() / RAND_MAX * (sizeof charset - 1);
        *dest++ = charset[index];
    }
    *dest = '\0';
}

int main () {
    #pragma omp parallel
    {
        char* random100;
        random100 = malloc (sizeof (char) * 100);
        char* random200;
        random200 = malloc (sizeof (char) * 200);
        char* random300;
        random300 = malloc (sizeof (char) * 300);
        char* random400;
        random400 = malloc (sizeof (char) * 400);
        printf ("set autocommit=0;\n");
        for (;;) {
            rand_str (random100, 100);
            rand_str (random200, 200);
            rand_str (random300, 300);
            rand_str (random400, 400);
//             change the format if needed
            printf ("insert into complex values ('%s', '%s', '%s', '%s');\n",random100, random200, random300, random400);
        }
    }
}

And write it's output to a file. This's going to run forever so you've to interrupt it to stop it. Go to the end of this file and remove any partial queries that were generated cause of the interrupt. This part is optional.

Append 'commit;' to the end of the file, otherwise you wont see any changes.

The import is pretty fast too.

Migrating OS across hardware (including VM to bare metal and opposite).

In the OS which is to be migrated/replicated if you've not tampered with the udev configuration or have not done something which forces some modules to be loaded in the kernel which causes problems with the new hardware, you can migrate with ease.

First you need to take backup of the OS which you need to migrate. This is best done offline, however it is possible if the system is up and files/directories etc... belonging to the OS are NOT being upgraded. Reason being the files which are read to start the OS successfully will generally not be updated while the OS is operational; dynamically modified files are re-read anyway because changes are common there (like in /var).

When backing up a live system, you have to exclude  psudo (commonly /dev, /sys/, /proc and maybe run /run too (tmpfs)) filesystems. WARNING -- the /dev directory has device nodes which are in the rootFS and are used when devfs/udev is not mounted on early boot. You NEED these to boot the system especially if it does not have an initramfs (commonly Gentoo setups).  One tip is to mount --bind the RootFS; the mount is not recursive (not rbind) so you get the core rootFS exposed (which includes the contents of the /dev directory). Backup this mount --bind ed FS.

For an offline backup, I recommend systemrescuecd.


Backup can be done using squashfs or tar (use switches which preserve xattr/selinux (security.selinux namespace), permissions. With tar DO NOT use --attrs. It has bugs which I have to report (another pending bug report :( ).

Make partitions and FS to migrate the OS.

Mount the FS to the directory tree where you'll extract the backed up OS and extract it (note: tar swithces).

Fix fstab entries to reflect the changes in your new setup.

If the bootloader is not managed by another OS on the same system, then you've to update/install it too. First step is regenerating the grub.cfg (using grub2-mkconfig) or manually modify menu.lst. Then finally install the bootloader (grub2-install); you may have to repeat these steps a few time otherwise you may get a crippled boot menu.

Whenever I say 'rootfs' I mean all the FS across which OS files exist. Thus if you have a separate /usr, /boot etc... partitions

How is this supposed to work? Question is why will it not work. Udev do not store persistent information about the hardware -- it detects it every time you boot your Linux OS. So new hardware doesn't matter.

Of course if the target machine's hardware is not support or has driver issues, then the usual problems imply as with a fresh install.

This is the official installation technique of Gentoo and my preferred technique. I first install a template OS in a VM, customize it to my needs and then migrate it to the bare metal or other VMs.

Sending mails to ip addresses (and ipv6 addresses).

This simple thing is not discussed on the Internet anywhere. If it is, the questions are still open.

Answer --

anemailaccount@[88.4.3.54]

For ipv6

testemail@[ipv6:ff::fe]

Speeing up MTP (android) transfers.

MTP is inherently flawed, it having it's origins with the developers of Microsoft.

It provides slow linear data transfer, but when it comes to transferring individual files (the efficiency of the protocol is tested here), it's the slowest thing you've ever seen.

So if you have a number of files, just make an archive off it, transfer it to the MTP device, an extract it there.

ZTE MF197 under Linux and review.

First -- This modem has a bug. Some wont see some (very popular) networks and the company refuses to fix or admit the problem.

This's a one of a kind modem. It has a built in NAT; so you don't have to dial it internally everything is in the web interface at 192.168.0.1.

Very good design.

For the installation, remove usb_modeswitch; otherwise this wont work.

And that's it. In generic distros, you'll get a new network interface on plugging; then operate the modem through 192.168.0.1.

I've not seen any other modem like this in the market.

HP K45 review.

I've to say this's the first non-mechanical keyboard which actually made it! For the first few months i.e.

Otherwise I've never encountered a keyboard which has lasted more than 2 months without a replacement.

The only complaint is that it's keys have become a little bit hard now; but that's not much of a problem (I'm a man) and despite the fact that this's not a mechanical keyboard, it doesn't have any multimedia keys.

Although HP claims this keyboard only works with Windows, I can confirm it's a standard plug and play keyboard.

Logitech MK220/M150 review

This Microsoft keyboard from Logitech is only intended for people with either 2 fingers or people who type with 2 fingers.

This 100% Microsoft, and Microsoft only compatible keyboard and mouse ensures tired arms cause --

1) It does not contain a palm rest
2) The keyboard does not have a lower edge boundary. The lowermost rows of keys make up the lower boundary of the keyboard.

Thus, if you attempt to rest your arms on the table any one of atrl, fn, alt, space, arrow keys and Logitech's favourite Windows keys will be pressed accidentally.

This's not the first time I've encountered a bad ergonomic keyboard from Logitech.

Another stupidity the company has done is that the mouse takes in 2 AA batteries making it heavy and keyboard takes in 2 AAA batteries in an attempt to make it light.

Anything more than 1M and the keyboard mouse starts giving communication issues. Maybe cause I made it work with Linux.

The DPI of the mouse is too low. So it has NO application in gaming or designing space. Buy a Microsoft keyboard/mouse for that . Logitech sells the same (it's only purpose being manufacturing HID devices to operate Windows) thing but at a higher price.

Booting GPT disks with Gigabyte's hybrid EFI.

I stated in my previous post, how horrible Gigabyte motherboards are. I continue to do the same now.

Basically Microsoft Gigabyte's hybrid EFI is nothing but a piece of Windows software which enables Windows to access 3+TB hard drive.

There's nothing EFI in the BIOS. It cannot boot GPT disks, it cannot run EFI applications and it does not provide any EFI API to the OS.