tps.py, a python script using curl to measure the time per transaction requested to your website
curl --connect-timeout 10 --max-time 10 -o /dev/null -s -w "%{http_code},%{size_download},%{time_appconnect},%{time_connect},%{time_namelookup},%{time_pretransfer},%{time_starttransfer},%{time_total}" -k#-*- coding: UTF-8 -*-
import commands
import getopt
import os
import re
import sys
import thread
import threading
import time
def help(code):
print 'tps.py -A "useragent" -C "cookies" -H "header" -P "period" -T "tps" -U "url" -O "outputdir"'
print 'examples:'
print 'tps.py -U https://www.amazon.com -T 20 -P PT5M'
print 'tps.py -U https://www.amazon.com -T 20 -P PT1H'
sys.exit(code)
def check_args(a, c, h, p, t, u, o):
if u == '':
print 'no URL specified!'
help(2)
if t == '':
print 'no TPS specified!'
help(2)
if p == '':
print 'no PERIOD specified!'
help(2)
cmd = 'curl --connect-timeout 10 --max-time 10 -o /dev/null -s -w "%{http_code},%{size_download},%{time_appconnect},%{time_connect},%{time_namelookup},%{time_pretransfer},%{time_starttransfer},%{time_total}" -k'
if not a == '':
cmd = cmd + " -A \"" + a + "\""
if not c == '':
cmd = cmd + " -b \"" + c + "\""
if not h == '':
cmd = cmd + " -H \"" + h + "\""
cmd = cmd + " " + u
fp = os.getcwd() + '/tps.csv'
if not o == '':
fp = o + "/tps.csv"
tpspt(cmd, t, p, fp)
def tpspt(cmd, t, p, fp):
f = open(fp, 'w+')
if not f:
print "fail to create tps.csv under ", os.getcwd()
sys.exit(1)
m = re.match('^PT(\d+)[MH]$', p)
if not m:
print ' wrong PERIOD pattern!', p , 'not match "PT\d+[MH]"'
help(2)
m = int(m.group(1))
if p[-1] == 'H':
m = m * 60 * 60
else:
m = m * 60
t = float(t)
print "-tps", t, "-period", p, ",", int(m * t), "transactions will be executed!"
print cmd
start = time.time()
wq = []
ths = []
maxm = m
if t < 1:
maxm = int(m*t)
for x in range(0, maxm):
if t < 1:
thargs = [cmd, t, wq, int(x/t) + 1]
th = threading.Timer(int(x/t) + 1, tps, thargs)
else:
thargs = [cmd, t, wq, x + 1]
th = threading.Timer(x + 1, tps, thargs)
th.start()
ths.append(th)
for th in ths:
th.join()
end = time.time()
print "Finish ", int(m * t), "transactions in", int(end - start), "seconds!"
print "Writing results to tps.csv ......"
f.write("id,http_code,size_download,time_appconnect,time_connect,time_namelookup,time_pretransfer,time_starttransfer,time_total")
f.write('\n')
for line in wq:
f.write(line)
f.close()
print "Done! Please check results in", fp
def tps(cmd, t, wq, x):
print "execute", t, "transactions at the ", x, "seconds"
while t > 0:
starttime = time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time.time()))
status, output = commands.getstatusoutput(cmd)
# sample output
# http_code,size_download,time_appconnect,time_connect,time_namelookup,time_pretransfer,time_starttransfer,time_total
# 200,79059,0.011196,0.000273,0.000214,0.011224,1.091032,5.055447
cells = output.split(",")
if len(cells) < 8:
print "ERROR!!!", output
continue;
cell0 = cells[0]
if not cell0 == '200':
print "WARNING!!!", cell0, "is not wanted response code!!!"
cell1 = cells[1]
cells = map(lambda x: float(x) * 1000, cells[2:])
cells.insert(0, float(cell1)/1024)
cells.insert(0, int(cell0))
cells = map(lambda x: str(x), cells)
cells.insert(0, starttime)
wq.append(','.join(cells))
wq.append('\n')
t = t - 1;
def main(argv):
useragent = ''
cookie = ''
header = ''
period = ''
tps = ''
url = ''
outputdir = ''
try:
opts, args = getopt.getopt(argv,"hA:C:H:O:P:T:U:",["useragent=","cookie=", "header=", "outputdir=", "period=", "tps=", "url="])
except getopt.GetoptError:
help(1)
for opt, arg in opts:
if opt == '-h':
help(0)
elif opt in ("-A", "--useragent"):
useragent = arg
elif opt in ("-C", "--cookie"):
cookie = arg
elif opt in ("-H", "--header"):
header = arg
elif opt in ("-P", "--period"):
period = arg
elif opt in ("-T", "--tps"):
tps = arg
elif opt in ("-U", "--url"):
url = arg
elif opt in ("-O", "--outputdir"):
outputdir = arg
check_args(useragent, cookie, header, period, tps, url, outputdir)
if __name__ == "__main__":
main(sys.argv[1:])
example usage:
./tps.py -T 10 -P "PT3M" -U https://errong.win
-tps 10.0 -period PT3M , 1800 transactions will be executed!
curl --connect-timeout 10 --max-time 10 -o /dev/null -s -w "%{http_code},%{size_download},%{time_appconnect},%{time_connect},%{time_namelookup},%{time_pretransfer},%{time_starttransfer},%{time_total}" -k https://errong.win
.......
Finish 1800 transactions in 184 seconds!
Writing results to tps.csv ......
Done! Please check results in tps.csv
TPS Graph
As you can found, errong.win average responding time is about 420ms, not too bad.
curl -w manual
Make curl display information on stdout after a completed transfer. The format is a string that may contain plain text mixed with any number of variables. The format can be specified as a literal "string", or you can have curl read the format from a file with "@filename" and to tell curl to read the format from stdin you write "@-".The variables present in the output format will be substituted by the value or text that curl thinks fit, as described below. All variables are specified as %{variable_name} and to output a normal % you just write them as %%. You can output a newline by using \n, a carriage return with \r and a tab space with \t.
NOTE: The %-symbol is a special symbol in the win32-environment, where all occurrences of % must be doubled when using this option.
The variables available are:
- content_type
The Content-Type of the requested document, if there was any. - filename_effective
The ultimate filename that curl writes out to. This is only meaningful if curl is told to write to a file with the -O, --remote-name or -o, --output option. It's most useful in combination with the -J, --remote-header-name option. - ftp_entry_path
The initial path curl ended up in when logging on to the remote FTP server. - http_code
The numerical response code that was found in the last retrieved HTTP(S) or FTP(s) transfer. - http_connect
The numerical code that was found in the last response (from a proxy) to a curl CONNECT request. (Added in 7.12.4) - http_version
The http version that was effectively used. - local_ip
The IP address of the local end of the most recently done connection - can be either IPv4 or IPv6 (Added in 7.29.0) - local_port
The local port number of the most recently done connection (Added in 7.29.0) - num_connects
Number of new connects made in the recent transfer. (Added in 7.12.3) - num_redirects
Number of redirects that were followed in the request. (Added in 7.12.3) - proxy_ssl_verify_result
The result of the HTTPS proxy's SSL peer certificate verification that was requested. 0 means the verification was successful. (Added in 7.52.0) - redirect_url
When an HTTP request was made without -L to follow redirects, this variable will show the actual URL a redirect would take you to. (Added in 7.18.2) - remote_ip
The remote IP address of the most recently done connection - can be either IPv4 or IPv6 (Added in 7.29.0) - remote_port
The remote port number of the most recently done connection (Added in 7.29.0) - scheme
The URL scheme (sometimes called protocol) that was effectively used (Added in 7.52.0) - size_download
The total amount of bytes that were downloaded. - size_header
The total amount of bytes of the downloaded headers. - size_request
The total amount of bytes that were sent in the HTTP request. - size_upload
The total amount of bytes that were uploaded. - speed_download
The average download speed that curl measured for the complete download. Bytes per second. - speed_upload
The average upload speed that curl measured for the complete upload. Bytes per second. - ssl_verify_result
The result of the SSL peer certificate verification that was requested. 0 means the verification was successful. (Added in 7.19.0) - time_appconnect
The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0) - time_connect
The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed. - time_namelookup
The time, in seconds, it took from the start until the name resolving was completed. - time_pretransfer
The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved. - time_redirect
The time, in seconds, it took for all redirection steps including name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3) - time_starttransfer
The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result. - time_total
The total time, in seconds, that the full operation lasted. - url_effective
The URL that was fetched last. This is most meaningful if you've told curl to follow location
