How we tested the Russian speedtest

Everyone at least once in their life used the speedtest.net service to measure the speed of Internet access at home or at work. Accuracy is not high, but free. I personally know more than 20 similar services, both Russian and foreign.

The idea arose to evaluate how these services show the “truth” with respect to the real picture. For this purpose, a stand was assembled that simulates the “last mile” of subscriber connection using technologies: FTTx, GPON and xDSL. The logical diagram of the assembled stand is shown in the figure below.
')


As an experimental service, we took the service, which is published on the website Banks.ru .

To limit the speed, a Cisco 7600 router was selected, where traffic shaping policies (speeds) for speeds from 1 to 200 Mbps were configured. The following 2 configurations were used:

Configuration 1:

sh running-config policy-map 1m-2s-out
Building configuration ...

Current configuration: 170 bytes
!
policy-map 1m-2s-out
class class-default
police cir 1000000 bc 250000
conform-action set-dscp-transmit cs1
exceed-action drop
violate-action drop
!
end

Configuration 2:

sh running-config policy-map 1024k-mg-out
Building configuration ...

Current configuration: 176 bytes
!
policy-map 1024k-mg-out
class class-default
police cir 1048500 bc 131072 be 196608 conform-action set-dscp-transmit cs1 exceed-action drop violate-action drop
!
end

The following machines were used as test terminals:

Sony

OS: Windows 7 x64
Processor: Intel Core i5 (2 cores)
Random access memory (RAM): 4GB
Web browser: Chrome 44.0.2403.125 m
FlashPlayer version: 18.0.0.209
Network card: 1GE

HP 8470

OS: Fedora x64
Processor: Intel Core i7 (4 cores)
Random access memory (RAM): 4GB
Web browser: Chrome 44.0.2403.125
FlashPlayer version: 18.0.0.209
Network card: 1GE

Dell Insperon 11 series 3138

OS: Windows 8 x64
Processor: Intel Celeron N2815 (2 cores)
Random access memory (RAM): 4GB
Web browser: Chrome 43.0.2357.134 m
FlashPlayer version: 18.0.0.209
Network card: 1GE

Notebook Asus s220n

OS: Windows XP
Processor: Intel Pentium 1000Mhz (2 cores)
RAM (RAM): 256 MB
WEB browser: Mozilla FireFox ver.26
FlashPlayer version: 11.9.900.170
Network card: 100FE

For each speed limit configuration, the test was repeated 3 times. Below are the aggregated measurement results.

Sony

Connection Type: FTTx

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s-in; 1m-2s-out	1.00	0.97	0%	3%
1,024	1024k-mgts-in; 1024k-mgts-out	1.02	1.01	0%	2%
five	5m-2s-in; 5m-2s-out	4.98	4.96	0%	one%
5.12	5120k-mgts-in; 5120k-mgts-out	5.21	5.24	2%	2%
ten	10m-2s-in; 10m-2s-out	9.90	9.98	one%	0%
ten	10m-mgts-in; 10m-mgts-out	10.13	10.29	one%	3%
20	20m-2s-in; 20m-2s-out	19.67	19.85	2%	one%
20	20m-mgts-in; 20m-mgts-out	20.16	20.37	one%	2%
thirty	30m-2s-in; 30m-2s-out	29.81	30.18	one%	one%
thirty	30m-mgts-in; 30m-mgts-out	29.89	29.77	0%	one%
50	50m-2s-in; 50m-2s-out	49.61	50.18	one%	0%
50	50m-mgts-in; 50m-mgts-out	50.53	51.25	one%	3%
70	70m-2s-in; 70m-2s-out	68.74	68,60	2%	2%
90	90m-2s-in; 90m-2s-out	88.86	89.47	one%	one%
100	100m-2s-in; 100m-2s-out	98.92	99.74	one%	0%
100	100m-mgts-in; 100m-mgts-out	98.98	100.40	one%	0%
150	150m-2s-in; 150m-2s-out	149.15	155.79	one%	four%
150	150m-mgts-in; 150m-mgts-out	149.05	153.17	one%	2%
200	200m-2s-in; 200m-2s-out	197.41	208.35	one%	four%
200	200m-mgts-in; 200m-mgts-out	201.34	194.20	one%	3%

Connection Type: GPON

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	0.97	1.02	3%	2%
1,024	1024k-mg	0.98	1.01	four%	one%
5.12	5120k-2s	5.05	5.10	one%	0%
5.12	5120k-mg	5.27	5.21	3%	2%
15	15m-mg	15.45	15.19	3%	one%
20	20m-2s	20.16	19.82	one%	one%
20	20m-mg	19.78	20.53	one%	3%
50	50m-2s	48.77	47.91	four%	four%
50	50m-mg	49.62	48.07	one%	four%
70	70m-2s	67.97	68.72	3%	2%
90	90m-mg	87.95	87,12	2%	3%

Connection Type: SHDSL

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	0.95	1.04	five%	four%
1,024	1024k-mgts	1.02	1.01	0%	one%
3,072	3072k-mgts	3.17	3.14	3%	2%
5.12	5120k-mgts	5.28	5.08	3%	one%
5.12	5120k-2s	5.22	4.99	2%	3%
15	15m-mgts	14.96	14.53	0%	3%
20	20m-mgts	19.84	20.23	one%	one%
20	20m-2s	19.79	20.17	one%	one%

HP 8470

Connection Type: FTTx

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s-in; 1m-2s-out	1.01	0.97	one%	one%
1,024	1024k-mgts-in; 1024k-mgts-out	1.02	1.01	one%	one%
five	5m-2s-in; 5m-2s-out	4.98	5.01	0%	one%
5.12	5120k-mgts-in; 5120k-mgts-out	5.28	5.24	3%	2%
ten	10m-2s-in; 10m-2s-out	9.90	9.95	one%	one%
ten	10m-mgts-in; 10m-mgts-out	10.13	10.20	2%	one%
20	20m-2s-in; 20m-2s-out	19.67	19.73	one%	0%
20	20m-mgts-in; 20m-mgts-out	20.19	20.37	one%	2%
thirty	30m-2s-in; 30m-2s-out	29.82	30.18	one%	0%
thirty	30m-mgts-in; 30m-mgts-out	29.80	29.77	one%	0%
50	50m-2s-in; 50m-2s-out	49.60	50.18	one%	0%
50	50m-mgts-in; 50m-mgts-out	50.82	51.25	2%	3%
70	70m-2s-in; 70m-2s-out	68.95	68,60	2%	0%
90	90m-2s-in; 90m-2s-out	89.23	89.47	one%	0%
100	100m-2s-in; 100m-2s-out	98.97	99.74	one%	0%
100	100m-mgts-in; 100m-mgts-out	99.08	100.40	one%	0%
150	150m-2s-in; 150m-2s-out	149.70	155.79	0%	0%
150	150m-mgts-in; 150m-mgts-out	149.75	153.17	0%	0%
200	200m-2s-in; 200m-2s-out	197.54	208.35	one%	0%
200	200m-mgts-in; 200m-mgts-out	202.06	194.20	one%	2%

Connection Type: GPON

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	0.96	0.98	four%	2%
1,024	1024k-mg	1.02	1.04	0%	2%
5.12	5120k-2s	5.11	5.06	0%	one%
5.12	5120k-mg	4.99	5.03	3%	2%
15	15m-mg	15,15	15.19	one%	one%
20	20m-2s	19.51	20.17	2%	one%
20	20m-mg	19.33	19.88	3%	one%
50	50m-2s	47.89	40.15	four%	2%
50	50m-mg	49.36	48.87	one%	2%
70	70m-2s	68.92	67.58	2%	3%
90	90m-mg	87.15	88.63	3%	2%

Connection Type: SHDSL

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	0.99	1.01	one%	one%
1,024	1024k-mgts	1.01	1.01	one%	one%
3,072	3072k-mgts	3.03	2.95	one%	four%
5.12	5120k-mgts	4.98	5.01	3%	2%
5.12	5120k-2s	5.21	4.95	2%	3%
15	15m-mgts	14.61	14.87	3%	one%
20	20m-mgts	19.76	19.84	one%	one%
20	20m-2s	20.14	20.09	one%	0%

Dell Insperon 11 series 3138

Connection Type: FTTx

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s-in; 1m-2s-out	1.03	1.00	3%	0%
1,024	1024k-mgts-in; 1024k-mgts-out	1.03	1.01	3%	0%
five	5m-2s-in; 5m-2s-out	4.98	5.00	0%	0%
5.12	5120k-mgts-in; 5120k-mgts-out	5.27	5.24	2%	3%
ten	10m-2s-in; 10m-2s-out	9.92	10.02	one%	0%
ten	10m-mgts-in; 10m-mgts-out	10.18	10.25	2%	2%
20	20m-2s-in; 20m-2s-out	19.82	20,12	one%	one%
20	20m-mgts-in; 20m-mgts-out	20.31	20.40	2%	2%
thirty	30m-2s-in; 30m-2s-out	29.98	30.42	0%	one%
thirty	30m-mgts-in; 30m-mgts-out	29.98	30.63	0%	2%
50	50m-2s-in; 50m-2s-out	49.95	51.28	0%	one%
50	50m-mgts-in; 50m-mgts-out	50.91	51.61	2%	3%
70	70m-2s-in; 70m-2s-out	69.06	71.43	one%	2%
90	90m-2s-in; 90m-2s-out	89.47	89.38	one%	one%
100	100m-2s-in; 100m-2s-out	99.57	99.81	0%	0%
100	100m-mgts-in; 100m-mgts-out	99.54	100.41	0%	0%
150	150m-2s-in; 150m-2s-out	149.96	153.13	0%	2%
150	150m-mgts-in; 150m-mgts-out	150.06	149.11	0%	one%
200	200m-2s-in; 200m-2s-out	204.37	201.76	2%	one%
200	200m-mgts-in; 200m-mgts-out	207.15	204.81	four%	2%

Connection Type: GPON

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	0.97	0.96	3%	four%
1,024	1024k-mg	0.99	1.04	3%	2%
5.12	5120k-2s	5.06	5.11	one%	0%
5.12	5120k-mg	5.17	4.97	one%	3%
15	15m-mg	14.79	15.06	one%	0%
20	20m-2s	19.77	20.37	one%	2%
20	20m-mg	19.82	19.73	one%	one%
50	50m-2s	48.99	48.71	2%	3%
50	50m-mg	49.57	49.93	one%	0%
70	70m-2s	69.24	67.98	one%	3%
90	90m-mg	89.23	88.17	one%	2%

Connection Type: SHDSL

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	1.03	0.96	five%	four%
1,024	1024k-mgts	1.00	1.07	0%	one%
3,072	3072k-mgts	2.91	3.17	3%	2%
5.12	5120k-mgts	5.11	5.05	3%	one%
5.12	5120k-2s	4.99	5.09	2%	3%
15	15m-mgts	14.94	15.02	0%	3%
20	20m-mgts	19,99	19.95	one%	one%
20	20m-2s	19.83	21.09	one%	one%

Notebook Asus s220n

Connection Type: FTTx

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s-in; 1m-2s-out	1.02	0.99	2%	one%
1,024	1024k-mgts-in; 1024k-mgts-out	1.05	1.04	3%	2%
five	5m-2s-in; 5m-2s-out	4.98	4.95	0%	one%
5.12	5120k-mgts-in; 5120k-mgts-out	5.18	5.26	one%	3%
ten	10m-2s-in; 10m-2s-out	9.90	9.80	one%	2%
ten	10m-mgts-in; 10m-mgts-out	10.12	10.18	one%	2%
20	20m-2s-in; 20m-2s-out	19.90	19.36	0%	3%
20	20m-mgts-in; 20m-mgts-out	20.46	19.83	2%	one%
thirty	30m-2s-in; 30m-2s-out	29.97	28.85	0%	four%
thirty	30m-mgts-in; 30m-mgts-out	30.02	28.48	0%	five%
50	50m-2s-in; 50m-2s-out	49,83	46.77	0%	6%
50	50m-mgts-in; 50m-mgts-out	50.81	46.75	2%	6%
70	70m-2s-in; 70m-2s-out	68.80	64.96	2%	eight%
90	90m-2s-in; 90m-2s-out	89.23	94.48	one%	five%
100	100m-2s-in; 100m-2s-out	96.95	96,86	3%	3%

Connection Type: GPON

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	0.99	0.99	one%	one%
1,024	1024k-mg	1.02	1.01	0%	one%
5.12	5120k-2s	5.07	5.05	one%	one%
5.12	5120k-mg	5.30	5.14	four%	0%
15	15m-mg	15.37	15.33	2%	2%
20	20m-2s	20.08	19.97	0%	0%
20	20m-mg	20.35	20.09	2%	0%
50	50m-2s	48.40	47.85	3%	four%
50	50m-mg	49.55	48.32	one%	3%
70	70m-2s	68.47	67.72	2%	3%
90	90m-mg	87,88	88.43	2%	2%
97.5	with no restrictions	95.62	94.38	2%	3%

Connection Type: SHDSL

Mbit / s speed	Service policy	Download Result	Upload Result	Average loading error,%	The average return error,%
one	1m-2s	1.03	1.05	3%	five%
1,024	1024k-mgts	1.03	1.01	one%	one%
3,072	3072k-mgts	3.21	3.10	four%	one%
5.12	5120k-mgts	5.18	5.10	one%	0%
5.12	5120k-2s	5.12	4.98	0%	3%
15	15m-mgts	14.83	14.60	one%	3%
20	20m-mgts	19.79	20.39	one%	2%
20	20m-2s	19.47	20.07	3%	0%

* The port has a physical limit of 100 Mbps, which gives the maximum speed at the third (IP) level of ~ 97.5 Mbps with MTU = 1500.

findings

The measurement results were pleasantly surprised. The measurement error in most cases did not exceed 5% at different speeds and for different subscriber access technologies, which is acceptable for such a service. Even on the oldest Asus laptop with 256 MB RAM and a Pentium 1000 MHz processor, the error did not exceed 8%.