NNSquad - Network Neutrality Squad
[ NNSquad ] Why I'm Skeptical of the FCC's Call for User Broadband Testing
Why I'm Skeptical of the FCC's Call for User Broadband Testing
http://lauren.vortex.com/archive/000688.html
Greetings. The FCC has issued a call for Internet users to test their
broadband connections and report the results back to the FCC for
analysis ( http://bit.ly/cc3LcY [Wired] ).
After inspecting the associated site and testing tools, I'm must admit
that I am extremely skeptical about the overall value of the data
being collected by their project, except in the sense of the most
gross of statistics.
In random tests against my own reasonably well-calibrated tools, the
FCC tools showed consistent disparities of 50% to 85%! Why isn't this
surprising?
I'm a big fan of Google's M-Lab project (and was involved in the
meeting at Google that served as the genesis for the project
itself -- http://bit.ly/9EFjdW [GCTIP] -- but I must question the use
of the Java-based M-Lab testing tool (and the other Java tool provided
by the FCC site) in this particular manner. (Ironically, the FCC site
stipulates that the M-Lab tool won't run under Google's own Chrome
browser!)
The FCC testing regime ( http://bit.ly/9IuQeC [FCC] ) provides for no
control related to other activity on users' connections. How many
people will (knowingly or not) run the tests while someone else in the
home or business is watching video, downloading files, or otherwise
significantly affecting the overall bandwidth behavior?
No obvious clues are provided to users regarding the underlying server
testing infrastructure. As anyone who uses speed tests is aware, the
location of servers used for these tests will dramatically affect
results. The ability of the server infrastructure to control for
these disparities can be quite limited depending on ISPs' own network
topologies.
And of course, on-demand, manually-run tests cannot provide any sort
of reasonable window into the wide variations in performance that
users commonly experience on different days of the week, times of day,
and so on.
Users are required to provide their street address information with
the tests, but there's nothing stopping anyone from entering any
address that they might wish, suggesting that such data could often be
untrustworthy compared with (much coarser) already available IP
address-based location info.
While these tests under this methodology may serve to help categorize
users into very broad classes of Internet service tiers, it's hard to
see how their data could be reasonably trusted beyond that level.
ISPs may be justifiably concerned that the data collected from these
tests by this FCC effort may be unrepresentative in significant ways.
There are certainly methods available to collect meaningful,
longitudinal data in manners that would provide genuinely useful
insight into the real-world characteristics of users' broadband
connections' performance.
I discussed my own proposals along these lines a couple of years ago
in:
Practical Issues of the Proposed "Global Internet Measurement
Analysis Array":
http://bit.ly/cWXH7r (Lauren's Blog)
and:
Proposal for Breaking the Internet Network Neutrality Deadlock:
http://bit.ly/9QWWYB (Lauren's Blog)
I still feel very strongly that a methodology of the type that I
suggested in those documents -- or something similar -- is an
appropriate way to collect truly meaningful broadband statistical
data, in contrast to the FCC's currently promoted approach that I
believe to be of relatively limited value.
--Lauren--
Lauren Weinstein
lauren@vortex.com
Tel: +1 (818) 225-2800
http://www.pfir.org/lauren
Co-Founder, PFIR
- People For Internet Responsibility - http://www.pfir.org
Co-Founder, NNSquad
- Network Neutrality Squad - http://www.nnsquad.org
Founder, GCTIP - Global Coalition
for Transparent Internet Performance - http://www.gctip.org
Founder, PRIVACY Forum - http://www.vortex.com
Member, ACM Committee on Computers and Public Policy
Lauren's Blog: http://lauren.vortex.com
Twitter: https://twitter.com/laurenweinstein