This paper poses and solves a new problem, the rankability problem, which refers to a dataset's inherent ability to produce a meaningful ranking of its items. Ranking is a fundamental data science task. Its applications are numerous and include web search, data mining, cybersecurity, machine learning, and statistical learning theory. Yet little attention has been paid to the question of whether a dataset is suitable for ranking. As a result, when a ranking method is applied to an unrankable dataset, the resulting ranking may not be reliable. The rankability problem asks the following: How can rankability be quantified? At what point is a dynamic, time-evolving graph rankable? If a dataset has low rankability, can modifications be made and which most improve the graph's rankability? We present a combinatorial approach to a rankability measure and then compare several algorithms for computing this new measure. Finally, we apply our new measure to several datasets.


  1. ranking
  2. rankability
  3. linear program
  4. integer program
  5. combinatorial optimization
  6. relaxation

MSC codes

  1. 90C08
  2. 90C10
  3. 52B12
  4. 90C35

Get full access to this article

View all available purchase options and get full access to this article.

Supplementary Material

PLEASE NOTE: These supplementary files have not been peer-reviewed.

Index of Supplementary Materials

Title of paper: The Rankability of Data

Authors: Paul E. Anderson, Timothy P. Chartier, and Amy N. Langville

File: Supplementary_Materials_1.pdf

Type: PDF

Contents: An example using real data from the 2009 ACC college basketball season of games between n = 12 teams.


A. Adolfsson, M. Ackerman, and N. C. Brownstein, To cluster, or not to cluster: How to answer the question, in Proceedings of Knowledge Discovery from Data, Halifax, Nova Scotia, Canada, 2017, pp. 1--9.
I. Ali, W. D. Cook, and M. Kress, On the minimum violations ranking of a tournament, Management Sci., 32 (1986), pp. 660--672.
R. D. Armstrong, W. D. Cook, and L. M. Seiford, Priority ranking and consensus formation: The case of ties, Management Sci., 28 (1982), pp. 638--645.
C. R. Cassady, L. M. Maillart, and S. Salman, Ranking sports teams: A customizable quadratic assignment approach, Interfaces, 35 (2005), pp. 497--510.
T. P. Chartier, E. Kreutzer, A. N. Langville, and K. E. Pedings, Sensitivity and stability of ranking vectors, SIAM J. Sci. Comput., 33 (2011), pp. 1077--1102, https://doi.org/10.1137/090772745.
T. P. Chartier, A. N. Langville, K. Hutson, and M. W. Berry, Identifying influential edges in a directed network: Big events, upsets, and non-transitivity, J. Complex Netw., 2 (2014), pp. 87--109.
B. J. Coleman, Minimizing game score violations in college football rankings, Interfaces, 35 (2005), pp. 483--496.
W. D. Cook and L. M. Seiford, Priority ranking and consensus formation, Management Sci., 24 (1978), pp. 1721--1732.
K. Fukuda and A. Prodon, Double description method revisited, in Combinatorics and Computer Science, Springer, Berlin, Heidelberg, 1996, pp. 91--111.
M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, San Francisco, 1979.
M. Grötschel, M. Junger, and G. Reinelt, A cutting plane algorithm for the linear ordering problem, Oper. Res., 32 (1984), pp. 1195--1220.
X. Jiang, L.-H. Lim, Y. Yao, and Y. Ye, Statistical ranking and combinatorial Hodge theory, Math. Program., 127 (2011), pp. 203--244.
M. G. Kendall and B. B. Smith, On the method of paired comparisons, Biometrika, 31 (1940), pp. 324--345.
A. N. Langville and C. D. Meyer, Who's \#1: The Science of Rating and Ranking Items, Princeton University Press, Princeton, NJ, 2012.
A. N. Langville, K. Pedings, and Y. Yamamoto, A minimum violations ranking method, Optim. Eng., 13 (2012), pp. 349--370.
R. Marti and G. Reinelt, The Linear Ordering Problem: Exact and Heuristic Methods in Combinatorial Optimization, Appl. Math. Sci. 175, Springer, Heidelberg, 2011.
S. Mehrotra and Y. Ye, Finding an interior point in the optimal face of linear programs, Math. Programming, 62 (1993), pp. 497--515.
C. D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, Philadelphia, 2000.
J. Moore and M. Ackerman, Foundations of perturbation robust clustering, in Proceedings of the 16th International IEEE Conference on Data Mining, 2016.
T. Motzkin, H. Raiffa, G. L. Thompson, and R. M. Thrall, The double description method, in Contributions to the Theory of Games, Vol. 2, H. W. Kuhn and A. W. Tucker, eds., Princeton University Press, Princeton, NJ, 1953, pp. 51--73.
A. Newman and S. Vempala, Fences are futile: On relaxations for the linear ordering problem, in Integer Programming and Combinatorial Optimization, Springer, Berlin, Heidelberg, 2001, pp. 333--347.
M. Oswald, G. Reinelt, and H. Seitz, Applying mod-k-cuts for solving linear ordering problems, TOP, 17 (2009), pp. 158--170.
J. Park, On Minimum Violations Ranking in Paired Comparisons, preprint, https://arxiv.org/abs/physics/0510242, 2005.
G. Reinelt, The Linear Ordering Problem: Algorithms and Applications, Heldermann Verlag, Lemgo, Gemany, 1985.
G. Reinelt, M. Grötschel, and M. Jünger, Facets of the linear ordering polytope, Math. Programming, 33 (1985), pp. 43--60.
S. Schenkerman, Inducement of nonexistent order by the analytic hierarchy process, Decis. Sci., 28 (1997), pp. 475--482.

Information & Authors


Published In

cover image SIAM Journal on Mathematics of Data Science
SIAM Journal on Mathematics of Data Science
Pages: 121 - 143
ISSN (online): 2577-0187


Submitted: 26 April 2018
Accepted: 11 January 2019
Published online: 12 February 2019


  1. ranking
  2. rankability
  3. linear program
  4. integer program
  5. combinatorial optimization
  6. relaxation

MSC codes

  1. 90C08
  2. 90C10
  3. 52B12
  4. 90C35



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options


View PDF







Copy the content Link

Share with email

Email a colleague

Share on social media