Abstract
This paper poses and solves a new problem, the rankability problem, which refers to a dataset's inherent ability to produce a meaningful ranking of its items. Ranking is a fundamental data science task. Its applications are numerous and include web search, data mining, cybersecurity, machine learning, and statistical learning theory. Yet little attention has been paid to the question of whether a dataset is suitable for ranking. As a result, when a ranking method is applied to an unrankable dataset, the resulting ranking may not be reliable. The rankability problem asks the following: How can rankability be quantified? At what point is a dynamic, time-evolving graph rankable? If a dataset has low rankability, can modifications be made and which most improve the graph's rankability? We present a combinatorial approach to a rankability measure and then compare several algorithms for computing this new measure. Finally, we apply our new measure to several datasets.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
Supplementary Material
PLEASE NOTE: These supplementary files have not been peer-reviewed.
Index of Supplementary Materials
Title of paper: The Rankability of Data
Authors: Paul E. Anderson, Timothy P. Chartier, and Amy N. Langville
File: Supplementary_Materials_1.pdf
Type: PDF
Contents: An example using real data from the 2009 ACC college basketball season of games between n = 12 teams.
References
1.
A. Adolfsson, M. Ackerman, and N. C. Brownstein, To cluster, or not to cluster: How to answer the question, in Proceedings of Knowledge Discovery from Data, Halifax, Nova Scotia, Canada, 2017, pp. 1--9.
2.
I. Ali, W. D. Cook, and M. Kress, On the minimum violations ranking of a tournament, Management Sci., 32 (1986), pp. 660--672.
3.
R. D. Armstrong, W. D. Cook, and L. M. Seiford, Priority ranking and consensus formation: The case of ties, Management Sci., 28 (1982), pp. 638--645.
4.
C. R. Cassady, L. M. Maillart, and S. Salman, Ranking sports teams: A customizable quadratic assignment approach, Interfaces, 35 (2005), pp. 497--510.
5.
T. P. Chartier, E. Kreutzer, A. N. Langville, and K. E. Pedings, Sensitivity and stability of ranking vectors, SIAM J. Sci. Comput., 33 (2011), pp. 1077--1102, https://doi.org/10.1137/090772745.
6.
T. P. Chartier, A. N. Langville, K. Hutson, and M. W. Berry, Identifying influential edges in a directed network: Big events, upsets, and non-transitivity, J. Complex Netw., 2 (2014), pp. 87--109.
7.
B. J. Coleman, Minimizing game score violations in college football rankings, Interfaces, 35 (2005), pp. 483--496.
8.
W. D. Cook and L. M. Seiford, Priority ranking and consensus formation, Management Sci., 24 (1978), pp. 1721--1732.
9.
K. Fukuda and A. Prodon, Double description method revisited, in Combinatorics and Computer Science, Springer, Berlin, Heidelberg, 1996, pp. 91--111.
10.
M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, San Francisco, 1979.
11.
M. Grötschel, M. Junger, and G. Reinelt, A cutting plane algorithm for the linear ordering problem, Oper. Res., 32 (1984), pp. 1195--1220.
12.
X. Jiang, L.-H. Lim, Y. Yao, and Y. Ye, Statistical ranking and combinatorial Hodge theory, Math. Program., 127 (2011), pp. 203--244.
13.
M. G. Kendall and B. B. Smith, On the method of paired comparisons, Biometrika, 31 (1940), pp. 324--345.
14.
A. N. Langville and C. D. Meyer, Who's \#1: The Science of Rating and Ranking Items, Princeton University Press, Princeton, NJ, 2012.
15.
A. N. Langville, K. Pedings, and Y. Yamamoto, A minimum violations ranking method, Optim. Eng., 13 (2012), pp. 349--370.
16.
R. Marti and G. Reinelt, The Linear Ordering Problem: Exact and Heuristic Methods in Combinatorial Optimization, Appl. Math. Sci. 175, Springer, Heidelberg, 2011.
17.
S. Mehrotra and Y. Ye, Finding an interior point in the optimal face of linear programs, Math. Programming, 62 (1993), pp. 497--515.
18.
C. D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, Philadelphia, 2000.
19.
J. Moore and M. Ackerman, Foundations of perturbation robust clustering, in Proceedings of the 16th International IEEE Conference on Data Mining, 2016.
20.
T. Motzkin, H. Raiffa, G. L. Thompson, and R. M. Thrall, The double description method, in Contributions to the Theory of Games, Vol. 2, H. W. Kuhn and A. W. Tucker, eds., Princeton University Press, Princeton, NJ, 1953, pp. 51--73.
21.
A. Newman and S. Vempala, Fences are futile: On relaxations for the linear ordering problem, in Integer Programming and Combinatorial Optimization, Springer, Berlin, Heidelberg, 2001, pp. 333--347.
22.
M. Oswald, G. Reinelt, and H. Seitz, Applying mod-k-cuts for solving linear ordering problems, TOP, 17 (2009), pp. 158--170.
23.
J. Park, On Minimum Violations Ranking in Paired Comparisons, preprint, https://arxiv.org/abs/physics/0510242, 2005.
24.
G. Reinelt, The Linear Ordering Problem: Algorithms and Applications, Heldermann Verlag, Lemgo, Gemany, 1985.
25.
G. Reinelt, M. Grötschel, and M. Jünger, Facets of the linear ordering polytope, Math. Programming, 33 (1985), pp. 43--60.
26.
S. Schenkerman, Inducement of nonexistent order by the analytic hierarchy process, Decis. Sci., 28 (1997), pp. 475--482.
Information & Authors
Information
Published In
SIAM Journal on Mathematics of Data Science
Pages: 121 - 143
DOI: 10.1137/18M1183595
ISSN (online): 2577-0187
Copyright
© 2019, Society for Industrial and Applied Mathematics.
History
Submitted: 26 April 2018
Accepted: 11 January 2019
Published online: 12 February 2019
Keywords
MSC codes
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- A supervised learning approach to rankabilityComputers & Operations Research, Vol. 1 | 1 Mar 2025
- Neural Improvement Heuristics for Graph Combinatorial Optimization ProblemsIEEE Transactions on Neural Networks and Learning Systems, Vol. 35, No. 12 | 1 Dec 2024
- A rankability-based fuzzy decision making procedure for oil supplier selectionApplied Soft Computing, Vol. 149 | 1 Dec 2023
- Rankability and linear ordering problem: Probabilistic insight and algorithmsComputers & Operations Research, Vol. 159 | 1 Nov 2023
- Developing a Ranking Problem Library (RPLIB) from a data-oriented perspectiveOptimization and Engineering, Vol. 24, No. 3 | 23 September 2022
- Genetic Algorithm-enhanced Rank aggregation model to measure the performance of Pulp and Paper IndustriesComputers & Industrial Engineering, Vol. 172 | 1 Oct 2022
- Fairness and the set of optimal rankings for the linear ordering problemOptimization and Engineering, Vol. 23, No. 3 | 23 June 2021
- The rankability of weighted data from pairwise comparisonsFoundations of Data Science, Vol. 3, No. 1 | 1 Jan 2021
- On the linear ordering problem and the rankability of dataFoundations of Data Science, Vol. 3, No. 2 | 1 Jan 2021
- Ranking with hamiltonian dynamicsPhysica D: Nonlinear Phenomena, Vol. 413 | 1 Dec 2020
- On the graph Laplacian and the rankability of dataLinear Algebra and its Applications, Vol. 588 | 1 Mar 2020
- Erratum: The Rankability of DataSIAM Journal on Mathematics of Data Science, Vol. 1, No. 3 | 26 September 2019
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].