Multi-dataset Low-rank Matrix Factorization

Hossein Valavi, Peter Jeffrey Ramadge

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Low-rank matrix factorization can reveal fundamental structure in data. For example, joint-PCA on multi-datasets can find a joint, lower-dimensional representation of the data. Recently other similar matrix factorization methods have been introduced for multi-dataset analysis, e.g., the shared response model (SRM) and hyperalignment (HA). We provide a comparison of these methods with joint-PCA that highlights similarities and differences. Necessary and sufficient conditions under which the solution set to SRM and HA can be derived from the joint-PCA are identified. In particular, if there exists a common template and a set of generalized rotation matrices through which datasets can be exactly aligned to the template, then for any number of features, SRM and HA solutions can be readily derived from the joint-PCA of datasets. Not surprisingly, this assumption fails to hold for complex multi-datasets, e.g., multi-subject fMRI datasets. We show that if the desired conditions are not satisfied, joint-PCA can easily over-fit to the training data when the dimension of the projected space is high (∼> 50). We also examine how well low-dimensional matrix factorization can be computed using gradient descent-type algorithms using Google's TensorFlow library.

Original languageEnglish (US)
Title of host publication2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728111513
DOIs
StatePublished - Apr 16 2019
Event53rd Annual Conference on Information Sciences and Systems, CISS 2019 - Baltimore, United States
Duration: Mar 20 2019Mar 22 2019

Publication series

Name2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019

Conference

Conference53rd Annual Conference on Information Sciences and Systems, CISS 2019
CountryUnited States
CityBaltimore
Period3/20/193/22/19

Fingerprint

Factorization

All Science Journal Classification (ASJC) codes

  • Information Systems

Cite this

Valavi, H., & Ramadge, P. J. (2019). Multi-dataset Low-rank Matrix Factorization. In 2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019 [8692932] (2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CISS.2019.8692932
Valavi, Hossein ; Ramadge, Peter Jeffrey. / Multi-dataset Low-rank Matrix Factorization. 2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019).
@inproceedings{9b28cb17aa014019a00944c7640e2e7e,
title = "Multi-dataset Low-rank Matrix Factorization",
abstract = "Low-rank matrix factorization can reveal fundamental structure in data. For example, joint-PCA on multi-datasets can find a joint, lower-dimensional representation of the data. Recently other similar matrix factorization methods have been introduced for multi-dataset analysis, e.g., the shared response model (SRM) and hyperalignment (HA). We provide a comparison of these methods with joint-PCA that highlights similarities and differences. Necessary and sufficient conditions under which the solution set to SRM and HA can be derived from the joint-PCA are identified. In particular, if there exists a common template and a set of generalized rotation matrices through which datasets can be exactly aligned to the template, then for any number of features, SRM and HA solutions can be readily derived from the joint-PCA of datasets. Not surprisingly, this assumption fails to hold for complex multi-datasets, e.g., multi-subject fMRI datasets. We show that if the desired conditions are not satisfied, joint-PCA can easily over-fit to the training data when the dimension of the projected space is high (∼> 50). We also examine how well low-dimensional matrix factorization can be computed using gradient descent-type algorithms using Google's TensorFlow library.",
author = "Hossein Valavi and Ramadge, {Peter Jeffrey}",
year = "2019",
month = "4",
day = "16",
doi = "https://doi.org/10.1109/CISS.2019.8692932",
language = "English (US)",
series = "2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019",
address = "United States",

}

Valavi, H & Ramadge, PJ 2019, Multi-dataset Low-rank Matrix Factorization. in 2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019., 8692932, 2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019, Institute of Electrical and Electronics Engineers Inc., 53rd Annual Conference on Information Sciences and Systems, CISS 2019, Baltimore, United States, 3/20/19. https://doi.org/10.1109/CISS.2019.8692932

Multi-dataset Low-rank Matrix Factorization. / Valavi, Hossein; Ramadge, Peter Jeffrey.

2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8692932 (2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Multi-dataset Low-rank Matrix Factorization

AU - Valavi, Hossein

AU - Ramadge, Peter Jeffrey

PY - 2019/4/16

Y1 - 2019/4/16

N2 - Low-rank matrix factorization can reveal fundamental structure in data. For example, joint-PCA on multi-datasets can find a joint, lower-dimensional representation of the data. Recently other similar matrix factorization methods have been introduced for multi-dataset analysis, e.g., the shared response model (SRM) and hyperalignment (HA). We provide a comparison of these methods with joint-PCA that highlights similarities and differences. Necessary and sufficient conditions under which the solution set to SRM and HA can be derived from the joint-PCA are identified. In particular, if there exists a common template and a set of generalized rotation matrices through which datasets can be exactly aligned to the template, then for any number of features, SRM and HA solutions can be readily derived from the joint-PCA of datasets. Not surprisingly, this assumption fails to hold for complex multi-datasets, e.g., multi-subject fMRI datasets. We show that if the desired conditions are not satisfied, joint-PCA can easily over-fit to the training data when the dimension of the projected space is high (∼> 50). We also examine how well low-dimensional matrix factorization can be computed using gradient descent-type algorithms using Google's TensorFlow library.

AB - Low-rank matrix factorization can reveal fundamental structure in data. For example, joint-PCA on multi-datasets can find a joint, lower-dimensional representation of the data. Recently other similar matrix factorization methods have been introduced for multi-dataset analysis, e.g., the shared response model (SRM) and hyperalignment (HA). We provide a comparison of these methods with joint-PCA that highlights similarities and differences. Necessary and sufficient conditions under which the solution set to SRM and HA can be derived from the joint-PCA are identified. In particular, if there exists a common template and a set of generalized rotation matrices through which datasets can be exactly aligned to the template, then for any number of features, SRM and HA solutions can be readily derived from the joint-PCA of datasets. Not surprisingly, this assumption fails to hold for complex multi-datasets, e.g., multi-subject fMRI datasets. We show that if the desired conditions are not satisfied, joint-PCA can easily over-fit to the training data when the dimension of the projected space is high (∼> 50). We also examine how well low-dimensional matrix factorization can be computed using gradient descent-type algorithms using Google's TensorFlow library.

UR - http://www.scopus.com/inward/record.url?scp=85065180310&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065180310&partnerID=8YFLogxK

U2 - https://doi.org/10.1109/CISS.2019.8692932

DO - https://doi.org/10.1109/CISS.2019.8692932

M3 - Conference contribution

T3 - 2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019

BT - 2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Valavi H, Ramadge PJ. Multi-dataset Low-rank Matrix Factorization. In 2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8692932. (2019 53rd Annual Conference on Information Sciences and Systems, CISS 2019). https://doi.org/10.1109/CISS.2019.8692932