TY - GEN
T1 - Picture-to-amount (PITA)
T2 - 25th International Conference on Pattern Recognition, ICPR 2020
AU - Li, Jiatong
AU - Han, Fangda
AU - Guerrero, Ricardo
AU - Pavlovic, Vladimir
N1 - Publisher Copyright: © 2020 IEEE
PY - 2020
Y1 - 2020
N2 - Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems. Although these systems may recognize the ingredients, a detailed analysis of their amounts in the meal, which is paramount for estimating the correct nutrition, is usually ignored. In this paper, we study the novel and challenging problem of predicting the relative amount of each ingredient from a food image. We propose PITA, the Picture-to-Amount deep learning architecture to solve the problem. More specifically, we predict the ingredient amounts using a domain-driven Wasserstein loss from image-to-recipe cross-modal embeddings learned to align the two views of food data. Experiments on a dataset of recipes collected from the Internet show the model generates promising results and improves the baselines on this challenging task. A demo of our system and our data is available at: foodai.cs.rutgers.edu.
AB - Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems. Although these systems may recognize the ingredients, a detailed analysis of their amounts in the meal, which is paramount for estimating the correct nutrition, is usually ignored. In this paper, we study the novel and challenging problem of predicting the relative amount of each ingredient from a food image. We propose PITA, the Picture-to-Amount deep learning architecture to solve the problem. More specifically, we predict the ingredient amounts using a domain-driven Wasserstein loss from image-to-recipe cross-modal embeddings learned to align the two views of food data. Experiments on a dataset of recipes collected from the Internet show the model generates promising results and improves the baselines on this challenging task. A demo of our system and our data is available at: foodai.cs.rutgers.edu.
UR - http://www.scopus.com/inward/record.url?scp=85110495176&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85110495176&partnerID=8YFLogxK
U2 - https://doi.org/10.1109/ICPR48806.2021.9412828
DO - https://doi.org/10.1109/ICPR48806.2021.9412828
M3 - Conference contribution
T3 - Proceedings - International Conference on Pattern Recognition
SP - 10343
EP - 10350
BT - Proceedings of ICPR 2020 - 25th International Conference on Pattern Recognition
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 10 January 2021 through 15 January 2021
ER -