An interpretable produce price forecasting system for small and marginal farmers in India using collaborative filtering and adaptive nearest neighbors

Wei Ma, Kendall Nowocin, Niraj Marathe, George H. Chen

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

4 Citations (Scopus)

Abstract

Small and marginal farmers, who account for over 80% of India's agricultural population, often sell their harvest at low, unfavorable prices before spoilage. These farmers often lack access to either cold storage or market forecasts. In particular, by having access to cold storage, farmers can store their produce for longer and thus have more flexibility as to when they should sell their harvest by. Meanwhile, by having access to market forecasts, farmers can more easily identify which markets to sell at and when. While affordable cold storage solutions have become more widely available, there has been less work on produce price forecasting. A key challenge is that in many regions of India, predominantly in rural and remote areas, we have either very limited or no produce pricing data available from public online sources. In this paper, we present a produce price forecasting system that pulls data from the Indian Ministry of Agriculture and Farmers Welfare's website Agmarknet, trains a model of prices using over a thousand markets, and displays interpretable price forecasts in a web application viewable from a mobile phone. Due to the pricing data being extremely sparse, our method first imputes missing entries using collaborative filtering to obtain a dense dataset. Using this imputed dense dataset, we then train a decision-tree-based classifier to predict whether the price for a specific produce at a specific market will go up, stay the same, or go down. In terms of interpretability, we display the most relevant historical pricing data that drive each forecasted price trend, where we take advantage of the fact that a wide family of decision-tree-based ensemble learning methods are adaptive nearest neighbor methods. We also show how our approach generalizes to forecasting exact produce prices and constructing heuristic price uncertainty intervals. We validate forecast accuracy on data from Agmarknet and a small field survey of a few markets in Odisha.

Original languageEnglish
Title of host publicationProceedings of the 10th International Conference on Information and Communication Technologies and Development, ICTD 2019
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450361224
DOIs
Publication statusPublished - 4 Jan 2019
Externally publishedYes
Event10th International Conference on Information and Communication Technologies and Development, ICTD 2019 - Ahmedabad, India
Duration: 4 Jan 20197 Jan 2019

Publication series

NameACM International Conference Proceeding Series

Conference

Conference10th International Conference on Information and Communication Technologies and Development, ICTD 2019
Country/TerritoryIndia
CityAhmedabad
Period4/01/197/01/19

Keywords

  • Collaborative filtering
  • ICTD
  • India
  • Nearest neighbors
  • Price forecasting
  • Small and marginal farmers
  • Web application

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Cite this