HarvestAPI: A Building Block to Better Agriculture Data Interfaces

Cross-posted from original Blog here

This blog post forms part two of a three-part series in which we will be sharing the key insights and lessons learned from our work thus far re-imagining Jamaica’s agriculture infrastructure. In part one, I provided an overview of the ADS project and its underlying theory of change for addressing some of the critical challenges that prevent the development of Jamaica’s agricultural sector. In the last post, I’ll be sharing what is next for the ADS initiative and some of the key questions that the work has raised.

In Jamaica, the Rural Agricultural Development Authority (RADA), an agency within the Ministry of Agriculture, is responsible for farmer registration and the provision of extension services to the agricultural community. As the custodian of the farmer registry, RADA regularly receives information requests from a wide variety of stakeholders. These range from policy makers developing reports and briefs, banks evaluating loan credibility of farmers, and, most frequently, individuals or organizations seeking farmers who produce particular crops or other criteria. The farmer registry also plays a key role in Government’s social protection and benefit programmes targeting farmers.

Understanding existing data interfaces

However, the channels, procedures, and rules through which this information can be accessed are not always clear or well-defined. And servicing these requests can be time-consuming for RADA staff. To better understand these challenges we conducted a 6 week problem discovery exploration surveying farmers and extension officers; RADA and Ministry of Agriculture staff; and other public and private sector data consumers. We also undertook an independent service trial of the data-requesting channels identified by participants to gain a first-hand understanding of the user-experience of accessing agricultural data.

We uncovered some interesting insights on the challenges that both data consumers and data custodians faced. A subset of these are included below:

  • Data gaps have thus far been unavoidable. With an estimated 130 extension staff serving more than 180,000 registered farmers, maintaining updated data records on agricultural activity has been a costly and difficult task for RADA. While these challenges reflect the realities of operational constraints, the data gaps have also undermined the confidence of data consumers in the information RADA provides.
  • Responses to data requests vary significantly across channels and within channels over time. There is a lack of clarity around what data can be shared, who should have access to this information and what were the appropriate channels through which to access it. While data consumers understood RADA’s mandate as both a provider of technical expertise and statistical data on the agricultural sector, the experience of accessing and using data from RADA was eroding confidence in the organization’s ability to fulfill this mandate. In our research, we found that even responses to the same information requests to a specific extension officer varied over time.
  • The use of formal and informal channels to access agricultural information was common among consumers with the latter most effective for meeting information needs. However, this favoured entities with existing personal relationships and often led to frustration for new agricultural sector entrants, particularly new businesses and young innovators.
  • Data consumers identified common factors for evaluating the trustworthiness of a data source. These included: the currency of the data, the frequency of updates, accessibility (meta-data, presentation, exportability), data coverage and perceived domain expertise of the source. However, the existing data interfaces often did not provide sufficient context to support this evaluation when information was shared.
  • Friction exists between RADA’s policy and constitutional requirement to protect farmers’ personal information and the operational realities of supporting the development of the agricultural sector, with farmers as key beneficiaries. At the operational level, access to personal information was often key to facilitating market linkages, channeling agricultural support to those most in need, and the activities of partner agency activities such as anti-praedial larceny efforts. However, no overarching Government of Jamaica or Ministry of Agriculture policy exists to inform key actors on how to perform their duties in serving the organization’s mission.

In responding to these and other insights identified, it became clear that there was a need to rethink how the farmer registry was managed and the design of the interfaces (systems and people) through which it was accessed. A key outcome of these discussions was the decision to develop the HarvestAPI platform.

Extending Traditional Data Infrastructure. Designing New Interfaces

HarvestAPI is an open source farmer registry platform that facilitates the sharing of agricultural data across government agencies and with the public. In its initial release, it acts as an extension to the existing Agricultural Business Information System, which houses the farmer registry and production data, to significantly improve the accessibility of information RADA collects. 

In designing the HarvestAPI platform a key set of principles were synthesized from the user research to guide the team's design decisions. These included:

  • Open Data should be infrastructural - Typically, open data and data sharing initiatives are seen as separate activities from core data collection systems. This often leads to duplicate or extra work for already stretched public servants and invariably gets deprioritized. As a result, we worked with RADA to ensure the data publication value chain and data collection value chain were directly linked. 
  • Data context is important - Data consumers wanted context on the information that was being shared to inform how it should be interpreted. For example, it is currently not unusual for a response to an information request (e.g farmers who produce carrot) to be shared without details on when the support data records were last updated.
  • Monitoring and controlling access to the farmer registry is key to protecting farmer data rights - This control is the basis of progressive data privacy policies and consent mechanisms which enable RADA to track how and who accesses the farmer registry information and lays the groundwork for empowering farmers to understand how their information is being used.
  • Data access channels should be transparent, and, where possible, frictionless - Guidelines about what information is available and how it can be accessed should be transparent and easy to understand. Digital tools and platforms can also assist in making responding easier for both data providers and consumers.

Image of the Developer Dashboard

Looking ahead

RADA is currently piloting the HarvestAPI platform in a private Beta with data consumers. In parallel, the SlashRoots team is working with the Praedial Larceny Prevention Unit to apply these design principles and other lessons from the research to revise the data interface used by police officers during their anti-praedial field operations. The research and the adoption of the HarvestAPI platform have also raised important questions that will need to be tackled.

  • While registers are expected to be authoritative, canonical references on specific topics, how do these characteristics translate to an environment where resource and operational realities make them impossible for any single entity to fulfill? Can distributed or shared data infrastructure aid in spreading the burden of data collection and maintenance across multiple actors to increase scale? How does the role of the registry custodian change in this scenario?
  • How do you operationalize data rights in resource-constrained and low literacy environments? What role can ^“trusted?” intermediaries play in bridging the gap?

By Matthew McNaughton, Executive Director of the SlashRoots Foundation, a civic tech non-profit building technology and communities to address development issues in the Caribbean.