The technologies associated with the Internet of Things have great potential for application in the domain of food and agriculture, especially in view of the societal and environmental challenges faced by this sector. From farm to fork, IoT technologies could transform the sector, contributing to food safety, and the reduction of agricultural inputs and food waste. A major step toward greater uptake of these technologies will be the execution of IoT-based large-scale pilots (LSPs) in the entire supply chain. This article outlines the challenges and constraints that an LSP deployment of IoT in this domain must consider. Sectoral and technological challenges are described in order to identify a set of technological and agrifood requirements. An architecture based on a system of systems approach is briefly presented, the importance of addressing the interoperability challenges faced by this sector is highlighted, and we elaborate on requirements for new business models, security, privacy, and data governance. A description of the technologies and solutions involved in designing pilots for four agrifood domains (dairy, fruit, arable, meat and vegetable supply chain) is eventually provided. In conclusion, it is noted that for IoT to be successful in this domain, a significant change of culture is needed.