LSU Student Team’s FarmSmart App Helps Farmers Manage Crops, Weeds With AI

Team to compete in the SEC Student Pitch Competition on October 17, hosted by the University of Tennessee, Knoxville, this week.

LSU students and alumni have created a new, AI-powered tool called FarmSmart that puts decades of LSU AgCenter research right under the green thumbs of Louisiana’s farmers. With a few clicks on their smartphones or computers, farmers and gardeners can quickly access actionable intelligence on how to best manage their crops and get rid of weeds.

“Our goal is to be as good or better than the best agricultural consultants,” said Colin Raby, LSU alumnus and CEO of FarmSmart, which won LSU’s leading pitch competition, the J. Terrell Brown Venture Challenge, last spring and now will compete in the SEC Student Pitch Competition on October 17, hosted by the University of Tennessee, Knoxville.

While there are several apps on the market that already do plant recognition based on photos, FarmSmart goes many steps further.

“You can upload a picture of a plant or weed, and our app won’t just identify them, but tell you how confident it is in its answer and provide alternatives—also, you can ask it any question about the plant, and it will provide recommendations for how to best manage that crop or weed based on the most up-to-date research, while citing the exact documents it pulled that answer from, so you can do your own fact-checking,” Raby said. “In short, we’re taking high-quality LSU agricultural research, synthesizing it using AI and large language models, and putting it in your pocket.”

Raby was one of the first students to take LSU’s Large Language Model Development and Deployment for Real-World Applications Honors course that was jointly developed by LSU Executive Vice President and Provost Roy Haggerty, Assistant Professor of Computer Science James Ghawaly, and Baton Rouge entrepreneur Henry Hays in 2023. The idea for FarmSmart came from a meeting and conversation with LSU AgCenter weed expert and associate professor Daniel Stephenson.

“Dr. Stephenson told us he’s getting farmers, especially younger farmers who are taking over family farms, coming to him asking, ‘How do I manage this thing about my crops?’ and Dr. Stephenson going, ‘Well, you have to read our guide,’” Raby said. “So, the idea of creating an interactive chatbot really came from him, to make the information in LSU AgCenter’s annual Louisiana Suggested Chemical Weed Management Guide more accessible and useful. At least, that’s where we started.”

“As a land-grant university, our research-based recommendations are free and available to the public, but the classic method of disseminating this information has been through a sizeable annual publication, either in hard copy or as an online PDF people can look at or download,” said Daniel Stephenson, weed scientist and LSU AgCenter associate professor and regional director based in Alexandria, Louisiana, where he serves as field crops coordinator for the Dean Lee Research Station. “Our primary goal has always been to help our producers be profitable. To accomplish that, the LSU AgCenter is dedicated to disseminating information in new and effective ways, and these students did an amazing job with FarmSmart.”

When he’s not working on developing FarmSmart, Raby serves as one of the nation’s first congressional AI specialists in Washington, DC, helping the legislative branch of the U.S. government work more efficiently by improving workflows and advising on policy. Raby graduated from LSU last May with a bachelor’s degree in mechanical engineering and minors in economics and aerospace engineering. He served as a teaching assistant for the second offering of LSU’s Large Language Model Development course last spring and started researching AI on his own about three years ago.

“The more I learned about AI, the deeper I knew I needed to go,” Raby said. “Everywhere I looked, I was like, why don’t we use this technology to make this better, make that better?”

With extended family who are farmers in Louisiana, Raby is proud to develop a solution that combines agriculture and AI to increase farmers’ profits while optimizing the use of pesticides and herbicides, which protects pocketbooks as well as the environment.

“We’ve incorporated massive amounts of data that no human could ever memorize,” Raby said. “First of all, it’s LSU research, but also research from other land-grant universities and data from the U.S. Department of Agriculture and the Environmental Protection Agency.”

Raby’s team includes LSU engineering students Julius Pallotta and Grant Muslow. Their biggest challenge in developing FarmSmart team was figuring out how to train a large language model on data that is structured somewhat randomly, such as herbicide and pesticide labels, for which there is no established template.

“Getting all of this information into a format the AI system could understand was actually very hard,” Raby said. “We had to build our own system for how to chunk the data.”

That’s one of the reasons Raby isn’t particularly worried about others coming along and stealing his team’s idea.

“Code isn’t patentable, so someone could, of course, come along and put in the time and effort to do what we’ve already done,” Raby said. “But from our perspective, the best protection comes from doing something really well—from the quality of the product.”

LSU Executive Vice President and Provost Roy Haggerty said FarmSmart exemplifies the kind of impact the university aims to achieve through its ongoing investment in AI and its practical applications for Louisiana.

“When we launched the Honors AI course in 2023, our goal was clear: to build a generation of experts who not only understand AI but use it to solve the unique challenges faced by our state,” Haggerty said. “FarmSmart is a perfect illustration of this vision. Colin Raby and his team’s work in developing AI-driven solutions for Louisiana farmers is an early example of how we are leveraging this emerging technology to directly contribute to the state’s economy and the well-being of its people. By the end of 2024, we will have trained nearly 50 students to tackle real-world problems with AI, addressing everything from agriculture to healthcare.”

Assistant Professor James Ghawaly, who co-teaches the Honors AI course and leads the AI-powered data science research for a $25 million collaboration between LSU and several national labs to protect the nation from nuclear threats, agreed.

“Within a short few months, Colin Raby and his team developed an entirely new AI system that will have a real impact on agriculture applications in Louisiana and beyond,” Ghawaly said. “They were able to couple state-of-the-art AI large language models and computer vision models to enable farmers to photograph a weed, automatically identify it, and recommend the appropriate herbicide to apply. Their project is a clear demonstration of the technical talent being developed through LSU’s College of Engineering.”

Project team member Grant Muslow, an LSU graduate student in electrical and computer engineering from Shreveport, Louisiana, earned his bachelor’s degree in computer science and engineering from LSU last May.

“FarmSmart is important to me because it provides farmers with access to information that would cost them thousands of dollars otherwise,” said Muslow, referencing the current state of the agricultural consulting industry. “This information should be accessible at a fair price.”

Team member Julius Pallotta from Mechanicsburg, Pennsylvania, is majoring in biological and agricultural engineering at LSU.

“Originally, my plan was to work in the automotive or aerospace industry. That was until I started to learn about the innovation in using technology to assist farmers,” Pallotta said. “The agriculture industry is the most impactful on our survival as a society and I noticed there are a lot of needs farmers have that can be addressed with automation, imaging, software solutions, and other technological advancements. Farm Smart is addressing one of the oldest and most prevalent problems farmers have always faced.”

The FarmSmart team expects their app to become available through the AppStore in coming weeks. For now, you can learn more about FarmSmart at farmsmart.ai.