Voice-controlled app to help track calorie intake

MIT researchers have developed a new voice-controlled app that may help obese people to lose weight by logging calorie counts and other nutritional information after every meal.

The system parses the description and automatically retrieves the pertinent nutritional data from an online database maintained by the US Department of Agriculture (USDA), researchers said.

The data is displayed together with images of the corresponding foods and pull-down menus that allow the user to refine their descriptions - selecting, for instance, precise quantities of food. But those refinements can also be made verbally, researchers said.

A user who begins by saying, "for breakfast, I had a bowl of oatmeal, bananas, and a glass of orange juice" can then make the amendment, "I had half a banana," and the system will update the data it displays about bananas while leaving the rest unchanged, they said.

Researchers at the Massachusetts Institute of Technology (MIT) concentrated on two problems. One is identifying words' functional role - the system needs to recognise that if the user records the phrase "bowl of oatmeal," nutritional information on oatmeal is pertinent, but if the phrase is "oatmeal cookie," it is not.

The other problem is reconciling the user's phrasing with the entries in the USDA database. For instance, the USDA data on oatmeal is recorded under the heading "oats"; the word "oatmeal" shows up nowhere in the entry.

To address the first problem, researchers used machine learning. Through the Amazon Mechanical Turk crowd-sourcing platform, they recruited workers who simply described what they had eaten at recent meals, then labelled the pertinent words in the description as names of foods, quantities, brand names, or modifiers of the food names.

In "bowl of oatmeal," "bowl" is a quantity and "oatmeal" is a food, but in "oatmeal cookie," oatmeal is a modifier. Once they had roughly 10,000 labelled meal descriptions, researchers used machine-learning algorithms to find patterns in the syntactic relationships between words that would identify their functional roles.

To translate between users' descriptions and the labels in the USDA database, the researchers used an open-source database called Freebase, which has entries on more than 8,000 common food items, many of which include synonyms.

"I think logging is enormously helpful for many people. A spoken-language system that you can use with your phone would allow people to log food wherever they are eating it, with less work," said Susan Roberts from Tufts University which came up with the idea of the spoken-language app.