Introduction

Welcome to my weekly report documenting my journey during Google Summer of Code 2023 with the Linux Foundation! In this project, I am working on enhancing the existing speech-to-text feature of Automotive Grade Linux (AGL) by introducing a Natural Language Intent engine and implementing software daemons/controllers to execute the extracted intent. This endeavor aims to significantly improve the user experience and functionality of the speech-to-text feature in automotive environments. Throughout this report, I will share my progress, challenges faced, and achievements made as I contribute to the development of AGL and pave the way for more intuitive and intelligent voice interactions in automobiles.

Summary of the week

This week was highly productive and successful for my GSoC project. I tackled and resolved the issues related to the "snips-nlu-parsers" and "snips-nlu-utils" recipes (thanks to the help from my mentors), successfully building them. Additionally, I dedicated time to crafting a dataset and training the NLU Engine model for crucial intents such as Dashboard Turn Signal, HVAC Temperature, HVAC Fan Speed, Media Player (Play, Pause, Prev, Next), Navigation (To/Stop), and Car Window Controls. Moreover, I expanded the project's capabilities by adding the "python-crfsuite" and "sklearn-crfsuite" recipes.

Tasks completed

• Fixed rust's network based issues and built the following recipes successfully:
  • snips-nlu-parsers
  • snips-nlu-utils
• Crafted dataset and trained NLU Engine model for the following intents:
  • Dashboard Turn Signal
  • HVAC Temperature
  • HVAC Fan Speed
  • Media Player
  • Navigation (requires more work)
  • Car Window Controls
• Added following new recipes to complete the Snips NLU dependencies:
  • python-crfsuite
  • sklearn-crfsuite

The above fixed and new recipes have been incorporated into the existing meta-offline-voice-agent. The trained NLU Engine along with its dataset can be found at this GitHub repository.

Tasks leftover

The following tasks were leftover this week:

• SciPy and Scikit-Learn recipe errors need to be resolved. (requires assistance from mentors)

Next steps

Looking ahead to the next week, I have identified the following tasks as my focus:

• Complete building of RASA: A crucial component of the project that requires careful attention, configuration, and more dependency recipes.
• Integrate Snips NLU: Finalize the integration of the Snips NLU into the meta-offline-voice-agent layer, ensuring seamless communication and functionality between the components.

Conclusion

Overall, this week was productive, and I am satisfied with the progress made in achieving the goals outlined for the week. I am excited to continue my GSoC journey and further enhance the speech-to-text feature in Automotive Grade Linux.

Resources

There were no resources found attached to this post.