Introduction

Welcome to my weekly report documenting my journey during Google Summer of Code 2023 with the Linux Foundation! In this project, I am working on enhancing the existing speech-to-text feature of Automotive Grade Linux (AGL) by introducing a Natural Language Intent engine and implementing software daemons/controllers to execute the extracted intent. This endeavor aims to significantly improve the user experience and functionality of the speech-to-text feature in automotive environments. Throughout this report, I will share my progress, challenges faced, and achievements made as I contribute to the development of AGL and pave the way for more intuitive and intelligent voice interactions in automobiles.

Summary of the week

This week marked significant progress in the Voice Assistant Flutter App and Voice Agent gRPC Service. The Flutter app now includes a Dart client for streamlined communication with the gRPC server, improved UI elements for enhanced user experience, and robust error handling. Meanwhile, the Voice Agent gRPC Service saw notable advancements in GStreamer pipeline management, the addition of crucial RPC methods, and optimizations for improved performance. Project documentation has also been initiated. The next steps involve rigorous testing on the AGL target machine (x86) and continued documentation efforts to ensure project success.

Tasks completed

• Voice Assistant Flutter App

  • Dart Client Development: Developed a Dart client within the Flutter app to facilitate communication with the gRPC server. This client ensures efficient data exchange between the app and the service.
  • Integration with gRPC Server: Successfully integrated the Flutter app with the gRPC server, establishing a robust connection for seamless voice assistant functionality.
  • UI Enhancement: Improved and enhanced the user interface for both "manual" and "wake word" modes. This enhancement aims to provide a more user-friendly experience.
  • Configuration Handling: Implemented a configuration handler capable of loading gRPC configuration parameters (e.g., port, host) from a config.json file, enhancing configurability.
  • Error Handling: Introduced robust error handling mechanisms throughout the Flutter app to handle various scenarios gracefully. The app now verifies the availability of the gRPC server before proceeding.

• Voice Agent gRPC Service

  • GStreamer Pipeline Improvements: Enhanced the creation and deallocation methods of the GStreamer pipeline in the Voice Agent gRPC Service. These improvements contribute to more efficient resource management.
  • Service Restructuring: Restructured the service to incorporate additional RPC methods, such as ServiceStatus (for checking service availability) and ExecuteCommand (for executing intents using Kuksa), expanding the service's functionality.
  • Optimizations and Bug Fixes: Performed optimizations on the overall service, addressing identified bugs to ensure its reliability and performance.

Flutter App UI

Tasks leftover

No tasks were leftover this week.

Next steps

In the upcoming week, the following tasks will be undertaken:

• Testing on AGL Target Machine (x86): Conduct thorough testing of the Flutter app on the AGL target machine (x86). Consider exploring Flutter workspace automation further to streamline the testing process and ensure compatibility.
• Documentation Continuation: Continue working on the comprehensive documentation of the project. This documentation will serve as a valuable resource for understanding the project's architecture, usage, and configuration.

These next steps align with the project's goals and will contribute to its successful completion.

Conclusion

Overall, this week was productive, and I am satisfied with the progress made in achieving the goals outlined for the week. I am excited to continue my GSoC journey and further enhance the speech-to-text feature in Automotive Grade Linux.

Resources

There were no resources found attached to this post.