Introduction

Hello guys! Whether we've crossed paths before or not, let's get acquainted. I'm Malik Talha, a final-year Computer Science student at National University of Computer & Emerging Sciences (FAST - NUCES) in Islamabad. I am the epitome of a model student, tirelessly dedicating my days to the betterment of society - Can you even imagine? 😂. Well, let's cut to the chase and drop the act, shall we? In reality, I'm just like any other person out there — I indulge in gaming, get lost in the captivating world of anime, and probably spend a bit too much time aimlessly scrolling through the vast abyss of the internet (Gotta make the time wasted on shorts and reels sound cool xD).

Everything aside, here's a delightful surprise: amidst all that, I've discovered a true passion for designing and developing innovative software applications. This passion has always led me into unknown origins and sometimes situations with no exit (referring to my time doing semester projects at university). Should I let you in on one of those captivating incidents? Well, brace yourself because it might just turn into a delightful time-waster. But hey, it's my personal blog, I am allowed to do whatever I want 😉. (You can skip the following paragraph if you're not interested to know about the incident - your loss ☹️)

In my fourth semester at university, I enrolled in the intriguing course "Operating Systems." As part of the curriculum, we were tasked with developing a 2D Ludo game using C++, OpenGL, and OS concepts. In a daring move, I decided to explore the realm of 3D game development, despite lacking experience, a tight deadline, and the project's weight on my final grade. Undeterred, I dove headfirst into learning about game development, graphics pipelines, shaders, and 2D/3D rendering. Alongside my studies, I crafted the foundations of my own "Game Engine" to programmatically render graphics and load 3D models. However, as the deadline loomed just three days away, panic struck. I realized I hadn't actually developed the Ludo game itself—a critical oversight! With time slipping away, I made the tough decision to pivot back to a 2D version and meet the deadline. Fortunately, my group mate came to my rescue, contributing to 50% of the game's development just in time. This experience taught me valuable lessons about balancing ambition with practicality, adapting under pressure, and the power of teamwork. On the bright side I was able to develop a bare minimum game engine graphics rendering pipeline which you can find on my GitHub.

Exploring the depths of the unknown has always been a catalyst for my growth and expertise. Continuing my passion for software development, I decided to apply for Google Summer of Code 2023. This prestigious and highly competetive program by Google offers students and new developers a chance to enhance their programming skills and engage with open-source communities globally. I'm delighted to share that I've been selected 🥳 for The Linux Foundation's Automotive Grade Linux (AGL) project—an exceptional opportunity to contribute to a standardized and customizable software platform for automotive applications. Join me on this exciting journey as I collaborate with the talented developers at Linux Foundation! I will be working under the mentorship of Linux Foundation members: Jan-Simon Möller (Release Manager) and Walt Miner (Community Manager). In the following section I'll dive deeper into my selection process.

Selection Process

Every March, GSoC reveals the Open Source organizations participating on their website, offering project details and seeking developers. Interested individuals submit detailed proposals outlining their project completion plans before the deadline which is usually first week of April. Mentors from participating organizations evaluate the proposals, and the winners are announced in May. The coding period begines after the winners are announced.

In my application process, the organization I applied to (The Linux Foundation) introduced a filtering quiz. It required setting up Automotive Grade Linux (AGL) OS locally, developing and publishing a basic app within the AGL environment. The quiz proved challenging, but with the invaluable guidance from the Linux Foundation, I successfully completed it within the given timeframe. Based on my quiz performance and the quality of my proposal I got selected into GSoC 2023.

Community Bonding

Following my selection into GSoC, the community bonding period commenced in May. This phase allows GSoC contributors to establish connections with their mentors and familiarize themselves with the organization. Currently, I am engaging in separate weekly meetings with my mentors and the AGL community to deepen my understanding of their expertise and grasp the development workflow of AGL. The AGL community comprises of exceptional individuals from whom I anticipate valuable learning experiences. It is a privilege to be a part of this remarkable community. Once the bonding period concludes, I will start work on my designated project.

My Project for GSoC 2023

Introductions and details about GSoC aside, let's discuss about what project I will be working on. Before I dive into my project details further, I'll give a brief description about AGL:

Automotive Grade Linux (AGL) is an open-source software platform specifically designed for automotive applications. It provides a standardized and customizable solution for connected cars and infotainment systems. AGL offers a comprehensive set of features, including a Linux-based operating system, middleware, and application framework. It enables automotive manufacturers and developers to collaborate, accelerate innovation, and create secure and reliable software for vehicles. AGL fosters an ecosystem of industry stakeholders, promoting interoperability and driving the adoption of open-source software in the automotive industry.

Overview

My project is to enhance the existing speech recognition engine of AGL by developing and integrating a natural language intent engine that understands and executes users' voice commands.

Goals and Objectives

The goal of my project is to develop a car's voice assistant which can understand voice commands and take necessary action based on them (e.g. turn down the volume, increase AC temperature, play a song, etc.). There are two main objectives of my project:

• Objective # 01: Integration and development of Natural Language Understanding Intent Engine with existing Vosk implementation.
• Objective # 02: Ability to execute the intent extracted from the NLU engine using some sort of interface that will communicate with the APIs.

Tools and Technologies

I will be working with the following tools and technologies as part of my project:

• Linux: AGL is built on a Linux-based operating system.
• Yocto Project: AGL utilizes the Yocto Project build system for creating customized Linux distributions.
• Bitbake: Bitbake is a build tool used in the Yocto Project for creating customized Linux distributions.
• Wayland/Weston: AGL uses the Wayland protocol and Weston compositor for graphical display and user interface rendering.
• Flutter: Flutter is a cross-platform framework for building native mobile and web applications.
• C/C++: AGL involves developing software components and applications using C/C++.
• Python: The Natural Language Understanding Intent engine will be developed in python.
• Collaborative Tools: Communication and collaboration are vital in open-source projects. Tools like mailing lists, IRC (Internet Relay Chat), and project management platforms (e.g., Jira) are commonly used in the AGL community.

Challenges and Expectations

I am expecting to face following challenges:

• Natural Language Understanding: Understanding the intent behind user commands requires robust natural language understanding (NLU) capabilities. NLU involves parsing and extracting relevant information from user inputs, which can be challenging, especially when dealing with complex or ambiguous commands.
• Integration with Car APIs: To execute commands such as adjusting car volume, your voice assistant needs to interact with the underlying car APIs and systems. Understanding and integrating with these APIs may require in-depth knowledge of the car's software architecture and protocols.
• Context Awareness: A voice assistant should be context-aware to provide relevant and accurate responses. Understanding the context of the user's request, such as the current state of the car or previous interactions, can enhance the user experience but requires careful implementation.
• Performance and Latency: Voice assistants should respond quickly and perform tasks efficiently to provide a smooth user experience. Optimizing performance and minimizing latency can be challenging, especially when dealing with resource-constrained embedded systems.

Conclusion

In conclusion, my journey into the world of GSoC 2023 and Automotive Grade Linux has been nothing short of exhilarating. From the thrill of being selected to the engaging community bonding period, I am ready to embark on this remarkable project. With the support of my mentors and the vibrant AGL community, I am confident that this experience will be filled with learning, growth, and exciting developments. Stay tuned as I dive into the realm of voice assistants, bringing innovation to the automotive landscape. Thank you for joining me on this adventure, and let's make some magic happen together!

(P.S. If you're interested about the actual proposal I submitted for GSoC 2023 let me know and I'll share it as a blog post.)