This guy made his own open-source Jarvis-like virtual assistant

Tanay Pant, our top Dev Diner Emerging Tech Inspiration for the Internet of Things and Artificial Intelligence in 2016, has put together his own virtual assistant called Melissa and open-sourced it for others to learn from! Here’s his story.

Tanay’s book cover and Tanay himself!

Tanay was inspired to “build a better and more natural interface between humans and the IoT”. This developer literally just felt that the IoT should be smarter and decided to put together his own Jarvis-like personal assistant like Tony Stark had in the Iron Man movies. In fact, he says the Iron Man movies were his primary source of inspiration! Thus, Melissa was born.

What is Melissa?

Melissa is a virtual assistant that runs on Windows, Linux and MacOS. It can recognise the user’s speech online via Google STT or offline via CMU’s Sphinx. It uses either MacOS’s Say, eSpeak or Ivona TTS (text-to-speech) for speaking. Currently, Tanay says that Melissa can do a whole lot of things, including:

• take notes
• retrieve system status
• tell your horoscope
• find your iPhone
• tell its battery level
• tell your system’s uptime
• list network interfaces
• control lights
• retrieve information from Wikipedia
• read business news
• tell jokes
• play music
• upload images
• tweet for you
• get weather updates
• and… self-destruct (I’m not sure if I’ll try that one…).

Tanay’s aim is to “make Melissa as personalised for the users as possible by allowing them to customise it. You can name it anything and if you mark its gender as male, it will talk back to you in a male voice”.

Why use Python?

The next big question here is — why did Tanay choose to use Python for this huge endeavour?

Tanay had good reasoning for this decision. His response —

“Python, in my opinion is the most easy to use language and it allows us to focus on the ideas rather than the programming itself. Building Melissa was my primary aim and my secondary aim was to teach others how to build their own virtual assistant from scratch. Python was the perfect language for this job also because of its huge list of wonderful modules.”

The benefits of open-source and hackathons!

Tanay’s project is open-source and actually had a whole lot of contributions during Hacktoberfest last year which pushed its capabilities even further. You can see the whole project over on GitHub right now! Tanay says that a lot of people discovered the project during Hacktoberfest because he’d used the Hacktoberfest tag on his project’s issues. This really drove people to help out with his project where he needed it most. The lesson here? Don’t be afraid to let others contribute to your project — if it’s something you can open-source, try getting a hackathon or other dev event to help spur on some contributions. Tanay had great success with this:

“The best and most memorable part of Hacktoberfest was that one of the contributors, Rachmadani Haryono helped us take the test coverage of the project up to 92%. That was a really valuable contribution to the project.” — Tanay Pant

Look at all those resolved GitHub issues!

Hey Mark, want to compare notes?

Mark Zuckerberg, the founder of Facebook, spent 2016 making his own virtual assistant he called JARVIS. He even got Morgan Freeman to voice it! When I asked Tanay about his thoughts on Mark’s creation, he said, “I can’t comment on the AI or the sophistication of the software purely on the basis of the video but it looked pretty interesting. Except for the facial recognition, Melissa too can perform all those tasks seamlessly. I’d like to invite Mark Zuckerberg to compare notes if he wants.”

So what voice would Tanay choose for his personal assistant if he could choose anyone?

“Scarlett Johansson’s voice would be the ideal voice of Melissa in my opinion. However, it would give a much more personalised and human experience if the voice was of one of my family members. To fill in this gap, I have been working on Margaery which is a custom TTS engine generator albeit highly experimental which will allow you to have personalised voices for the TTS component of Melissa.”

A custom text to speech generator sounds pretty darn cool! Tanay is also working on integrating a futuristic user interface for Melissa, named Melisandre and on integrating a full-fledged home automation system with Melissa. He’s a busy dude!

He also has a book he is working on entitled “Getting Started with Web-based Virtual Reality”, so I hope to see him integrate Melissa into a WebVR experience some day soon with Melisandre. That’d be pretty neat!

Want to get into building AI?

There are plenty of resources out there. Tanay recommends two main resources:

• Online courses on Coursera
• Building a Virtual Assistant for Raspberry Pi — his book which he says is “a good starting point for developers wanting to build cool stuff that they can even expand to construct home automation systems.”

Final pieces of advice

For those looking to get into developing for AI and the IoT, Tanay points out that it’s okay to make mistakes. In fact, it’s necessary:

“It’s good to make mistakes whether they are common or obscure, we learn more from our own mistakes while working rather than from any book or lecture.” — Tanay Pant

Finally, Tanay finished up with the ever important quote from Steve Jobs:

“Stay hungry, stay foolish!” — Steve Jobs

Thank you to Tanay for sharing his AI development story with us today! You can find out more about him on his Wikipedia page and watch his Twitter at @tanay1337. Got someone you think deserves a mention for their incredible project? Someone who’d have some great knowledge to share? Let us know!