A thought on improving voice user interface while ensuring privacy

Voice user interface is going to be one of the ways we interact with our devices as we go about our daily lives. It is just a very intuitive way for us because we communicate primarily via voice with text and images to complement.

But there still are various problems that need people to work on them to improve the overall experience. One of it is related to how the AI behind voice user interface can interact with us more naturally, like how we interact with fellow human beings.

A premium Medium article written by Cheryl Platz got me thinking about that. It also covered a little on privacy and why it is a contributing factor that make it difficult for current generation of AIs to speak more naturally and understand the context when we speak. Unless, companies don’t give a shit about our privacy and start collecting even more data.

In this article, I am going to share what I thought could help improve the AI and ensure user privacy.

Current Implementations and Limitations

What an AI needs to be better at understanding and responding in ways most useful to us are processing power, a good neural network that allows it to self-learn, and a database to store whatever it has learnt.

The cloud is the best way for an AI to gain access to a processing power and huge enough database. Companies like Amazon and Microsoft offer cloud computing and storage services via their AWS and Azure platform respectively at very low cost. Even Google offers such services via their Compute Engine.

The problem with the cloud is reduced level of confidence when privacy is involved. Anything you store up there is vulnerable, available for retrieval through security flaws or misconfigurations. Companies could choose to encrypt those data via end-to-end encryption to help with protect user’s privacy but the problem is the master keys are owned by said companies. They could decrypt those data whenever they want.

Or you could do it like what Apple did with Siri, storing data locally, and use Differential Privacy to help ensure anonymity but it reduces the AI capabilities because it doesn’t have access to sufficient amount of personal data. Two, Siri runs on devices like Apple Watch, iPhones and iPads, which could be a problem when it comes to processing and compute capabilities, and having enough information to understand the user.

Although those devices have more processing power than room-sized mainframes from decades ago, it’s still not enough, energy-efficiency and capability wise, to handle highly complex neural networks for better experience with voice user interfaces.

Apple did try to change that with its A11 Bionic SoC that has a neural engine. Companies like Qualcomm, Imagination Technologies, and even NVIDIA are also contributing to increase local processing power with energy efficiency for AI through their respective CPU and GPU products.

Possible Solution

The work on the hardware by companies should continue so that there will be even more powerful and energy efficient processors for AI to use.

In addition to that, what we need is a standard, wireless-based protocol (maybe bluetooth) for the AI on our devices, irrespective of companies, to talk to each other when they are near to each other and in our home network. This way, the AI on each of those devices can share information and perform distributed computing, thereby improving its accuracy, overall understanding of the user, and respond accordingly.

A common software kernel is also necessary to provide different implementation of neural network a standardized way of doing distributed computing efficiently and effectively.

So now, imagine Siri talking to Alexa, Google Assistant or even Cortana via this protocol and vice versa.

Taking privacy into account, information exchanged via this protocol should be encrypted by default with keys owned only by the user. Any data created or stored should only reside on device also encrypted and nowhere else. Taking a page out of Apple’s playbook, the generated keys should come from some kind of hardware-based “Secure Enclave”.

To further improve the neural network, Differential Privacy should be applied on any query or information sent by the AI to the cloud for processing.

Conclusion

The above is really just a thought of how current the AIs powering voice user interfaces can be improved.

At the end, it’s really up to the companies to decide if they want to come together and improve all our lives taking into account our privacy and security.

Thought Snapshot #2

Snippet #1

Being un-busy doesn’t mean not doing any shit. It simply means doing the things that matter, the things that are essentials. Everything else is just noise. Don’t clogged up your calendar.

Snippet #2

Alcohol is a useful substance that help me minimise or neutralise my neuroticism. It remove my automatic, excessive self-control and allow me to be freer, happier, more extroverted.

Snippet #3

People put money on-top of everything else. I put environment conservation on-top. Thus I recycle.

Thoughts Snapshot #1

Introduction

Every day, from time to time, I would have these thoughts or comments about the world that I will publish straight to my Facebook account. But since my Facebook is mostly private, I decided to consolidate and share these thoughts or comments with the world as a snapshot entry. Previously, I have always published them as collections of random thought snippets under the Journal category. Now, I felt that they should be in their own category on my blog. In some cases, you may find that they are more suitable as a journal entry but I prefer to keep them as pure as they come. And if I don’t write them down somewhere, they are lost forever until one day, if I’m lucky, I will remember them.

Snippet #1

Most software companies aren’t design driven. They always apply engineering thinking to everything. They are all about the tangible output, fast cash, and I think that is why these companies are not amongst the most valuable companies in the world. It is especially so of system integrators. They will remain that way until they shut down, go out of business or bought over. They don’t bring to the table solutions that cover the emotive aspect of how one might use a piece of software. It shows. Humans intuitively know whether tender loving care (TLC) was applied to a product they buy. Of course, there is nothing inherently wrong with being pragmatic, being all about the objectives but your end product could become way more valuable to the “normal” people you are selling your product to. It is the subjective experience that will bring the most money in.

Snippets #2

An ex-colleague saw me today and commented how much happier I look. I’m happier than ever before because I remain steadfast to my values, doing things and acting in accordance to my own value system. To the people around me, you will also get something tangible. E.g. if I’m happy at work, my output is consistent and stable. If I’m happy with life, you get more smiley face or more jovial actions from me. But if you step on any of my values, everything flips. So, to external parties, the equation is simple and very easy to balance.

Snippets #3

If you want to get a LV bag, do you complain about the price? If you want to get a Rolex, do you complain about the price? Apple products are the Rolexes or LVs of the technology world. You want it, then shut up and pay for it or don’t buy it.

Snippets #4

Want to do product design or design a great user experience? Make sure you have an enthographer or someone with background in anthropology giving advice.

Snippets #5

It’s perfectly alright if you want to spend up to 14 hours every day, Monday to Sunday working. But don’t complain after when everything else in your life starts falling apart and you feel extremely miserable. Like someone smarter, more hardworking, and tried stuff, compared to me, once said, “You pick up the boulder, you can put it down any time you want.”

Raising awareness about basic web security

Disclaimer: I am not a certified or experienced security expert but the issues raised in this entry has been repeated many times by many other security experts and are so easy to solve. Yet these problems constantly resurfaced.

Over the past ten years, our lives have increasingly move to the web. Unless you are a complete computer illiterate, lived in an extremely rural area and has no internet access, chances are you would have used computers and did some stuff online.

With that, security of any website that collect user information, including password, is of utmost important. At the minimum, sites that need user to login with some kind of username and password should use HTTPS throughout. Even HTTPS is not perfect. There should be additional security mechanism in place such as encrypting username and password even before you hit your web server.

I’m not going to claim that I am a security expert but there are just some things that irks me when it comes to the implementation of web security by some local sites.

So let’s take a look at some exhibits.

Exhibit A: VR-Zone

VR-Zone is one of the more popular tech site that comes with a forum. I stopped using the site ever since I noticed that they don’t implement HTTPS at all for their forums.

And to think that the login page is just a div modal on the forums. It’s obvious when you mouseover the login button, it shows at the bottom of the page.

So after you clicked it, the login dialog pops up.

Now let’s take a look at the source for the dialog box. Let’s see the section I’ve highlighted.

I’m confused. Is it because the password is md5-hashed onsubmit that the site owner thought HTTPS isn’t necessary? Even so, MD5 hash is obsolete in this day and age. I can just use any off-the-shell CPU and brute force the password once I have sniffed the traffic between the browser and the server. Without HTTPS make all that so much more easier.

Now let’s assume that the site owner wakes up and decides the whole site should use HTTPS.

But the protocol for the form action is hardcoded to use HTTP instead of using relative path. Using relative path would have automatically ensure the form submission use whatever the original protocol is.

Put it this way, humans make mistakes and are near sighted. Therefore, there is a chance that people forget to clean up their codes or that the site owner don’t give a shit. Besides, not all web developers are good at their job. And when they are rushing to meet deadline, security isn’t even one of the priorities. Then combine the fact Singapore is particularly famous for outsourcing instead of getting the right people for the job… See where I’m going with this?

I don’t even know how the site owner store user credentials or data. Since the “security” of the site is almost non-existent, I doubt the security of the backend components. Also, VBulletin is well-known for having a fairly long list of security issues. Some of my personal data had been leaked in the past as part of data breaches of forums that uses VBulletin. Here you can see the amount of security flaws or bugs found with the software.

I also found at least one feedback about this HTTP/HTTPS issue in the forum itself. So I’m not the only one.

Since the post was made in April and there has been no response from the site owner. It looks like the security of user data is not the main concern. Also, I’m assuming the site is hosted in Singapore. If that’s the case, they could, and I mean could, be in breach of the Personal Data Protection Act.

Exhibit A: Hardware Zone

Hardware Zone is owned by Singapore Press Holding. I am expecting the security to be better than most sites hosted in Singapore.

The main page does not use HTTPS.

The forum does not use HTTPS.

When you hover your mouse over the login icon.

This is what you get. You will be redirected to a HTTPS page to do login.

Since the main site is not protected by HTTPS, it would be relatively easy to modify the page content before it is send to your browser. Some malicious person may just decide to change the login URL and present you a lookalike login form. You enter your login credentials and that’s it. Considering that Hardware Zone does collect personal information such as your age, address, name and many more, if hackers do manage to get your login credential stolen, well, it means problem for you.

The login form itself is protected by HTTPS.

Let’s look at the source code.

At least they used relative path for the form action. Looks like the MD5 hashing is a default thing for VBulletin. But still, MD5? It’s time to seriously look into SHA-256 or SHA-512.

After you login however, you are redirected back to HTTP again. Even when you access your user profile, it’s on HTTP too. This is like a half-hearted attempt to do HTTPS. Shame on you SPH.

Thoughts about Microsoft Build 2017

Well, I’m late to the party. Other news sites, bloggers, and vloggers have all covered it. They have their own thoughts about it. I’m going to do my own take on it, on only features that I’m interested in

For those who don’t already know, Microsoft held an event in Seattle between May 10 to 12, 2017 that introduced to the world a whole bunch of new things for Windows 10 that we all should be excited about — by that I mean techies.

To summarize, during the event, Microsoft pointed out several trends that’s going on that most of us already know: the rise of IOT devices, Artificial Intelligence, Serverless Architecture. They also talked about what they are doing regarding those trends. They are digging in deeper, delivering better solutions or services that customers need.

I’m more interested in their new Fluent Design. User experience and graphics is important to me (though I can’t really draw very well). Microsoft has attempted over the past decade to deliver a nice interface. They introduced Aero with Windows Vista, which was a massive flop. It was slow and sluggish. I’ve personally used it and don’t really like it. Then came Windows 7 and they improved Aero. It was much snappier and still look great.

Then came Windows 8 where they introduced what was known then as Metro design, subsequently known as Modern UI and then Microsoft Design Language. I personally liked it more than the pseudo-3D of Aero in Windows 7 because it is simplistic and minimalistic. It was improved Windows 8.1 and then in Windows 10.

However, the biggest gripe about Microsoft’s Windows was that the user interface isn’t consistent. There are some glaring defects that any good user interface designer should pick up on and fix it. Then, there is the fact that there are thousands if not millions of applications that runs on it. Many of which are legacy application and their user interface hasn’t been updated in ages. On the other hand, the web user interfaces have gotten so much better though every webpage is starting to look like each other due to the extensive use of common frameworks like bootstrap.

With Fluent Design, Microsoft aim to make Windows look great. As a design language, there are five main features:
1. Light
2. Depth
3. Motion
4. Material
5. Scale

You can read more about the Fluent Design System here.

When you combine all five of them properly in your application, the experience the user get will be so much better, richer and immerse them. This design language is in part, I believe due to the increase use and popularity of mixed-reality devices.

As someone who has been using Apple for more than year and already buy into the ecosystem, I’m somewhat excited to see how future applications made for Microsoft Windows will look like and how it will affect the way we as users interact with our devices. My concern was that, there is always the issue where the third-party developers do not ensure their app is consistent with the overall look and feel of the Windows running on the user’s computer.

Right now, the Fluent Design System is implemented in a somewhat beta form with the insiders build of Windows 10. If you are interested, you can download that and try it. Until it is released fully and used by many of the applications in Windows, I will stick with the experience I’m getting from Apple’s ecosystem.